A pruning method and device for a multi-task neural network model

Abstract

An embodiment of the present invention discloses a pruning method and device for a multi-task neural network model, the method comprising: obtaining a model to be pruned; executing steps A-C for each convolutional layer of the model to be pruned to determine the sensitivity information of each convolutional layer; A pre-pruning the convolutional layer based on the preset layer pruning rate and each convolution kernel in the convolutional layer for each preset layer pruning rate; B determining, based on first training data, the loss value of each task corresponding to the pre-pruned model to be pruned corresponding to the preset layer pruning rate; C determining, based on the loss value of each task, the sensitivity information of the convolutional layer at the preset layer pruning rate; grouping the convolution kernels of the model to be pruned based on channel information corresponding to the model to be pruned; and determining a target pruning model based on the current model pruning rate, the grouping of the convolution kernels, the pruning norm value corresponding to each convolution kernel, and the sensitivity information of each convolution layer, so as to achieve pruning of the multi-task neural network model.

Description

Technical Field

[0001] The present invention relates to the field of computer technology, and in particular to a pruning method and device for a multi-task neural network model. Background Art

[0002] The development of deep learning has led to the increasing application of deep neural networks in computer vision tasks such as image recognition, detection, and tracking, and network models are increasingly being designed to be wider and deeper. The success of deep learning relies heavily on the large number of model parameters and powerful computing devices. However, the huge memory requirements and computational overhead of deep neural networks make it difficult to deploy them on low-memory, low-power hardware platforms (such as mobile devices), which greatly limits their application. Therefore, studying how to effectively compress neural network models while maintaining the performance of existing deep neural network models is an important issue that needs to be addressed.

[0003] Therefore, how to provide a pruning method for multi-task neural network models becomes an urgent problem to be solved. Summary of the Invention

[0004] The present invention provides a method and apparatus for pruning a multi-task neural network model to achieve pruning of the multi-task neural network model. The specific technical solution is as follows:

[0005] In a first aspect, an embodiment of the present invention provides a pruning method for a multi-task neural network model, the method comprising:

[0006] Obtain the trained multi-task neural network model as the model to be pruned;

[0007] For each convolutional layer of the model to be pruned, perform the following steps AC to determine the sensitivity information of each convolutional layer under different preset layer pruning rates;

[0008] Step A: Pre-pruning the convolutional layer based on the preset layer pruning rate and the convolution kernels in the convolutional layer;

[0009] Step B: Based on the first training data, determining the loss value of each task corresponding to the pre-pruned to-be-pruned model corresponding to the preset layer pruning rate;

[0010] Step C: Based on the loss values ​​of each task corresponding to the pre-pruned model, determine the sensitivity information of the convolutional layer at the preset layer pruning rate;

[0011] Grouping the convolution kernels in the convolution layer of the model to be pruned based on the channel information corresponding to the model to be pruned;

[0012] The target pruning model is determined based on the current model pruning rate, the grouping of the convolution kernels of the to-be-pruned model, the pruning norm value corresponding to each convolution kernel in each convolution layer, and the sensitivity information of each convolution layer at different layer pruning rates.

[0013] Optionally, the step of determining a target pruning model based on the current model pruning rate, the grouping of the convolution kernels of the to-be-pruned model, the pruning norm value corresponding to each convolution kernel in each convolution layer, and the sensitivity information of each convolution layer at different layer pruning rates includes:

[0014] Pruning the model to be pruned based on the current model pruning rate, the grouping of the convolution kernels of the model to be pruned, the pruning norm value corresponding to each convolution kernel in each convolution layer, and the sensitivity information of each convolution layer under different layer pruning rates to obtain the current pruned model;

[0015] Determining, based on the second training data, detection information corresponding to each second training data corresponding to the current pruning model;

[0016] If, based on the detection information corresponding to each second training data corresponding to the current pruning model and the calibration information in each second training data, it is determined that the current pruning model meets the preset pruning conditions, the current pruning model is determined as the target pruning model.

[0017] Optionally, the method further includes:

[0018] If, based on the detection information corresponding to each second training data corresponding to the current pruning model and the calibration information in each second training data, it is determined that the current pruning model does not meet the preset pruning condition, the value of the current model pruning rate is adjusted to obtain a new current model pruning rate;

[0019] Based on the new current model pruning rate, the grouping of the convolution kernels of the to-be-pruned model, the pruning norm value corresponding to each convolution kernel in each convolution layer, and the sensitivity information of each convolution layer under different layer pruning rates, the to-be-pruned model is pruned to obtain a new current pruning model;

[0020] Determining, based on the third training data, detection information corresponding to each third training data corresponding to the new current pruning model;

[0021] If, based on the detection information corresponding to each third training data corresponding to the new current pruning model and the calibration information in each third training data, it is determined that the new current pruning model meets the preset pruning condition, the new current pruning model is determined as the target pruning model;

[0022] If, based on the detection information corresponding to each third training data corresponding to the new current pruning model and the calibration information in each third training data, it is determined that the new current pruning model does not meet the preset pruning conditions, the step of adjusting the value of the current model pruning rate is returned to obtain a new current model pruning rate.

[0023] Optionally, the step of determining the sensitivity information of the convolutional layer at the preset layer pruning rate based on the loss values ​​of each task corresponding to the pre-pruned model includes:

[0024] Determine the loss value corresponding to the pre-pruned model based on the loss value of each task corresponding to the pre-pruned model and the preset objective function;

[0025] Based on the loss value corresponding to the to-be-pruned model after the pre-pruning, the sensitivity information of the convolutional layer at the preset layer pruning rate is determined; wherein the expression of the preset objective function is:

[0026]

[0027] Among them, Objective represents the loss value corresponding to the pruned model after pre-pruning, loss i represents the task loss value corresponding to the i-th task in the model to be pruned, w i represents the task weight corresponding to the i-th task in the model to be pruned; L represents the number of tasks in the model to be pruned.

[0028] Optionally, the step of pre-pruning the convolutional layer based on the preset layer pruning rate and each convolution kernel in the convolutional layer includes:

[0029] For each convolution kernel of the convolution layer, based on each weight parameter in the convolution kernel, determine a pruning norm value corresponding to the convolution kernel;

[0030] Based on the preset layer pruning rate and the pruning norm value corresponding to each convolution kernel in the convolution layer, the convolution layer is pre-pruned according to a preset pruning rule, wherein the preset pruning rule is: pre-pruning is performed preferentially on convolution kernels with small corresponding pruning norm values.

[0031] In a second aspect, an embodiment of the present invention provides a pruning device for a multi-task neural network model, the device comprising:

[0032] An acquisition module is configured to obtain a trained multi-task neural network model as a model to be pruned;

[0033] The first determination module is configured to determine, for each convolutional layer of the to-be-pruned model, sensitivity information of each convolutional layer under different preset layer pruning rates through the following units;

[0034] a pre-pruning unit configured to pre-prune the convolutional layer based on the preset layer pruning rate and each convolution kernel in the convolutional layer for each preset layer pruning rate;

[0035] A first determining unit is configured to determine, based on the first training data, a loss value of each task corresponding to the pre-pruned to-be-pruned model corresponding to the preset layer pruning rate;

[0036] The second determining unit is configured to determine the sensitivity information of the convolutional layer at the preset layer pruning rate based on the loss values ​​of each task corresponding to the to-be-pruned model after pre-pruning;

[0037] A grouping module is configured to group the convolution kernels in the convolution layer of the to-be-pruned model based on the channel information corresponding to the to-be-pruned model;

[0038] The second determination module is configured to determine the target pruning model based on the current model pruning rate, the grouping of the convolution kernels of the model to be pruned, the pruning norm value corresponding to each convolution kernel in each convolution layer, and the sensitivity information of each convolution layer at different layer pruning rates.

[0039] Optionally, the second determining module includes:

[0040] A first pruning unit is configured to prune the model to be pruned based on a current model pruning rate, a grouping of convolution kernels of the model to be pruned, a pruning norm value corresponding to each convolution kernel in each convolution layer, and sensitivity information of each convolution layer at different layer pruning rates to obtain a current pruned model;

[0041] A third determining unit is configured to determine, based on the second training data, detection information corresponding to each second training data corresponding to the current pruning model;

[0042] The fourth determination unit is configured to determine that the current pruning model meets the preset pruning conditions based on the detection information corresponding to each second training data corresponding to the current pruning model and the calibration information in each second training data, and determine the current pruning model as the target pruning model.

[0043] Optionally, the second determining module further includes:

[0044] an adjusting unit configured to, if it is determined that the current pruning model does not meet the preset pruning condition based on the detection information corresponding to each second training data corresponding to the current pruning model and the calibration information in each second training data, adjust the value of the current model pruning rate to obtain a new current model pruning rate;

[0045] The second pruning unit is configured to prune the model to be pruned based on the new current model pruning rate, the grouping of the convolution kernels of the model to be pruned, the pruning norm value corresponding to each convolution kernel in each convolution layer, and the sensitivity information of each convolution layer under different layer pruning rates to obtain a new current pruning model;

[0046] A fifth determining unit is configured to determine, based on the third training data, detection information corresponding to each third training data corresponding to the new current pruning model;

[0047] a sixth determining unit configured to, if determining that the new current pruned model meets a preset pruning condition based on the detection information corresponding to each third training data corresponding to the new current pruned model and the calibration information in each third training data, determine the new current pruned model as a target pruned model;

[0048] If, based on the detection information corresponding to each third training data corresponding to the new current pruning model and the calibration information in each third training data, it is determined that the new current pruning model does not meet the preset pruning condition, the adjustment unit is triggered.

[0049] Optionally, the second determining unit is specifically configured to determine the loss value corresponding to the pre-pruned model to be pruned based on the loss value of each task corresponding to the pre-pruned model to be pruned and a preset objective function;

[0050] Based on the loss value corresponding to the to-be-pruned model after the pre-pruning, the sensitivity information of the convolutional layer at the preset layer pruning rate is determined; wherein the expression of the preset objective function is:

[0051]

[0052] Among them, Objective represents the loss value corresponding to the pruned model after pre-pruning, loss i represents the task loss value corresponding to the i-th task in the model to be pruned, w i represents the task weight corresponding to the i-th task in the model to be pruned; L represents the number of tasks in the model to be pruned.

[0053] Optionally, the pre-pruning unit is specifically configured to determine, for each convolution kernel of the convolution layer, a pruning norm value corresponding to the convolution kernel based on each weight parameter in the convolution kernel;

[0054] Based on the preset layer pruning rate and the pruning norm value corresponding to each convolution kernel in the convolution layer, the convolution layer is pre-pruned according to a preset pruning rule, wherein the preset pruning rule is: pre-pruning is performed preferentially on convolution kernels with small corresponding pruning norm values.

[0055] From the above content, it can be seen that the embodiment of the present invention provides a pruning method and device for a multi-task neural network model, which obtains a trained multi-task neural network model as a model to be pruned; for each convolutional layer of the model to be pruned, the following steps AC are performed to determine the sensitivity information of each convolutional layer under different preset layer pruning rates; Step A: for each preset layer pruning rate, based on the preset layer pruning rate and each convolution kernel in the convolutional layer, pre-prune the convolutional layer; Step B: Based on the first training data, determine the sensitivity of the preset layer pruning rate to the convolutional layer; Step C: Based on the first training data, determine the sensitivity of the preset layer pruning rate to the convolutional layer; Step D ... D: Based on the first training data, determine the sensitivity of the preset layer pruning rate to the convolutional layer; Step D: Based on the first training data, determine the sensitivity of the preset layer pruning rate to the convolutional layer; Step D: The loss value of each task corresponding to the pre-pruned model to be pruned; Step C: Based on the loss value of each task corresponding to the pre-pruned model to be pruned, determine the sensitivity information of the convolution layer under the preset layer pruning rate; Based on the channel information corresponding to the model to be pruned, group the convolution kernels in the convolution layer of the model to be pruned; Based on the current model pruning rate, the grouping of the convolution kernels of the model to be pruned, the pruning norm value corresponding to each convolution kernel in each convolution layer, and the sensitivity information of each convolution layer under different layer pruning rates, determine the target pruning model.

[0056] By applying the embodiment of the present invention, the task loss values ​​corresponding to the multiple tasks in the network to be pruned are used to jointly determine the sensitivity information of each convolution layer under different preset layer pruning rates. Then, based on the current model pruning rate, the grouping of the convolution kernels of the model to be pruned, the pruning norm values ​​corresponding to the convolution kernels in each convolution layer, and the sensitivity information of each convolution layer under different layer pruning rates, the target pruning model is determined to achieve pruning of the multi-task neural network model. In addition, the task loss values ​​corresponding to the multiple tasks are used as pruning targets, and the sensitivity analysis and pruning are performed by comprehensively considering the accuracy of multiple tasks, effectively retaining the generalization performance of the sparse model, being able to balance the accuracy loss of multiple tasks of the pruned model, and reducing the difficulty of retraining. Of course, implementing any product or method of the present invention does not necessarily require achieving all the advantages described above at the same time.

[0057] The innovative features of the embodiments of the present invention include:

[0058] 1. Using the task loss values ​​corresponding to multiple tasks in the network to be pruned, the sensitivity information of each convolutional layer under different preset layer pruning rates is jointly determined. Then, based on the current model pruning rate, the grouping of the convolution kernels in the pruned model, the pruning norm values ​​corresponding to each convolution kernel in each convolutional layer, and the sensitivity information of each convolution layer under different layer pruning rates, the target pruning model is determined to achieve pruning of the multi-task neural network model. In addition, the task loss values ​​corresponding to multiple tasks are used as pruning targets. The sensitivity analysis and pruning of the model are comprehensively considered based on the accuracy of multiple tasks. This effectively preserves the generalization performance of the sparse model, balances the accuracy loss of multiple tasks in the pruned model, and reduces the difficulty of retraining.

[0059] 2. Set the preset pruning conditions. Based on the current loss value corresponding to the current pruned model, determine whether the current pruned model meets the preset pruning conditions to determine whether the pruned model is successfully pruned. If it is determined that the pruned model is successfully pruned, determine the current pruned model as the target pruned model. If it is determined that the pruned model is not successfully pruned, adjust the value of the current pruned model and re-prune the pruned model until it is determined that the pruned model is successfully pruned to obtain the target pruned model, so as to obtain a target pruned model with good detection information and small storage space requirements.

[0060] 3. Using the preset objective function, based on the loss value of each task and its corresponding task weight, determine the loss value corresponding to the pruned model after pre-pruning, and then determine the sensitivity information of the convolutional layer under the preset layer pruning rate, so as to achieve sensitivity analysis and pruning of the model by comprehensively considering the accuracy of multiple tasks, and realize effective pruning of the model. BRIEF DESCRIPTION OF THE DRAWINGS

[0061] To more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are merely some embodiments of the present invention. Those skilled in the art can derive other drawings based on these drawings without inventive effort.

[0062] Figure 1 A schematic diagram of a flow chart of a pruning method for a multi-task neural network model provided by an embodiment of the present invention;

[0063] Figure 2 A schematic diagram of the structure of a pruning device for a multi-task neural network model provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0064] The following will provide a clear and complete description of the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of them. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0065] It should be noted that the terms "including" and "having" and any variations thereof in the embodiments of the present invention and the accompanying drawings are intended to cover non-exclusive inclusions. For example, a process, method, system, product, or apparatus comprising a series of steps or units is not limited to the listed steps or units, but may optionally include steps or units not listed, or may optionally include other steps or units inherent to the process, method, product, or apparatus.

[0066] The present invention provides a method and apparatus for pruning a multi-task neural network model, so as to implement pruning of the multi-task neural network model. The embodiments of the present invention are described in detail below.

[0067] Figure 1 A flow chart of a pruning method for a multi-task neural network model provided by an embodiment of the present invention. The method may include the following steps:

[0068] S101: Obtain a trained multi-task neural network model as a model to be pruned.

[0069] The pruning method for a multi-task neural network model provided in an embodiment of the present invention can be applied to any electronic device with computing capabilities, which may be a terminal or a server.

[0070] The multi-task neural network model is a neural network model that can implement at least two detection tasks. The detection tasks may include but are not limited to tasks such as classification, localization, and semantic segmentation. The present invention does not limit the specific task types of the multi-task neural network model.

[0071] The trained multi-task neural network model may refer to: a multi-task neural network model that has been trained based on training data and whose detection accuracy of detection information is not lower than a preset accuracy threshold.

[0072] S102: For each convolutional layer of the model to be pruned, execute the following steps S1021-S1023 to determine the sensitivity information of each convolutional layer under different preset layer pruning rates.

[0073] S1021: Pre-pruning the convolutional layer based on the preset layer pruning rate and each convolution kernel in the convolutional layer;

[0074] S1022: Determine, based on the first training data, the loss value of each task corresponding to the pre-pruned model to be pruned corresponding to the preset layer pruning rate;

[0075] S1023: Based on the loss values ​​of each task corresponding to the to-be-pruned model after pre-pruning, determine the sensitivity information of the convolutional layer at the preset layer pruning rate.

[0076] The preset layer pruning rate is used to indicate the proportion of convolutional layers that need to be pruned during the pre-pruning process for each convolutional layer. That is, the ratio of the convolution kernels that need to be pre-pruned in the pre-pruned convolutional layer to the total convolution kernels in the convolutional layer. The value range of the preset layer pruning rate is (0, 1). There are multiple preset layer pruning rates, and the difference between each two adjacent preset layer pruning rates in the multiple preset layer pruning rates can be the same.

[0077] In this step, the electronic device can determine the sensitivity information of each convolutional layer under different preset layer pruning rates for each convolutional layer of the model to be pruned, that is, for each convolutional layer of the model to be pruned, in turn, for each preset layer pruning rate, based on the preset layer pruning rate and each convolution kernel in the convolutional layer, pre-prune the convolutional layer, that is, based on each convolution kernel in the convolutional layer, determine the convolution kernel to be pre-pruned corresponding to the preset layer pruning rate, and pre-prune the convolution kernel to be pre-pruned corresponding to the preset layer pruning rate, for example: set the preset layer pruning rate The weight value corresponding to the corresponding convolution kernel to be pre-pruned is 0, thereby pre-pruning the convolution kernel to be pre-pruned corresponding to the preset layer pruning rate; further, based on the first training data, the loss value of each task corresponding to the pre-pruned model to be pruned corresponding to the preset layer pruning rate is determined, that is, the task loss value corresponding to each of the multiple tasks; based on the loss value of each task corresponding to the pre-pruned model to be pruned, the sensitivity information of the convolution layer at the preset layer pruning rate is determined, so as to determine the sensitivity information of each convolution layer of the model to be pruned at each preset layer pruning rate. For example, if there are 10 convolution layers in the model to be pruned, and the preset layer pruning rate includes an integer percentage between 1% and 99%, that is, 99, then the number of sensitivity information of all convolution layers of the model to be pruned includes 10*99=990.

[0078] Among them, the first training data can be part of the training data for training the model to be pruned, or it can be other training data corresponding to the model to be pruned, which is all possible. The first training data may include a sample image and its corresponding calibration information, wherein the sample image includes the target that the model to be pruned needs to detect, and the calibration information corresponding to the sample image includes the specific content of the detection label corresponding to each task of the model to be pruned. For example: the target that the model to be pruned needs to detect is a person, and the detection label corresponding to each task of the model to be pruned is the identity of the person and the location of the person. Accordingly, the sample image includes each person, and the calibration information corresponding to the sample image includes: the identity information of the included person and the location information in the image.

[0079] Accordingly, in one implementation, the process of determining the loss value of each task corresponding to the pre-pruned model to be pruned corresponding to the preset layer pruning rate based on the first training data may be: inputting each sample image in the first training data into the pre-pruned model to be pruned corresponding to the preset layer pruning rate to obtain prediction information corresponding to the sample image, wherein the prediction information includes prediction information corresponding to each task of the model to be pruned; for each task, based on the preset loss function corresponding to the task, the prediction information corresponding to the task corresponding to the sample image, and the calibration information corresponding to the task corresponding to the sample image in the first training data, determining the task sub-loss value corresponding to the task corresponding to the sample image, so as to obtain the task sub-loss value corresponding to the task corresponding to the sample image; and for each task, based on the task sub-loss value corresponding to the task corresponding to each sample image, determining the task loss value corresponding to the task. The preset loss function corresponding to each task may be any type of loss function in the related art and is not limited here.

[0080] Among them, the smaller the change in the task loss value corresponding to the task before and after pre-pruning, the smaller the impact of the convolution kernel corresponding to the preset layer pruning rate of the pre-pruning in the convolution layer on the detection information of the task corresponding to the task loss value, that is, the smaller the impact of the convolution kernel corresponding to the preset layer pruning rate of the pre-pruning on the task detection information corresponding to the task loss value. Accordingly, it can be characterized that for the task corresponding to the task loss value, the sensitivity of the convolution kernel corresponding to the preset layer pruning rate of the pre-pruning is not high.

[0081] In one implementation of the present invention, the step S1021 may include the following steps:

[0082] For each convolution kernel of the convolution layer, based on each weight parameter in the convolution kernel, determine a pruning norm value corresponding to the convolution kernel;

[0083] Based on the preset layer pruning rate and the pruning norm value corresponding to each convolution kernel in the convolution layer, the convolution layer is pre-pruned according to a preset pruning rule, wherein the preset pruning rule is: pre-pruning is performed preferentially on convolution kernels with small corresponding pruning norm values.

[0084] In this implementation, the electronic device may determine, for each convolution kernel of the convolution layer, a pruning norm value corresponding to the convolution kernel based on the weight parameters in the convolution kernel using the L1-norm algorithm. Specifically, the absolute values ​​of the weight parameters in the convolution kernel may be summed, and the result obtained is used as the pruning norm value corresponding to the convolution kernel. A larger pruning norm value corresponding to a convolution kernel indicates a more important convolution kernel, and a lower probability of selecting the convolution kernel for pruning during pre-pruning and subsequent pruning processes.

[0085] Subsequently, based on the preset layer pruning rate and the pruning norm values ​​corresponding to each convolution kernel in the convolution layer, the convolution layer is pre-pruned according to the preset pruning rules. Specifically, pre-pruning is performed preferentially on convolution kernels with small pruning norm values. Specifically, within the convolution layer, convolution kernels and their corresponding output feature map channels are pruned from the smallest to the largest pruning norm values ​​according to the preset layer pruning rate.

[0086] Among them, the above-mentioned pruning of the convolution kernel and its corresponding output feature map channel can be: setting the weight parameters contained in the convolution kernel to 0, or setting the weight value corresponding to the convolution kernel to 0, and setting the weight value corresponding to the output feature map channel corresponding to the convolution kernel to 0.

[0087] S103: Grouping the convolution kernels in the convolution layer of the to-be-pruned model based on the channel information corresponding to the to-be-pruned model.

[0088] In one implementation, there may be multiple convolutional layers in the model to be pruned that are connected by the ElementwiseLayer function. In view of the data processing method of the ElementwiseLayer function, for the multiple convolutional layers in the model to be pruned that are connected by the ElementwiseLayer function, when it is necessary to prune a convolution kernel in a certain layer, it is necessary to use a convolution kernel with the same channel identifier as the channel identifier corresponding to the convolution kernel to be pruned in other convolutional layers. In view of this, the electronic device can pre-determine the channel information corresponding to the model to be pruned, and then, based on the channel information corresponding to the model to be pruned, group the convolution kernels in the convolution layer of the model to be pruned. Among them, the convolution kernels corresponding to the same channel identifier are divided into one group, the convolution kernels corresponding to different channel identifiers are divided into different groups, and the convolution kernels of the convolution layers that are not connected by the ElementwiseLayer function, that is, the convolution kernels that do not correspond to the channel identifier can be divided into one group.

[0089] S104: Determine the target pruning model based on the current model pruning rate, the grouping of the convolution kernels of the model to be pruned, the pruning norm value corresponding to each convolution kernel in each convolution layer, and the sensitivity information of each convolution layer at different layer pruning rates.

[0090] The electronic device may pre-store a current model pruning rate locally or in a connected storage device. The model pruning rate is used to limit the ratio of the convolution kernels and their corresponding output feature map channels subtracted during the pruning process of the model to be pruned, that is, to limit the ratio of the total number of the convolution kernels and their corresponding output feature map channels subtracted during the pruning process of the model to be pruned to the total number of the convolution kernels and their corresponding output feature map channels included in the model to be pruned.

[0091] The pruning norm value corresponding to each convolution kernel in each convolution layer is: a value determined based on the weight parameters in the convolution kernel, which is used to characterize the importance of the convolution kernel in its convolution layer. The larger the pruning norm value corresponding to the convolution kernel in the convolution layer, the more important the convolution kernel is, and the smaller the probability of choosing to prune the convolution kernel in the pre-pruning and subsequent pruning processes.

[0092] After the electronic device groups the convolution kernels in the convolution layer of the model to be pruned, the grouping situation of the convolution kernels of the model to be pruned can be obtained, wherein, for the convolution kernels corresponding to the same channel identifier, they need to be pruned or retained at the same time, and for the convolution kernels that do not correspond to the channel identifier, they are pruned or retained based on their corresponding pruning norm values.

[0093] The electronic device can determine the pruning rate corresponding to each convolution layer based on the current model pruning rate and the sensitivity information of each convolution layer at different pruning rates, wherein the sensitivity information of the convolution layer indicates that the less sensitive it is, the higher its corresponding pruning rate. Furthermore, the electronic device determines the specific convolution kernels and their corresponding output feature map channels that need to be pruned for each convolution layer based on the pruning rate corresponding to the convolution layer, the grouping of the convolution kernels of the model to be pruned, and the pruning norm value corresponding to each convolution kernel in each convolution layer; based on the specific convolution kernels and their corresponding output feature map channels that need to be pruned for each convolution layer, pruning is performed to obtain a target pruned model.

[0094] Among them, the above principles for determining the specific convolution kernels and their corresponding output feature map channels that need to be pruned in each convolution layer may include: giving priority to pruning the convolution kernels with small corresponding pruning norm values ​​in the convolution layer, and when a convolution kernel in a convolution layer needs to be pruned, and the convolution kernel belongs to a convolution kernel group corresponding to a channel identifier, all corresponding convolution kernels in the convolution kernel group and their corresponding output feature map channels need to be pruned.

[0095] By applying the embodiment of the present invention, the task loss values ​​corresponding to multiple tasks in the network to be pruned are used to jointly determine the sensitivity information of each convolutional layer under different preset layer pruning rates. Then, based on the current model pruning rate, the grouping of the convolution kernels of the model to be pruned, the pruning norm values ​​corresponding to each convolution kernel in each convolutional layer, and the sensitivity information of each convolution layer under different layer pruning rates, the target pruning model is determined to achieve pruning of the multi-task neural network model. In addition, the task loss values ​​corresponding to multiple tasks are used as pruning targets, and sensitivity analysis and pruning are performed by comprehensively considering the accuracy of multiple tasks, effectively retaining the generalization performance of the sparse model, balancing the accuracy loss of multiple tasks in the pruned model, and reducing the difficulty of retraining.

[0096] Taking into account that the current model pruning rate may not be set appropriately, the current model pruning rate may be set too low. Theoretically, under the constraint of detection accuracy, the model to be pruned can also prune more convolution kernels and / or their corresponding output feature map channels, that is, a model with a smaller amount of data after pruning and a relatively high detection accuracy can be obtained; or the current model pruning rate may be set too high, and the amount of data of the model obtained after pruning is reduced, but the detection accuracy of the model obtained after pruning is not high. The models obtained after pruning in the above cases can all be considered as inappropriate models. In view of this, in another embodiment of the present invention, the S104 may include the following steps:

[0097] Based on the current model pruning rate, the grouping of the convolution kernels of the to-be-pruned model, the pruning norm values ​​corresponding to the convolution kernels in each convolution layer, and the sensitivity information of each convolution layer under different pruning rates, the to-be-pruned model is pruned to obtain the current pruned model.

[0098] Determine, based on the second training data, detection information corresponding to each second training data corresponding to the current pruning model;

[0099] If it is determined that the current pruned model meets the preset pruning conditions based on the detection information corresponding to each second training data corresponding to the current pruned model and the calibration information in each second training data, the current pruned model is determined as the target pruned model.

[0100] The second training data may be part of the training data used to train the model to be pruned, or other training data corresponding to the model to be pruned. The second training data may include the same data as the first training data, or may include data different from the first training data.

[0101] The above process of determining the detection information corresponding to each second training data corresponding to the current pruning model based on the second training data can be: inputting the sample image in each second training data into the current pruning model to obtain the detection information of each task corresponding to each second training data.

[0102] The above-mentioned process of determining whether the current pruned model meets the preset pruning conditions based on the detection information corresponding to each second training data corresponding to the current pruned model and the calibration information in each second training data can be: based on the model obtained by pruning after the increased value of the current model pruning rate, the second training data, the detection information of each task corresponding to each determined second training data, the calibration information in each second training data, and the preset accuracy determination method corresponding to the task, determining the accuracy of the new pruned model of the task; if it is determined that the difference between the accuracy of the task corresponding to the to-be-pruned model and the accuracy of the new pruned model is not within an acceptable accuracy reduction range, and the difference is greater than the upper limit of the acceptable accuracy reduction range, it can be determined that the current pruned model meets the preset pruning conditions.

[0103] It can be understood that the preset accuracy determination methods corresponding to different tasks may be different. The specific preset accuracy determination method can refer to the accuracy determination method of each task of the neural network model in the relevant technology.

[0104] Taking the tasks of the model to be pruned as examples, including target detection and semantic segmentation, the pruning accuracy corresponding to the target detection task can be represented by the mAP (Mean Average Precision) value; the pruning accuracy corresponding to the semantic segmentation task can be represented by the mIOU (Mean Intersection-Over-Union) value. The calculation process of the mAP value and the mIOU value can be referred to the calculation process of the mAP value of the target detection task and the mIOU value of the semantic segmentation task in the neural network model in the related art, and will not be repeated here.

[0105] In another embodiment of the present invention, the step S104 may further include the following steps:

[0106] If, based on the detection information corresponding to each second training data corresponding to the current pruning model and the calibration information in each second training data, it is determined that the current pruning model does not meet the preset pruning condition, the value of the current model pruning rate is adjusted to obtain a new current model pruning rate;

[0107] Based on the new current model pruning rate, the grouping of the convolution kernels of the to-be-pruned model, the pruning norm value corresponding to each convolution kernel in each convolution layer, and the sensitivity information of each convolution layer under different layer pruning rates, the to-be-pruned model is pruned to obtain a new current pruning model;

[0108] Determine, based on the third training data, detection information corresponding to each third training data corresponding to a new current pruning model;

[0109] If, based on the detection information corresponding to each third training data corresponding to the new current pruned model and the calibration information in each third training data, it is determined that the new current pruned model meets the preset pruning condition, the new current pruned model is determined as the target pruned model;

[0110] If, based on the detection information corresponding to each third training data corresponding to the new current pruning model and the calibration information in each third training data, it is determined that the new current pruning model does not meet the preset pruning conditions, return to the step of adjusting the value of the current model pruning rate to obtain the new current model pruning rate.

[0111] If the electronic device determines that the difference between the accuracy of the task corresponding to the to-be-pruned model and the accuracy of the new pruned model is not within an acceptable accuracy degradation range, it can be determined that the current pruned model does not meet the preset pruning conditions.

[0112] Accordingly, the value of the current model pruning rate can be adjusted to obtain a new current model pruning rate. Based on the new current model pruning rate, the grouping of the convolution kernels of the model to be pruned, the pruning norm value corresponding to each convolution kernel in each convolution layer, and the sensitivity information of each convolution layer under different layer pruning rates, the model to be pruned is pruned to obtain a new current pruning model; based on the third training data, the detection information corresponding to each third training data corresponding to the new current pruning model is determined; if, based on the detection information corresponding to each third training data corresponding to the current pruning model and the calibration information in each third training data, it is determined that the new current pruning model meets the preset pruning conditions, the new current pruning model is determined as the target pruning model; if, based on the detection information corresponding to each third training data corresponding to the current pruning model and the calibration information in each third training data, it is determined that the new current pruning model does not meet the preset pruning conditions, return to the step of adjusting the value of the current model pruning rate to obtain the new current model pruning rate.

[0113] Among them, if it is judged that the difference between the accuracy of the task corresponding to the model to be pruned and the new pruning accuracy is not within the acceptable accuracy reduction range, and the difference between the accuracy of the task corresponding to the model to be pruned and the accuracy of the new pruned model exceeds the upper limit of the acceptable accuracy reduction range, then the value of the current model pruning rate can be reduced to obtain a new current model pruning rate; if it is judged that the difference between the accuracy of the task corresponding to the model to be pruned and the accuracy of the new pruned model is within the acceptable accuracy reduction range, and the difference between the accuracy of the task corresponding to the model to be pruned and the accuracy of the new pruned model is lower than the upper limit of the acceptable accuracy reduction range, then the value of the current model pruning rate can be increased to obtain a new current model pruning rate.

[0114] The third training data may be the same as the second training data, or may be data corresponding to the model to be pruned and different from the second training data.

[0115] In another embodiment of the present invention, the step S1023 may include the following steps:

[0116] Determine the loss value corresponding to the pre-pruned model based on the loss value of each task corresponding to the pre-pruned model and the preset objective function;

[0117] Based on the loss value corresponding to the to-be-pruned model after pre-pruning, the sensitivity information of the convolutional layer under the preset layer pruning rate is determined; wherein the expression of the preset objective function is:

[0118]

[0119] Among them, Objective represents the loss value corresponding to the pruned model after pre-pruning, loss i represents the task loss value corresponding to the i-th task in the model to be pruned, w i represents the task weight corresponding to the i-th task in the model to be pruned; L represents the number of tasks in the model to be pruned.

[0120] In one case, the change in loss value corresponding to the model to be pruned before and after pre-pruning can be determined as the sensitivity information at the preset layer pruning rate. For example, the difference between the loss value corresponding to the model to be pruned after pre-pruning and the loss value corresponding to the model to be pruned before pre-pruning can be determined as the sensitivity information of the convolutional layer at the preset layer pruning rate. Accordingly, the smaller the increase in the loss value corresponding to the model to be pruned after pre-pruning compared to the loss value corresponding to the model to be pruned before pre-pruning, the smaller the impact of the convolutional layer on the accuracy of the detection information of the pruned model at the preset layer pruning rate, and the lower the sensitivity of the convolutional layer at the preset layer pruning rate.

[0121] In one case, the task weights corresponding to the tasks can be selected according to the range of the task loss values ​​corresponding to the tasks, so that the weighted task loss values ​​are at the same data level.

[0122] Corresponding to the above method embodiment, the embodiment of the present invention provides a pruning device for a multi-task neural network model, such as Figure 2 As shown, the device may include:

[0123] An acquisition module 210 is configured to obtain a trained multi-task neural network model as a model to be pruned;

[0124] The first determination module 220 is configured to determine, for each convolutional layer of the to-be-pruned model, sensitivity information of each convolutional layer under different preset layer pruning rates through the following units;

[0125] A pre-pruning unit 221 is configured to pre-prune the convolutional layer based on the preset layer pruning rate and each convolution kernel in the convolutional layer;

[0126] The first determining unit 222 is configured to determine, based on the first training data, the loss value of each task corresponding to the pre-pruned to-be-pruned model corresponding to the preset layer pruning rate;

[0127] The second determining unit 223 is configured to determine the sensitivity information of the convolutional layer at the preset layer pruning rate based on the loss values ​​of each task corresponding to the to-be-pruned model after pre-pruning;

[0128] A grouping module 230 is configured to group the convolution kernels in the convolution layer of the to-be-pruned model based on the channel information corresponding to the to-be-pruned model;

[0129] The second determination module 240 is configured to determine the target pruning model based on the current model pruning rate, the grouping of the convolution kernels of the model to be pruned, the pruning norm value corresponding to each convolution kernel in each convolution layer, and the sensitivity information of each convolution layer at different layer pruning rates.

[0130] By applying the embodiment of the present invention, the task loss values ​​corresponding to multiple tasks in the network to be pruned are used to jointly determine the sensitivity information of each convolutional layer under different preset layer pruning rates. Then, based on the current model pruning rate, the grouping of the convolution kernels of the model to be pruned, the pruning norm values ​​corresponding to each convolution kernel in each convolutional layer, and the sensitivity information of each convolution layer under different layer pruning rates, the target pruning model is determined to achieve pruning of the multi-task neural network model. In addition, the task loss values ​​corresponding to multiple tasks are used as pruning targets, and sensitivity analysis and pruning are performed by comprehensively considering the accuracy of multiple tasks, effectively retaining the generalization performance of the sparse model, balancing the accuracy loss of multiple tasks in the pruned model, and reducing the difficulty of retraining.

[0131] In another embodiment of the present invention, the second determining module 240 includes:

[0132] A first pruning unit (not shown in the figure) is configured to prune the model to be pruned based on the current model pruning rate, the grouping of the convolution kernels of the model to be pruned, the pruning norm value corresponding to each convolution kernel in each convolution layer, and the sensitivity information of each convolution layer under different layer pruning rates to obtain a current pruned model;

[0133] A third determining unit (not shown in the figure) is configured to determine, based on the second training data, detection information corresponding to each second training data corresponding to the current pruning model;

[0134] A fourth determination unit (not shown in the figure) is configured to determine that the current pruning model meets the preset pruning conditions based on the detection information corresponding to each second training data corresponding to the current pruning model and the calibration information in each second training data, and determine the current pruning model as the target pruning model.

[0135] In another embodiment of the present invention, the second determining module 240 further includes:

[0136] an adjustment unit (not shown in the figure), configured to adjust a value of a current model pruning rate to obtain a new current model pruning rate if it is determined that the current pruning model does not meet a preset pruning condition based on detection information corresponding to each second training data corresponding to the current pruning model and calibration information in each second training data;

[0137] A second pruning unit (not shown) is configured to prune the model to be pruned based on the new current model pruning rate, the grouping of the convolution kernels of the model to be pruned, the pruning norm value corresponding to each convolution kernel in each convolution layer, and the sensitivity information of each convolution layer under different layer pruning rates to obtain a new current pruning model;

[0138] a fifth determining unit (not shown in the figure), configured to determine, based on the third training data, detection information corresponding to each third training data corresponding to the new current pruning model;

[0139] a sixth determining unit (not shown in the figure) configured to, if determining that the new current pruned model meets a preset pruning condition based on the detection information corresponding to each third training data corresponding to the new current pruned model and the calibration information in each third training data, determine the new current pruned model as a target pruned model;

[0140] If, based on the detection information corresponding to each third training data corresponding to the new current pruning model and the calibration information in each third training data, it is determined that the new current pruning model does not meet the preset pruning condition, the adjustment unit is triggered.

[0141] In another embodiment of the present invention, the second determining unit 223 is specifically configured to determine the loss value corresponding to the pre-pruned model to be pruned based on the loss value of each task corresponding to the pre-pruned model to be pruned and a preset objective function;

[0142] Based on the loss value corresponding to the to-be-pruned model after the pre-pruning, the sensitivity information of the convolutional layer at the preset layer pruning rate is determined; wherein the expression of the preset objective function is:

[0143]

[0144] Among them, Objective represents the loss value corresponding to the pruned model after pre-pruning, loss i represents the task loss value corresponding to the i-th task in the model to be pruned, w i represents the task weight corresponding to the i-th task in the model to be pruned; L represents the number of tasks in the model to be pruned.

[0145] In another embodiment of the present invention, the pre-pruning unit 221 is specifically configured to determine, for each convolution kernel of the convolution layer, a pruning norm value corresponding to the convolution kernel based on each weight parameter in the convolution kernel;

[0146] Based on the preset layer pruning rate and the pruning norm value corresponding to each convolution kernel in the convolution layer, the convolution layer is pre-pruned according to a preset pruning rule, wherein the preset pruning rule is: pre-pruning is performed preferentially on convolution kernels with small corresponding pruning norm values.

[0147] The above-mentioned system and device embodiments correspond to the system embodiment and have the same technical effects as the method embodiment. For detailed descriptions, please refer to the method embodiment. The device embodiment is obtained based on the method embodiment. For detailed descriptions, please refer to the method embodiment section and will not be repeated here. It should be understood by those skilled in the art that the accompanying drawings are only schematic diagrams of one embodiment, and the modules or processes in the accompanying drawings are not necessarily required to implement the present invention.

[0148] Those skilled in the art will appreciate that the modules in the apparatuses of the embodiments may be distributed in the apparatuses of the embodiments as described in the embodiments, or may be located in one or more apparatuses different from the embodiments with corresponding changes. The modules in the above embodiments may be combined into one module or further divided into multiple sub-modules.

[0149] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A pruning method for a multi-task neural network model, characterized in that: The method comprises: Obtaining a trained multi-task neural network model as a model to be pruned, wherein the multi-task neural network model is a neural network model capable of implementing at least two detection tasks; For each convolutional layer of the model to be pruned, perform the following steps AC to determine the sensitivity information of each convolutional layer under different preset layer pruning rates; Step A: Pre-pruning the convolutional layer based on the preset layer pruning rate and the convolution kernels in the convolutional layer; Step B: Based on the first training data, determine the loss value of each task corresponding to the pre-pruned model to be pruned corresponding to the preset layer pruning rate, the first training data includes sample images and their corresponding calibration information, wherein the sample images include the target to be detected by the model to be pruned, and the calibration information corresponding to the sample images includes the specific content of the detection labels corresponding to each task of the model to be pruned, and the step B includes: inputting each sample image in the first training data into the pre-pruned model to be pruned corresponding to the preset layer pruning rate to obtain prediction information corresponding to the sample image, wherein the prediction information includes prediction information corresponding to each task of the model to be pruned; for each task, based on the preset loss function corresponding to the task, the prediction information corresponding to the task corresponding to the sample image, and the calibration information corresponding to the task corresponding to the sample image in the first training data, determine the task sub-loss value corresponding to the task corresponding to the sample image, so as to obtain the task sub-loss value corresponding to the task corresponding to the sample image; and for each task, determine the task loss value corresponding to the task based on the task sub-loss value corresponding to the task corresponding to each sample image; Step C: Based on the loss values ​​of each task corresponding to the pre-pruned model, determine the sensitivity information of the convolutional layer at the preset layer pruning rate; Based on the channel information corresponding to the model to be pruned, the convolution kernels in the convolution layer of the model to be pruned are grouped, wherein, when there are multiple convolution layers connected by the ElementwiseLayer function in the model to be pruned, the convolution kernels corresponding to the same channel identifier are grouped together, the convolution kernels corresponding to different channel identifiers are grouped together, and the convolution kernels of the convolution layers not connected by the ElementwiseLayer function are grouped together, where the convolution kernels of the convolution layers not connected by the ElementwiseLayer function are convolution kernels that do not correspond to the channel identifiers; Based on the current model pruning rate, the grouping of the convolution kernels of the model to be pruned, the pruning norm value corresponding to each convolution kernel in each convolution layer, and the sensitivity information of each convolution layer under different layer pruning rates, the target pruning model is determined, wherein, for the convolution kernels corresponding to the same channel identifier, they are pruned or retained at the same time, and for the convolution kernels that do not correspond to the channel identifier, they are pruned or retained based on their corresponding pruning norm values.

2. The method according to claim 1, wherein The step of determining the target pruning model based on the current model pruning rate, the grouping of the convolution kernels of the to-be-pruned model, the pruning norm value corresponding to each convolution kernel in each convolution layer, and the sensitivity information of each convolution layer under different layer pruning rates includes: Pruning the model to be pruned based on the current model pruning rate, the grouping of the convolution kernels of the model to be pruned, the pruning norm value corresponding to each convolution kernel in each convolution layer, and the sensitivity information of each convolution layer under different layer pruning rates to obtain the current pruned model; Determining, based on the second training data, detection information corresponding to each second training data corresponding to the current pruning model; If, based on the detection information corresponding to each second training data corresponding to the current pruning model and the calibration information in each second training data, it is determined that the current pruning model meets the preset pruning conditions, the current pruning model is determined as the target pruning model.

3. The method according to claim 2, wherein The method further comprises: If, based on the detection information corresponding to each second training data corresponding to the current pruning model and the calibration information in each second training data, it is determined that the current pruning model does not meet the preset pruning condition, the value of the current model pruning rate is adjusted to obtain a new current model pruning rate; Based on the new current model pruning rate, the grouping of the convolution kernels of the to-be-pruned model, the pruning norm value corresponding to each convolution kernel in each convolution layer, and the sensitivity information of each convolution layer under different layer pruning rates, the to-be-pruned model is pruned to obtain a new current pruning model; Determining, based on the third training data, detection information corresponding to each third training data corresponding to the new current pruning model; If, based on the detection information corresponding to each third training data corresponding to the new current pruning model and the calibration information in each third training data, it is determined that the new current pruning model meets the preset pruning condition, the new current pruning model is determined as the target pruning model; If, based on the detection information corresponding to each third training data corresponding to the new current pruning model and the calibration information in each third training data, it is determined that the new current pruning model does not meet the preset pruning conditions, the step of adjusting the value of the current model pruning rate is returned to obtain a new current model pruning rate.

4. The method according to any one of claims 1 to 3, wherein The step of determining the sensitivity information of the convolutional layer at the preset layer pruning rate based on the loss values ​​of each task corresponding to the pre-pruned model includes: Determine the loss value corresponding to the pre-pruned model based on the loss value of each task corresponding to the pre-pruned model and the preset objective function; Based on the loss value corresponding to the to-be-pruned model after the pre-pruning, the sensitivity information of the convolutional layer at the preset layer pruning rate is determined; wherein the expression of the preset objective function is: ； in, Represents the loss value corresponding to the model to be pruned after pre-pruning, represents the task loss value corresponding to the i-th task in the model to be pruned, represents the task weight corresponding to the i-th task in the model to be pruned; Indicates the number of tasks of the model to be pruned.

5. The method according to any one of claims 1 to 3, wherein The step of pre-pruning the convolutional layer based on the preset layer pruning rate and each convolution kernel in the convolutional layer includes: For each convolution kernel of the convolution layer, based on each weight parameter in the convolution kernel, determine a pruning norm value corresponding to the convolution kernel; Based on the preset layer pruning rate and the pruning norm value corresponding to each convolution kernel in the convolution layer, the convolution layer is pre-pruned according to a preset pruning rule, wherein the preset pruning rule is: pre-pruning is performed preferentially on convolution kernels with small corresponding pruning norm values.

6. A pruning device for a multi-task neural network model, characterized in that: The device comprises: An acquisition module is configured to obtain a trained multi-task neural network model as a model to be pruned, wherein the multi-task neural network model is a neural network model capable of implementing at least two detection tasks; The first determination module is configured to determine, for each convolutional layer of the to-be-pruned model, sensitivity information of each convolutional layer under different preset layer pruning rates through the following units; a pre-pruning unit configured to pre-prune the convolutional layer based on the preset layer pruning rate and each convolution kernel in the convolutional layer for each preset layer pruning rate; The first determining unit is configured to determine the loss value of each task corresponding to the pre-pruned model to be pruned corresponding to the preset layer pruning rate based on the first training data, the first training data including the sample image and the corresponding calibration information, wherein the sample image includes the target to be detected by the model to be pruned, and the calibration information corresponding to the sample image includes the specific content of the detection label corresponding to each task of the model to be pruned, and the determining of the loss value of each task corresponding to the pre-pruned model to be pruned corresponding to the preset layer pruning rate based on the first training data includes: inputting each sample image in the first training data into the image corresponding to the preset layer pruning rate In the pre-pruned model to be pruned, prediction information corresponding to the sample image is obtained, wherein the prediction information includes prediction information corresponding to each task of the model to be pruned; for each task, based on a preset loss function corresponding to the task, the prediction information corresponding to the task corresponding to the sample image, and the calibration information corresponding to the task corresponding to the sample image in the first training data, a task sub-loss value corresponding to the task corresponding to the sample image is determined to obtain a task sub-loss value corresponding to the task corresponding to the sample image; and for each task, a task loss value corresponding to the task is determined based on the task sub-loss value corresponding to the task corresponding to each sample image; The second determining unit is configured to determine the sensitivity information of the convolutional layer at the preset layer pruning rate based on the loss values ​​of each task corresponding to the to-be-pruned model after pre-pruning; a grouping module configured to group the convolution kernels in the convolution layer of the to-be-pruned model based on the channel information corresponding to the to-be-pruned model, wherein, when there are multiple convolution layers connected by an ElementwiseLayer function in the to-be-pruned model, the convolution kernels corresponding to the same channel identifier are grouped together, the convolution kernels corresponding to different channel identifiers are grouped together, and the convolution kernels of the convolution layers not connected by the ElementwiseLayer function are grouped together, where the convolution kernels of the convolution layers not connected by the ElementwiseLayer function are convolution kernels that do not correspond to the channel identifiers; The second determination module is configured to determine the target pruning model based on the current model pruning rate, the grouping of the convolution kernels of the model to be pruned, the pruning norm value corresponding to each convolution kernel in each convolution layer, and the sensitivity information of each convolution layer under different layer pruning rates, wherein, for the convolution kernels corresponding to the same channel identifier, they are pruned or retained at the same time, and for the convolution kernels that do not correspond to the channel identifier, they are pruned or retained based on their corresponding pruning norm values.

7. The device according to claim 6, characterized in that The second determining module includes: A first pruning unit is configured to prune the model to be pruned based on a current model pruning rate, a grouping of convolution kernels of the model to be pruned, a pruning norm value corresponding to each convolution kernel in each convolution layer, and sensitivity information of each convolution layer at different layer pruning rates to obtain a current pruned model; A third determining unit is configured to determine, based on the second training data, detection information corresponding to each second training data corresponding to the current pruning model; The fourth determination unit is configured to determine that the current pruning model meets the preset pruning conditions based on the detection information corresponding to each second training data corresponding to the current pruning model and the calibration information in each second training data, and determine the current pruning model as the target pruning model.

8. The device according to claim 7, wherein The second determining module further includes: an adjusting unit configured to, if it is determined that the current pruning model does not meet the preset pruning condition based on the detection information corresponding to each second training data corresponding to the current pruning model and the calibration information in each second training data, adjust the value of the current model pruning rate to obtain a new current model pruning rate; The second pruning unit is configured to prune the model to be pruned based on the new current model pruning rate, the grouping of the convolution kernels of the model to be pruned, the pruning norm value corresponding to each convolution kernel in each convolution layer, and the sensitivity information of each convolution layer under different layer pruning rates to obtain a new current pruning model; A fifth determining unit is configured to determine, based on the third training data, detection information corresponding to each third training data corresponding to the new current pruning model; a sixth determining unit configured to, if determining that the new current pruned model meets a preset pruning condition based on the detection information corresponding to each third training data corresponding to the new current pruned model and the calibration information in each third training data, determine the new current pruned model as a target pruned model; If, based on the detection information corresponding to each third training data corresponding to the new current pruning model and the calibration information in each third training data, it is determined that the new current pruning model does not meet the preset pruning condition, the adjustment unit is triggered.

9. The device according to any one of claims 6 to 8, characterized in that The second determining unit is specifically configured to determine the loss value corresponding to the pre-pruned model to be pruned based on the loss value of each task corresponding to the pre-pruned model to be pruned and a preset objective function; Based on the loss value corresponding to the to-be-pruned model after the pre-pruning, the sensitivity information of the convolutional layer at the preset layer pruning rate is determined; wherein the expression of the preset objective function is: ； in, Represents the loss value corresponding to the model to be pruned after pre-pruning, represents the task loss value corresponding to the i-th task in the model to be pruned, represents the task weight corresponding to the i-th task in the model to be pruned; Indicates the number of tasks of the model to be pruned.

10. The device according to any one of claims 6 to 8, characterized in that The pre-pruning unit is specifically configured to determine, for each convolution kernel of the convolution layer, a pruning norm value corresponding to the convolution kernel based on each weight parameter in the convolution kernel; Based on the preset layer pruning rate and the pruning norm value corresponding to each convolution kernel in the convolution layer, the convolution layer is pre-pruned according to a preset pruning rule, wherein the preset pruning rule is: pre-pruning is performed preferentially on convolution kernels with small corresponding pruning norm values.

Images