Model training method, processing method, device, electronic device and storage medium

Abstract

The embodiment of the present invention provides a model training method, a processing method, an apparatus, an electronic device and a storage medium. In the method, a first target model is created according to the model to be trimmed, and a dynamic activation function branch is set in the first target model. For the processing layer in the first target model, the importance characterization parameters corresponding to each channel in the input feature map of the processing layer are determined according to the dynamic activation function branch. The importance characterization parameters include the slope corresponding to the channel. The importance characterization parameters corresponding to different input feature maps are different. Then, the importance of each channel is determined according to the importance characterization parameters corresponding to each channel, and the channels of the input feature map are trimmed according to the importance of each channel. Finally, based on the trimmed input feature map and the first target model, the second target model is trained and obtained. In this way, the prediction effect can be ensured while improving efficiency to a certain extent.

Description

Technical Field

[0001] The present invention belongs to the field of network technology, and in particular relates to a model training method, a processing method, a device, an electronic device and a storage medium. Background Art

[0002] In order to improve the processing efficiency of the model, it is sometimes necessary to trim the model. When trimming, the channels of the feature map are often trimmed according to the importance of the channels of the feature map during the model processing.

[0003] When determining the importance of a channel, the prior art often uses the scaling factor parameter of the batch normalization layer of the model to determine the importance of the channel. However, the scaling factor parameter of the batch normalization layer is also used to determine the scaling size of the feature. Therefore, when adjusting the scaling factor parameter, it is also necessary to take into account the feature scaling at the same time, which will result in the scaling factor parameter being unable to accurately characterize the importance of the channel. When using the importance determined by the scaling factor parameter for cropping, some important channels may be cropped, which in turn leads to poor prediction results of the model. Summary of the invention

[0004] In view of the above problems, the embodiments of the present invention propose a model training method, a processing method, a device, an electronic device and a storage medium to overcome the above problems or at least partially solve the above problems. In order to solve the above technical problems, the present invention is implemented as follows:

[0005] A first aspect of an embodiment of the present invention provides a model training method, the method comprising:

[0006] Creating a first target model according to the model to be trimmed; a dynamic activation function branch is provided in the first target model;

[0007] For the processing layer in the first target model, determining, according to the dynamic activation function branch, an importance characterization parameter corresponding to each channel in the input feature map of the processing layer; the importance characterization parameter includes a slope corresponding to the channel;

[0008] Determine the importance of each of the channels according to the importance characterization parameters corresponding to each of the channels;

[0009] Crop the channels of the input feature map according to the importance of each channel;

[0010] Based on the cropped input feature map and the first target model, a second target model is trained and acquired.

[0011] A second aspect of an embodiment of the present invention provides a model processing method, the method comprising:

[0012] For the processing layer in the second target model, determining the importance characterization parameters corresponding to each channel in the input feature map of the processing layer according to the dynamic activation function branch in the second target model; the importance characterization parameters include the slopes corresponding to the channels;

[0013] Determine the importance of each of the channels according to the importance characterization parameters corresponding to each of the channels;

[0014] Crop the channels of the input feature map according to the importance of each channel;

[0015] Performing subsequent processing based on the cropped input feature map;

[0016] Wherein, the second target model is trained according to any one of the methods described in claims 1 to 9.

[0017] A third aspect of an embodiment of the present invention provides a model training device, the device comprising:

[0018] A creation module, used to create a first target model according to the model to be cut; the first target model is provided with a dynamic activation function branch;

[0019] A first determination module is used to determine, for a processing layer in the first target model, an importance characterization parameter corresponding to each channel in an input feature map of the processing layer according to the dynamic activation function branch; the importance characterization parameter includes a slope corresponding to the channel;

[0020] A second determination module, used to determine the importance of each of the channels according to the importance characterization parameters corresponding to each of the channels;

[0021] A cropping module, used for cropping the channels of the input feature map according to the importance of each channel;

[0022] A training module is used to train and obtain a second target model based on the cropped input feature map and the first target model.

[0023] A fourth aspect of the embodiments of the present invention provides a model processing device, the device comprising:

[0024] A first determination module is used to determine, for a processing layer in a second target model, importance characterization parameters corresponding to each channel in an input feature map of the processing layer according to a dynamic activation function branch in the second target model; the importance characterization parameters include slopes corresponding to the channels;

[0025] A second determination module, used to determine the importance of each of the channels according to the importance characterization parameters corresponding to each of the channels;

[0026] A cropping module, used for cropping the channels of the input feature map according to the importance of each channel;

[0027] A processing module, used for performing subsequent processing based on the cropped input feature map;

[0028] The second target model is obtained by training using the device described in the third aspect of the embodiment of the present invention.

[0029] According to a fifth aspect of an embodiment of the present invention, an electronic device is disclosed, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes, the method described in the first aspect or the second aspect of the embodiment of the present invention is implemented.

[0030] According to a sixth aspect of the embodiments of the present invention, a computer-readable storage medium is disclosed, wherein a computer program stored in the medium enables a processor to execute the method described in the first aspect or the second aspect of the embodiments of the present invention.

[0031] The embodiments of the present invention include the following advantages:

[0032] In an embodiment of the present invention, a first target model is created according to a model to be pruned, and a dynamic activation function branch is provided in the first target model. For a processing layer in the first target model, importance characterization parameters corresponding to each channel in the input feature map of the processing layer are determined according to the dynamic activation function branch. The importance characterization parameters include the slope corresponding to the channel, and the importance characterization parameters corresponding to different input feature maps are different. Then, the importance of each channel is determined according to the importance characterization parameters corresponding to each channel, and the channels of the input feature map are pruned according to the importance of each channel. Finally, based on the pruned input feature map and the first target model, a second target model is obtained through training. In this way, by using the slope determined by the dynamic activation function branch to determine the importance of the channel during the training process, the model can learn how to prune the channel based on the slope. Since the slope corresponding to the channel does not need to take into account feature scaling, it can improve efficiency to a certain extent while ensuring the prediction effect. BRIEF DESCRIPTION OF THE DRAWINGS

[0033] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0034] Figure 1 is a flowchart of the steps of a model training method provided by an embodiment of the present invention;

[0035] Figure 2 is a schematic diagram of a dynamic activation function branch provided by an embodiment of the present invention;

[0036] Figure 3 is a schematic diagram of a processing process of an embodiment of the present invention;

[0037] Figure 4 is a flow chart of steps of a model processing method provided by an embodiment of the present invention;

[0038] Figure 5 is a flowchart of the steps of an image processing method provided by an embodiment of the present invention;

[0039] Figure 6 is a block diagram of a model training device provided by an embodiment of the present invention;

[0040] Figure 7 is a block diagram of a model processing device provided by an embodiment of the present invention;

[0041] Figure 8 It is a block diagram of an image processing device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0042] In order to make the above-mentioned purposes, features and advantages of the present invention more obvious and easy to understand, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0043] First, an application scenario involved in an embodiment of the present invention is described. Specifically, deep learning models based on convolutional neural networks have made great progress in the field of computer vision. For example, deep learning models can achieve image classification, target detection, semantic segmentation and other tasks with better prediction performance. However, the powerful prediction performance of convolutional neural networks requires greater computing power support. For example, in image processing scenarios, sufficient graphics processor (Graphics Processing Unit, GPU) computing power support is required. This imposes greater restrictions on the equipment used to deploy convolutional neural networks. Therefore, the model compression algorithm used to improve the model prediction efficiency without reducing the model prediction performance has become the object of widespread attention in academia and industry.

[0044] Among them, the model channel pruning algorithm is a relatively common and effective method in the model compression algorithm to improve the computational efficiency of the model. It reduces the floating-point calculation amount of each layer by reducing the number of input channels in each layer of the model, thereby improving the overall computational efficiency of the model. Existing model channel pruning algorithms can be divided into two categories: static model channel pruning and dynamic model channel pruning. Static model pruning is that the samples on the entire data set share a model structure, while dynamic model pruning is to retain the complete expression ability of the model before pruning. During training and prediction, it is necessary to dynamically select the model substructure according to the input, that is, dynamically select the channel of the feature map in the model. Compared with static model pruning, dynamic model channel pruning retains the complete expression ability of the model structure and will not cause prediction performance loss. It can also better adapt to input features. It improves the generalization ability of the model, and therefore has been widely used.

[0045] At present, dynamic model pruning algorithms all judge the importance of channels based on the scaling factor parameters of the batch normalization layer. However, the scaling factor parameters of the batch normalization layer also have another function, which is to scale the output features of each layer to a suitable distribution. Since the optimization directions of the two are not consistent, the process of optimizing the scaling factor parameters may fall into a local optimum, which may cause the scaling factor parameters obtained by the final optimization to fail to accurately represent the importance of the channels. When pruning using the importance determined by the scaling factor parameters, some important channels are pruned, which in turn affects the final prediction performance of the model.

[0046] To this end, an embodiment of the present invention provides a model training method to solve the above problems.

[0047] Figure 1 is a flow chart of steps of a model training method provided by an embodiment of the present invention, such as Figure 1 As shown, the method may include:

[0048] Step 101: Create a first target model according to the model to be cropped; a dynamic activation function branch is set in the first target model.

[0049] In an embodiment of the present invention, the model to be cropped may be a model that needs to be cropped to reduce computational complexity. The specific tasks that need to be completed by the model to be cropped may be set according to actual needs, and the embodiment of the present invention is not limited to this. For example, the model to be cropped may be a model for implementing image classification tasks, target detection tasks, video classification tasks, image enhancement tasks, and motion tracking tasks. Furthermore, the first target model may be a model including a dynamic activation function branch constructed based on the model to be cropped, and the dynamic activation function branch in the embodiment of the present invention may be a module.

[0050] Step 102: For the processing layer in the first target model, determine the importance characterization parameters corresponding to each channel in the input feature map of the processing layer according to the dynamic activation function branch; the importance characterization parameters include the slope corresponding to the channel.

[0051] In an embodiment of the present invention, the processing layer may be a layer in the first target model that needs to process the feature map. For example, the processing layer may be a convolutional layer. The input feature map may be a feature map output to the processing layer by a layer above the processing layer. The input feature map may be a feature map of C*H*W dimensions. Wherein, C represents the number of channels of the input feature map, H represents the height of the input feature map, and W represents the width of the input feature map. Each channel of the input feature map includes elements, and all elements constitute the input feature map.

[0052] The importance characterization parameter can be used to characterize the importance of the channel. Specifically, the importance characterization parameter may include a slope determined by a dynamic activation function branch. The corresponding slope can be determined for each channel according to the characteristic value of the element in the feature map by setting the dynamic activation function branch. The characteristic values ​​of the elements in different input feature maps are different. Accordingly, the slopes determined by the dynamic activation function branch are often different, and the subsequent channels clipped based on the slope are also different. Further, in actual application scenarios, when processing the feature map, it is often calculated based on the slope corresponding to the element in the feature map and the characteristic value of the element. Therefore, if the slope of an element is larger, it can reflect to a certain extent that the importance of the element is higher. Accordingly, in an embodiment of the present invention, by setting the dynamic activation function branch to determine the corresponding slope for each channel from the dimension of the channel, the importance of the channel can be more accurately characterized by the slope corresponding to each channel. Of course, it should be noted that the importance characterization parameter may also include other parameters, which are not limited in the embodiment of the present invention.

[0053] Step 103: Determine the importance of each channel according to the importance characterization parameter corresponding to each channel.

[0054] Step 104: Crop the channels of the input feature map according to the importance of each channel.

[0055] In the embodiment of the present invention, since the importance characterization parameter can characterize the importance of each channel, the importance of the channel can be quantified based on the importance characterization parameter to obtain the importance of each channel. Then, the channels with lower importance can be trimmed. In this way, by trimming some channels, the amount of calculation when processing the input feature map can be reduced, thereby reducing the overall computational complexity of the model.

[0056] Step 105: Based on the cropped input feature map and the first target model, train and obtain a second target model.

[0057] Since the number of channels of the cropped input feature map is reduced, accordingly, when training to obtain the second target model based on the cropped input feature map, the amount of model calculation during the training process can be reduced.

[0058] The model training method provided by the embodiment of the present invention creates a first target model based on the model to be pruned. The first target model is provided with a dynamic activation function branch. For the processing layer in the first target model, the importance characterization parameters corresponding to each channel in the input feature map of the processing layer are determined according to the dynamic activation function branch. The importance characterization parameters include the slope corresponding to the channel. The importance characterization parameters corresponding to different input feature maps are different. Then, the importance of each channel is determined according to the importance characterization parameters corresponding to each channel, and the channels of the input feature map are pruned according to the importance of each channel. Finally, based on the pruned input feature map and the first target model, the second target model is trained and obtained. In this way, by using the slope determined by the dynamic activation function branch to determine the importance of the channel during the training process, the model can learn how to prune the channel based on the slope. Since the slope corresponding to the channel does not need to take into account feature scaling, it can improve efficiency to a certain extent while ensuring the prediction effect.

[0059] At the same time, the additional dynamic activation function branch introduced in the embodiment of the present invention introduces a small amount of model parameters and calculations into the model, so that the model training method provided by the embodiment of the present invention can be implemented at a low cost to improve the prediction ability of the model. In addition, the model training method provided by the embodiment of the present invention is independent of the specific tasks that the model to be trimmed needs to complete, and can be applied to models that complete different tasks, and has a wide range of applications.

[0060] Optionally, in an implementation of the embodiment of the present invention, the step of determining the importance characterization parameters corresponding to each channel in the input feature map of the processing layer according to the dynamic activation function branch may include the following sub-steps:

[0061] Sub-step (1): Based on the model parameters defined in the dynamic activation function branch and the characteristic values ​​of the elements in each channel, a slope and an intercept are assigned to each channel of the input feature map; the elements in the same channel share the slope and intercept of the channel.

[0062] Sub-step (2): Obtain importance characterization parameters corresponding to each of the channels based on the slope and intercept corresponding to each of the channels.

[0063] In an embodiment of the present invention, different channels may contain different eigenvalues ​​of elements, and therefore, the slopes and intercepts assigned to different channels may be different. During the training process, as the model's capabilities improve during the learning process, the various model parameters in the first target model will be continuously optimized, and the accuracy of each model parameter will become higher and higher. Accordingly, through training, the dynamic activation function branch can eventually learn the ability to accurately assign slopes and intercepts to different channels. At the same time, since cropping is often based on the channel dimension, in an embodiment of the present invention, the slope and intercept are assigned to each channel in the channel dimension through the dynamic activation function branch, so that the importance characterization parameters of each channel in the channel dimension can be easily obtained later, thereby improving the acquisition efficiency to a certain extent.

[0064] Further, when obtaining the importance characterization parameters corresponding to each channel according to the slope and intercept corresponding to each channel, the slope corresponding to each channel can be extracted from the slope and intercept corresponding to each channel as the importance characterization parameters corresponding to each channel. Further, in actual application scenarios, when processing the feature map, the intercept corresponding to the element in the feature map is often combined for calculation. Therefore, the intercept can also reflect the importance to a certain extent. Accordingly, in an embodiment of the present invention, the slope and intercept corresponding to each channel can also be extracted as the importance characterization parameters corresponding to each channel. It should be noted that although the intercept can also characterize the importance to a certain extent, the proportion of the slope is often greater than the proportion of the intercept when calculating in combination with the slope and the intercept. Therefore, the importance cannot be accurately characterized by the intercept alone. In an embodiment of the present invention, on the basis of characterizing the importance with the slope, the intercept is introduced to ensure the accuracy of the importance characterization parameters to a certain extent. Of course, in another optional embodiment of the present invention, only the intercept corresponding to each channel can be extracted as the importance characterization parameter, and the embodiment of the present invention is not limited to this.

[0065] In an embodiment of the present invention, the dynamic activation function branch may include a global pooling layer, at least one fully connected layer, a rectified linear unit (RELU) activation function layer and a normalization layer. For example, the dynamic activation function branch may be specifically: global pooling layer → fully connected layer → ReLU activation function layer → fully connected layer → normalization layer. It should be noted that when the slope and intercept are assigned to each channel of the input feature map based on the model parameters defined in the dynamic activation function branch and the characteristic values ​​of the elements in each channel, the characteristic values ​​of the elements in each channel in the input feature map can be used as the dynamic activation function branch, and each layer in the dynamic activation function branch processes the input in combination with the parameters defined in each layer to obtain the slope and intercept of the dynamic activation function branch output, and finally, the output slope and intercept can be directly used as the assigned slope and intercept. Alternatively, the sum of the slope and intercept of the output and the preset constant term can be calculated as the assigned slope and intercept. In this way, by adding a preset constant term to the output slope and intercept for correction and then using them as the allocated slope and intercept, the accuracy of the allocated slope and intercept can be ensured to a certain extent.

[0066] For example, Figure 2 is a schematic diagram of a dynamic activation function branch provided by an embodiment of the present invention, such as Figure 2 As shown, X l-1 Represents the input of the dynamic activation function branch, "Global Pool" represents the global pooling layer, "Full Connected Layer" represents the fully connected layer, "RELU" represents the RELU activation function layer, and "Normalize" represents the normalization layer. After the global pooling layer, fully connected layer, RELU activation function layer, fully connected layer, and normalization layer process the input in sequence, the preset constant term α can be added to the output slope respectively. 1：k , which is the output intercept plus the preset constant term β 1：k , and get the slope a and intercept b assigned to the channel.

[0067] Optionally, the step of creating a first target model according to the model to be cropped in the embodiment of the present invention can be implemented by the following operation: replacing the original activation function layer in the model to be cropped with the dynamic activation function branch.

[0068] Specifically, since the original activation function layer often allocates slope and intercept for each element in the feature map, and often allocates a fixed slope and intercept for each element. In order to dynamically allocate channels according to the actual conditions of each dimension in the feature map from the channel dimension, the embodiment of the present invention can replace the original activation function layer in the model to be cut with a dynamic activation function branch. For example, when replacing, the original activation function layer in the model to be cut can be removed first, and then the dynamic activation function branch can be connected to the model to be cut, and a dynamic activation function branch can be set for each convolution layer in the model. In the embodiment of the present invention, by replacing the original activation function layer in the model to be cut with a dynamic activation function branch, the model can be based on the ability to classify slopes and intercepts from the channel dimension, thereby facilitating the subsequent acquisition of importance characterization parameters based on the dynamic activation function branch. At the same time, since the activation function itself has the characteristics of filtering unimportant features to a certain extent, the characteristics of the activation function fully utilized in the embodiment of the present invention, and the use of the activation function for channel selection is more meaningful, and the accuracy of the channel selection can be ensured to a certain extent. Further, in an embodiment of the present invention, the importance characterization parameter characterizing the importance of the channel and the scaling factor parameter in the batch normalization layer that determines the feature scaling size are separated, so that the scaling factor parameter in the batch normalization layer is only used to determine the feature scaling size, and the importance characterization parameter is only used to determine the importance of the channel. In this way, the optimization direction is inconsistent, resulting in poor results. It should be noted that when the original activation function is a RELU activation function layer, the global pooling layer, at least one fully connected layer and the normalization layer can be directly connected on the basis of the original RELU activation function layer to obtain a dynamic activation function branch. Further, the global pooling layer and the fully connected layer in the dynamic activation function branch can be used to vectorize the input, so that subsequent calculations can be facilitated by vectorization, thereby improving the calculation efficiency to a certain extent. The normalization layer can be used to normalize the slope and intercept of the RELU activation function layer to a preset interval. For example, the normalization layer can be composed of a Sigmoid activation function, and the processing logic of the normalization layer can be characterized as 2*Sigmoid(x)-1, and the preset interval can be [-1, 1]. In this way, through normalization, the values ​​of the slope and intercept finally output by the dynamic activation function branch can be made more stable and easier to handle.

[0069] Optionally, in an implementation of an embodiment of the present invention, the dimension of the slope and intercept assigned to each channel by the dynamic activation function branch may be [N, C, K]. Wherein, N represents the number of input feature maps, C represents the number of input feature map channels, and K represents the number of linear transformations. The specific values ​​of N, C, and K may be set according to actual needs. For example, in a batch computing training scenario, the specific value of N may be determined by the number of samples contained in a mini-batch subset. Further, K may be the number of elements contained in a channel, which is not limited in the embodiment of the present invention. Accordingly, a channel may correspond to K slopes, and the step of determining the importance of each of the channels according to the importance characterization parameters corresponding to each of the channels may include: for any of the channels, taking the maximum slope corresponding to the channel as the importance of the channel. Specifically, the maximum slope among the K slopes may be directly extracted as the importance of the channel, so that the importance of the channel can be obtained by reducing the operations, and the determination efficiency is high. Alternatively, the average value of the K slopes corresponding to the channel is used as the importance of the channel; specifically, the average value of the K slopes can be calculated as the importance of the channel according to a preset average value calculation formula, and the average value can be used to characterize the overall situation of the K slopes, thereby making the determined importance more representative. Alternatively, the Euclidean distance between the K slopes corresponding to the channel is used as the importance of the channel. Specifically, the Euclidean distance between the corresponding K slopes can be calculated by calculating the Euclidean distance (L2 norm) of the tensor composed of the K slopes in K dimensions. Since the Euclidean distance between the slopes can more accurately characterize the overall situation of the K slopes, the determined importance can be more representative, thereby ensuring the accuracy of subsequent clipping based on importance. Furthermore, by solving each channel of each input feature map, an importance feature with a dimension of [N, C] can be obtained, which can characterize the importance of each channel in the feature map output from the previous layer to the processing layer.

[0070] Optionally, the operation of training to obtain the second target model in an embodiment of the present invention may include N training stages, wherein the sub-target computational complexity of each training stage decreases successively, and the sub-target computational complexity of the last training stage is equal to the target computational complexity of the second target model.

[0071] The specific N can be set according to actual needs. For example, N can be 4 or 5. The target computational complexity of the second target model is less than the computational complexity of the model to be cut. The target computational complexity can be set according to actual needs. For example, the computational complexity of the model to be cut can be used as the benchmark computational complexity. The target computational complexity can be 50%, 60%, etc. of the benchmark computational complexity. The embodiment of the present invention does not limit this. Furthermore, the sub-target computational complexity of each training stage can be determined based on the target computational complexity that is ultimately desired to be achieved. For example, assuming that 10% of the number of channels in each layer is cut off in each training stage, N is 4, and the target computational complexity is 60% of the baseline computational complexity, then the sub-target computational complexity of the first training stage can be set to 90% of the baseline computational complexity, the sub-target computational complexity of the second training stage can be set to 80% of the baseline computational complexity, the sub-target computational complexity of the third training stage can be set to 70% of the baseline computational complexity, and the sub-target computational complexity of the fourth training stage can be set to 60% of the baseline computational complexity. After the four stages of training, a model whose computational complexity reaches the required target computational complexity can be obtained. In an embodiment of the present invention, by dividing the training process into multiple training stages, and setting the computational complexity required to be achieved in each training stage in a decreasing manner, the complexity of the model can be reduced more smoothly and stably during the training process, avoiding sudden changes in the computational complexity of the model in a short period of time, thereby improving the training effect of the model to a certain extent.

[0072] Accordingly, in an optional implementation of the embodiment of the present invention, the operation of clipping the channels of the input feature map according to the importance of each channel may include:

[0073] Sub-step (3): Determine the sub-goal computational complexity of the current training stage according to the sub-goal computational complexity of each of the training stages, and obtain the current goal complexity.

[0074] Sub-step (4): Determine the number n of channels to be pruned corresponding to the processing layer according to the current target complexity; the number n is negatively correlated with the current target complexity.

[0075] Sub-step (5): based on the channel pruning branch, prune the first n channels with the lowest importance in the input feature map.

[0076] The channel pruning branch can be set for the first target module when creating the first target model. The channel pruning branch can be a module. In specific implementation, it can be recorded after each training stage is completed, and the current training stage is determined according to the completed training stage, and then the current target complexity is determined. Since the computational complexity of the sub-goals required to be achieved in different training stages is different, the number of channels that need to be pruned in each training stage is different. Accordingly, it can be determined according to the current target complexity how many channels need to be pruned at present. Specifically, the model may contain multiple processing layers. As the layers are processed layer by layer, the channels of the input feature map received by the processing layer at the later position will be fewer and fewer. Therefore, when determining n, the number of channels to be pruned corresponding to the processing layer can be found and determined according to the preset complexity and the corresponding relationship between the number of layers of the processing layer and the number of pruned. Among them, the larger the number of layers of the processing layer, the later the position. In the corresponding relationship, the number of layers and complexity of the processing layer are negatively correlated with the corresponding number of pruned. Furthermore, the channels may be sorted in order from low to high importance, and then the first n channels may be pruned.

[0077] In an embodiment of the present invention, by determining the current target complexity, the number of channels n to be pruned is determined and pruned according to the current target complexity, so as to ensure that the current target complexity can be obtained in the current training stage, thereby ensuring the training effect to a certain extent.

[0078] Optionally, in an implementation of the embodiment of the present invention, the step of training and acquiring the second target model based on the cropped input feature map and the first target model may include:

[0079] Sub-step (6): Calculate the sparse constraint value of each processing layer according to the slope corresponding to the input feature map of each processing layer in the first target model.

[0080] Sub-step (7): Determine the sum of the sparse constraint values ​​as the sparse constraint loss value of the first target model.

[0081] In the embodiment of the present invention, the Group Lasso sparse constraint value of each processing layer in the first target model can be calculated according to the slope corresponding to the input feature map of each processing layer and according to the preset Group Lasso algorithm. Then, the sum of the Group Lasso sparse constraint values ​​of all processing layers is calculated to obtain the sparse constraint loss value of the first target model.

[0082] Sub-step (8): Based on the sparse constraint loss value of the first target model, adjust the model parameters in the first target model.

[0083] Specifically, the basic loss value of the first target model can also be calculated. For example, the basic loss value of the first target model can be calculated by the cross entropy loss function. The model parameters are adjusted in combination with the sparse constraint loss value and the basic loss value. For example, the weighted sum of the two loss values ​​can be calculated to obtain a comprehensive loss value. According to the comprehensive loss value, the model parameters are optimized and adjusted using the stochastic gradient descent method. Of course, the model parameters can also be optimized and adjusted using the stochastic gradient descent method based only on the sparse constraint loss value. The embodiments of the present invention are not limited to this.

[0084] In an embodiment of the present invention, the sparse constraint loss value of the first target model is calculated, the model parameters are adjusted, and sparse constraints are introduced during the training process, so that when the trained model is running, the slope value determined for each channel is small and has better sparsity, which facilitates subsequent slope-based processing.

[0085] Sub-step (9): when the first target model satisfies the model convergence condition of the current training stage and the current training stage is the last training stage, the first target model is determined as the second target model.

[0086] In an embodiment of the present invention, the model convergence condition may be set according to actual needs. For example, the model convergence condition may be to use all samples in the training set for training N times, the loss value of the model falls within a preset range, the number of adjustments to the model parameters reaches a preset threshold, and so on. If the model converges in the current training stage, and the current training stage is the last training stage, it can be considered that the model has currently reached the required computational complexity. Therefore, the first target model that converges in the current training stage can be determined as the second target model. Optionally, if the first target model meets the model convergence condition of the current training stage, and the current training stage is not the last training stage, the next training stage of the current training stage can be entered to continue training to ensure that a model with the target computational complexity is trained.

[0087] Optionally, the first n channels with the lowest importance in the input feature map are cropped, including: setting the importance of the first n channels with the lowest importance to 0.

[0088] The processing layer may include a convolution processing unit and a maximum (maxout) activation function unit. Specifically, each convolution processing unit in the first target model may be connected to a maxout activation function unit. Accordingly, an embodiment of the present invention may also include: performing convolution processing on the input feature map according to the convolution processing unit to obtain a convolution processing result; based on the maxout activation function unit, obtaining the output feature map of the processing layer according to the importance tensor of the cropped input feature map and the convolution processing result; the importance tensor includes the importance corresponding to each channel in the cropped input feature map; and using the output feature map of the processing layer as the input feature map of the next processing layer in the first target model.

[0089] Specifically, the convolution processing result can be processed according to the batch normalization layer in the first target model, and then the output of the batch normalization layer and the importance tensor are multiplied together as the maxout activation function unit. Finally, the output of the maxout activation function unit is used as the final output feature map of the processing layer, as the input feature map of the next processing layer, and output to the next processing layer for processing. In an embodiment of the present invention, the convolution processing result is processed by using the importance tensor of the cropped input feature map, the output feature map is obtained and used as the input feature map of the next processing layer, so that the number of channels of the input feature map of the next processing layer can be reduced, thereby reducing the subsequent calculation amount and improving the calculation efficiency.

[0090] For example, Figure 3 is a schematic diagram of a processing process of an embodiment of the present invention, such as Figure 3 As shown, for the input feature map X of the convolutional layer l-1 , the convolution layer can perform convolution processing (Convolution) on the input feature map to obtain the convolution processing result Conv(Xi-1), and then the convolution processing result can be processed by the batch normalization layer (BN) to obtain the output of the batch normalization layer BN(Conv(Xi-1)). At the same time, for the input feature map Xi-1, the slope corresponding to each channel in the input feature map can be determined by the dynamic RELU module (Dynamic RELU), and then the L2norm of the slope can be calculated to obtain the importance of each channel. Then, the top n channels with the lowest importance can be selected (Choose top-n channel predictor), and the importance of these n channels can be set to 0 to obtain the importance tensor ( Figure 3 The covered channels are set to 0, and the uncovered channels are not set to 0). After that, the product of the importance tensor and BN(Conv(Xi-1)) can be calculated. After the product is processed by the maxout activation function layer, the input feature map of the next processing layer can be obtained. It should be noted that Figure 3 The number of channels of the input feature map of the current convolutional layer has not been reduced. Therefore, for the current convolutional layer, the amount of floating-point calculations in its calculation process has not been reduced. Furthermore, in order to improve processing efficiency, the information flow can be divided into Figure 3 That is, convolution processing is performed according to the cropped input feature map, thereby reducing the amount of calculation in the convolution processing and improving the calculation efficiency.

[0091] The following is a specific example to illustrate the model training method provided by the embodiment of the present invention. Specifically, taking the model to be trimmed as a ResNet-50 model applied to the image recognition dataset (ImageNet) to implement the classification task, and taking the goal of trimming 50% of the computational complexity of the ResNet-50 model as an example, the operation shown in the following steps can be implemented:

[0092] S1: Get the trained ResNet-50 model on the ImageNet dataset.

[0093] S2: Based on the pre-trained ResNet-50 model, the ReLU layer in each convolutional layer (block) in the model is replaced with a dynamic activation function branch, and a channel pruning branch is introduced.

[0094] S3: Based on the cross entropy objective function of the classification task, the Group Lasso sparse constraint is introduced. Based on the Group Lasso sparse constraint, the GroupLass sparse constraint value is calculated according to the slope a output by the dynamic activation function module in each convolutional layer. The sparse constraint loss value of the model is the sum of the Group Lasso sparse constraint values ​​in all convolutional layers. The model parameters are adjusted based on the cross entropy loss value calculated by the cross entropy objective function and the sparse constraint loss value.

[0095] S4: Introduce iterative training. For example, the initial cropping rate is 0, the first iteration process is to retain 90% of the channels in each convolution layer for training, and the remaining 10% of the channels are set to 0 until the training of the iterative process converges, the second iteration is to retain 80% of the channels in each convolution layer for training, and the remaining 20% ​​of the channels are set to 0 until the training of the iterative process converges, and so on, until the iterative process corresponding to the target computational complexity is reached, and the training of the iterative process converges.

[0096] In this specific example, by introducing a dynamic activation function module to trim channels, the prediction effect of the model can be ensured while reducing the computational complexity and improving the model's computational efficiency. By introducing sparse constraints, when the trained model is running, the slope value determined for each channel is small, with good sparsity, which facilitates slope-based processing. By dividing the process into multiple rounds of iterations, that is, dividing the training phase, the complexity of the model can be reduced more smoothly and stably during the training process, avoiding sudden changes in the computational complexity of the model in the short term, thereby improving the training effect of the model to a certain extent.

[0097] Figure 4 is a flow chart of steps of a model processing method provided by an embodiment of the present invention, such as Figure 4 As shown, the method may include:

[0098] Step 201: For the processing layer in the second target model, determine the importance characterization parameters corresponding to each channel in the input feature map of the processing layer according to the dynamic activation function branch in the second target model; the importance characterization parameters include the slope corresponding to the channel.

[0099] Step 202: Determine the importance of each channel according to the importance characterization parameter corresponding to each channel.

[0100] Step 203: Crop the channels of the input feature map according to the importance of each channel.

[0101] Step 204: Perform subsequent processing based on the cropped input feature map.

[0102] In an embodiment of the present invention, the second target model can be obtained by training through the above-mentioned model training embodiment. Through the above-mentioned training method, the second target model learns the ability to accurately assign importance characterization parameters to each channel in the feature map based on the dynamic activation function branch, and then can ensure the importance determined for each channel based on the importance characterization parameter. Accordingly, when using the second target model, the channels of the input feature map can be accurately cropped according to the importance. In this way, when subsequent processing is performed based on the cropped input feature map, it is possible to achieve a better balance between computing efficiency and prediction accuracy while reducing processing complexity and improving processing efficiency.

[0103] Figure 5 is a flowchart of the steps of an image processing method provided by an embodiment of the present invention, such as Figure 5 As shown, the method may include:

[0104] Step 301: Obtain an image to be processed.

[0105] In the embodiment of the present invention, the image to be processed may be an image input by a user, an image downloaded from a network, an image captured by a device, or an image in a video captured by a device, and the like.

[0106] Step 302: Use the image to be processed as an input of a second target model, and use the second target model to process the image to be processed to obtain an image processing result.

[0107] In an embodiment of the present invention, the image to be processed can be input into the second target model. Accordingly, the second target module can extract the image features of the image to be processed according to the feature extraction layer, and generate an input feature map corresponding to the image to be processed. Then, the input feature map can be input into the processing layer in the second target model. The processing layer can determine the importance characterization parameters corresponding to each channel in the input feature map of the processing layer according to the dynamic activation function branch in the second target model, and determine the importance of each channel according to the importance characterization parameters corresponding to each channel. Then, the channels of the input feature map are cropped according to the importance of each channel. Finally, subsequent processing is performed based on the cropped input feature map. After subsequent processing, the image processing result of the image to be processed can be obtained.

[0108] Furthermore, the second target model may be a model for performing operations such as target detection, image segmentation, and image classification on an image. Accordingly, the image processing result obtained may be a target object contained in the detected image to be processed, the segmented image to be processed, the category to which the image to be processed belongs, and the like. It should be noted that the specific operation implemented by the second target model on the image to be processed in the embodiment of the present invention does not affect the process of cropping the input feature map corresponding to the image to be processed by the second target model, and the specific operation implemented by the second target model may be set according to actual needs.

[0109] In an embodiment of the present invention, the second target model can be obtained by training through the above-mentioned model training embodiment. Through the above-mentioned training method, the second target model learns the ability to accurately assign importance characterization parameters to each channel in the feature map based on the dynamic activation function branch, and then can ensure the importance determined for each channel based on the importance characterization parameter. Accordingly, when the second target model is used to process the image to be processed, the second target model can accurately crop the channels of the feature map corresponding to the image to be processed according to the importance. In this way, when subsequent processing is performed based on the cropped feature map, it can be achieved while reducing the complexity of image processing and improving the efficiency of image processing, while ensuring the image processing effect.

[0110] Figure 6 is a block diagram of a model training device provided by an embodiment of the present invention, such as Figure 6 As shown, the device 50 may include:

[0111] A creation module 501 is used to create a first target model according to the model to be cut; the first target model is provided with a dynamic activation function branch;

[0112] A first determination module 502 is used to determine, for a processing layer in the first target model, an importance characterization parameter corresponding to each channel in an input feature map of the processing layer according to the dynamic activation function branch; the importance characterization parameter includes a slope corresponding to the channel;

[0113] A second determination module 503 is used to determine the importance of each channel according to the importance characterization parameter corresponding to each channel;

[0114] A clipping module 504, configured to clip the channels of the input feature map according to the importance of each channel;

[0115] The training module 505 is used to train and obtain a second target model based on the cropped input feature map and the first target model.

[0116] Optionally, the first determining module 502 is specifically configured to:

[0117] Based on the model parameters defined in the dynamic activation function branch and the feature values ​​of the elements in each of the channels, a slope and an intercept are assigned to each of the channels of the input feature map; the elements in the same channel share the slope and intercept of the channel;

[0118] According to the slope and intercept corresponding to each of the channels, the importance characterization parameters corresponding to each of the channels are obtained.

[0119] Optionally, the creation module 501 is specifically used for:

[0120] Replacing the original activation function layer in the model to be pruned with the dynamic activation function branch;

[0121] Among them, the dynamic activation function branch includes a global pooling layer, at least one fully connected layer, a rectified linear unit RELU activation function layer and a normalization layer.

[0122] Optionally, one channel corresponds to K slopes; the second determination module 503 is specifically configured to:

[0123] For any of the channels, the maximum slope corresponding to the channel is used as the importance of the channel;

[0124] Alternatively, the average value of K slopes corresponding to the channel is taken as the importance of the channel;

[0125] Alternatively, the Euclidean distance between the K slopes corresponding to the channel is used as the importance of the channel.

[0126] Optionally, the training operation of obtaining the second target model includes N training stages; wherein the sub-target computational complexity of each training stage decreases successively, and the sub-target computational complexity of the last training stage is equal to the target computational complexity of the second target model.

[0127] Optionally, the first target model further includes a channel pruning branch; the pruning module 504 is specifically used to:

[0128] Determine the sub-goal computational complexity of the current training stage according to the sub-goal computational complexity of each of the training stages, and obtain the current goal complexity;

[0129] According to the current target complexity, determining the number n of channels to be clipped corresponding to the processing layer; wherein n is negatively correlated with the current target complexity;

[0130] The first n channels with the lowest importance in the input feature map are pruned based on the channel pruned branch.

[0131] Optionally, the training module is specifically used for:

[0132] Calculating the sparse constraint value of each processing layer according to the slope corresponding to the input feature map of each processing layer in the first target model;

[0133] Determining the sum of the sparse constraint values ​​as the sparse constraint loss value of the first target model;

[0134] Adjusting model parameters in the first target model based on the sparse constraint loss value of the first target model;

[0135] When the first target model satisfies the model convergence condition of the current training stage and the current training stage is the last training stage, the first target model is determined as the second target model.

[0136] Optionally, the clipping module 504 is further specifically configured to: set the importance of the first n channels with the lowest importance to 0.

[0137] Accordingly, the processing layer includes a convolution processing unit and a maximum maxout activation function unit, and the device 50 also includes:

[0138] A processing module, configured to perform convolution processing on the input feature map according to the convolution processing unit to obtain a convolution processing result;

[0139] An acquisition module, configured to acquire the output feature map of the processing layer based on the maxout activation function unit and the importance tensor of the cropped input feature map and the convolution processing result; the importance tensor includes the importance corresponding to each channel in the cropped input feature map;

[0140] An output module is used to use the output feature map of the processing layer as the input feature map of the next processing layer in the first target model.

[0141] Optionally, the training module 504 is further specifically used for:

[0142] If the first target model meets the model convergence condition of the current training stage, and the current training stage is not the last training stage, then entering the next training stage of the current training stage.

[0143] The model training device provided by the embodiment of the present invention will create a first target model according to the model to be pruned. The first target model is provided with a dynamic activation function branch. For the processing layer in the first target model, the importance characterization parameters corresponding to each channel in the input feature map of the processing layer are determined according to the dynamic activation function branch. The importance characterization parameters include the slope corresponding to the channel. The importance characterization parameters corresponding to different input feature maps are different. Then, the importance of each channel is determined according to the importance characterization parameters corresponding to each channel, and the channels of the input feature map are pruned according to the importance of each channel. Finally, based on the pruned input feature map and the first target model, the second target model is trained and obtained. In this way, by using the slope determined by the dynamic activation function branch to determine the importance of the channel during the training process, the model can learn how to prune the channel based on the slope. Since the slope corresponding to the channel does not need to take into account feature scaling, it can improve efficiency to a certain extent while ensuring the prediction effect.

[0144] Figure 7 is a block diagram of a model processing device provided by an embodiment of the present invention, such as Figure 7 As shown, the device 60 may include:

[0145] A first determination module 601 is used to determine, for a processing layer in a second target model, importance characterization parameters corresponding to each channel in an input feature map of the processing layer according to a dynamic activation function branch in the second target model; the importance characterization parameters include slopes corresponding to the channels;

[0146] A second determination module 602 is used to determine the importance of each channel according to the importance characterization parameter corresponding to each channel;

[0147] A clipping module 603, used for clipping the channels of the input feature map according to the importance of each channel;

[0148] A processing module 604 is used to perform subsequent processing based on the cropped input feature map;

[0149] Wherein, the second target model is trained according to the above-mentioned model training device.

[0150] Figure 8 is a block diagram of an image processing device provided by an embodiment of the present invention, such as Figure 8 As shown, the device 70 may include:

[0151] An acquisition module 701 is used to acquire an image to be processed;

[0152] A processing module 702 is used to use the image to be processed as an input of a second target model, and use the second target model to process the image to be processed to obtain an image processing result;

[0153] Wherein, the second target model is trained according to the above-mentioned model training device.

[0154] As for the device embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts and the effects that can be achieved can be referred to the partial description of the corresponding method embodiment.

[0155] An embodiment of the present invention also provides an electronic device, which may include: one or more processors; and one or more machine-readable media having instructions stored thereon, which, when executed by the one or more processors, enable the device to perform the steps of one or more method embodiments described in the embodiments of the present invention.

[0156] An embodiment of the present invention further provides a computer-readable storage medium, which stores a computer program that enables a processor to execute the steps of one or more method embodiments described in the embodiments of the present invention.

[0157] The various embodiments in this specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the various embodiments can be referenced to each other.

[0158] It will be appreciated by those skilled in the art that the embodiments of the present invention may be provided as methods, devices, or computer program products. Therefore, the embodiments of the present invention may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the embodiments of the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program codes.

[0159] The embodiments of the present invention are described with reference to the flowcharts and / or block diagrams of the methods, terminal devices (systems), and computer program products according to the embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of the processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing terminal device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing terminal device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0160] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing terminal device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce a manufactured product including an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0161] These computer program instructions can also be loaded onto a computer or other programmable data processing terminal device so that a series of operating steps are executed on the computer or other programmable terminal device to produce a computer-implemented process, thereby providing instructions for executing on the computer or other programmable terminal device to implement the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0162] Although the preferred embodiments of the present invention have been described, those skilled in the art may make additional changes and modifications to these embodiments once they have learned the basic creative concept. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications that fall within the scope of the embodiments of the present invention.

[0163] Finally, it should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or terminal device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or terminal device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the existence of other identical elements in the process, method, article or terminal device including the elements.

[0164] The above is a detailed introduction to a model training method, device, electronic device and storage medium provided by the present invention. Specific examples are used in this article to illustrate the principles and implementation methods of the present invention. The description of the above embodiments is only used to help understand the method of the present invention and its core idea; at the same time, for general technical personnel in this field, according to the idea of ​​the present invention, in specific implementation methods.

Claims

1. A model training method, characterized in that: The method comprises: According to the model to be cropped, a first target model is created; a dynamic activation function branch is set in the first target model; the model to be cropped includes: a model that implements at least one of an image classification task, a video classification task, and an image enhancement task; For the processing layer in the first target model, determining, according to the dynamic activation function branch, an importance characterization parameter corresponding to each channel in the input feature map of the processing layer; the importance characterization parameter includes a slope corresponding to the channel; Determine the importance of each of the channels according to the importance characterization parameters corresponding to each of the channels; Crop the channels of the input feature map according to the importance of each channel; Based on the cropped input feature map and the first target model, training to obtain a second target model; Wherein, the dynamic activation function branch includes: a global pooling layer, at least one fully connected layer, a rectified linear unit activation function layer and a normalization layer; Determining the importance characterization parameters corresponding to each channel in the input feature map of the processing layer according to the dynamic activation function branch includes: Based on the model parameters defined in the dynamic activation function branch and the feature values ​​of the elements in each of the channels, a slope and an intercept are assigned to each of the channels of the input feature map; the elements in the same channel share the slope and intercept of the channel; According to the slope and intercept corresponding to each of the channels, the importance characterization parameters corresponding to each of the channels are obtained.

2. The method according to claim 1, characterized in that: The step of creating a first target model according to the model to be cut includes: The original activation function layer in the model to be pruned is replaced by the dynamic activation function branch.

3. The method according to claim 1, characterized in that: One channel corresponds to K slopes; Determining the importance of each of the channels according to the importance characterization parameters corresponding to each of the channels includes: For any of the channels, the maximum slope corresponding to the channel is used as the importance of the channel; Alternatively, the average value of K slopes corresponding to the channel is taken as the importance of the channel; Alternatively, the Euclidean distance between the K slopes corresponding to the channel is used as the importance of the channel.

4. The method according to claim 1, characterized in that: The training operation of obtaining the second target model includes N training stages; wherein the sub-target computational complexity of each training stage decreases successively, and the sub-target computational complexity of the last training stage is equal to the target computational complexity of the second target model.

5. The method according to claim 4, characterized in that: The first target model also includes a channel pruning branch; pruning the channels of the input feature map according to the importance of each channel includes: Determine the sub-goal computational complexity of the current training stage according to the sub-goal computational complexity of each of the training stages, and obtain the current goal complexity; According to the current target complexity, determining the number n of channels to be clipped corresponding to the processing layer; wherein n is negatively correlated with the current target complexity; The first n channels with the lowest importance in the input feature map are pruned based on the channel pruned branch.

6. The method according to claim 4 or 5, characterized in that: The training and obtaining of the second target model based on the cropped input feature map and the first target model includes: Calculating the sparse constraint value of each processing layer according to the slope corresponding to the input feature map of each processing layer in the first target model; Determining the sum of the sparse constraint values ​​as the sparse constraint loss value of the first target model; Adjusting model parameters in the first target model based on the sparse constraint loss value of the first target model; When the first target model meets the model convergence condition of the current training stage and the current training stage is the last training stage, the first target model is determined as the second target model.

7. The method according to claim 6, characterized in that: The clipping of the first n channels with the lowest importance in the input feature map comprises: The importance of the first n channels with the lowest importance is set to 0; Accordingly, the processing layer includes a convolution processing unit and a maximum maxout activation function unit, and the method further includes: Performing convolution processing on the input feature map according to the convolution processing unit to obtain a convolution processing result; Based on the maxout activation function unit, according to the importance tensor of the cropped input feature map and the convolution processing result, the output feature map of the processing layer is obtained; the importance tensor includes the importance corresponding to each channel in the cropped input feature map; The output feature map of the processing layer is used as the input feature map of the next processing layer in the first target model.

8. The method according to claim 6, characterized in that: The method further comprises: If the first target model meets the model convergence condition of the current training stage, and the current training stage is not the last training stage, then entering the next training stage of the current training stage.

9. A model processing method, characterized in that: The method comprises: For the processing layer in the second target model, determining the importance characterization parameters corresponding to each channel in the input feature map of the processing layer according to the dynamic activation function branch in the second target model; the importance characterization parameters include the slopes corresponding to the channels; Determine the importance of each of the channels according to the importance characterization parameters corresponding to each of the channels; Crop the channels of the input feature map according to the importance of each channel; Performing subsequent processing based on the cropped input feature map; Wherein, the second target model is trained according to any one of the methods described in claims 1 to 8.

10. An image processing method, characterized in that: The method comprises: Get the image to be processed; Using the image to be processed as an input of a second target model, and using the second target model to process the image to be processed to obtain an image processing result; Wherein, the second target model is trained according to any one of the methods described in claims 1 to 8.

11. A model training device, characterized in that: The device comprises: A creation module, used to create a first target model according to the model to be cropped; the first target model is provided with a dynamic activation function branch; the model to be cropped includes: a model for implementing at least one of an image classification task, a video classification task, and an image enhancement task; A first determination module is used to determine, for a processing layer in the first target model, an importance characterization parameter corresponding to each channel in an input feature map of the processing layer according to the dynamic activation function branch; the importance characterization parameter includes a slope corresponding to the channel; A second determination module, used to determine the importance of each of the channels according to the importance characterization parameters corresponding to each of the channels; A cropping module, used for cropping the channels of the input feature map according to the importance of each channel; A training module, used for training and acquiring a second target model based on the cropped input feature map and the first target model; Wherein, the dynamic activation function branch includes: a global pooling layer, at least one fully connected layer, a rectified linear unit activation function layer and a normalization layer; The first determining module is specifically configured to: Based on the model parameters defined in the dynamic activation function branch and the feature values ​​of the elements in each of the channels, a slope and an intercept are assigned to each of the channels of the input feature map; the elements in the same channel share the slope and intercept of the channel; According to the slope and intercept corresponding to each of the channels, the importance characterization parameters corresponding to each of the channels are obtained.

12. A model processing device, characterized in that: The device comprises: A first determination module is used to determine, for a processing layer in a second target model, importance characterization parameters corresponding to each channel in an input feature map of the processing layer according to a dynamic activation function branch in the second target model; the importance characterization parameters include slopes corresponding to the channels; A second determination module, used to determine the importance of each of the channels according to the importance characterization parameters corresponding to each of the channels; A cropping module, used for cropping the channels of the input feature map according to the importance of each channel; A processing module, used for performing subsequent processing based on the cropped input feature map; Wherein, the second target model is trained according to the device described in claim 11.

13. An image processing device, characterized in that: The device comprises: An acquisition module, used for acquiring an image to be processed; A processing module, used for taking the image to be processed as an input of a second target model, and processing the image to be processed using the second target model to obtain an image processing result; Wherein, the second target model is trained according to the device described in claim 11.

14. An electronic device, characterized in that: The method comprises a memory, a processor and a computer program stored in the memory and executable on the processor, wherein when the processor executes the method, the method according to any one of claims 1 to 11 is implemented.

15. A computer-readable storage medium, characterized in that: The computer program stored therein enables the processor to execute the method according to any one of claims 1 to 10.

Images