Self-adaptive pruning method based on priority experience playback

Abstract

The invention discloses a self-adaptive pruning method based on priority experience playback. The method comprises the steps of training a to-be-pruned image classification model, converting a pruning process of a neural network into a Markov decision process, and designing a corresponding state space, an action space and a reward function; constructing and initializing a value network, a strategy network, a target value network and a target strategy network of a depth deterministic strategy gradient algorithm; pruning the to-be-pruned image classification model; storing the pruning process of the to-be-pruned image classification model into an experience playback pool, and setting a priority for the pruning process; updating the sampling probability of each sample according to the time difference error; updating the value network and the strategy network, updating the target value network and the target strategy network, and carrying out loop training until a set maximum number of iterations is reached; and carrying out fine tuning on the image classification model to be pruned of image classification. According to the method, the pruning proportion of each layer of the network can be adaptively generated, and adaptive compression of the network to-be-pruned image classification model is realized.

Description

Technical Field

[0001] The present invention relates to the technical field of pruning, and in particular to an adaptive pruning method based on priority experience playback. Background Art

[0002] With the rapid development of artificial intelligence technology, the complexity and scale of neural network image classification models to be pruned are showing an increasing trend. Today, the development of software and hardware lags far behind the growth of storage and computing power requirements for neural network image classification models to be pruned. Therefore, how to achieve lightweighting of neural network image classification models to be pruned and their computing carriers while ensuring accuracy, and form a lightweight artificial intelligence computing system suitable for deployment on low-power devices, has become a hot topic in current AI research. Network pruning is the main method to achieve lightweighting of neural network image classification models to be pruned. It removes redundant connections, filters, neurons or layers in the network to obtain a lighter network without affecting performance. Traditional network pruning methods include three stages: training, pruning, and fine-tuning. The latter two stages require manual design of the pruning ratio, which has the disadvantages of high labor costs and insufficient compression rate. Therefore, it is necessary to design new pruning methods to address these shortcomings.

[0003] Patent application number CN202011561741.9 is a deep neural network compression method based on joint dynamic pruning. It introduces two hyperparameters, the dynamic pruning rate of the convolution kernel and the dynamic compression rate of the channel, to dynamically prune the convolution kernel and the channel during training. However, this process relies on artificially designed indicators such as the L1 norm, and it is difficult to adaptively learn the optimal pruning strategy, which may lead to a large loss of accuracy in the image classification model to be pruned. Patent application number CN202311851063.3 is a lightweight image classification method and system based on neural network pruning. It prunes all weight parameters of the convolutional neural network and uses the L1 regularization term to constrain the sparsity of the image classification model to be pruned. However, it lacks the distinction between the importance of different layers or channels, and may prune parameters that are critical to the performance of the image classification model to be pruned. Summary of the invention

[0004] In view of this, the present invention provides an adaptive pruning method based on priority experience playback.

[0005] The present invention discloses an adaptive pruning method based on priority experience playback, which comprises:

[0006] Step 1: Train the image classification model to be pruned, transform the pruning process of the neural network into a Markov decision process, and design the corresponding state space, action space, and reward function;

[0007] Step 2: Build and initialize the value network, policy network, target value network, and target policy network of the deep deterministic policy gradient algorithm;

[0008] Step 3: Encode the attributes of each layer of the image classification model to be pruned to obtain the corresponding state, add noise to the action output by the policy network as the pruning ratio of each layer of the image classification model to be pruned, and prune the image classification model to be pruned;

[0009] Step 4: Count the accuracy, number of parameters, amount of computation, and inference delay of the image classification model to be pruned, calculate the reward according to the designed reward function, store the pruning process of the image classification model to be pruned into the experience replay pool, and set a priority for it;

[0010] Step 5: Based on the sampling probability of historical experience, sample state transition samples from the experience replay pool, use importance sampling to calculate the importance sampling weight and time difference error of each state transition sample, and update the sampling probability of each sample based on the time difference error;

[0011] Step 6: Update the value network and policy network, update the target value network and target policy network, and repeat the training until the maximum number of iterations is reached.

[0012] Step 7: Fine-tune the image classification model to be pruned so that its accuracy is restored to a specified degree.

[0013] Furthermore, the step 1 comprises:

[0014] Step 11: Train the image classification model M to be pruned on the data set; encode the network layer features of each layer of the image classification model M to be pruned as states; the network layer features include: the index l of the network layer, the number of filters N of the l-th layer network, the number of channels C, height H, width W of the input feature map, the size K of the convolution kernel, the convolution step S, the floating-point calculation amount F of the l-th layer, the number of parameters P of the l-th layer, the floating-point calculation amount RF that has been reduced before the l-th layer, the number of parameters RP that has been reduced before the l-th layer, and the action a taken on the previous layer l-1 , which is the pruning ratio of the previous layer; the state of the l-th layer network of the image classification model M to be pruned is (l,N,C,H,W,K,S,F,P,RF,RP,a l-1 );

[0015] Step 12: Take the pruning ratio of each layer of the image classification model M to be pruned as the action a of the agent l , and satisfies a l ∈[0,1); define the reward function R according to the accuracy loss e of the image classification model to be pruned, the number of parameters P, the amount of floating-point calculation F, and the inference delay D of the image classification model to be pruned;

[0016] Step 13: Initialize the value network Q of the deep deterministic policy gradient algorithm ω The parameters ω of (·) and the policy network π θ (·) parameter θ;

[0017] Step 14: Set up a target value network Q with the same structure ω' (·) and value network Q ω (·), setting the target policy network π with the same structure θ' (·) with the policy network π θ (·).

[0018] Furthermore, in step 11, the calculation formula of the floating point calculation amount F and the parameter amount P is:

[0019] F=K×K×C×N×H×W

[0020] P=K×K×C×N

[0021] In step 12, the reward function R is expressed as:

[0022]

[0023] Among them, e represents the difference in accuracy between the pruned image classification model and the original pruned image classification model, ρ, σ, Respectively represent the parameter amount, floating point calculation amount, and weight factor of the inference delay of the image classification model to be pruned; P is the parameter amount of the lth layer of the image classification model to be pruned at the current moment, P max is the maximum value of the parameter of the lth layer of the image classification model to be pruned; F is the floating-point calculation amount of the lth layer of the image classification model to be pruned at the current moment, F max is the maximum value of the floating-point calculation of the lth layer of the image classification model to be pruned at the current moment; D is the inference delay of the image classification model to be pruned, D max is the maximum value of the inference delay of the image classification model to be pruned;

[0024] Target Value Network Q ω' The parameters ω' and the target policy network π θ' The parameter θ' initialization strategy of (·) is:

[0025] ω'←ω

[0026] θ'←θ.

[0027] Furthermore, the step 2 comprises:

[0028] Constructing the value network Q of the deep deterministic policy gradient algorithm ω (·), policy network π θ (·), target value network Qω' (·) and the target policy network π θ' (·), respectively three fully connected layers; initialize Q ω (·),π θ (·), Q ω' (·) and π θ' (·).

[0029] Furthermore, the step 3 comprises:

[0030] Step 31: Encode the attributes of the lth layer of the image classification model M to be pruned to obtain the state s l , will s l As a policy network π θ (·) input; for the policy network π θ Gaussian noise is added to the output of (·) to obtain action a l ;

[0031] Step 32: Calculate the importance of each channel of the lth layer of the image classification model M to be pruned, and use the sum of the weight values ​​in the channel as a criterion for determining the importance of the feature channel;

[0032] Step 33: Perform channel pruning on the lth layer of the image classification model M to be pruned. According to the importance of each channel, prune the channel with the lowest importance until the pruning ratio of this layer is reached.

[0033] Furthermore, in step 31, action a l The expression is:

[0034] a l =clip(π θ (s l )+ε,0,1)

[0035] ε~N(0,σ 2 )

[0036] Among them, clip represents the truncated action value after adding noise to the range of [0,1), and then obtains the pruning ratio of the lth layer of the model M, ε represents zero mean and variance σ 2 Gaussian noise;

[0037] The step 32 comprises:

[0038] The N filters of size C×K×K are split into C weight sets of size N×K×K in the channel dimension. 1 ,W 2 ,…,W C}, for the i-th channel, its importance I i The calculation formula is:

[0039]

[0040] in, Representation Rights Reorganization i The weight value in ; when the channel of any layer is pruned, the input features connected to the channel become redundant parameters.

[0041] Furthermore, the step 4 comprises:

[0042] Step 41: testing the classification accuracy loss E of the image classification model M to be pruned after pruning on the classification data set, and calculating the parameter quantity P, floating point calculation quantity F, and inference delay D of the image classification model M to be pruned, and calculating the reward value according to the reward function R;

[0043] Step 42: The state s of the lth layer of the image classification model M to be pruned l 、Action a l , reward value R l , successor state s l+1 As a state transition sequence (s l ,a l ,R l ,s l+1 ), and record it as T;

[0044] Step 43: Using the target policy network π θ' (·) and the target value network Q ω' (·), calculate the time difference target y of the state transition sequence T l and the timing difference error δ l ;

[0045] Step 44: Set the priority L for the state transition sequence T according to the absolute value of the timing differential error T , and store T into the experience replay pool E.

[0046] Furthermore, in step 43, the time series difference target y l and the timing difference error δ l The calculation formula is:

[0047] δ l =Q ω (s l ,a l )-y l

[0048] y l =R l +γQ ω' (s l+1 ,a l+1 )

[0049] al+1 =π θ' (s l+1 )

[0050] Among them, γ represents the discount factor of the return, γ∈[0,1];

[0051] In step 44, the priority L T The calculation formula is:

[0052]

[0053] Among them, rank represents the state transition sequence T with the time difference error δ l When the absolute value of is the standard, the ranking in the experience replay pool E.

[0054] Furthermore, the step 5 comprises:

[0055] For the state transition sample T in the experience replay pool E, its sampling priority is L T , calculate the sampling probability p of T s (T) and importance sampling weight W T , based on the weighted sampling probability p w (T) Sample n state transition samples from the experience replay pool E;

[0056] For n state transition samples, calculate the timing difference error δ of each sample l , according to δ l The absolute value of updates the priority and sampling probability of each sample.

[0057] Furthermore, the sampling probability p s (T), importance sampling weight W T and weighted sampling probability p w The calculation formulas for (T) are:

[0058]

[0059] p w (t) = W T p s (t)

[0060] in, Represents the sampling priority L T The degree of use is cumulative. Represents any sampling priority L k , t represents the time, α represents the priority usage, S represents the size of the experience replay pool E, and χ represents the correction factor of the importance sampling weight.

[0061] Further, the step 6 comprises:

[0062] Step 61: Based on the n state transition samples, derive the loss function L(Q) of the value network;

[0063] Step 62: Set the learning rate α, average the derivatives of the loss function L(Q) obtained from n samples, and update the value network Q ω (·) parameter ω;

[0064] Step 63: For the kth sample, let its form be (s l ,a l ,R l ,s l+1 ), then the derivative of the loss function L(Q) is taken;

[0065] Step 64: According to the deterministic policy gradient theorem, the policy network π θ (·) is updated by gradient ascent of the parameter θ;

[0066] Step 65: Target value network Q ω' (·) parameter ω', target policy network π θ' (·) to update the parameter θ';

[0067] Step 66: If the current number of iterations is less than the maximum number of iterations, return to step 3, otherwise execute step 7.

[0068] Furthermore, in step 61, the loss function L(Q) of the value network is derived by the following formula:

[0069]

[0070]

[0071] in, represents the derivative of the loss function L(Q), Represents the value network Q ω (·) parameter ω to find the gradient, Q ω (s,a) represents the value network Q in any layer of model M. ω (·) The estimated value of the agent performing action a in state s, δ represents the temporal difference error, and y represents the temporal difference target;

[0072] In step 62, the value network Q is updated by the following formula ω The parameter ω of (·):

[0073]

[0074] Among them, n represents the total number of sampled samples. The derivative of the loss function representing the value network The value of the k-th sample, L k (Q) represents the value of the loss function of the value network on the kth sample;

[0075] In step 63, the calculation formula for deriving the loss function L(Q) is:

[0076]

[0077] q k =Q ω (s l ,a l )

[0078] y k =R l +γQ ω' (s l+1 ,a l+1 )

[0079] a l+1 =π θ' (s l+1 )

[0080] Among them, q k Representing the value network Q ω (·) For the agent in state s l Next, perform action a l The estimated value, y k represents the time difference target, q k With y k The difference is the time difference error;

[0081] In step 64, the policy network π θ The update method for gradient ascent update of the parameter θ of (·) is:

[0082]

[0083] Among them, β represents the policy network π θ (·) is the learning rate, Represents the policy network π θ (·) to find the gradient of the parameter θ, π θ (s l ) represents the policy network π θ (·) In state s l The action value output when represents the gradient of the action value, Representing the value network Q ω (·) For the agent in state s l Next action The estimated value, Representing the policy network π θ (·) For the agent in state s l The predicted action;

[0084] In step 65, the target value network Q is updated by the following formula ω' (·) parameter ω', target policy network π θ' The parameter θ' of (·):

[0085] ω'←(1-τ)ω+τω'

[0086] θ'←(1-τ)θ+τθ'

[0087] Among them, τ represents the ratio of the soft update of the parameters of the two target networks.

[0088] Further, the step 7 comprises:

[0089] The learning rate of the image classification model M to be pruned is set to decay and updated, and the image classification model M to be pruned after pruning is trained on the data set for fine-tuning.

[0090] Due to the adoption of the above technical solution, the present invention has the following advantages:

[0091] 1. The present invention uses priority experience playback and importance sampling technology to compress the size of the image classification model to be pruned to the greatest extent while ensuring the minimum loss of accuracy of the image classification model to be pruned, thereby achieving lightweight image classification model to be pruned.

[0092] 2. The present invention can automatically learn the optimal pruning ratio of each layer through a reinforcement learning algorithm to achieve adaptive pruning without manual intervention and repeated experiments, thereby reducing the labor cost of the pruning process and improving pruning efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0093] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in the embodiments of the present invention. For ordinary technicians in this field, other drawings can also be obtained based on these drawings.

[0094] Figure 1 A schematic diagram of a flow chart of an adaptive pruning method based on priority experience playback according to an embodiment of the present invention;

[0095] Figure 2 A schematic diagram of channel pruning of an image classification model to be pruned for image classification according to an embodiment of the present invention;

[0096] Figure 3A schematic diagram of neural network parameter update of a deep deterministic policy gradient algorithm according to an embodiment of the present invention;

[0097] Figure 4 The figure is a schematic diagram of the overall training process of an adaptive pruning method based on priority experience playback according to an embodiment of the present invention. DETAILED DESCRIPTION

[0098] The present invention is further described in conjunction with the accompanying drawings and embodiments, and the described embodiments are only a part of the embodiments of the present invention, rather than all the embodiments. All other embodiments obtained by ordinary technicians in this field should fall within the scope of protection of the embodiments of the present invention.

[0099] The purpose of the present invention is to provide an adaptive pruning method based on priority experience replay, which converts the pruning process of the neural network into a Markov decision process in advance, and designs the corresponding state space, action space and reward function. In the iterative pruning process, the pruning ratio of each layer of the network is first output through the deep deterministic policy gradient algorithm; then, the current reward value is calculated according to the accuracy, parameter quantity and other information of the network image classification model to be pruned, and the state transition sequence is stored in the experience replay pool, and the corresponding priority is set for it; then, according to the sampling probability of historical experience, sampling is performed from the experience replay pool, and the value network and policy network in the deep deterministic policy gradient algorithm are updated; finally, the image classification model to be pruned is fine-tuned to achieve the best accuracy of the image classification model to be pruned. This method can adaptively generate the pruning ratio of each layer of the network and realize the adaptive compression of the network image classification model to be pruned.

[0100] Specifically, see Figures 1 to 4 The present invention provides an embodiment of an adaptive pruning method based on priority experience playback, which comprises the following steps:

[0101] Step 1: Train the image classification model to be pruned, transform the pruning process of the neural network into a Markov decision process, and design the state space, action space, and reward function. Specifically include:

[0102] (1) Selecting an image classification model M to be pruned and training the image classification model M on the data set;

[0103] (2) Encode the network layer features of each layer of the image classification model M to be pruned as states, including: the index l of the network layer, the number of filters N of the l-th layer network, the number of channels C, height H, width W of the input feature map, the size K of the convolution kernel, the convolution step S, the floating-point calculation amount F of the l-th layer, the number of parameters P of the l-th layer, the floating-point calculation amount RF that has been reduced before the l-th layer, the number of parameters RP that has been reduced before the l-th layer, and the action a taken on the previous layer l-1, this action is the pruning ratio of the previous layer, then the state of the l-th layer network is (l,N,C,H,W,K,S,F,P,RF,RP,a l-1 ), where the calculation formulas for the floating-point calculation amount F and the parameter amount P are as follows:

[0104] F=K×K×C×N×H×W

[0105] P=K×K×C×N

[0106] (3) The pruning ratio of each layer of the image classification model M to be pruned is used as the action a of the intelligent agent. l , and satisfies a l ∈[0,1);

[0107] (4) A reward function R is defined based on the accuracy loss e of the image classification model to be pruned, the number of parameters P, the floating-point calculation amount F, and the inference delay D of the image classification model to be pruned. The reward calculation formula is as follows:

[0108]

[0109] Among them, e represents the difference in accuracy between the pruned image classification model and the original pruned image classification model, ρ, σ, Respectively represent the parameter amount, floating point calculation amount, and weight factor of the inference delay of the image classification model to be pruned; P is the parameter amount of the lth layer of the image classification model to be pruned at the current moment, P max is the maximum value of the parameter of the lth layer of the image classification model to be pruned; F is the floating-point calculation amount of the lth layer of the image classification model to be pruned at the current moment, F max is the maximum value of the floating-point calculation of the lth layer of the image classification model to be pruned at the current moment; D is the inference delay of the image classification model to be pruned, D max is the maximum value of the inference delay of the image classification model to be pruned;

[0110] (5) Initialize the value network Q of the deep deterministic policy gradient algorithm ω The parameters ω of (·) and the policy network π θ (·) parameter θ;

[0111] (6) Set the target value network Q with the same structure ω' (·) and value network Q ω (·), setting the target policy network π with the same structure θ' (·) with the policy network π θ (·);

[0112] Among them, Q ω' The parameters ω' and π of (·) θ' The initialization strategy of the parameter θ' of (·) is as follows:

[0113] ω'←ω

[0114] θ'←θ

[0115] Step 2: Build and initialize the value network, policy network, target value network, and target policy network of the deep deterministic policy gradient algorithm. Specifically include:

[0116] (1) Constructing the value network Q of the deep deterministic policy gradient algorithm ω (·), policy network π θ (·), target value network Q ω' (·) and the target policy network π θ' (·), respectively three fully connected layers;

[0117] (2) Initialize Q ω (·),π θ (·), Q ω' (·) and π θ' (·).

[0118] Step 3: Encode the attributes of each layer of the image classification model to be pruned to obtain the corresponding state, add noise to the action output by the policy network as the pruning ratio of each layer of the image classification model to be pruned, and prune the image classification model to be pruned;

[0119] (1) Encode the attributes of the lth layer of the image classification model M to be pruned to obtain the state s l , use sl as the policy network π θ (·) input;

[0120] (2) Policy network π θ Gaussian noise is added to the output of (·) to obtain action a l , the calculation formula is as follows:

[0121] a l =clip(π θ (s l )+ε,0,1)

[0122] ε~N(0,σ 2 )

[0123] Among them, clip represents the truncated action value after adding noise to the range of [0,1), and then obtains the pruning ratio of the lth layer of the model M, ε represents zero mean and variance σ 2 Gaussian noise;

[0124] (3) Calculate the importance of each channel in the lth layer of the image classification model M to be pruned, and use the sum of the weight values ​​in the channel as the criterion for determining the importance of the feature channel. The process is as follows:

[0125] The N filters of size C×K×K are split into C weight sets of size N×K×K in the channel dimension. 1 ,W 2 ,…,W C}, for the i-th channel, its importance I i The calculation formula is as follows:

[0126]

[0127] in, Representation Rights Reorganization i When a channel of a layer is pruned, the input features connected to the channel become redundant parameters and can be pruned, so the corresponding filters in the upper network can also be pruned;

[0128] (4) Perform channel pruning on the lth layer of the pruned image classification model M. According to the importance of each channel, prune the channel with the lowest importance until the pruning ratio of this layer is reached. Assume that the input feature map dimension is H×W×C, the filter size is C×K×K, and the number of filters N depends on the number of features to be extracted. Channel pruning performs regularized pruning on the channels. If channel pruning changes the number of channels of the network layer from C to C', the number of network layer parameters is reduced from H×W×C×K×K×N to H×W×(1-a l )×C×K×K×N, channel pruning compresses the network parameters to 1-a of the original network l times.

[0129] Step 4: Count the accuracy, number of parameters, amount of computation, and inference delay of the image classification model to be pruned, calculate the reward based on the designed reward function, store the pruning process of the image classification model to be pruned into the experience replay pool, and set a priority for it. Specifically include:

[0130] (1) Testing the classification accuracy loss E of the image classification model M to be pruned on the classification data set after pruning, and calculating the parameter quantity P, floating-point calculation quantity F, and inference delay D of the image classification model M to be pruned, and calculating the reward value according to the reward function R designed in step 1 (4);

[0131] (2) The state s of the lth layer of the image classification model M to be pruned l 、Action a l , reward value R l , successor state s l+1 As a state transition sequence (s l ,a l ,R l ,s l+1 ), and record it as T;

[0132] With the help of target strategy network π θ' (·) and the target value network Q ω' (·) Calculate the time difference target y of the state transition sequence T l and the timing difference error δ l , the calculation formula is as follows:

[0133] δ l =Q ω (s l ,a l )-y l

[0134] y l =R l +γQ ω' (s l+1 ,a l+1 )

[0135] a l+1 =π θ' (s l+1 )

[0136] Among them, γ represents the discount factor of the return, γ∈[0,1];

[0137] (3) Set the priority L for the state transition sequence T according to the absolute value of the timing difference error T , and store T in the experience replay pool E. The calculation formula is as follows:

[0138]

[0139] Among them, rank represents the state transition sequence T with the time difference error δ l When the absolute value of is the standard, the ranking in the experience replay pool E.

[0140] Step 5: Based on the sampling probability of historical experience, sample state transition samples from the experience replay pool, use importance sampling to calculate the importance sampling weight and time difference error of each state transition sample, and update the sampling probability of each sample based on the time difference error. Specifically include:

[0141] (1) For the state transition sample T in the experience replay pool E, its sampling priority is L T , calculate the sampling probability p of T s (T) and importance sampling weight W T , based on the weighted sampling probability p w (T) Sample n state transition samples from the experience replay pool E. The calculation formula is as follows:

[0142]

[0143] p w (t) = W T p s (t)

[0144] in, Represents the sampling priority L T The degree of use is cumulative. Represents any sampling priority L k The usage degree of , t represents time, α represents the usage degree of priority, S represents the size of the experience replay pool E, and χ represents the correction factor of the importance sampling weight;

[0145] (2) For n state transition samples, calculate the timing difference error δ of each sample l , according to δ l The absolute value of updates the priority and sampling probability of each sample.

[0146] Step 6: Update the value network and policy network, update the target value network and target policy network, and repeat the training until the maximum number of iterations is reached. Specifically, it includes:

[0147] (1) Based on n state transition samples, the loss function L(Q) of the value network is derived. The calculation formula is as follows:

[0148]

[0149]

[0150] in, represents the derivative of the loss function L(Q), Represents the value network Q ω (·) parameter ω to find the gradient, Q ω (s,a) represents the value network Q in any layer of model M. ω (·) The estimated value of the agent performing action a in state s, δ represents the temporal difference error, and y represents the temporal difference target;

[0151] Set the learning rate α, average the derivatives of the loss function L(Q) obtained by n sampling samples, and update the value network Q ω The parameter ω of (·) is as follows:

[0152]

[0153] Among them, n represents the total number of sampled samples. The derivative of the loss function representing the value network The value of the k-th sample, L k(Q) represents the value of the loss function of the value network on the kth sample. For the kth sample, let its form be (s l ,a l ,R l ,s l+1 ), then the calculation formula for the derivative of the loss function L(Q) is as follows:

[0154]

[0155] q k =Q ω (s l ,a l )

[0156] y k =R l +γQ ω' (s l+1 ,a l+1 )

[0157] a l+1 =π θ' (s l+1 )

[0158] Among them, q k Representing the value network Q ω (·) For the agent in state s l Next, perform action a l The estimated value, y k represents the time difference target, q k With y k The time difference error

[0159] (2) According to the deterministic policy gradient theorem, the policy network π θ The parameter θ of (·) is updated by gradient ascent, and the update method is as follows:

[0160]

[0161] Among them, β represents the policy network π θ (·) is the learning rate, Represents the policy network π θ (·) to find the gradient of the parameter θ, π θ (s l ) represents the policy network π θ (·) In state s l The action value output when represents the gradient of the action value, Representing the value network Q ω (·) For the agent in state s l Next action The estimated value, Representing the policy network π θ (·) For the agent in state s l The predicted action;

[0162] (3) Target value network Q ω' (·) parameter ω', target policy network π θ' The parameter θ' of (·) is updated as:

[0163] ω'←(1-τ)ω+τω'

[0164] θ'←(1-τ)θ+τθ'

[0165] Among them, τ represents the ratio of the parameters of the two target networks to be updated;

[0166] (4) Repeat steps 3 to 6 until the maximum number of iterations is reached.

[0167] Step 7: Fine-tune the image classification model to be pruned to restore the accuracy to a preset degree.

[0168] (1) Set the learning rate decay of the image classification model M to be pruned and update it, and keep other parameters the same as during training;

[0169] (2) The image classification model M to be pruned after pruning is trained on the dataset and fine-tuned to improve the classification accuracy.

[0170] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the above embodiments, ordinary technicians in the relevant field should understand that the specific implementation methods of the present invention can still be modified or replaced by equivalents. Any modification or equivalent replacement that does not depart from the spirit and scope of the present invention should be covered within the scope of protection of the claims of the present invention.

Claims

1. An adaptive pruning method based on priority experience playback, characterized in that: include: Step 1: Train the image classification model to be pruned, transform the pruning process of the neural network into a Markov decision process, and design the corresponding state space, action space, and reward function; Step 2: Build and initialize the value network, policy network, target value network, and target policy network of the deep deterministic policy gradient algorithm; Step 3: Encode the attributes of each layer of the image classification model to be pruned to obtain the corresponding state, add noise to the action output by the policy network as the pruning ratio of each layer of the image classification model to be pruned, and prune the image classification model to be pruned; Step 4: Count the accuracy, number of parameters, amount of computation, and inference delay of the image classification model to be pruned, calculate the reward according to the designed reward function, store the pruning process of the image classification model to be pruned into the experience replay pool, and set a priority for it; Step 5: Based on the sampling probability of historical experience, sample state transition samples from the experience replay pool, use importance sampling to calculate the importance sampling weight and time difference error of each state transition sample, and update the sampling probability of each sample based on the time difference error; Step 6: Update the value network and policy network, update the target value network and target policy network, and repeat the training until the maximum number of iterations is reached. Step 7: Fine-tune the image classification model to be pruned so that its accuracy is restored to a preset degree.

2. The method according to claim 1, characterized in that: The step 1 comprises: Step 11: Train the image classification model M to be pruned on the data set; encode the network layer features of each layer of the image classification model M to be pruned as states; the network layer features include: the index l of the network layer, the number of filters N of the l-th layer network, the number of channels C, height H, width W of the input feature map, the size K of the convolution kernel, the convolution step S, the floating-point calculation amount F of the l-th layer, the number of parameters P of the l-th layer, the floating-point calculation amount RF that has been reduced before the l-th layer, the number of parameters RP that has been reduced before the l-th layer, and the action a taken on the previous layer l-1 , which is the pruning ratio of the previous layer; the state of the l-th layer network of the image classification model M to be pruned is (l,N,C,H,W,K,S,F,P,RF,RP,a l-1 ); Step 12: Take the pruning ratio of each layer of the image classification model M to be pruned as the action a of the agent l , and satisfies a l ∈[0,1); define the reward function R according to the accuracy loss e of the image classification model to be pruned, the number of parameters P, the amount of floating-point calculation F, and the inference delay D of the image classification model to be pruned; Step 13: Initialize the value network Q of the deep deterministic policy gradient algorithm ω The parameters ω of (·) and the policy network π θ (·) parameter θ; Step 14: Set up a target value network Q with the same structure ω' (·) and value network Q ω (·), setting the target policy network π with the same structure θ' (·) with the policy network π θ (·).

3. The method according to claim 2, characterized in that In step 11, the calculation formula of the floating point calculation amount F and the parameter amount P is: F=K×K×C×N×H×W P=K×K×C×N In step 12, the reward function R is expressed as: Where e represents the difference in accuracy between the pruned image classification model and the original image classification model, ρ, σ, and ζ represent the weight factor of the parameter amount, floating-point calculation amount, and inference delay of the pruned image classification model, respectively; P is the parameter amount of the lth layer of the image classification model to be pruned at the current moment, and P max is the maximum value of the parameter of the lth layer of the image classification model to be pruned; F is the floating-point calculation amount of the lth layer of the image classification model to be pruned at the current moment, F max is the maximum value of the floating-point calculation of the lth layer of the image classification model to be pruned at the current moment; D is the inference delay of the image classification model to be pruned, D max is the maximum value of the inference delay of the image classification model to be pruned; Target Value Network Q ω' The parameters ω' and the target policy network π θ' The parameter θ' initialization strategy of (·) is: ω'←ω θ'←θ.

4. The method according to any one of claims 2 to 4, characterized in that: The step 2 comprises: Constructing the value network Q of the deep deterministic policy gradient algorithm ω (·), policy network π θ (·), target value network Q ω' (·) and the target policy network π θ' (·), respectively three layers of fully connected layers; initialize Q ω (·),π θ (·), Q ω' (·) and π θ' (·).

5. The method according to claim 4, characterized in that The step 3 comprises: Step 31: Encode the attributes of the lth layer of the image classification model M to be pruned to obtain the state s l , will s l As a policy network π θ (·) input; for the policy network π θ Gaussian noise is added to the output of (·) to obtain action a l , this action is the pruning ratio of the lth layer of model M; Step 32: Calculate the importance of each channel of the lth layer of the image classification model M to be pruned, and use the sum of the weight values ​​in the channel as a criterion for determining the importance of the feature channel; Step 33: Perform channel pruning on the lth layer of the image classification model M to be pruned. According to the importance of each channel, prune the channel with the lowest importance until the pruning ratio of this layer is reached.

6. The method according to claim 5, characterized in that In step 31, action a l The expression is: a l =clip(π θ (s l )+ε,0,1) ε~N(0,σ 2 ) Among them, clip represents the truncated action value after adding noise to the range of [0,1), and then obtains the pruning ratio of the lth layer of the model M, ε represents zero mean and variance σ 2 Gaussian noise; The step 32 comprises: The N filters of size C×K×K are split into C weight sets of size N×K×K in the channel dimension. 1 ,W 2 ,…,W C }, for the i-th channel, its importance I i The calculation formula is: in, Representation Rights Reorganization i The weight value in ; when the channel of any layer is pruned, the input features connected to the channel become redundant parameters.

7. The method according to claim 5 or 6, characterized in that: The step 4 comprises: Step 41: testing the classification accuracy loss E of the image classification model M to be pruned after pruning on the classification data set, and calculating the parameter quantity P, floating point calculation quantity F, and inference delay D of the image classification model M to be pruned, and calculating the reward value according to the reward function R; Step 42: The state s of the lth layer of the image classification model M to be pruned l 、Action a l , reward value R l , successor state s l+1 As a state transition sequence (s l ,a l ,R l ,s l+1 ), and record it as T; Step 43: Using the target policy network π θ' (·) and the target value network Q ω' (·), calculate the time difference target y of the state transition sequence T l and the timing difference error δ l ; Step 44: Set the priority L for the state transition sequence T according to the absolute value of the timing differential error T , and store T into the experience replay pool E.

8. The method according to claim 7, characterized in that In step 43, the time series difference target y l and the timing difference error δ l The calculation formula is: δ l =Q ω (s l ,a l )-y l y l =R l +γQ ω' (s l+1 ,a l+1 ) a l+1 =π θ' (s l+1 ) Where γ represents the discount factor of the return, γ∈[0,1]; In step 44, the priority L T The calculation formula is: Among them, rank represents the state transition sequence T with the time difference error δ l When the absolute value of is the standard, the ranking in the experience replay pool E.

9. The method according to claim 1, characterized in that: The step 5 comprises: For the state transition sample T in the experience replay pool E, its sampling priority is L T , calculate the sampling probability p of T s (T) and importance sampling weight W T , based on the weighted sampling probability p w (T) Sample n state transition samples from the experience replay pool E; For n state transition samples, calculate the timing difference error δ of each sample l , according to δ l The absolute value of updates the priority and sampling probability of each sample.

10. The method according to claim 9, characterized in that Sampling probability p s (T), importance sampling weight W T and weighted sampling probability p w The calculation formulas for (T) are: p w (t)=W T p s (t) in, Represents the sampling priority L T The degree of use is cumulative. Represents any sampling priority L k , t represents the time, α represents the priority usage, S represents the size of the experience replay pool E, and χ represents the correction factor of the importance sampling weight.

11. The method according to claim 1, characterized in that The step 6 comprises: Step 61: Based on the n state transition samples, derive the loss function L(Q) of the value network; Step 62: Set the learning rate α, average the derivatives of the loss function L(Q) obtained from n samples, and update the value network Q ω (·) parameter ω; Step 63: For the kth sample, let its form be (s l ,a l ,R l ,s l+1 ), then the derivative of the loss function L(Q) is taken; Step 64: According to the deterministic policy gradient theorem, the policy network π θ (·) is updated by gradient ascent of the parameter θ; Step 65: Target value network Q ω' (·) parameter ω', target policy network π θ' (·) to update the parameter θ'; Step 66: If the current number of iterations is less than the maximum number of iterations, return to step 3, otherwise execute step 7.

12. The method according to claim 11, characterized in that In step 61, the loss function L(Q) of the value network is derived by the following formula: in, represents the derivative of the loss function L(Q), Represents the value network Q ω (·) parameter ω to find the gradient, Q ω (s,a) represents the value network Q in any layer of model M. ω (·) The estimated value of the agent performing action a in state s, δ represents the temporal difference error, and y represents the temporal difference target; In step 62, the value network Q is updated by the following formula ω The parameter ω of (·): Where n represents the total number of samples. The derivative of the loss function representing the value network The value of the k-th sample, L k (Q) represents the value of the loss function of the value network on the kth sample; In step 63, the calculation formula for deriving the loss function L(Q) is: q k =Q ω (s l ,a l ) y k =R l +γQ ω' (s l+1 ,a l+1 ) a l+1 =π θ' (s l+1 ) Among them, q k Representing the value network Q ω (·) The estimated value of the agent performing action al in state sl, y k represents the time difference target, q k With y k The difference is the time difference error; In step 64, the policy network π θ The update method for gradient ascent update of the parameter θ of (·) is: Among them, β represents the policy network π θ (·) is the learning rate, Represents the policy network π θ (·) to find the gradient of the parameter θ, π θ (s l ) represents the policy network π θ (·) In state s l The action value output when represents the gradient of the action value, Representing the value network Q ω (·) For the agent in state s l Next action The estimated value, Representing the policy network π θ (·) For the agent in state s l The predicted action; In step 65, the target value network Q is updated by the following formula ω' (·) parameter ω', target policy network π θ' The parameter θ' of (·): ω'←(1-τ)ω+τω' θ'←(1-τ)θ+τθ' Among them, τ represents the ratio of the parameters of the two target networks to be updated.

13. The method according to claim 1, characterized in that The step 7 comprises: The learning rate of the image classification model M to be pruned is set to decay and updated, and the image classification model M to be pruned after pruning is trained on the data set for fine-tuning.

Images