A neural network automatic pruning method based on GRPO reinforcement learning

Abstract

The application belongs to the technical field of artificial intelligence, and particularly relates to a neural network automatic pruning method based on GRPO reinforcement learning, which comprises introducing a dynamic scaling factor into a batch normalization layer of a neural network to be pruned, and combining an attention mechanism to calculate the importance score of each convolution layer channel of the neural network to be pruned; based on the importance calculation result of step S1, a multi-dimensional state vector containing layer structure features is constructed; the multi-dimensional state vector constructed in S2 is input into a policy network of a GRPO reinforcement learning agent, and the policy network generates a pruning action according to the state information of the current network layer, and the action is defined to represent the pruning rate of the layer; the method adopts the GRPO reinforcement learning algorithm, discards the traditional Critic model, simplifies the policy calculation process through the "group sampling-relative advantage estimation" mechanism, and significantly reduces the memory occupation.

Description

Technical Field

[0001] This invention belongs to the field of artificial intelligence technology, specifically relating to an automated pruning method for neural networks based on GRPO reinforcement learning. Background Technology

[0002] As artificial intelligence technology penetrates edge computing scenarios such as IoT terminals and embedded devices, deep neural networks, due to their large number of parameters and high computational overhead, are difficult to deploy efficiently on resource-constrained edge devices. To achieve lightweight models, neural network pruning techniques have become a key approach. The core of this technique is to remove redundant structures to reduce computational and storage costs while maintaining model accuracy.

[0003] Existing automated pruning methods mostly employ reinforcement learning algorithms such as DDPG or PPO, but these methods have certain limitations: DDPG requires maintaining an Actor-Critic dual network structure, which consumes a lot of memory; while PPO relies on the Critic model to estimate the value function, introducing additional computational redundancy. Summary of the Invention

[0004] To address the aforementioned shortcomings of existing technologies, this invention provides an automated neural network pruning method based on GRPO reinforcement learning, comprising:

[0005] S1. Introduce a dynamic scaling factor into the batch normalization layer of the neural network to be pruned, and combine it with an attention mechanism to calculate the importance score of each convolutional layer channel of the neural network to be pruned. ;

[0006] S2, Importance score based on step S1 Construct a multidimensional state vector containing layer structure features. ;

[0007] S3, convert the multidimensional state vector The policy network input to the GRPO reinforcement learning agent By this policy network Generate pruning actions based on the current network layer state information. At the same time, based on pruning actions Set the pruning rate for this layer. ;

[0008] S4. Based on the GRPO strategy output of the network at each layer in S3, calculate the pruning rate. Combined with the channel importance score calculated in S1 Sort the redundant channels from low to high importance and remove the corresponding proportion; after completing the pruning of the current layer, update the network structure state and advance the processing flow to the next layer until all layers to be pruned have been traversed.

[0009] S5. Adjust the parameters of the lightweight neural network that has been pruned in S4, and use the validation set to calculate the accuracy (Acc) of the lightweight neural network after pruning and the change in floating-point operations (FLOPs) before and after pruning. Evaluate the pruning effect based on the accuracy (Acc) and the floating-point operations (FLOPs), and feed the pruning effect back to the GRPO policy agent in S3 for the next round of policy iteration optimization.

[0010] Further improvements to this technical solution include step S1, which includes:

[0011] S11. Introduce a dynamic scaling factor in the batch normalization layer of the neural network to be pruned. Dynamic scaling factor is adjusted using L1 regularization. Constraints are applied to achieve sparsity of channel weights; simultaneously, a calibration matrix is ​​constructed based on the maximum activation feature maps of various types of samples in the training dataset. ;

[0012] S12. Input the preprocessed training dataset samples into the neural network to be pruned, activate each convolutional layer and batch normalization layer through forward propagation, and obtain the feature vectors of each channel. Calculate the eigenvector With calibration matrix eigenvectors cosine similarity The formula is:

[0013] ;

[0014] in, The element in the i-th dimension of the feature map vector; The element in the i-th dimension of the calibration matrix; The dimension of the feature vector;

[0015] S13. Extract the dynamic scaling factor constrained by L1 regularization from the batch normalization layer. Attention weights output by the attention mechanism The cosine similarity obtained in step S12 and convolutional layer channel weights The sum of the L2 norms is used to calculate the channel importance score. The formula is:

[0016] ;

[0017] in, The disturbance factor has a value range of 1. ; Let be the L2 norm of the weights of the j-th filter and the k-th input channel; n is the number of filters, i.e., the number of output channels; and C is the number of input channels.

[0018] Further improvements to this technical solution include step S11, which further includes:

[0019] Based on cosine similarity The loss function for the batch normalization layer is defined as follows:

[0020] ;

[0021] in, It is an L1 regularization function; The regularization coefficient is used. The original loss function; For training samples and labels; For the convolutional layer weight tensor; For the Sigmoid function; It is a tiny constant; The set of dynamic scaling factors γ introduced in all convolutional layers to be pruned;

[0022] Further improvements to this technical solution include step S2, which includes:

[0023] S21. Extract the basic structural features of each convolutional layer of the neural network to be pruned, including the current layer index. Number of filters Number of input channels (C), Input data height Input data width and kernel size This forms a subset of basic features;

[0024] S22. Calculate the dynamic performance characteristics of each convolutional layer, including the floating-point operation volume of the current layer. The cumulative reduction in floating-point operations Total floating-point operations of the remaining layers and the pruning rate of the upper layer This forms a subset of performance characteristics;

[0025] S23. Calculate the channel importance score obtained in step S1. The feature set is incorporated and integrated with the basic feature subset from step S21 and the performance feature subset from step S22 to construct an 11-dimensional state vector. The formula is:

[0026] .

[0027] Further improvements to this technical solution include step S3, which includes:

[0028] S31. Based on the multidimensional state vector constructed in step S2, determine the boundary conditions of the current pruning task, including the range of the layers to be pruned and the accuracy loss threshold; call the parallel sampling module of the GRPO reinforcement learning agent to sample G groups of candidate pruning rate action sequences in parallel for each pruning task, each group of action sequences containing the pruning rate corresponding to each layer to be pruned. The action sequence of candidate pruning rates in group G is denoted as Each action sequence independently generates pruning strategy candidate schemes;

[0029] S32. Apply each candidate pruning rate action sequence to the neural network to be pruned, obtain the neural network performance after each action is executed through forward inference, and construct a reward function based on the precision-resource dual objective to calculate the reward value of each action. Then calculate the reward value for all groups. Intra-group standardization was performed to obtain the relative advantage of each group of actions. Eliminate single-group reward estimation bias; neural network performance includes validation set accuracy and floating-point computational complexity.

[0030] S33. Update the current policy network of the GRPO reinforcement learning agent. ;

[0031] S34. Utilizing relative advantages Construct the objective function for updating the GRPO policy, and update the GRPO reinforcement learning agent policy network using gradient ascent. Parameters;

[0032] S35. After the update is completed, select the relative advantage from the candidate pruning rate action sequences of group G. The largest set is determined as the optimal pruning action sequence for the current pruning task, and the pruning rate of each layer in this sequence is the final pruning action generated by the policy network.

[0033] Further improvements to this technical solution include a reward function as follows:

[0034] ;

[0035] in, To reduce the accuracy of the neural network after pruning; This represents the floating-point computation cost of the neural network before pruning. This represents the floating-point computation cost of the pruned neural network.

[0036] Further improvements to this technical solution include its relative advantages. The calculation formula is:

[0037] ;

[0038] in, This is the group reward mean, that is, the arithmetic mean of the rewards for group G; The standard deviation of the group reward; This is the reward value for a single group.

[0039] Further improvements to this technical solution include updating the objective function of the GRPO strategy as follows:

[0040] ;

[0041] in, The objective function for optimizing the GRPO strategy; For parameter vectors; For the pruning task; This represents the probability distribution for the pruning task; Number of samples per group; For the first Action sequences obtained from group sampling; For the first The specific action generated in the action sequence at step t; For the first All historical actions in the sequence prior to step t; For the new strategy network to be optimized; For old policy networks; The relative advantage of a group is the relative advantage function of the i-th action sequence at step t. These are the KL regularization coefficients; This is the clipping function.

[0042] Further improvements to this technical solution include step S4, which includes:

[0043] S41. For the t-th layer of the neural network to be pruned, extract the pruning rate of that layer output in step S3. The importance scores of all channels calculated in step S1 are sorted from low to high to obtain the sorted channel index sequence.

[0044] S42. Based on the pruning rate Calculate the number of channels k to be pruned in the current layer, select the first k channels as redundant channels from the sorting results in step S41, and set their output features to zero by setting a mask vector during the forward propagation of the network to achieve channel pruning;

[0045] S43. After completing the pruning of the t-th layer, recalculate the number of output channels, floating-point operation volume and feature map size of the layer, and update the structural features and performance features related to the layer in the multidimensional state vector constructed in step S2.

[0046] S44. Proceed to the next layer and repeat steps S41 to S43 until all convolutional layers of the neural network to be pruned are traversed to obtain the lightweight network structure after preliminary pruning.

[0047] Further improvements to this technical solution include step S5, which includes:

[0048] S51. The parameters of the lightweight neural network obtained in step S4 are adjusted using a pre-stored fine-tuning strategy. Backpropagation is performed using a batch of samples from the training dataset. The learning rate is fixed at 1 / 10 of the initial training phase. Iterate for 5 to 10 epochs to recover the accuracy loss caused by pruning.

[0049] S52. Input the preprocessed validation set samples into the fine-tuned lightweight neural network, calculate the classification accuracy (Acc) of the model on the validation set through forward inference, and simultaneously calculate the floating-point operation cost of the model before and after pruning. and And calculate the reward value corresponding to the current pruning strategy based on the reward function. ;

[0050] S53. Transfer the reward value obtained in step S52. The multidimensional state vector from step S2 and the pruning action sequence from step S3 are associated and stored as empirical data input to the GRPO reinforcement learning agent's experience replay pool, which is used to drive the policy network. The next round of parameter updates will enable iterative optimization of the pruning strategy.

[0051] The beneficial effects of this invention are as follows:

[0052] Reduced memory and computational overhead: This method employs the GRPO reinforcement learning algorithm, abandoning the traditional Critic model. It simplifies the policy calculation process through a "group sampling-relative advantage estimation" mechanism, significantly reducing memory usage and computational overhead. Compared to the DDPG algorithm, this method reduces memory consumption and is more suitable for deployment on resource-constrained edge devices.

[0053] Improving pruning accuracy and efficiency: By introducing a dynamic scaling factor and an attention mechanism, this method can more accurately calculate channel importance, thereby identifying and removing redundant structural units. Combined with GRPO's "group relative advantage" mechanism, this method effectively reduces reward estimation bias while reducing the number of model parameters and floating-point operations.

[0054] Achieving end-to-end automated pruning: This method constructs a fully automated framework from importance calculation to GRPO strategy formulation, and then to pruning and edge deployment, which can complete the pruning process without human intervention.

[0055] Optimizing Policy Exploration and Stability: The GRPO algorithm improves the efficiency and stability of policy exploration through group sampling and relative advantage estimation. Compared with DDPG and PPO algorithms, this method reduces variance and bias during policy updates, making the pruning policy more stable and reliable. Attached Figure Description

[0056] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, for those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0057] Figure 1 This is a schematic flowchart illustrating a method according to an embodiment of the present invention. Detailed Implementation

[0058] To make the objectives, features, and advantages of this invention more apparent and understandable, the technical solutions of this invention will be clearly and completely described below with reference to the accompanying drawings of the specific embodiments. Obviously, the embodiments described below are only some embodiments of this invention, and not all embodiments. Based on the embodiments in this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0059] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains. The terminology used herein in the description of the invention is for the purpose of describing particular embodiments only and is not intended to be limiting of the invention.

[0060] Figure 1 This is a schematic flowchart of an automated neural network pruning method based on GRPO reinforcement learning provided by the present invention. The order of the steps in this flowchart can be changed, and some steps can be omitted, depending on different requirements.

[0061] like Figure 1 As shown, the method includes:

[0062] S1. Introduce a dynamic scaling factor into the batch normalization layer of the neural network to be pruned, and combine it with an attention mechanism to calculate the importance score of each convolutional layer channel of the neural network to be pruned. ;

[0063] S2, Importance score based on step S1 Construct a multidimensional state vector containing layer structure features. ;

[0064] S3, convert the multidimensional state vector The policy network input to the GRPO reinforcement learning agent By this policy network Generate pruning actions based on the current network layer state information. At the same time, based on pruning actions Set the pruning rate for this layer. ;

[0065] S4. Based on the GRPO strategy output of the network at each layer in S3, calculate the pruning rate. Combined with the channel importance score calculated in S1 Sort the redundant channels from low to high importance and remove the corresponding proportion; after completing the pruning of the current layer, update the network structure state and advance the processing flow to the next layer until all layers to be pruned have been traversed.

[0066] S5. Adjust the parameters of the lightweight neural network that has been pruned in S4, and use the validation set to calculate the accuracy (Acc) of the lightweight neural network after pruning and the change in floating-point operations (FLOPs) before and after pruning. Evaluate the pruning effect based on the accuracy (Acc) and the floating-point operations (FLOPs), and feed the pruning effect back to the GRPO policy agent in S3 for the next round of policy iteration optimization.

[0067] To facilitate understanding of the present invention, the following description further illustrates the automatic pruning method for neural networks based on GRPO reinforcement learning, using the principle of the present invention and the process of automatically pruning neural networks based on GRPO reinforcement learning in the embodiments.

[0068] First, step S1 includes:

[0069] S11. Introduce a dynamic scaling factor in the batch normalization layer of the neural network to be pruned. Dynamic scaling factor is adjusted using L1 regularization. Constraints are applied to achieve sparsity of channel weights; simultaneously, a calibration matrix is ​​constructed based on the maximum activation feature maps of various types of samples in the training dataset. ;

[0070] S12. Input the preprocessed training dataset samples into the neural network to be pruned, activate each convolutional layer and batch normalization layer through forward propagation, and obtain the feature vectors of each channel. Calculate the eigenvector With calibration matrix eigenvectors cosine similarity The formula is:

[0071] ;

[0072] in, The element in the i-th dimension of the feature map vector; The element in the i-th dimension of the calibration matrix; The dimension of the feature vector;

[0073] S13. Extract the dynamic scaling factor constrained by L1 regularization from the batch normalization layer. Attention weights output by the attention mechanism The cosine similarity obtained in step S12 and convolutional layer channel weights The sum of the L2 norms is used to calculate the channel importance score. The formula is:

[0074] ;

[0075] in, The disturbance factor has a value range of 1. ; Let be the L2 norm of the weights of the j-th filter and the k-th input channel; n is the number of filters, i.e., the number of output channels; and C is the number of input channels.

[0076] Furthermore, step S11 also includes:

[0077] Based on cosine similarity The loss function for the batch normalization layer is defined as follows:

[0078] ;

[0079] in, It is an L1 regularization function; The regularization coefficient is used. The original loss function; For training samples and labels; For the convolutional layer weight tensor; For the Sigmoid function; It is a tiny constant; The set of dynamic scaling factors γ introduced in all convolutional layers to be pruned; It serves as a benchmark for feature validity (usually the minimum value of feature similarity within the category is taken to ensure stable feature validity assessment).

[0080] Secondly, step S2 includes:

[0081] S21. Extract the basic structural features of each convolutional layer of the neural network to be pruned, including the current layer index. Number of filters Number of input channels (C), Input data height Input data width and kernel size This forms a subset of basic features;

[0082] S22. Calculate the dynamic performance characteristics of each convolutional layer, including the floating-point operation volume of the current layer. The cumulative reduction in floating-point operations Total floating-point operations of the remaining layers and the pruning rate of the upper layer This forms a subset of performance characteristics;

[0083] S23. Calculate the channel importance score obtained in step S1. The feature set is incorporated and integrated with the basic feature subset from step S21 and the performance feature subset from step S22 to construct an 11-dimensional state vector. The formula is:

[0084] .

[0085] Next, step S3 includes:

[0086] S31. Based on the multidimensional state vector constructed in step S2, determine the boundary conditions of the current pruning task, including the range of the layers to be pruned and the accuracy loss threshold; call the parallel sampling module of the GRPO reinforcement learning agent to sample G groups of candidate pruning rate action sequences in parallel for each pruning task, each group of action sequences containing the pruning rate corresponding to each layer to be pruned. The action sequence of candidate pruning rates in group G is denoted as Each action sequence independently generates pruning strategy candidate schemes;

[0087] S32. Apply each candidate pruning rate action sequence to the neural network to be pruned, obtain the neural network performance after each action is executed through forward inference, and construct a reward function based on the precision-resource dual objective to calculate the reward value of each action. Then calculate the reward value for all groups. Intra-group standardization was performed to obtain the relative advantage of each group of actions. Eliminate single-group reward estimation bias; neural network performance includes validation set accuracy and floating-point computational complexity.

[0088] S33. Update the current policy network of the GRPO reinforcement learning agent. ;

[0089] S34. Utilizing relative advantages Construct the objective function for updating the GRPO policy, and update the GRPO reinforcement learning agent policy network using gradient ascent. Parameters;

[0090] S35. After the update is completed, select the relative advantage from the candidate pruning rate action sequences of group G. The largest set is determined as the optimal pruning action sequence for the current pruning task, and the pruning rate of each layer in this sequence is the final pruning action generated by the policy network.

[0091] This involves calculating the reward value for each set of actions. The reward function is:

[0092] ;

[0093] in, To reduce the accuracy of the neural network after pruning; This represents the floating-point computation cost of the neural network before pruning. This represents the floating-point computation cost of the pruned neural network.

[0094] Furthermore, relative advantage The calculation formula is:

[0095] ;

[0096] in, This is the group reward mean, that is, the arithmetic mean of the rewards for group G; The standard deviation of the group reward; This is the reward value for a single group.

[0097] Furthermore, the objective function for updating the GRPO policy is:

[0098] ;

[0099] in, The objective function for optimizing the GRPO strategy; For parameter vectors; For the pruning task; This represents the probability distribution for the pruning task; Number of samples per group; For the first Action sequences obtained from group sampling; For the first The specific action generated in the action sequence at step t; For the first All historical actions in the sequence prior to step t; For the new strategy network to be optimized; For old policy networks; The relative advantage of a group is the relative advantage function of the i-th action sequence at step t. These are the KL regularization coefficients; This is the clipping function.

[0100] Then, step S4 includes:

[0101] S41. For the t-th layer of the neural network to be pruned, extract the pruning rate of that layer output in step S3. The importance scores of all channels calculated in step S1 are sorted from low to high to obtain the sorted channel index sequence.

[0102] S42. Based on the pruning rate Calculate the number of channels k to be pruned in the current layer, select the first k channels as redundant channels from the sorting results in step S41, and set their output features to zero by setting a mask vector during the forward propagation of the network to achieve channel pruning;

[0103] S43. After completing the pruning of the t-th layer, recalculate the number of output channels, floating-point operation volume and feature map size of the layer, and update the structural features and performance features related to the layer in the multidimensional state vector constructed in step S2.

[0104] S44. Proceed to the next layer and repeat steps S41 to S43 until all convolutional layers of the neural network to be pruned are traversed to obtain the lightweight network structure after preliminary pruning.

[0105] Finally, step S5 includes:

[0106] S51. The parameters of the lightweight neural network obtained in step S4 are adjusted using a pre-stored fine-tuning strategy. Backpropagation is performed using a batch of samples from the training dataset. The learning rate is fixed at 1 / 10 of the initial training phase. Iterate for 5 to 10 epochs to recover the accuracy loss caused by pruning.

[0107] S52. Input the preprocessed validation set samples into the fine-tuned lightweight neural network, calculate the classification accuracy (Acc) of the model on the validation set through forward inference, and simultaneously calculate the floating-point operation cost of the model before and after pruning. and And calculate the reward value corresponding to the current pruning strategy based on the reward function. ;

[0108] S53. Transfer the reward value obtained in step S52. The multidimensional state vector from step S2 and the pruning action sequence from step S3 are associated and stored as empirical data input to the GRPO reinforcement learning agent's experience replay pool, which is used to drive the policy network. The next round of parameter updates will enable iterative optimization of the pruning strategy.

[0109] Furthermore, the method also includes converting the pruned lightweight neural network into TensorRT format and deploying it to edge devices to perform inference tasks. The specific methods include:

[0110] S61. Perform format preprocessing on the pruned lightweight neural network after optimization and verification in step S5. First, export the neural network from the original training framework format to the ONNX general format. During the export process, fix the neural network input dimension, operator type and weight parameters.

[0111] S62. Start the TensorRT SDK tool on the NVIDIA Jetson Xavier NX edge device, load the ONNX format neural network exported in step S61, parse the neural network computation graph through the TensorRT model parser, automatically complete operator fusion, accuracy calibration and memory optimization, and generate TensorRT engine files adapted to the edge device hardware architecture.

[0112] S63. Configure the operating environment of the edge device, install the CUDA Toolkit, cuDNN library and Python / C++ inference dependency package that match the TensorRT engine, and load the TensorRT engine file generated in step S62 into the memory of the edge device.

[0113] Specifically, GRPO (Group Relative Policy Optimization) is a variant of PPO (Proximal Policy Optimization) designed to improve the performance of language models on mathematical reasoning tasks while significantly reducing memory consumption during training. Its core idea is to abandon the traditional critic model and instead estimate the advantage function by using the intra-group relative rewards of multiple sampled outputs for the same question, thereby reducing reliance on extra-value function models.

[0114] (1) Differences from PPO:

[0115] PPO uses GAE (Generalized Advantage Estimation) based on value functions. Computational advantages GRPO uses the standardized within-group reward as the advantage estimate, eliminating the need to train an additional value function model.

[0116] GRPO objective function:

[0117] GRPO is a variant of PPO, the core of which is to abandon the Critic model and calculate the relative advantage through group sampling. The objective function is as follows:

[0118] Symbol definition: For pruning tasks (such as channel filtering). For the first Group output, For the number of samples in a group, As a group's relative advantage, The KL regularization coefficients are... As a reference model (initial SFT pruning model);

[0119] Group relative advantage calculation (outcome supervision scenario): Reward for each group Normalization Value estimation is not required using a Critic model.

[0120] Advantage function Calculation:

[0121] Results monitoring:

[0122] For each output Calculate rewards ;

[0123] Intra-group standardization: ;

[0124] The advantage of all tokens (tasks) is set as follows: .

[0125] Process Supervision:

[0126] For each reasoning step Calculate rewards ;

[0127] Intra-group standardization: ;

[0128] The advantage of the token is the sum of subsequent rewards: .

[0129] KL divergence estimation:

[0130] GRPO uses an unbiased estimator to calculate the KL divergence:

[0131] .

[0132] Channel / filter importance calculation:

[0133] Channel Importance: Introducing Small Perturbation Factors Avoid values ​​of 0: ;

[0134] in, is the scaling factor for the BN layer (L1 regularization sparsity). The similarity of the channel's interest targets (cosine similarity, with values ​​ranging from 1 to 10). ), This represents the channel weight.

[0135] Filter importance: Introducing a rank scaling factor for global filter ranking. : ;

[0136] in, ( (mean rank of the convolutional layer) These are learnable parameters.

[0137] Overview of model pruning techniques:

[0138] Based on the different units of model pruning, model pruning methods can be divided into structured pruning and unstructured pruning. Structured pruning typically targets complete filters. Compared to the fourth layer, the filters in the third layer of the initial model still maintain fully connected mappings. However, when the complete filters of this layer are removed, all input channels connected to them are also pruned, resulting in a sparse network structure. This method generally includes the following pruning steps:

[0139] (1) Model training: First, train the original neural network to make it converge on the target dataset, and retain the trained parameters as the basis for subsequent pruning;

[0140] (2) Filter pruning: Filters are ranked by importance according to the preset evaluation criteria, and filters with lower importance are removed;

[0141] (3) Model reconstruction: After pruning, the model is reconstructed and fine-tuned to restore its original accuracy, so that some of the removed parameters are reconstructed to a certain extent. The second and third steps of iterative optimization can be repeated.

[0142] After structured pruning, the resulting lightweight model shows significant improvements in both inference speed and model size. However, due to the removal of filters, model accuracy usually suffers some loss, and redundant parameters often remain in the model that have not been completely removed.

[0143] Traditional model pruning primarily involves pre-training a selected model, but this cannot be used as the core step in pruning a network model; rather, it can only serve as a preparatory step. Therefore, traditional model pruning can be divided into three parts: selecting filters, selecting pruning methods, and fine-tuning the model.

[0144] However, traditional pruning techniques have many limitations. First, the setting of the pruning ratio heavily relies on expert experience. In practice, a thorough understanding of the model structure, combined with the number of filters, application scenario, and computing power of the deployed devices, is necessary to reasonably determine the pruning ratio. Improper pruning can easily lead to a significant decrease in model accuracy, usually requiring fine-tuning to restore performance. However, fine-tuning is not always effective in compensating for accuracy loss and often only reduces expected performance. Because pruning strategies are highly dependent on experience, the entire process is usually time-consuming and parameter tuning is difficult. Second, the current pruning process itself has efficiency and universality issues. Soft pruning methods require longer computation time than hard pruning, and during iteration, if some parameters are set to zero early on, they may not be able to be updated later due to the activation function, thus becoming permanently invalid. While hard pruning is computationally efficient, its adaptability across devices and scenarios is poor: often, before the target compression rate is reached, the variance between the remaining filters is already too small and the confidence difference is insufficient, leading to a sharp drop in accuracy after pruning, which is difficult to recover even with fine-tuning.

[0145] Automated pruning algorithm based on channel importance:

[0146] Soft pruning method:

[0147] Pruning channel selection:

[0148] From a mathematical perspective, the weights of a convolutional layer in a neural network can be represented as a four-dimensional tensor. Let the weights of the first convolutional layer be... The weight tensor of the convolutional layer is Its dimension can be represented as:

[0149] ;

[0150] in, Indicates the number of input channels. For the number of filters, and These represent the height and width of the filter, respectively.

[0151] In convolution operations, the input and output feature maps can also be represented as four-dimensional tensors. Taking a single sample as an example, let the first... The input feature map of the layer is The output feature map is Then the convolution operation can be expressed as:

[0152] ;

[0153] in, For input tensors; Batch size; Input the number of channels; The height of the input data; The width of the input data; To output a tensor; Number of output channels; This is the height of the output feature map; This is the width of the output feature map.

[0154] The core of channel pruning lies in identifying and removing redundant channels. To this end, we propose a channel importance evaluation metric that combines scaling factors with similarity information from the attention mechanism. Specifically, channel importance... Defined as:

[0155] ;

[0156] in, This is the scaling factor in the batch normalization layer. This refers to the similarity of the channel interest target detection obtained through the attention mechanism.

[0157] Introducing scaling factor For the batch normalization layer, L1 regularization can be used to promote channel sparsity. After training, according to... The magnitude of the value determines the importance of a channel; a smaller value indicates higher channel redundancy. The loss function introduces the concept of [missing information - likely a specific parameter or parameter]. Regular terms:

[0158] ;

[0159] Attention mechanisms are used to enhance the ability to identify important channels. We construct a calibration matrix using predefined calibrators (such as maximum activation feature maps of various image types). And calculate the input feature vector. and Cosine similarity between them:

[0160] ;

[0161] Finally, by combining the scaling factor and similarity information, an improved loss function is constructed:

[0162] ;

[0163] in, For the Sigmoid function, It is a small constant to avoid division by zero errors.

[0164] Reinforcement learning methods:

[0165] The entire method takes image data as input, introduces masks on each channel of the convolutional layer, and initializes the masks using a normal distribution. The attention mechanism module, built based on prior information, is used to identify regions in each channel that respond significantly to a specific target. We measure the correlation between channel features and pre-defined calibrators using cosine similarity, thereby generating a channel importance score, which is then embedded into the state features.

[0166] During the policy search process, the DDPG agent calculates reward signals based on the characteristics of the current environment to evaluate the effectiveness of the pruning strategy and stores these experiences in the replay cache. Through continuous iteration, the agent gradually optimizes the policy space and finally determines the number of channels to be pruned in each layer based on the preset global pruning rate. Specifically, by comprehensively evaluating the scaling factor and channel interest similarity, the system automatically selects an appropriate number of redundant channels for pruning. Finally, the fully connected layer is decomposed, and the optimized model is deployed to an embedded device.

[0167] The importance of information contained in the channels of each layer in the network varies. Using a uniform pruning ratio might inadvertently remove critical channels, leading to a decrease in model accuracy. Therefore, we set an independent pruning rate for each layer, calculated using the following formula:

[0168] ;

[0169] in, In the t-th layer, the first... Importance score of each channel This represents the total number of channels in this layer. This represents the set of important channels (usually the top 50% of channels are selected based on their importance). It is a tiny constant introduced to avoid the denominator being zero.

[0170] The construction of the state space needs to fully describe the structure and environmental information of the current layer. We use a state vector containing 11 features. To represent the state of each layer:

[0171] ;

[0172] The meanings of each symbol are as follows:

[0173] : The index of the current layer;

[0174] , The number of filters and the number of input channels;

[0175] , : Input the height and width of the data;

[0176] : Kernel size;

[0177] The number of floating-point operations in the current layer;

[0178] : The cumulative amount of computation has been reduced;

[0179] Total computational cost of the remaining layers.

[0180] In action space, we use continuous actions. This represents the pruning ratio at each layer to avoid the curse of dimensionality in the discrete action space, thereby improving the efficiency of policy search.

[0181] The pruning process is guided by reinforcement learning, and the agent adjusts its behavior based on the current state. Generate feature representations The system then outputs the pruning rate for that layer. Based on this pruning rate, the system removes redundant channels and updates the environmental state accordingly. The process is iterated until the last layer of the network. After each round of pruning, the reward value is calculated using the validation set accuracy and the reduction in computational cost.

[0182] ;

[0183] objective function Set to:

[0184] ;

[0185] in As a baseline reward, For the average reward, This is a discount factor. Noise is introduced into the actions during the exploration process. To enhance strategy diversity:

[0186] ;

[0187] The noise decays over time, gradually converging to the optimal strategy.

[0188] Core Algorithm:

[0189] Input: training data:

[0190] 1: Initialize: experience replay buffer B (capacity=1e5); GRPO policynetwork π_θ; value network V_φ

[0191] 2: for t = 1 to Episode (Episode=200) do

[0192] 3:Update model parameter w via SGD on X; η = η0; Update state S_t,feature vector φ(S_t)

[0193] 4:Add mask to channels in layer t (mask ~ N(0,1), clipped to [0,1])

[0194] 5:for i = 1 to G (G=number of convolution layers) do

[0195] 6:Compute weight norm ||W_j,k||2 via Eq.3.5; Compute similarity s viaEq.3.7

[0196] 7:Calculate channel importance I via Eq.3.5; Embed I into φ(S_t)

[0197] 8:Sample action A_t (pruning rate η_t) from π_θ(φ(S_t)) + N_s (N_s~N(0,0.1), exponential decay)

[0198] 9:Prune channels in layer i by η_t; Compute FLOPs and Acc afterpruning

[0199] 10:Calculate reward R_t via Eq.3.13; Store (S_t, A_t, R_t, S_{t+1})into B

[0200] 11:if |B|>batch_size (batch_size=64) then

[0201] 12:Sample mini-batch from B; Compute GAE advantage A_GAE via Eq.3.14

[0202] 13:Update V_φ by minimizing MSE: L_V = E[(V_φ(S_t) - (R_t + γV_φ(S_{t+1})))²]

[0203] 14:Update π_θ by policy gradient: ∇θJ(θ) = E[∇θlogπ_θ(A_t|S_t) ·A_GAE]

[0204] 15:end if

[0205] 16:end for

[0206] 17:Update η = η_t+1; S = S_t+1; φ(S) = φ(S_{t+1})

[0207] 18: if Reward > Reward_max then

[0208] 19:Save η of each layer; Save optimal model parameter w

[0209] 20:end if

[0210] 21: end for

[0211] Output: Compact model and its parameter w

[0212] GRPO Strategy Development (Core Steps):

[0213] Group sampling: For each pruning task q (such as channel filtering at a certain level), from the old strategy Sample G=32 outputs, each containing the candidate pruning rate η∈(0,1) for that layer;

[0214] Reward Calculation: Design the reward function with a dual objective of "accuracy-resource". Where FLOPsinit is the initial model computation cost, β=0.03 (KL regularization coefficient, adapted for pruning scenarios);

[0215] Strategy update: Single-round update is adopted. (This simplifies clip operations, with a batch size of 128 and a learning rate of 0.01.)

[0216] 1. Environment Setup

[0217] The system is Ubuntu 18.04, and the training environment is built using PyTorch. The hardware configuration is Intel Xeon Silver4210 @ 2.20GHz, NVIDIA GeForce RTX 3090. The software configuration is CUDA Toolkit v11.1, Anaconda3, and PyCharm 2020.

[0218] For hyperparameter settings during training, the batch size was 128, and the SGD algorithm was chosen as the optimization method. The initial learning rate was set to 0.01, the decay rate to 0.0005, and the momentum to 0.9. The total training epochs were set to 150. After pruning, we recorded the accuracy at each epoch, then saved the optimal weights and retrained to test its accuracy.

[0219] To verify the applicability of the proposed method in different image task scenarios, experiments were conducted using two representative public datasets—CIFAR-10 and ImageNet—for systematic validation. CIFAR-10 is suitable for preliminary performance validation of lightweight models, while ImageNet, as a mainstream benchmark for evaluating the performance of deep convolutional neural networks (CNNs), can further verify the effectiveness of the method in complex image scenarios.

[0220] In terms of model selection, the experiments adopted three mainstream CNN architectures: VGG, ResNet, and MobileNetV2. VGG represents a traditional dense convolutional model with a large number of parameters and computational cost. ResNet solves the gradient vanishing problem in deep networks with residual connections and is a typical representative of medium and large-scale models. MobileNetV2 is based on a depthwise separable convolutional design and is a lightweight model commonly used in edge devices. The three models cover CNN structures of different scales and design ideas, which can comprehensively verify the versatility of the method.

[0221] In terms of evaluation dimensions, the experiment used Top-1 accuracy, floating-point operations (FLOPs), and parameter count as core indicators, corresponding to the quantitative analysis of model performance, computational overhead, and storage requirements, respectively: Top-1 accuracy is used to measure the accuracy of the model's classification results, and the higher the value, the better the classification effect; FLOPs reflect the computational cost of a single forward inference of the model, and a decrease in this value means that the model can achieve a more significant speed improvement in actual operation, and the greater the reduction, the more obvious the acceleration effect; the number of parameters directly reflects the amount of storage space occupied by the neural network, and the parameter reduction rate is calculated by the difference between the number of parameters and the original model, and presents the parameter compression effect of the model in a percentage form.

[0222] Server-side experiments and analysis:

[0223] (1) Experimental Environment

[0224] Hardware: Intel Xeon Silver 4210 @2.20GHz, NVIDIA GeForce RTX 3090, 8GB LPDDR4;

[0225] Software: Ubuntu 18.04, PyTorch 1.9.0, CUDA Toolkit v11.1, Anaconda3, PyCharm2020;

[0226] Hyperparameters: Batch size 128, SGD optimizer (initial learning rate 0.01, decay rate 0.0005, momentum=0.9), total training epochs=200; GRPO specific hyperparameters: γ=0.99, λ=0.95, policy network learning rate 3e-4, value network learning rate 1e-3, experience replay pool capacity 1e5, batch size 64.

[0227] (2) Dataset and Network Model

[0228] Dataset:

[0229] CIFAR-10: 10 categories of images (aircraft, vehicles, etc.), 6000 images per category, training / test set ratio 5:1;

[0230] ImageNet: 1000 classes of images, with nearly 1000 images per class in the training set and 50,000 images in the test set, with a resolution of 224×224;

[0231] Network models: VGG-19, ResNet-56, ResNet-50, MobileNetV2;

[0232] Evaluation metrics: Top-1 accuracy (classification accuracy), FLOPs (computational cost, remaining percentage = compressed FLOPs / original FLOPs × 100%), and number of parameters (remaining percentage = compressed number of parameters / original number of parameters × 100%).

[0233] Experimental results on the CIFAR-10 dataset:

[0234] The pruning rate was set to 0.5 (after multiple adjustments, this pruning rate achieved the optimal balance between accuracy loss and compression rate), and the experimental results are shown in the table. Compared to the original DDP-based method (Ours), the GRPO-based method (Ours (GRPO)) improved accuracy by 0.13%-0.21%, with a slight reduction in FLOPs and parameters, demonstrating the optimization advantages of the GRPO strategy. The experimental results for VGG-19 and ResNet-56 on the CIFAR-10 dataset are shown in Table 1.

[0235] Table 1 shows the experimental results of VGG-19 and ResNet-56 on the CIFAR-10 dataset.

[0236]

[0237] Results analysis:

[0238] VGG-19: The GRPO method has an accuracy only 0.01% lower than the baseline, with FLOPs remaining at 47.82% (0.24% lower than the original Ours) and 2.01M parameters (0.02M lower than the original Ours). Because GRPO more accurately identifies redundant channels, it reduces the accidental deletion of critical channels.

[0239] ResNet-56: The GRPO method has 0.21% higher accuracy than the original Ours method, with FLOPs remaining at 49.50% (0.52% lower), and the number of parameters is the same as the original Ours method, demonstrating the improved adaptability of GRPO to residual structures.

[0240] Experimental results on the ImageNet dataset:

[0241] The ResNet-50 pruning rate was set to 0.4 (to reduce redundancy in large datasets; a high pruning rate can easily lead to accuracy loss), and the results are shown in Table 2. The GRPO method achieves 0.27% higher accuracy than the original Ours method, with further reductions in FLOPs and parameter count, validating its effectiveness on large models.

[0242] Table 2 shows the experimental results of ResNet-50 on the ImageNet dataset.

[0243]

[0244] The pruning rate for MobileNetV2 was set to 0.4, and the results are shown in Table 3. The GRPO method has 0.28% higher accuracy than the original Ours method, with slightly lower FLOPs and parameter count, and is well-suited to the inverted residual structure of MobileNetV2.

[0245] Table 3 shows the experimental results of MobileNetV2 on the ImageNet dataset (GRPO version).

[0246]

[0247] Edge device deployment:

[0248] The device used is the NVIDIA Jetson Xavier NX edge device. Operating system: Ubuntu 18.04; Hardware: 6-core 64-bit ARM architecture CPU, 384-core integrated GPU, 8GB LPDDR4; Acceleration tools: cuDNN, TensorRT SDK (inference acceleration).

[0249] Test data: 50 images of 10 classes (cars, flowers, etc.) from ImageNet were selected, and the experimental results were the average of 5 measurements; Test models: VGG-19, ResNet-50, MobileNetV2 (a lightweight model after GRPO pruning).

[0250] The evaluation metrics have been expanded to include "Inference Speed ​​(ms / frame)" and "Frame Rate (FPS)," with results shown in Table 4. The GRPO-pruned model is 5%-8% faster inference on edge devices than the original Ours model, with a 3-5 FPS improvement. Due to the further reduction in model parameters, it is more efficient for TensorRT acceleration.

[0251] Table 4 shows the experimental results of edge device deployment.

[0252]

[0253] Results analysis:

[0254] VGG-19: The GRPO model inference speed is 2.25ms faster than the original Ours, with an FPS improvement of 1, and a FLOPs reduction of 0.17G.

[0255] ResNet-50: The GRPO model has a 2.35ms faster inference speed, a 2 FPS improvement, and a better pruning strategy to adapt to residual structures;

[0256] MobileNetV2: The GRPO model has a 1.08ms faster inference speed and a 2 FPS (Frames Per Second) improvement, meeting the real-time detection requirements of edge devices (FPS>60).

[0257] To address the problems of traditional pruning methods, such as "empirical dependence, significant accuracy loss, and weak versatility," an automated pruning algorithm based on channel importance and GRPO is proposed. The core conclusions are as follows:

[0258] Algorithm innovation: Channel importance index is constructed by fusing the scaling factor of BN layer with attention similarity, GRPO is used instead of DDPG, and GAE is used to reduce policy update variance, thereby improving sample efficiency and pruning strategy stability;

[0259] Experimental results show that on the CIFAR-10 and ImageNet datasets, the GRPO method improves accuracy by 0.13%-0.28%, reduces FLOPs by 0.24%-0.52%, and reduces the number of parameters by 0.5%-1.0% compared to the original DDPG method.

[0260] Edge deployment: The GRPO pruned model achieves a 5%-8% improvement in inference speed and a 3-5 FPS improvement on NVIDIA Jetson Xavier NX, meeting the needs of real-time applications;

[0261] Limitations and directions for improvement: GRPO still requires manual intervention in hyperparameter tuning (such as γ, λ). In the future, it can be combined with AutoML to further automate hyperparameter selection, while exploring the integration and compression of quantization and knowledge distillation.

[0262] Although the present invention has been described in detail with reference to the accompanying drawings and preferred embodiments, the present invention is not limited thereto. Various equivalent modifications or substitutions can be made to the embodiments of the present invention by those skilled in the art without departing from the spirit and essence of the invention, and such modifications or substitutions should all be within the scope of the present invention. Any variations or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in the present invention should also be covered within the protection scope of the present invention.

Claims

1. An automated neural network pruning method based on GRPO reinforcement learning, characterized in that, include: S1. Introduce a dynamic scaling factor into the batch normalization layer of the neural network to be pruned, and combine it with an attention mechanism to calculate the importance score of each convolutional layer channel of the neural network to be pruned. ; S2, Importance score based on step S1 Construct a multidimensional state vector containing layer structure features. ; S3, convert the multidimensional state vector The policy network input to the GRPO reinforcement learning agent By this policy network Generate pruning actions based on the current network layer state information. At the same time, based on pruning actions Set the pruning rate for this layer. ; S4. Based on the GRPO strategy output of the network in S3, the pruning rate of each layer. Combined with the channel importance score calculated in S1 Sort the redundant channels from low to high importance and remove the corresponding proportion; after completing the pruning of the current layer, update the network structure state and advance the processing flow to the next layer until all layers to be pruned have been traversed. S5. Adjust the parameters of the lightweight neural network that has been pruned in S4, and use the validation set to calculate the accuracy Acc of the lightweight neural network after pruning and the change in floating-point operation volume FLOPs before and after pruning; evaluate the pruning effect based on the accuracy Acc and the floating-point operation volume FLOPs, and feed the pruning effect back to the GRPO policy agent in S3 for the next round of policy iteration optimization. The method also includes converting the pruned lightweight neural network into TensorRT format and deploying it to edge devices to perform inference tasks. Specifically, the method includes: S61. Perform format preprocessing on the pruned lightweight neural network after optimization and verification in step S5. First, export the neural network from the original training framework format to the ONNX general format. During the export process, fix the neural network input dimension, operator type and weight parameters. S62. Start the TensorRT SDK tool on the NVIDIA Jetson Xavier NX edge device, load the ONNX format neural network exported in step S61, parse the neural network computation graph through the TensorRT model parser, automatically complete operator fusion, accuracy calibration and memory optimization, and generate TensorRT engine files adapted to the edge device hardware architecture. S63. Configure the operating environment of the edge device, install the CUDA Toolkit, cuDNN library and Python / C++ inference dependency package that match the TensorRT engine, and load the TensorRT engine file generated in step S62 into the memory of the edge device.

2. The automated neural network pruning method based on GRPO reinforcement learning according to claim 1, characterized in that, Step S1 includes: S11. Introduce a dynamic scaling factor in the batch normalization layer of the neural network to be pruned. Dynamic scaling factor is adjusted using L1 regularization. Constraints are applied to achieve sparsity of channel weights; simultaneously, a calibration matrix is ​​constructed based on the maximum activation feature maps of various types of samples in the training dataset. ; S12. Input the preprocessed training dataset samples into the neural network to be pruned, activate each convolutional layer and batch normalization layer through forward propagation, and obtain the feature vectors of each channel. Calculate the eigenvector With calibration matrix eigenvectors cosine similarity The formula is: ； in, The element of the i-th dimension of the feature map vector; The element in the i-th dimension of the calibration matrix; The dimension of the feature vector; S13. Extract the dynamic scaling factor constrained by L1 regularization from the batch normalization layer. Attention weights output by the attention mechanism The cosine similarity obtained in step S12 and convolutional layer channel weights The sum of L2 norms is used to calculate the channel importance score. The formula is: ； in, The disturbance factor has a value range of 1. ; Let be the L2 norm of the weights of the j-th filter and the k-th input channel; n is the number of filters, i.e., the number of output channels; and C is the number of input channels.

3. The automated neural network pruning method based on GRPO reinforcement learning according to claim 2, characterized in that, Step S11 also includes: Based on cosine similarity The loss function for the batch normalization layer is defined as follows: ； in, It is an L1 regularization function; The regularization coefficient is used. The original loss function; For training samples and labels; For the convolutional layer weight tensor; For the Sigmoid function; It is a tiny constant; This is the set of dynamic scaling factors γ introduced in all convolutional layers to be pruned.

4. The automated neural network pruning method based on GRPO reinforcement learning according to claim 1, characterized in that, Step S2 includes: S21. Extract the basic structural features of each convolutional layer of the neural network to be pruned, including the current layer index. Number of filters Number of input channels (C), Input data height Input data width and kernel size This forms a subset of basic features; S22. Calculate the dynamic performance characteristics of each convolutional layer, including the floating-point operation volume of the current layer. The cumulative reduction in floating-point operations Total floating-point operations of the remaining layers and the pruning rate of the upper layer This forms a subset of performance characteristics; S23. Calculate the channel importance score obtained in step S1. The feature set is incorporated and integrated with the basic feature subset from step S21 and the performance feature subset from step S22 to construct an 11-dimensional state vector. The formula is: 。 5. The automated neural network pruning method based on GRPO reinforcement learning according to claim 1, characterized in that, Step S3 includes: S31. Based on the multidimensional state vector constructed in step S2, determine the boundary conditions of the current pruning task, including the range of the layers to be pruned and the accuracy loss threshold; call the parallel sampling module of the GRPO reinforcement learning agent to sample G groups of candidate pruning rate action sequences in parallel for each pruning task, each group of action sequences containing the pruning rate corresponding to each layer to be pruned. The action sequence of candidate pruning rates in group G is denoted as Each action sequence independently generates pruning strategy candidate schemes; S32. Apply each candidate pruning rate action sequence to the neural network to be pruned, obtain the neural network performance after each action is executed through forward inference, and construct a reward function based on the precision-resource dual objective to calculate the reward value of each action. Then calculate the reward value for all groups. Intra-group standardization was performed to obtain the relative advantage of each group of actions. Eliminate single-group reward estimation bias; neural network performance includes validation set accuracy and floating-point computational complexity. S33. Update the current policy network of the GRPO reinforcement learning agent. ; S34. Utilizing relative advantages Construct the objective function for updating the GRPO policy, and update the GRPO reinforcement learning agent policy network using gradient ascent. Parameters; S35. After the update is completed, select the relative advantage from the candidate pruning rate action sequences of group G. The largest set is determined as the optimal pruning action sequence for the current pruning task, and the pruning rate of each layer in this sequence is the final pruning action generated by the policy network.

6. The automated neural network pruning method based on GRPO reinforcement learning according to claim 5, characterized in that, The reward function is: ； in, To reduce the accuracy of the neural network after pruning; This represents the floating-point computation cost of the neural network before pruning. This represents the floating-point computation cost of the pruned neural network.

7. The automated neural network pruning method based on GRPO reinforcement learning according to claim 5, characterized in that, Relative advantage The calculation formula is: ； in, This is the group reward mean, that is, the arithmetic mean of the rewards for group G; The standard deviation of the group reward; This is the reward value for a single group.

8. The automated neural network pruning method based on GRPO reinforcement learning according to claim 5, characterized in that, The objective function for updating the GRPO policy is: ； in, The objective function for optimizing the GRPO strategy; For parameter vectors; For the pruning task; This represents the probability distribution of the pruning task; Number of samples per group; For the first Action sequences obtained from group sampling; For the first The specific action generated in the action sequence at step t; For the first All historical actions in the sequence prior to step t; For the new strategy network to be optimized; For old policy networks; The relative advantage of a group is the relative advantage function of the i-th action sequence at step t. These are the KL regularization coefficients; This is the clipping function.

9. The automated neural network pruning method based on GRPO reinforcement learning according to claim 4, characterized in that, Step S4 includes: S41. For the t-th layer of the neural network to be pruned, extract the pruning rate of that layer output in step S3. The importance scores of all channels calculated in step S1 are sorted from low to high to obtain the sorted channel index sequence. S42. Based on the pruning rate Calculate the number of channels k to be pruned in the current layer, select the first k channels as redundant channels from the sorting results in step S41, and set their output features to zero by setting a mask vector during the forward propagation of the network to achieve channel pruning; S43. After completing the pruning of the t-th layer, recalculate the number of output channels, floating-point operation volume and feature map size of the layer, and update the structural features and performance features related to the layer in the multidimensional state vector constructed in step S2. S44. Proceed to the next layer and repeat steps S41 to S43 until all convolutional layers of the neural network to be pruned are traversed to obtain the lightweight network structure after preliminary pruning.

10. The automated neural network pruning method based on GRPO reinforcement learning according to claim 6, characterized in that, Step S5 includes: S51. The parameters of the lightweight neural network obtained in step S4 are adjusted using a pre-stored fine-tuning strategy. Backpropagation is performed using a batch of samples from the training dataset. The learning rate is fixed at 1 / 10 of the initial training phase. Iterate for 5 to 10 epochs to recover the accuracy loss caused by pruning. S52. Input the preprocessed validation set samples into the fine-tuned lightweight neural network, calculate the classification accuracy (Acc) of the model on the validation set through forward inference, and simultaneously calculate the floating-point operation cost of the model before and after pruning. and And calculate the reward value corresponding to the current pruning strategy based on the reward function. ; S53. Transfer the reward value obtained in step S52. The multidimensional state vector from step S2 and the pruning action sequence from step S3 are associated and stored as empirical data input to the GRPO reinforcement learning agent's experience replay pool, which is used to drive the policy network. The next round of parameter updates will enable iterative optimization of the pruning strategy.

Images