Target detection model pruning method
By performing dynamic sparse processing and network pruning in the object detection model, combining strip pruning and mixed accuracy training, the problem of low pruning efficiency and insufficient accuracy of the object detection model in the existing technology is solved, and an efficient model is achieved Compression and performance improvements.
Patent Information
- Application Number
- CN202411301236.9
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-09-18
- Publication Date
- 2025-01-10
AI Technical Summary
The existing pruning method of object detection model is complicated, resulting in low pruning efficiency, insufficient model accuracy after pruning, and inability to balance model compression ratio and accuracy loss, limiting its application in edge devices and cloud computing environments.
By constructing and training the object detection model, dynamic sparse processing and network pruning, combining strip pruning method, pruning the convolution kernel skeleton matrix according to the preset proportion, and retraining using a mixed precision training method to ensure that the model is Achieve the expected accuracy before pruning, simplify the model structure and improve the computational efficiency.
It improves model performance, reduces computational cost and storage requirements, improves pruning processing efficiency, balances model compression ratio and accuracy loss, is more efficient than traditional methods, and ensures the accuracy and performance of the model after pruning.
Smart Images
Figure CN119294462A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of model pruning, and in particular to a target detection model pruning method. Background Art
[0002] Object detection is a core problem in the field of computer vision. Its task is to find all objects of interest in an image and determine their categories and locations. Object detection has always been a challenging task due to the influence of factors such as the appearance, shape, posture of the object, as well as lighting and occlusion during the imaging process. The object detection model extracts features from image or video data through deep neural networks, and applies classifiers and bounding box regressors on the extracted features to identify and locate objects.
[0003] Model pruning is a technology for optimizing neural network models. It reduces unnecessary redundant parameters and computational complexity in the model to improve the model's reasoning speed, reduce memory usage, and maintain the model's accuracy to a certain extent. Its main purpose is to optimize the neural network model to make it more efficient and lightweight. By removing parameters and computational complexity that contribute little or are redundant in the model, the model size can be significantly reduced, the model's reasoning speed and efficiency can be improved, and at the same time, the model's accuracy can be kept at a minimum or minimized.
[0004] Most of the existing target detection model pruning methods have complicated processes, resulting in low pruning efficiency of the target detection model, and the pruned model is also insufficient in accuracy. It is impossible to balance the model compression ratio and accuracy loss, thereby reducing the model performance of the target detection model and being unable to be effectively deployed and applied in various edge devices and cloud computing environments. In addition, therefore, the present invention proposes a target detection model pruning method to solve the problems existing in the prior art. Summary of the invention
[0005] In view of the above problems, the purpose of the present invention is to propose a target detection model pruning method to solve the problem that most of the existing target detection model pruning methods have complicated processes, resulting in low pruning efficiency of the target detection model, and the pruned model is also insufficient in accuracy, and cannot balance the model compression ratio and accuracy loss.
[0006] In order to achieve the purpose of the present invention, the present invention is implemented by the following technical scheme: a target detection model pruning method comprises the following steps:
[0007] Step 1: First build a target detection model, then train the built target detection model, and perform dynamic sparse processing on the target detection model during the training process to make the target detection model reach the expected accuracy;
[0008] Step 2: Prune the target detection model trained in step 1 based on the batch normalization layer scaling factor, and then use the strip pruning method to prune the target detection model to obtain the convolution kernel skeleton matrix of the target detection model;
[0009] Step 3: Sort the elements in the convolution kernel skeleton matrix obtained in step 2, set the pruning threshold, and then prune the strips in the corresponding convolution kernel skeleton matrix that are smaller than the pruning threshold according to the preset pruning ratio to obtain a structured strip pruning model;
[0010] Step 4: Use the mixed precision training method to retrain the structured strip pruning model obtained in step 3 to obtain a half-precision representation model, thus completing the model pruning of the target detection model.
[0011] A further improvement is that in step one, the specific steps for constructing and training the target detection model are: first, select a data set related to the target detection task, then perform image scaling, data enhancement, and annotation conversion preprocessing on the data in the data set in turn, then select a model architecture that is suitable for the target detection task, then configure the model parameters according to the selected model architecture, and finally use the preprocessed data to train the configured model until the model reaches the expected accuracy.
[0012] A further improvement is that in step one, the specific steps of dynamically sparsely processing the target detection model are: first, the weights of the target detection model are initialized using the Xavier initialization method, and then the expected sparsity level of the target detection model is determined, and then the sparsity of the model is regularly evaluated during the model training process, and the model weights are adjusted according to the expected sparsity level to achieve dynamic sparse processing of the target detection model.
[0013] A further improvement is that in the step 2, the specific steps of pruning the network of the target detection model based on the batch normalization layer scaling factor are: first analyze the scaling factor of each batch normalization layer in the target detection model, then determine a pruning threshold according to the distribution of the scaling factor, and then remove the batch normalization layer output channels whose scaling factors are less than the pruning threshold, and achieve network pruning of the model by modifying the number of output channels of the batch normalization layer.
[0014] A further improvement is that in step 2, the specific steps of using the strip pruning method to prune the target detection model are: first determine the pruning granularity of the target detection model, then calculate the skeleton matrix of the convolution kernel for each convolution layer of the target detection model, and then construct the convolution kernel skeleton matrix based on the calculation results.
[0015] A further improvement is that in step three, the elements in the convolution kernel skeleton matrix are sorted by importance by calculating the importance index of each element, and the pruning ratio is the ratio of the expected number of strips to be pruned to the total number of strips.
[0016] A further improvement is that in step three, for each convolution kernel in the convolution kernel skeleton matrix, the pruning threshold is dynamically adjusted to ensure that each convolution kernel retains the same number of strips.
[0017] A further improvement is that in step 4, when the structured strip pruning model is retrained, a full-precision network weight is first stored in the optimizer. During the training process, the structured strip pruning model uses half-precision weights to calculate the loss and the gradient of each network weight, and then the gradient is converted to full precision and copied to the optimizer. The optimizer then uses the full-precision gradient value to perform gradient descent on the parameters, and finally the updated full-precision weight is converted to half precision, replacing the network weights in the structured strip pruning model to obtain a model represented by half precision.
[0018] The beneficial effects of the present invention are as follows: the present invention trains and dynamically performs sparsification processing on the target detection model before pruning the target detection model, so that the target detection model can achieve the expected accuracy before pruning, thereby improving the model performance, reducing the subsequent computing cost and storage requirements, and improving the efficiency of subsequent pruning processing. The target detection model is pruned and trained by combining network pruning and strip pruning, and the strips smaller than the pruning threshold in the corresponding convolution kernel skeleton matrix are pruned according to a preset pruning ratio, thereby realizing the structuring of the convolution kernel, which can simplify the structure of the model and further improve the computing efficiency. At the same time, the structured strip pruning model is retrained through a mixed precision training method to complete the final model pruning process, which can balance the model compression ratio and precision loss. Compared with traditional pruning methods, it is more efficient and can ensure the accuracy and performance of the model after pruning. BRIEF DESCRIPTION OF THE DRAWINGS
[0019] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.
[0020] Figure 1 It is a flow chart of the target detection model pruning method of the present invention. DETAILED DESCRIPTION
[0021] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.
[0022] See also Figure 1 , this embodiment provides a target detection model pruning method, comprising the following steps:
[0023] Step 1: Build and train the target detection model and perform dynamic sparsification
[0024] First, build a target detection model and train the built target detection model. The specific steps are as follows: first, select a data set related to the target detection task. The data set contains a large number of annotated images. These annotations indicate the location and category of the target in the image. Then, perform image scaling, data enhancement, and annotation conversion preprocessing on the data in the data set in turn. Adjust the image to a uniform size through image scaling, enhance the image data through rotation, flipping and cropping to increase data diversity, and convert the annotations in the image data into a format suitable for model input through annotation conversion. Then, select a model architecture that is suitable for the target detection task. Then, configure the model parameters according to the selected model architecture, including input image size, anchor box setting, loss function, and optimizer. Finally, use the preprocessed data to train the configured model until the model reaches the expected accuracy, improves the model performance, and facilitates subsequent pruning.
[0025] In this embodiment, the target detection model is dynamically sparsely processed during the training process of the target detection model. The specific steps are: first, the weight of the target detection model is initialized by the Xavier initialization method, and then the expected sparsity level of the target detection model is determined. Then, the sparsity of the model is regularly evaluated during the model training process, and the model weight is adjusted according to the expected sparsity level, so as to realize dynamic sparse processing of the target detection model, so as to reduce the number of model parameters, improve the sparsity of the model, and thus reduce the computing cost and storage requirements;
[0026] Step 2: Prune the target detection model by combining network pruning and strip pruning. First, analyze the scaling factor of each batch normalization layer in the target detection model obtained in step 1, and then determine a pruning threshold according to the distribution of the scaling factor. Then, remove the batch normalization layer output channels whose scaling factor is less than the pruning threshold. The network pruning of the model is achieved by modifying the number of output channels of the batch normalization layer to further reduce the model complexity. Then, use the strip pruning method to prune the target detection model. The specific steps are as follows: first determine the pruning granularity of the target detection model, that is, how many continuous weights each strip contains, and then calculate the skeleton matrix of the convolution kernel for each convolution layer of the target detection model. Then, construct the convolution kernel skeleton matrix according to the calculation results to obtain the convolution kernel skeleton matrix of the target detection model.
[0027] Step 3: Structure the convolution kernel and obtain a structured strip pruning model
[0028] First, the importance index of each element in the convolution kernel skeleton matrix is calculated, and then the elements in the convolution kernel skeleton matrix are sorted by importance, and the pruning threshold is set. Then, the strips in the corresponding convolution kernel skeleton matrix that are smaller than the pruning threshold are pruned according to the preset pruning ratio, where the pruning ratio is the ratio of the expected number of pruned strips to the total number of strips, so that the convolution kernel is structured and a structured strip pruning model is obtained, which aims to maintain the structural consistency of the convolution kernel and ensure that each convolution kernel retains the same number of strips after pruning, thereby simplifying the structure of the model and possibly improving the computational efficiency.
[0029] In this embodiment, for each convolution kernel, the pruning threshold is dynamically adjusted to ensure that each convolution kernel retains the same number of stripes. This process is achieved by calculating the ranking of the weights of each convolution kernel and determining the pruning threshold according to the target number of stripes.
[0030] Step 4: Use mixed precision training method to retrain the model to complete pruning. Use mixed precision training method to retrain the structured strip pruning model. During training, first store a full-precision network weight in the optimizer. During training, the structured strip pruning model uses half-precision weights to calculate the loss and the gradient of each network weight. Then the gradient is converted to full precision and copied to the optimizer. Then the optimizer uses the full-precision gradient value to perform gradient descent on the parameters. Finally, the updated full-precision weight is converted to half precision, and the network weight in the structured strip pruning model is replaced to obtain a model represented by half precision. The model pruning of the target detection model is completed.
[0031] In order to avoid the gradient being ignored, the mixed precision training method in this embodiment also adopts loss scaling technology, which amplifies the half-precision loss value to prevent it from being ignored as 0 due to its small size. Before the full-precision gradient descends, the gradient value will be reduced by the same proportion, ensuring that the gradient value remains unchanged, reducing the difficulty of training, and improving the robustness of the model.
[0032] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principle of the present invention should be included in the protection scope of the present invention.
Claims
1. A target detection model pruning method, characterized in that: The following steps are involved: Step 1: First build a target detection model, then train the built target detection model, and perform dynamic sparse processing on the target detection model during the training process to make the target detection model reach the expected accuracy; Step 2: Prune the target detection model trained in step 1 based on the batch normalization layer scaling factor, and then use the strip pruning method to prune the target detection model to obtain the convolution kernel skeleton matrix of the target detection model; Step 3: Sort the elements in the convolution kernel skeleton matrix obtained in step 2, set the pruning threshold, and then prune the strips in the corresponding convolution kernel skeleton matrix that are smaller than the pruning threshold according to the preset pruning ratio to obtain a structured strip pruning model; Step 4: Use the mixed precision training method to retrain the structured strip pruning model obtained in step 3 to obtain a half-precision representation model, thus completing the model pruning of the target detection model.
2. The target detection model pruning method according to claim 1, characterized in that: In the step 1, the specific steps of building and training the target detection model are: first select a data set related to the target detection task, then perform image scaling, data enhancement and annotation conversion preprocessing on the data in the data set in turn, then select a model architecture that is adapted to the target detection task, then configure the model parameters according to the selected model architecture, and finally use the preprocessed data to train the configured model until the model reaches the expected accuracy.
3. The target detection model pruning method according to claim 1, characterized in that: In the step 1, the specific steps of dynamically sparsely processing the target detection model are: first, the weight of the target detection model is initialized by the Xavier initialization method, and then the expected sparsity level of the target detection model is determined, and then the sparsity of the model is regularly evaluated during the model training process, and the model weight is adjusted according to the expected sparsity level to achieve dynamic sparse processing of the target detection model.
4. The target detection model pruning method according to claim 1, characterized in that: In the step 2, the specific steps of performing network pruning on the target detection model based on the batch normalization layer scaling factor are: first analyzing the scaling factor of each batch normalization layer in the target detection model, then determining a pruning threshold according to the distribution of the scaling factor, and then removing the batch normalization layer output channels whose scaling factors are less than the pruning threshold, and achieving network pruning of the model by modifying the number of output channels of the batch normalization layer.
5. The target detection model pruning method according to claim 1, characterized in that: In the step 2, the specific steps of using the strip pruning method to perform pruning training on the target detection model are: first determine the pruning granularity of the target detection model, then calculate the skeleton matrix of the convolution kernel for each convolution layer of the target detection model, and then construct the convolution kernel skeleton matrix based on the calculation results.
6. The target detection model pruning method according to claim 1, characterized in that: In the step three, the elements in the convolution kernel skeleton matrix are sorted by importance by calculating the importance index of each element, and the pruning ratio is the ratio of the number of strips expected to be pruned to the total number of strips.
7. The target detection model pruning method according to claim 1, characterized in that: In the step three, for each convolution kernel in the convolution kernel skeleton matrix, the pruning threshold is dynamically adjusted to ensure that each convolution kernel retains the same number of strips.
8. The target detection model pruning method according to claim 1, characterized in that: In the step 4, when the structured strip pruning model is retrained, a full-precision network weight is first stored in the optimizer. During the training process, the structured strip pruning model uses half-precision weights to calculate the loss and the gradient of each network weight, and then the gradient is converted to full precision and copied to the optimizer. The optimizer then uses the full-precision gradient value to perform gradient descent on the parameters. Finally, the updated full-precision weight is converted to half precision, replacing the network weights in the structured strip pruning model to obtain a model represented by half precision.