General intelligent model compression and fine adjustment method
By pruning, quantizing and low-rank decomposition of deep learning models, the problem of insufficient resources on the embedded platform is solved, and the efficient operation of the model and controllable precision compression and fine-tuning of the model in an embedded environment are achieved.
Patent Information
- Application Number
- CN202510530957.5
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-04-25
- Publication Date
- 2025-08-01
AI Technical Summary
When running deep learning models on embedded platforms, we face the problems of high computational complexity and huge parameters, resulting in insufficient resources and inability to deploy efficiently and run in real time.
The general intelligent model compression and fine-tuning method is adopted to simplify the network structure and weight representation of the model through technologies such as pruning, quantization, low-rank decomposition and knowledge distillation, and reduce the computational complexity and parameter scale.
Under the premise that the accuracy loss is controllable, the calculation complexity and parameter scale of the model are reduced, so that it adapts to the resource limitations of the embedded environment and achieves efficient operation.
Smart Images
Figure CN120409565A_ABST
Abstract
Description
Technical Field
[0001] The present invention mainly relates to the technical field of software design, and particularly to a general intelligent model compression and fine-tuning method. Background Art
[0002] The performance optimization of the radar in-service system faces severe challenges in hardware resources, which are mainly reflected in the following aspects: First, limited by key indicators such as the power consumption budget, physical size, and memory capacity of the embedded platform, the system storage capacity and computing resources are often relatively limited. However, modern artificial intelligence models, especially algorithm models based on deep learning, usually contain millions or even hundreds of millions of parameter scales, posing extremely high requirements in terms of model storage space occupancy and real-time computing needs.
[0003] This contradiction is particularly prominent in intelligent application scenarios such as radar target recognition. With the in-depth application of deep learning technology in the fields of radar signal processing and target recognition, the system faces two key challenges: First, in the face of the recognition requirements of new targets, the traditional model has a long upgrade and iteration cycle and low efficiency; Second, due to the large number of parameters and high computational complexity of large-scale deep neural network models, it is difficult to be efficiently deployed and run in real time on resource-constrained embedded platforms.
[0004] To solve these problems, it is urgent to develop an intelligent model compression tool for the embedded environment, which supports automatic compression of the model through operations such as pruning and quantization, realizes online upgrade of system functions and capacity evolution, and improves the system's adaptability to unknown targets. Summary of the Invention
[0005] The purpose of the present invention is to provide a general intelligent model compression and fine-tuning method, which simplifies the network structure or weight representation method of the model, and reduces the computational complexity and parameter scale of the model on the premise of controllable accuracy loss, so as to solve the problem that deep learning models cannot run due to insufficient resources in the embedded environment.
[0006] To achieve the above object, the present invention provides a general intelligent model compression and fine-tuning method, including the following steps: Step 1: Initialization settings for model compression: Load model weights, training data, and training code, initialize the model, data loader, and model trainer required for automatic model compression, and verify whether the above initializations are successful; Step 2: Automatic model compression: Automatically compress the model by using the inferable model, data loader, and model trainer obtained by the initialization setting module: S1: Read the compression requirements, obtain the model, and create a pruning tool; S2: Use a pruning tool to perform forward inference and backpropagation on the model. According to the backpropagation path, create a model structure dependency graph, and group the model parameters based on the dependency graph. S3: Use the pruning tool to evaluate the parameter importance of the model according to the pruning method specified by the compression requirement. S4: Sort by importance within the same dependency group and use the pruning tool to prune the corresponding low-importance parameters according to the pruning rate of the compression requirement to obtain the compressed model. S5: Feed the compressed model into the automatic fine-tuning module. Step 3: Fine-tune the compressed model and automatically select the optimal hyperparameters to obtain the optimal result.
[0007] Furthermore, the model compression initialization settings in step 1 include the following steps: T1: Load the model parameters and perform model verification. T2: Load the training data and perform data verification. T3: Load the training code, create a model trainer, and perform training verification.
[0008] Furthermore, the model compression initialization settings in step 1 include the following methods: Model creation method: Read the parameters according to the path, create a model based on the model parameters, and load the parameters. Data loader creation method: Load the data according to the data path, create a data loader, and provide training data at any time. Model trainer creation method: Read the training code according to the training code path and create a model trainer to train the model. Loading verification method: Use the inferable model and the data loader as inputs, call the model trainer to train for 10 iterations, and return a report on whether the loading is successful.
[0009] Furthermore, the automatic model compression in step 2 includes the following methods: Compression parameter receiving and parsing method: Receive and parse the pruning rate, pruning method, and ignored area included in the compression parameters, and create a pruning tool. Structure dependency graph creation method: Establish a structure dependency graph according to the model backpropagation path and divide it into dependency groups. Parameter importance estimation method: Evaluate the importance of each parameter according to the selected pruning method. Redundant parameter pruning method: Sort the parameters by importance within the dependency group, prune the low-importance parameters, and save the pruned model.
[0010] Furthermore, fine-tuning the compressed model in step 3 includes the following steps: T4: Create a search space and train the compressed model for one epoch. T5: Traverse the search space to evaluate all hyperparameter combinations and select the optimal hyperparameter combination.
[0011] Further, the automated model fine-tuning in step 3 includes the following methods: Automated hyperparameter search method: Create a search space, traverse the search space to evaluate all hyperparameter combinations, and select the optimal hyperparameter combination; Model trainer method: Fine-tune and train the model according to the input training hyperparameters and save the finally fine-tuned model; Model evaluation method: Calculate the number of parameters of the compressed model and evaluate the model accuracy and return it to the user interface.
[0012] Beneficial effects: The present invention provides a general intelligent model compression and fine-tuning method, which can provide designers with an intelligent model compression tool based on an embedded environment. This tool can simplify the network structure or weight representation of the model, and reduce the computational complexity and parameter scale of the model on the premise of controllable accuracy loss, so as to solve the problem that deep learning models cannot run due to insufficient resources in the embedded environment. Brief Description of the Drawings
[0013] Figure 1 is a sequence diagram of the model automated compression and fine-tuning components involved in the embodiments of the present invention; Figure 2 is a processing sub-diagram of the model automated compression and fine-tuning components involved in the embodiments of the present invention; Figure 3 is a flowchart of the model compression initialization setting unit involved in the embodiments of the present invention; Figure 4 is a flowchart of the automated model compression unit involved in the embodiments of the present invention; Figure 5 is a flowchart of the automated model fine-tuning unit involved in the embodiments of the present invention. Detailed Embodiments
[0014] The following further describes the preferred mechanisms and implementation methods of the present invention in conjunction with the drawings and specific embodiments.
[0015] As Figures 1 to 5 shown, the embodiments of the present invention disclose a technical solution of a general intelligent model compression and fine-tuning method. In the field of deep neural networks, current methods for compressing various deep neural networks include pruning, quantization, low-rank decomposition, and knowledge distillation.
[0016] Pruning: Pruning is a powerful technique for reducing the number of parameters in a DNN. In a deep neural network, many parameters are redundant and contribute little to reducing error and generalizing the network during training. Therefore, after training, these parameters can be removed from the network with minimal impact on the network's accuracy. The main motivation for pruning is to reduce the storage requirements of the deep learning model, making it more storage-friendly. Also, if the model is small enough to fit in on-board memory, fewer memory accesses from DRAM will be required during inference, consuming less energy.
[0017] Quantization: In a deep neural network, weights are generally stored as 32-bit floating-point numbers. Reducing the number of bits used to represent weights and activations can significantly reduce the number of multiply-accumulate (MAC) operations required and decrease the size of the neural network after training. In quantization, weights are represented by reducing the number of bits needed to store each weight. This idea can be further extended to represent gradients and activations in a quantized form. Weights can be quantized to 16 bits, 8 bits, 4 bits, or even 1 bit (a special case of quantization where weights are represented using only binary values, called weight binaryization).
[0018] Knowledge Distillation: In knowledge distillation, a larger, cumbersome network (the teacher model) is generally first trained on a large dataset, and then the knowledge it has learned to generalize well on unseen data is transferred to a smaller, lighter network, called the student model. The teacher model can be a single large model or a collection of separately trained models. The main goal of training the student model from the teacher model is to make the student model learn the generalization ability of the teacher model while maintaining its lightweight nature. Knowledge distillation can significantly reduce the number of parameters, the number of multiply-add operations, and make the model smaller.
[0019] Low-Rank Decomposition: In low-rank decomposition, a weight matrix A with dimensions m×n and rank r is replaced by matrices of smaller dimensions. In a deep convolutional neural network, singular value decomposition (SVD) is a common and popular factorization scheme for reducing the number of parameters. SVD decomposes the original weight matrix into three smaller matrices to replace the original weight matrix. S is a diagonal matrix with singular values on the diagonal, and U and V are orthogonal matrices. Each entry in S is greater than the next diagonal entry. When it is necessary to reduce the model size, the low-rank decomposition technique achieves this by decomposing a large matrix into smaller matrices. Factorization of the dense layer matrix mainly improves the storage requirements, making the model more suitable for storage. While factorization of the convolutional filters can accelerate the inference process. Example 1
[0020] A general intelligent model compression and fine-tuning method, characterized by comprising the following steps: Step 1: Initial Settings for Model Compression: Load the model weights, training data, and training code, initialize the model, data loader, and model trainer required for automated model compression, and verify the success of the foregoing initializations.
[0021] The initial settings for model compression include the following steps: T1: Load the model parameters and perform model verification; T2: Load the training data and perform data verification; T3: Load the training code, create a model trainer, and perform training verification.
[0022] The initial settings for model compression in Step 1 include the following methods: Model creation method: Read the parameters according to the path, create a model based on the model parameters, and load the parameters; Data loader creation method: Load the data according to the data path, create a data loader, and provide training data at any time; Model trainer creation method: Read the training code according to the training code path and create a model trainer to train the model; Loading verification method: Use the inferable model and data loader as inputs, call the model trainer to train for 10 iterations, and return a report on whether the loading is successful.
[0023] Step 2: Automated Model Compression: Automatically compress the model using the inferable model, data loader, and model trainer obtained from the initialization settings module: S1: Read the compression requirements, obtain the model, and create a pruning tool; S2: Perform forward inference and backpropagation on the model using the pruning tool, create a model structure dependency graph based on the backpropagation path, and group the model parameters according to the dependency graph; S3: Use the pruning tool to evaluate the parameter importance of the model according to the pruning method specified by the compression requirements; S4: Sort by importance within the same dependency group and use the pruning tool to prune the corresponding low-importance parameters according to the pruning rate of the compression requirements to obtain the compressed model; S5: Send the compressed model to the automated fine-tuning module.
[0024] The automated model compression in Step 2 includes the following methods: Method for receiving and parsing compression parameters: Receive and parse the pruning rate, pruning method, and ignored area included in the compression parameters, and create a pruning tool; Method for creating a structure dependency graph: Establish a structure dependency graph according to the model backpropagation path and divide it into dependency groups; Method for parameter importance estimation: Evaluate the importance of each parameter according to the selected pruning method; Redundant parameter pruning method: Sort the parameters within the dependency group according to their importance, prune the parameters with low importance, and save the pruned model.
[0025] Step 3: Fine-tune the compressed model and automatically select the optimal hyperparameters to obtain the optimal result.
[0026] The fine-tuning of the compressed model in Step 3 includes the following steps: T4: Create a search space and train the compressed model for one epoch; T5: Traverse the search space to evaluate all hyperparameter combinations and select the optimal hyperparameter combination.
[0027] The automatic model fine-tuning in Step 3 includes the following methods: Automatic hyperparameter search method: Create a search space, traverse the search space to evaluate all hyperparameter combinations, and select the optimal hyperparameter combination; Model trainer method: Fine-tune and train the model according to the input training hyperparameters, and save the finally fine-tuned model; Model evaluation method: Calculate the number of parameters of the compressed model and evaluate the model accuracy and return it to the user interface.
[0028] The present invention provides a general intelligent model compression and fine-tuning method, which can provide designers with an intelligent model compression tool based on an embedded environment. This tool can simplify the network structure or weight representation of the model, and reduce the computational complexity and parameter scale of the model on the premise of controllable accuracy loss, so as to solve the problem that the deep learning model cannot run due to insufficient resources in the embedded environment.
[0029] Finally, it should be noted that the above are only the preferred embodiments of the present invention and are not used to limit the present invention. Although the present invention has been described in detail with reference to the embodiments, for those skilled in the art, they can still modify the technical solutions described in the foregoing embodiments, or perform equivalent replacements for some of the technical features. However, any modifications, equivalent replacements, improvements, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.
Claims
1. A general intelligent model compression and fine-tuning method, characterized in that It includes the following steps: Step 1: Initial settings for model compression: Load model weights, training data, and training code, initialize the model, data loader, and model trainer required for automatic model compression, and verify whether the above initializations are successful; Step 2: Automatic model compression: Automatically compress the model using the inferable model, data loader, and model trainer obtained from the initialization settings module: S1: Read the compression requirements, obtain the model, and create a pruning tool; S2: Use the pruning tool to perform forward inference and backpropagation on the model, create a model structure dependency graph based on the backpropagation path, and group the model parameters according to the dependency graph; S3: According to the pruning method specified in the compression requirements, use the pruning tool to evaluate the importance of the model parameters; S4: Sort by importance within the same dependency group and use the pruning tool to prune the corresponding low-importance parameters according to the pruning rate of the compression requirements to obtain the compressed model; S5: Send the compressed model to the automatic fine-tuning module; Step 3: Fine-tune the compressed model and automatically select the optimal hyperparameters to obtain the optimal result.
2. The general intelligent model compression and fine-tuning method according to claim 1, characterized in that The initial settings for model compression in Step 1 include the following steps: T1: Load model parameters and perform model verification; T2: Load training data and perform data verification; T3: Load training code, create a model trainer, and perform training verification.
3. The general intelligent model compression and fine-tuning method according to claim 1 or 2, characterized in that, The initial settings for model compression in Step 1 include the following methods: Model creation method: Read parameters according to the path, create a model based on the model parameters, and load the parameters; Data loader creation method: Load data according to the data path, create a data loader, and provide training data at any time; Model trainer creation method: Read the training code according to the training code path and create a model trainer to train the model; Loading verification method: Use the inferable model and data loader as inputs to call the model trainer to train for 10 iterations and return a report on whether the loading is successful.
4. The general intelligent model compression and fine-tuning method according to claim 1, characterized in that The automatic model compression in Step 2 includes the following methods: Compression parameter reception and parsing method: Receive and parse the pruning rate, pruning method, and ignored area included in the compression parameters, and create a pruning tool; Structure dependency graph creation method: Establish a structure dependency graph based on the model backpropagation path and divide it into dependency groups; Parameter importance estimation method: Evaluate the importance of each parameter according to the selected pruning method; Redundant parameter pruning method: Sort the parameters by importance within the dependency group, prune the low-importance parameters, and save the pruned model.
5. The general intelligent model compression and fine-tuning method according to claim 1, characterized in that The fine-tuning of the compressed model in Step 3 includes the following steps: T4: Create a search space and train the compressed model for one cycle; T5: Traverse the search space to evaluate all hyperparameter combinations and select the optimal hyperparameter combination.
6. The general intelligent model compression and fine-tuning method according to claim 1 or 5, characterized in that The automatic model fine-tuning in Step 3 includes the following methods: Automatic hyperparameter search method: Create a search space, traverse the search space to evaluate all hyperparameter combinations, and select the optimal hyperparameter combination; Model trainer method: Fine-tune the model according to the input training hyperparameters and save the finally fine-tuned model; Model evaluation method: Calculate the number of parameters of the compressed model and evaluate the model accuracy and return it to the user interface.