Model optimization system and method based on deep learning

Abstract

The invention discloses a model optimization system and method based on deep learning, the system comprises a model square module, a my model module, a model fine tuning module and a model optimization module, and the method comprises the following steps: S1, selecting an initial model; s2, applying an optimization algorithm; s21, a pruning algorithm is applied, and weights or neurons to be removed are determined; s22, a quantification algorithm is applied, and weight and activation are converted into low-precision representation from floating-point numbers; s23, knowledge distillation: selecting a teacher model, defining a student model, and training the student model by using the output of the teacher model as a guidance signal; and S3, selectively performing fine adjustment. Based on a leading edge deep learning method, the open source technology is fully utilized for improvement, and the target of low-cost deployment is achieved under the condition that performance is not lost basically; the model size is reduced, storage is saved, the method is particularly important for mobile equipment or an embedded system, and meanwhile the transmission speed and the deployment speed can be increased.

Description

Technical Field

[0001] The present invention relates to the field of large model technology, and in particular to a model optimization system and method based on deep learning. Background Art

[0002] Large model technology, especially large language models (LLMs) and visual models based on deep learning, has made significant progress in recent years. These models have achieved unprecedented performance levels in many fields such as natural language processing and image recognition by using a large number of parameters to capture complex patterns in the data. With the growth of computing power and the continuous expansion of training data sets, large models can complete more complex tasks and show capabilities close to or even exceeding those of humans. However, they also face some problems, including huge computing resources and energy consumption requirements, and significantly increased training and inference costs, which make it difficult for enterprises to widely apply these technologies and the models are too large to be deployed in mobile or embedded devices. Summary of the invention

[0003] The purpose of the present invention is to propose a model optimization system and method based on deep learning in response to the above-mentioned technical problems.

[0004] A model optimization system based on deep learning, including a model square module, a my model module, a model fine-tuning module, and a model optimization module; The Model Square module centrally displays and manages selected preset general large models, supports multi-dimensional screening, and supports fine-tuning, evaluation, deployment, and optimization of large models; The model fine-tuning module and the model optimization module fine-tuning and optimized models displayed and managed by the My Model module; The model fine-tuning module selects a model for parameter adjustment training according to business needs; The model optimization module defines the model optimization types of pruning algorithms, quantization, and knowledge distillation, and optimizes different types of model selections.

[0005] Furthermore, a model optimization system based on deep learning, wherein the model fine-tuning module includes a training preparation submodule, a model training submodule, and a model management submodule; The training preparation submodule loads data and corresponding models from the shared file system according to the configuration; The model training submodule uses the downloaded data and the selected general model to train the model according to the filled-in hyperparameters and computing power configuration parameters; The model obtained after training by the model management submodule is uploaded to the shared file system for use by the distributed system to publish the model and recorded in the My Model module.

[0006] Furthermore, a model optimization system based on deep learning, wherein the model optimization module includes a pruning algorithm optimization type, a model quantization optimization type, and a knowledge model distillation optimization type; The pruning algorithm optimization type removes weights or neurons in the neural network that contribute little to the final output, reducing the size and computational complexity of the model; The model quantization optimization type converts floating-point weights and activation values ​​in the neural network into low-precision integers, reducing the storage requirements and computational complexity of the model, speeding up inference and reducing energy consumption; The knowledge model distillation optimization type is model compression, which reduces the size and computational complexity of the model; The knowledge model distillation optimization types include teacher model and student model; The teacher model is a pre-trained large-scale neural network; The student model has a simpler structure and fewer parameters, and aims to learn the behavior of the teacher model.

[0007] Furthermore, a model optimization system based on deep learning, wherein the pruning algorithm optimization types include unstructured pruning, structured pruning, iterative pruning, adaptive pruning, and global pruning; The unstructured pruning removes weights from the network one by one, requiring hardware to exploit sparsity; The structured pruning removes entire neurons, convolution kernels or network structural units without resulting in a sparse matrix; The iterative pruning is performed step by step, and the pruning ratio is gradually increased through multiple iterations while maintaining the model performance; The adaptive pruning dynamically adjusts the pruning strategy during the training process to adapt to the training requirements at different stages; The global pruning uniformly applies pruning standards across the entire network.

[0008] Furthermore, a model optimization system based on deep learning, wherein the unstructured pruning includes pruning based on weight size, pruning based on gradient, and pruning based on optimization; The weight-based pruning sets a threshold according to the absolute value of the weight and removes weights below the threshold; The gradient-based pruning determines weight removal by analyzing the gradient information of the weights; The optimization-based pruning uses LASSO regression to directly learn a sparse weight vector; The structured pruning includes filter pruning, channel pruning, and layer pruning; The filter pruning removes the entire set of filters in the convolutional layer; The channel pruning removes input or output channels; The layer pruning removes entire network layers; The iterative pruning includes one-time pruning and progressive pruning; The one-time pruning prunes a large number of weights at one time and fine-tunes the remaining weights; The progressive pruning only prunes a small amount of weight each time, and reaches the final goal after multiple iterations; The adaptive pruning includes dynamic threshold pruning and loss function-based pruning; The dynamic threshold pruning dynamically adjusts the pruning threshold as the training progresses; The loss function-based pruning incorporates the pruning process into the loss function, so that the model naturally tends to be sparse during the training process; The global pruning includes global threshold pruning and global percentage pruning; The global threshold pruning sets a global threshold, and all weights below the threshold are removed; The global percentage pruning removes a certain percentage of the minimum weights in the entire network.

[0009] Furthermore, a model optimization system based on deep learning, wherein the model quantization optimization types include fixed-point quantization, dynamic quantization, mixed precision quantization, and quantization-aware training; The fixed-point quantization uses a linear mapping to map floating-point numbers into a fixed integer range; The dynamic quantization dynamically adjusts the quantization parameters according to the minimum and maximum values ​​of the input data before each inference, and performs quantization according to the statistical information of the current input data during inference; Mixed precision quantization: using different quantization precisions in different parts of the model; The quantization-aware training is defined as inserting pseudo-quantization nodes in the forward propagation, where the nodes simulate actual quantization operations so that the model considers the impact of quantization during the training phase.

[0010] A model optimization method based on deep learning, comprising the following sub-steps: S1: Select the initial model; S2: Apply optimization algorithm; S21: Apply pruning algorithm to determine weights or neurons to be removed; S22: Apply quantization algorithms to convert weights and activations from floating point numbers to low-precision representations; S23: Knowledge distillation, select the teacher model, define the student model, and use the output of the teacher model as a guidance signal to train the student model; S3: Optional fine-tuning.

[0011] Furthermore, in a model optimization method based on deep learning, the S21 selects the following sub-steps: Unstructured pruning: Train the original model until convergence, determine the weights to be removed based on the selected pruning criteria, remove the selected weights, set the corresponding connections to zero, and fine-tune the pruned model to restore performance; Structured pruning: Train the original model until convergence, and use L1 norm and activation statistics to evaluate the importance of each structural unit. Identify and remove less important structural units, and retrain or fine-tune the pruned model; Iterative pruning: Train the original model until convergence, set the initial pruning ratio and number of iterations, and prune a portion of the weights according to the rules in each round of iteration. After each pruning, fine-tune the model to restore performance. Adaptive pruning: Define a loss function that includes pruning constraints. During the training process, automatically update the weights and apply pruning according to the loss function. Regularly evaluate model performance and adjust the pruning strategy. Global pruning: Train the original model until convergence, determine a global threshold or percentage global pruning criterion, apply the pruning criterion, remove eligible weights, and fine-tune the pruned model to restore performance.

[0012] Furthermore, in a model optimization method based on deep learning, the S22 comprises the following sub-steps: S221: Select quantization scheme: According to the application requirements and the support of the hardware platform, select the quantization method in fixed-point quantization, dynamic quantization, mixed precision quantization, and quantization-aware training; S222: Prepare model: Prepare the trained model, making sure there is a trained floating point model as a starting point; S223: Calibration: For fixed-point quantization, a representative set of data is collected to determine the quantization parameters; For dynamic quantization, the calibration step is omitted and the quantization parameters are calculated in real time during inference; S224: Quantization Model: Convert floating point weights and activation values ​​to low precision representation using the selected quantization method; S225: Fine-tuning: If the model performance degrades, fine-tune the quantized model to restore the performance; S226: Evaluation and Optimization: Evaluate the performance of the quantized model on the validation set to ensure that the expected accuracy and indicators are met. If the standard is not met, adjust the quantization scheme and retrain the model.

[0013] Furthermore, in a model optimization method based on deep learning, S23 comprises the following sub-steps: S231: Prepare teacher model: Select or train a teacher model that performs well and has high performance on the target task; S232: Design student model: Design a student model with a simple structure and fewer parameters than the teacher model; S233: Setting hyperparameters: determining the temperature parameter T and the weight α of the soft target loss and the hard target loss; S234: Data preparation: Prepare the same dataset used to train the teacher model for training; S235: Train the student model: Use the soft labels and true labels of the teacher model to jointly train the student model. In each forward propagation, the soft target loss and the hard target loss are calculated at the same time. The two are combined to obtain the total loss, and back propagation is performed to update the parameters of the student model. S236: Evaluation and fine-tuning: After training is completed, the performance of the student model is evaluated on the validation set. If the performance is not up to standard, the hyperparameters or the architecture of the student model are adjusted and retrained; S237: Deployment: Deploy the optimized student model to actual applications to reduce model size and speed up inference.

[0014] The beneficial effects of the present invention are as follows: through a deep learning-based model optimization system and method, based on cutting-edge deep learning methods, full use of open source technology for improvement, and by applying technologies such as model distillation, model pruning and quantization loading, the goal of low-cost deployment is achieved without basically losing performance; the model size is reduced and storage is saved, which is particularly important for mobile devices or embedded systems, and at the same time, the transmission speed and deployment speed can be accelerated; the model's reasoning speed is greatly improved without sacrificing too much accuracy, thereby reducing computing resource consumption. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] Figure 1 It is a structural diagram of a model optimization system based on deep learning.

[0016] Figure 2 It is a flowchart of a model optimization method based on deep learning. DETAILED DESCRIPTION

[0017] In order to have a clearer understanding of the technical features, purposes and effects of the present invention, specific embodiments of the present invention are now described with reference to the accompanying drawings.

[0018] As attached Figure 1 As shown, a model optimization system based on deep learning includes a model square module, a my model module, a model fine-tuning module, and a model optimization module; The Model Square module centrally displays and manages selected preset general large models, supports multi-dimensional screening, and supports fine-tuning, evaluation, deployment, and optimization of large models; The model fine-tuning module and the model optimization module fine-tuning and optimized models displayed and managed by the My Model module; The model fine-tuning module selects a model for parameter adjustment training according to business needs; The model optimization module defines the model optimization types of pruning algorithms, quantization, and knowledge distillation, and optimizes different types of model selections.

[0019] Furthermore, a model optimization system based on deep learning, wherein the model fine-tuning module includes a training preparation submodule, a model training submodule, and a model management submodule; The training preparation submodule loads data and corresponding models from the shared file system according to the configuration; The model training submodule uses the downloaded data and the selected general model to train the model according to the filled-in hyperparameters and computing power configuration parameters; The model obtained after training by the model management submodule is uploaded to the shared file system for use by the distributed system to publish the model and recorded in the My Model module.

[0020] Furthermore, a model optimization system based on deep learning, wherein the model optimization module includes a pruning algorithm optimization type, a model quantization optimization type, and a knowledge model distillation optimization type; The pruning algorithm optimization type removes weights or neurons in the neural network that contribute little to the final output, reducing the size and computational complexity of the model; The model quantization optimization type converts floating-point weights and activation values ​​in the neural network into low-precision integers, such as 8-bit integers int8, to reduce the storage requirements and computational complexity of the model, speed up reasoning, and reduce energy consumption; Quantization can make the model more suitable for deployment on resource-constrained devices such as mobile phones, embedded systems, etc. without significantly sacrificing model performance.

[0021] The knowledge model distillation optimization type is model compression, which reduces the size and computational complexity of the model; The knowledge model distillation optimization types include teacher model and student model; The teacher model is a pre-trained large-scale neural network; The student model has a simpler structure and fewer parameters, and aims to learn the behavior of the teacher model.

[0022] Furthermore, a model optimization system based on deep learning, wherein the pruning algorithm optimization types include unstructured pruning, structured pruning, iterative pruning, adaptive pruning, and global pruning; The unstructured pruning removes weights from the network one by one, requiring hardware to exploit sparsity; The structured pruning removes entire neurons, convolution kernels or network structural units without resulting in a sparse matrix; The iterative pruning is performed step by step, and the pruning ratio is gradually increased through multiple iterations while maintaining the model performance; The adaptive pruning dynamically adjusts the pruning strategy during the training process to adapt to the training requirements at different stages; The global pruning uniformly applies pruning standards across the entire network.

[0023] Furthermore, a model optimization system based on deep learning, wherein the unstructured pruning includes pruning based on weight size, pruning based on gradient, and pruning based on optimization; The weight-based pruning sets a threshold according to the absolute value of the weight and removes weights below the threshold; The gradient-based pruning determines weight removal by analyzing the gradient information of the weights; The optimization-based pruning uses LASSO regression to directly learn a sparse weight vector; The structured pruning includes filter pruning, channel pruning, and layer pruning; The filter pruning removes the entire set of filters in the convolutional layer; The channel pruning removes input or output channels; The layer pruning removes entire network layers; The iterative pruning includes one-time pruning and progressive pruning; The one-time pruning prunes a large number of weights at one time and fine-tunes the remaining weights; The progressive pruning only prunes a small amount of weight each time, and reaches the final goal after multiple iterations; The adaptive pruning includes dynamic threshold pruning and loss function-based pruning; The dynamic threshold pruning dynamically adjusts the pruning threshold as the training progresses; The loss function-based pruning incorporates the pruning process into the loss function, so that the model naturally tends to be sparse during the training process; The global pruning includes global threshold pruning and global percentage pruning; The global threshold pruning sets a global threshold, and all weights below the threshold are removed; The global percentage pruning removes a certain percentage of the minimum weights in the entire network.

[0024] Furthermore, a model optimization system based on deep learning, wherein the model quantization optimization types include fixed-point quantization, dynamic quantization, mixed precision quantization, and quantization-aware training; The fixed-point quantization uses a linear mapping to map floating-point numbers into a fixed integer range; The dynamic quantization dynamically adjusts the quantization parameters according to the minimum and maximum values ​​of the input data before each inference, and performs quantization according to the statistical information of the current input data during inference; Mixed precision quantization: using different quantization precisions in different parts of the model; The quantization-aware training is defined as inserting pseudo-quantization nodes in the forward propagation, where the nodes simulate actual quantization operations so that the model considers the impact of quantization during the training phase.

[0025] As attached Figure 2 As shown, a model optimization method based on deep learning includes the following sub-steps: S1: Select the initial model; S2: Apply optimization algorithm; S21: Apply pruning algorithm to determine weights or neurons to be removed; S22: Apply quantization algorithms to convert weights and activations from floating point numbers to low-precision representations; S23: Knowledge distillation, select the teacher model, define the student model, and use the output of the teacher model as a guidance signal to train the student model; S3: Optional fine-tuning.

[0026] Furthermore, in a model optimization method based on deep learning, the S21 selects the following sub-steps: Unstructured pruning: Train the original model until convergence, determine the weights to be removed based on the selected pruning criteria, remove the selected weights, set the corresponding connections to zero, and fine-tune the pruned model to restore performance; Structured pruning: train the original model until convergence, and use L1 norm and activation statistics to evaluate the importance of each structural unit; Identify and remove less important structural units, and retrain or fine-tune the pruned model; Iterative pruning: Train the original model until convergence, set the initial pruning ratio and number of iterations, and prune a portion of the weights according to the rules in each round of iteration. After each pruning, fine-tune the model to restore performance. Adaptive pruning: Define a loss function that includes pruning constraints. During the training process, automatically update the weights and apply pruning according to the loss function. Regularly evaluate model performance and adjust the pruning strategy. Global pruning: Train the original model until convergence, determine a global threshold or percentage global pruning criterion, apply the pruning criterion, remove eligible weights, and fine-tune the pruned model to restore performance.

[0027] Furthermore, in a model optimization method based on deep learning, the S22 comprises the following sub-steps: S221: Select quantization scheme: According to the application requirements and the support of the hardware platform, select the quantization method in fixed-point quantization, dynamic quantization, mixed precision quantization, and quantization-aware training; S222: Prepare model: Prepare the trained model, making sure there is a trained floating point model as a starting point; S223: Calibration: For fixed-point quantization, a representative set of data is collected to determine the quantization parameters; For dynamic quantization, the calibration step is omitted and the quantization parameters are calculated in real time during inference; S224: Quantization Model: Convert floating point weights and activation values ​​to low precision representation using the selected quantization method; S225: Fine-tuning: If the model performance degrades, fine-tune the quantized model to restore the performance; S226: Evaluation and Optimization: Evaluate the performance of the quantized model on the validation set to ensure that the expected accuracy and indicators are met. If the standard is not met, adjust the quantization scheme and retrain the model.

[0028] Furthermore, in a model optimization method based on deep learning, S23 comprises the following sub-steps: S231: Prepare teacher model: Select or train a teacher model that performs well and has high performance on the target task; S232: Design student model: Design a student model with a simple structure and fewer parameters than the teacher model; S233: Setting hyperparameters: determining the temperature parameter T and the weight α of the soft target loss and the hard target loss; S234: Data preparation: Prepare the same dataset used to train the teacher model for training; S235: Train the student model: Use the soft labels and true labels of the teacher model to jointly train the student model. In each forward propagation, the soft target loss and the hard target loss are calculated at the same time. The two are combined to obtain the total loss, and back propagation is performed to update the parameters of the student model. S236: Evaluation and fine-tuning: After training is completed, the performance of the student model is evaluated on the validation set. If the performance is not up to standard, the hyperparameters or the architecture of the student model are adjusted and retrained; S237: Deployment: Deploy the optimized student model to actual applications to reduce model size and speed up inference.

[0029] Specific embodiment 1 The framework of the model training submodule: The model training submodule is based on the aMA-Factory framework; The LLaMA-Factory framework features include multi-model support, diverse integration methods, multiple precisions, and advanced algorithms; The multi-model support includes LLaMA, LLaVA, Mistral, Mixtral-MoE, Qwen, Qwen2-VL, Yi, Gemma, Baichuan, ChatGLM, and Phi; The integration methods are diverse, including incremental pre-training, multimodal instruction supervised fine-tuning, reward model training, PPO training, DPO training, KTO training, and ORPO training; The multiple precisions include 16-bit full parameter fine-tuning, frozen fine-tuning, LoRA fine-tuning, and QLoRA fine-tuning; The advanced algorithms include GaLore, BAdam, Adam-mini, DoRA, LongLoRA, LLaMA Pro, Mixture-of-Depths, LoRA+, LoftQ, PiSSA, and Agent fine-tuning.

[0030] Advantages: Compared with ChatGLM's official P-Tuning fine-tuning, LLaMA Factory's LoRA fine-tuning provides a 3.7x speedup ratio and achieves a higher Rouge score in the ad copy generation task. Combined with 4-bit quantization technology, LLaMA Factory's QLoRA fine-tuning further reduces GPU memory consumption.

[0031] Specific Example 2 Quantified advantages and disadvantages: Fixed-point quantization usually uses a linear mapping, for example, mapping floating-point numbers in the range [-6, 6] to int8 integers in the range [-127, 127]; Advantages: simple and easy to implement, high computational efficiency; Disadvantages: May not work well for data with a wide or non-uniform distribution of values.

[0032] Dynamic Quantization Advantages: Strong adaptability and better robustness to different inputs; Disadvantages: Requires additional computation to determine quantization parameters, may increase latency.

[0033] Mixed precision quantization uses int8 in some layers and float16 in other layers; Advantages: It can balance accuracy and performance and is suitable for situations where different layers have different requirements; Disadvantages: Increases implementation complexity and requires more detailed tuning.

[0034] Quantization-aware training Advantages: It can significantly improve the performance of the quantized model because the model has learned how to work under quantized conditions. Disadvantages: Increases the complexity and time cost of training.

[0035] Specific Example 3 Working principle of model distillation: Soft Targets: The output of the teacher model is usually a probability distribution processed by the softmax function. The probability distribution contains not only the information of the final classification result, but also the relative relationship between categories: During the distillation process, the output of the teacher model is used as soft targets instead of the original hard labels. The soft labels provide more information and help the student model learn more detailed inter-class distinctions.

[0036] Temperature: In order to make the soft labels smoother, a temperature parameter T is introduced. When T increases, the output of the softmax function becomes more uniform, which makes the probability difference between different categories smaller, thereby retaining more information; The formula is: softmax(z / T)softmax(z / T), where z is the output vector of the network and T is the temperature parameter; Loss Function: The loss function of model distillation usually consists of two parts: Soft target loss: the cross entropy loss between the output of the student model and the soft label of the teacher model; Hard target loss: the cross entropy loss between the output of the student model and the true label; The total loss can be expressed as the weighted sum of two parts of the loss: L=αLsoft+(1−α)LhardL=αLsoft+(1−α)Lhard, where α is the weight coefficient.

[0037] This solution uses a deep learning-based model optimization system and method, based on cutting-edge deep learning methods, and makes full use of open source technologies for improvement. By applying technologies such as model distillation, model pruning, and quantized loading, it achieves the goal of low-cost deployment without basically losing performance; reduces model size and saves storage, which is especially important for mobile devices or embedded systems, and can also speed up transmission and deployment speeds; greatly improves the model's inference speed without sacrificing too much accuracy, reducing computing resource consumption.

[0038] The above shows and describes the basic principles and main features of the present invention and the advantages of the present invention. It should be understood by those skilled in the art that the present invention is not limited to the above embodiments. The above embodiments and descriptions are only for explaining the principles of the present invention. Without departing from the spirit and scope of the present invention, the present invention may have various changes and improvements, which fall within the scope of the present invention to be protected. The scope of protection of the present invention is defined by the attached claims and their equivalents.

Claims

1. A model optimization system based on deep learning, characterized in that: Including model square module, my model module, model fine-tuning module, model optimization module; The Model Square module centrally displays and manages selected preset general large models, supports multi-dimensional screening, and supports fine-tuning, evaluation, deployment, and optimization of large models; The model fine-tuning module and the model optimization module fine-tuning and optimized models displayed and managed by the My Model module; The model fine-tuning module selects a model for parameter adjustment training according to business needs; The model optimization module defines the model optimization types of pruning algorithms, quantization, and knowledge distillation, and optimizes different types of model selections.

2. A model optimization system based on deep learning according to claim 1, characterized in that: The model fine-tuning module includes a training preparation submodule, a model training submodule, and a model management submodule; The training preparation submodule loads data and corresponding models from the shared file system according to the configuration; The model training submodule uses the downloaded data and the selected general model to train the model according to the filled-in hyperparameters and computing power configuration parameters; The model obtained after training by the model management submodule is uploaded to the shared file system for use by the distributed system to publish the model and recorded in the My Model module.

3. The model optimization system based on deep learning according to claim 1, characterized in that: The model optimization module includes pruning algorithm optimization type, model quantization optimization type, and knowledge model distillation optimization type; The pruning algorithm optimization type removes weights or neurons in the neural network that contribute little to the final output, reducing the size and computational complexity of the model; The model quantization optimization type converts floating-point weights and activation values ​​in the neural network into low-precision integers, reducing the storage requirements and computational complexity of the model, speeding up inference and reducing energy consumption; The knowledge model distillation optimization type is model compression, which reduces the size and computational complexity of the model; The knowledge model distillation optimization types include teacher model and student model; The teacher model is a pre-trained large-scale neural network; The student model has a simpler structure and fewer parameters, and aims to learn the behavior of the teacher model.

4. A model optimization system based on deep learning according to claim 3, characterized in that: The pruning algorithm optimization types include unstructured pruning, structured pruning, iterative pruning, adaptive pruning, and global pruning; The unstructured pruning removes weights from the network one by one, requiring hardware to exploit sparsity; The structured pruning removes entire neurons, convolution kernels or network structural units without resulting in a sparse matrix; The iterative pruning is performed step by step, and the pruning ratio is gradually increased through multiple iterations while maintaining the model performance; The adaptive pruning dynamically adjusts the pruning strategy during the training process to adapt to the training requirements at different stages; The global pruning uniformly applies pruning standards across the entire network.

5. A model optimization system based on deep learning according to claim 4, characterized in that: The unstructured pruning includes pruning based on weight size, pruning based on gradient, and pruning based on optimization; The weight-based pruning sets a threshold according to the absolute value of the weight and removes weights below the threshold; The gradient-based pruning determines weight removal by analyzing the gradient information of the weights; The optimization-based pruning uses LASSO regression to directly learn a sparse weight vector; The structured pruning includes filter pruning, channel pruning, and layer pruning; The filter pruning removes the entire set of filters in the convolutional layer; The channel pruning removes input or output channels; The layer pruning removes entire network layers; The iterative pruning includes one-time pruning and progressive pruning; The one-time pruning prunes a large number of weights at one time and fine-tunes the remaining weights; The progressive pruning only prunes a small amount of weight each time, and reaches the final goal after multiple iterations; The adaptive pruning includes dynamic threshold pruning and loss function-based pruning; The dynamic threshold pruning dynamically adjusts the pruning threshold as the training progresses; The loss function-based pruning incorporates the pruning process into the loss function, so that the model naturally tends to be sparse during the training process; The global pruning includes global threshold pruning and global percentage pruning; The global threshold pruning sets a global threshold, and all weights below the threshold are removed; The global percentage pruning removes a certain percentage of the minimum weights in the entire network.

6. A model optimization system based on deep learning according to claim 3, characterized in that: The model quantization optimization types include fixed-point quantization, dynamic quantization, mixed precision quantization, and quantization-aware training; The fixed-point quantization uses a linear mapping to map floating-point numbers into a fixed integer range; The dynamic quantization dynamically adjusts the quantization parameters according to the minimum and maximum values ​​of the input data before each inference, and performs quantization according to the statistical information of the current input data during inference; Mixed precision quantization: using different quantization precisions in different parts of the model; The quantization-aware training is defined as inserting pseudo-quantization nodes in the forward propagation, where the nodes simulate actual quantization operations so that the model considers the impact of quantization during the training phase.

7. A model optimization method based on deep learning, implemented by a model optimization system based on deep learning according to any one of claims 1 to 6, characterized in that: It includes the following sub-steps: S1: Select the initial model; S2: Apply optimization algorithm; S21: Apply pruning algorithm to determine weights or neurons to be removed; S22: Apply quantization algorithms to convert weights and activations from floating point numbers to low-precision representations; S23: Knowledge distillation, select the teacher model, define the student model, and use the output of the teacher model as a guidance signal to train the student model; S3: Optional fine-tuning.

8. The model optimization method based on deep learning according to claim 7, characterized in that: The S21 selects the following sub-steps: Unstructured pruning: Train the original model until convergence, determine the weights to be removed based on the selected pruning criteria, remove the selected weights, set the corresponding connections to zero, and fine-tune the pruned model to restore performance; Structured pruning: Train the original model until convergence, and use L1 norm and activation statistics to evaluate the importance of each structural unit. Identify and remove less important structural units, and retrain or fine-tune the pruned model; Iterative pruning: Train the original model until convergence, set the initial pruning ratio and number of iterations, and prune a portion of the weights according to the rules in each round of iteration. After each pruning, fine-tune the model to restore performance. Adaptive pruning: Define a loss function that includes pruning constraints. During the training process, automatically update the weights and apply pruning according to the loss function. Regularly evaluate model performance and adjust the pruning strategy. Global pruning: Train the original model until convergence, determine a global threshold or percentage global pruning criterion, apply the pruning criterion, remove eligible weights, and fine-tune the pruned model to restore performance.

9. The model optimization method based on deep learning according to claim 7, characterized in that: The S22 comprises the following sub-steps: S221: Select quantization scheme: According to the application requirements and the support of the hardware platform, select the quantization method in fixed-point quantization, dynamic quantization, mixed precision quantization, and quantization-aware training; S222: Prepare model: Prepare the trained model, making sure there is a trained floating point model as a starting point; S223: Calibration: For fixed-point quantization, a representative set of data is collected to determine the quantization parameters; For dynamic quantization, the calibration step is omitted and the quantization parameters are calculated in real time during inference; S224: Quantization Model: Convert floating point weights and activation values ​​to low precision representation using the selected quantization method; S225: Fine-tuning: If the model performance degrades, fine-tune the quantized model to restore the performance; S226: Evaluation and Optimization: Evaluate the performance of the quantized model on the validation set to ensure that the expected accuracy and indicators are met. If the standard is not met, adjust the quantization scheme and retrain the model.

10. The model optimization method based on deep learning according to claim 7, characterized in that: The S23 comprises the following sub-steps: S231: Prepare teacher model: Select or train a teacher model that performs well and has high performance on the target task; S232: Design student model: Design a student model with a simple structure and fewer parameters than the teacher model; S233: Setting hyperparameters: determining the temperature parameter T and the weight α of the soft target loss and the hard target loss; S234: Data preparation: Prepare the same dataset used to train the teacher model for training; S235: Train the student model: Use the soft labels and true labels of the teacher model to jointly train the student model. In each forward propagation, the soft target loss and the hard target loss are calculated at the same time. The two are combined to obtain the total loss, and back propagation is performed to update the parameters of the student model. S236: Evaluation and fine-tuning: After training is completed, the performance of the student model is evaluated on the validation set. If the performance is not up to standard, the hyperparameters or the architecture of the student model are adjusted and retrained; S237： Deployment: Deploy the optimized student model to actual applications to reduce model size and speed up inference.

Images