Visual large model channel pruning-distillation-quantization compensation cascade compression method

Abstract

The invention relates to the technical field of deep neural network model compression and acceleration, in particular to a visual large model channel pruning-distillation-quantization compensation cascade compression method. Differential sparse gating channel pruning, progressive contraction-expansion knowledge distillation and mixing precision quantification joint online compensation are sequentially executed, and hyper-parameter end-to-end joint optimization is achieved in cooperation with an error feedback loop and an MAML frame. A hardware-friendly structured sparse skeleton is generated through channel pruning, knowledge distillation meets a teacher-student capacity gap, and quantitative compensation corrects quantitative loss through a foldable QLoRA and an online error compensator. On ImageNet-1K, after the ViT-G model is compressed by 16.7 times through the method, the precision is only reduced by 0.48%, the Jetson Orin edge end reasoning delay is reduced to 34 ms from 590 ms, full-process automatic compression can be completed within 24 hours, an ONNX model capable of being directly deployed can be output, and high compression ratio, low precision loss and high hardware affinity are considered.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of deep neural network model compression and acceleration, in particular to a visual large model channel pruning-distillation-quantization compensation cascaded compression method. BACKGROUND

[0002] In just two years, visual large models have jumped from hundreds of millions of parameters to tens of billions of parameters, with accuracy records being broken constantly, but the storage, calculation and energy consumption pressure has also been pushed to the limit. Taking ViT-G as an example, a single 224x224 image inference requires 330 GFLOPs, the weight size is 3.6GB, and the peak power consumption is more than 50W, far exceeding the bearing range of mobile phones, vehicle-mounted or industrial cameras. Model compression has become a "must-answer question" for landing, but existing technologies are fragmented and difficult to balance compression rate, accuracy and deployment convenience.

[0003] The current three major compression paths all have obvious shortcomings. Channel pruning can directly reduce the number of parameters, but it is easy to generate irregular sparsity, and the GPU must rely on a special sparse library to realize acceleration; if the structure is forcibly maintained, the pruning ratio will be immediately locked, and the compression rate will be difficult to break through 4-5 times. When the capacity difference between the teacher and the student is one order of magnitude, knowledge distillation often appears "teacher-student gap", and the student network cannot reproduce the high-order features of the teacher, resulting in a cliff-like drop in accuracy. Post-training quantization seems simple and direct, but the long-tail distribution of visual large model channels is serious, and a small number of outliers can pull down the INT4 precision; existing offline calibration tools can only "patch" in the laboratory, and there is no way to deal with new distribution drifts in runtime.

[0004] More difficultly, the three procedures are usually completed offline in series: first pruning, then training, and then quantization. The errors generated in the previous stage cannot be traced back and corrected, and the hyperparameters can only be adjusted separately, with an experimental period of several weeks; the intermediate model generated needs to be repeatedly converted into a format and recalibrated, and the deployment link is long. The industry urgently needs a "pruning-distillation-quantization" deep integration cascaded pipeline that allows the three stages to work together in real time under the same optimization goal, so as to meet the stringent requirements of end-side deployment for "extreme compression, extremely low accuracy loss, and one-key online". SUMMARY

[0005] The present application provides a visual large model channel pruning-distillation-quantization compensation cascaded compression method, aiming to solve the industry pain points of insufficient compression rate, large accuracy loss, fragmented deployment link, and high parameter tuning cost in the prior art. The overall technical route takes end-to-end collaborative optimization as the core, and through three technical modules of differentiable sparse gating channel pruning, progressive shrinking-expanding knowledge distillation, and mixed precision quantization joint online compensation, combined with error feedback loop unified hyperparameter search, the whole process of automatic compression from the original tens of billions of parameters model to the deployable end-side engine is completed within 24 hours on a single GPU.

[0006] The technical scheme adopted by the present application to solve its technical problems is: a visual large model channel pruning-distillation-quantization compensation cascade compression method, comprising the following steps:

[0007] S1 channel pruning stage:

[0008] S101 For each volume or linear layer weight of the input visual large model , introduce trainable scaling factor and differentiable sparse gating , temperature is a hyperparameter;

[0009] S102 construct joint loss , wherein is the task loss, is the entropy regularization;

[0010] S103 calculate the dynamic sparse budget of each layer based on the Hessian trace estimation , and constrain the pruning proportion accordingly;

[0011] S104 update in the training stage using the Straight-Through Estimator, permanently remove the channels of in the inference stage, generate a structured sparse skeleton S and save it in CSR format {val, ptr, idx, mask};

[0012] S2 knowledge distillation stage: S201 takes the original large model as the teacher T, and the sparse skeleton as the student , and inserts a foldable Adapter-Bypass branch in each TransformerBlock, which consists of 1×1 convolution, SE module and residual connection. During training, it is activated, and during inference, it is folded through structural reparameterization;

[0013] S202 construct distillation loss , wherein P is a learnable projection;

[0014] S203 adopt curriculum temperature scheduling , ;

[0015] S3 quantization compensation stage:

[0016] S301 perform mixed precision post-training quantization on the distilled student network: search INT4 / INT8 mixed bits layer by layer for weights, and dynamically INT8 for activations per token;

[0017] S302 Introduce foldable quantized residual error LoRA for each layer of abnormal channel: , , , the inference stage is fused according to ;

[0018] S303 Deploy online quantization error compensator OQEC: each forward Batch calculation , if , trigger local dequantization-micro correction PDC, only for the k token channels with the largest error to back FP16;

[0019] S304 Search for bit width of each layer using reinforcement learning and , the reward function ;

[0020] S4 Hyperparameter joint optimization stage: S401 Construct unified loss , where is the meta-gradient error feedback loss, updated through the MAML framework ;

[0021] S402 Use differentiable architecture-quantization joint search DAQS on a single GPU to complete optimal hyperparameter determination within ≤1000 iterations;

[0022] S5 Output and deployment stage:

[0023] S501 Generate a compressed model file containing structured sparse descriptors, quantization scale factors, QLoRA residuals, and OQEC threshold tables, which can be directly loaded into TensorRT, ONNXRuntime, or OpenVINO for execution.

[0024] Specifically, the value range of in step S104 is [0.005, 0.05], and is determined on the validation set through grid search or Bayesian optimization.

[0025] Specifically, the Adapter-Bypass branch folding condition in step S201 is: when the running_var of BatchNorm <1e-5 and the scale parameter of the SE module.

[0026] Specifically, the MarginLoss in step S202 is defined as , where .

[0027] Specifically, the quantized residual error LoRA rank r in step S302 is searched in the set {4, 8, 16, 32} through reinforcement learning, and Symmetric quantization and scaling factor are used. .

[0028] Specifically, the OQEC threshold in step S303 It is a dynamic value. , , .

[0029] Specifically, the learning rate within the MAML loop in step S401 outer loop learning rate The outer loop is updated every 10 steps.

[0030] Specifically, the compressed model file output in step S501 is in ONNX format, and the additional custom operators include: SparseConv, which is used to perform CSR sparse convolution. , used to perform matrix multiplication with quantized residual LoRA; It is used to trigger local dequantization-microcorrection during inference.

[0031] The beneficial effects of this invention are:

[0032] Achieving both extreme compression and high accuracy: On ImageNet-1K, the weight of ViT-G with 1B parameters was reduced from 3.6GB to 216MB (16.7×) after processing by this pipeline, with a Top-1 accuracy decrease of only 0.48%, achieving for the first time <0.5% accuracy loss at a compression rate of >15×.

[0033] Significantly accelerated edge inference: In actual testing with NVIDIA Jetson Orin NX, single-frame latency was reduced from 590ms to 34ms (17.4×), while power consumption was reduced by 62%; the combined effect of 78% structured sparsity and INT4 / INT8 hybrid quantization resulted in a TensorRT core utilization rate of 92%.

[0034] End-to-end maintenance-free deployment: Hyperparameters for the three stages of pruning, distillation, and quantification are automatically searched by the error feedback loop, and an ONNX file that can be directly loaded is output within 24 hours, without the need for manual parameter tuning or secondary calibration, achieving one-click end-side deployment. Attached Figure Description

[0035] The present invention will be further described below with reference to the accompanying drawings and embodiments.

[0036] Fig. 1 This is a flowchart illustrating the compression method of the present invention.

[0037] Fig. 2 This is a flowchart illustrating the LoRA and OQEC workflows for quantitative residuals in this invention patent. Detailed Implementation

[0038] To make the technical means, creative features, objectives and effects of this invention easier to understand, the invention will be further described below in conjunction with specific embodiments.

[0039] like Figs. 1-2 As shown, the visual large model channel pruning-distillation-quantization compensation cascade compression method described in this invention adopts a "three-stage cascade, dual-loop optimization" overall architecture:

[0040] Outer loop: EFL meta gradient optimizer, responsible for collecting three-stage loss and hardware metrics in real time, and dynamically updating all hyperparameters such as pruning budget, distillation temperature, quantization bits, and compensation threshold through secondary backpropagation;

[0041] Inner loop: Each stage remains differentiable, ensuring that the gradient can be backpropagated across stages, thus minimizing error accumulation.

[0042] This architecture logically integrates the previously separate pruning, distillation, and quantization processes into a single optimization problem, enabling errors at any stage to be immediately fed back to the upstream module via gradient signals, thereby breaking through the information silos of traditional offline serial processes.

[0043] Channel pruning module (S1)

[0044] Differentiable sparse gating (DSG): For each convolutional / linear layer weight tensor Introducing a trainable scaling factor and through temperature Controlled Sigmoid function Generate a gating value between 0 and 1. The gating value is directly multiplied by the corresponding channel output to achieve "soft pruning". The value is set to 0.1 at the beginning of training, and then linearly annealed to 0.01 as the epoch increases to ensure stable convergence.

[0045] Joint loss function: ,in For the original task loss, L1 regularization prompts Approaching 0; entropy regularity Prevent excessive concentration of gating and maintain exploration capabilities. , The initial values ​​are 1e-4 and 1e-5, respectively, and are updated by EFL every 10 steps.

[0046] Dynamic sparse budget allocation: To avoid the accidental deletion of critical channels due to "one-size-fits-all" pruning, a random Hessian trace estimator is used to quickly approximate the target channel on a 2k sample subset. Define the pruning ratio for each layer. ,in , , This is the maximum trace value of the network. This mechanism is adjusted in real time during training to ensure that the network remains recoverable after pruning.

[0047] Structured pruning and formatting: In the reasoning stage, The channel has been permanently removed. The weights are determined on the validation set through Bayesian optimization, with a typical value of 0.012. The pruned weights are stored as CSR triples {val, ptr, idx} plus a channel-wise mask, which can be directly called by cuSPARSELt and MKL-Sparse without additional memory copying.

[0048] Knowledge distillation module (S2)

[0049] Gradual contraction-expansion architecture: The teacher network T maintains its original size; the student network... Inherit the pruning framework An Adapter-Bypass branch is inserted into each TransformerBlock. The branch consists of a 1×1 convolution (dimensionality reduction → dimensionality increase) + SE attention, with parameters accounting for only 2% of the original Block. During training, the branch is activated to increase capacity; during inference, when the running_var of BatchNorm is less than 1e-5 and SEscale ∈ [0.98, 1.02], the branch weights are merged into the main branch through structural reparameterization, achieving zero-overhead folding.

[0050] Multi-particle size distillation losses:

[0051] , where P is a learnable 1×1 projection; Defined as channel-residual distillation loss , This forces student networks to maintain robust boundaries in pruned dimensions.

[0052] Course-based temperature control: , , As the number of training steps decreases, it effectively alleviates the distribution gap between a large number of teachers and a small number of students.

[0053] Quantitative compensation module (S3)

[0054] Training quantization after mixed precision: weights search bits by layer Activation uses dynamic INT8 tokens per token; search space There are a total of 4×4=16 combinations. Through reinforcement learning, PPO can be completed within 800 steps, with the reward function... , The Pareto front was automatically calibrated to 1.5:0.5:0.8.

[0055] Foldable Quantization Residual LoRA: Introduced to each layer of anomalous channels , Symmetric quantization and scaling factor are used. ; Maintain FP16. Rank r is searched synchronously by RL in {4, 8, 16, 32}. Inference phase. The weights are collapsed back to their original values ​​in one go, without adding any extra latency.

[0056] Online Quantization Error Compensator (OQEC): Calculates global quantization error per forward batch. ;like , , , This triggers local dequantization-microcorrection PDC: only the abnormal channels of the top-k tokens in this batch are backed up to FP16 calculation, while the rest remain at INT4 / INT8. Actual measurements show that the percentage of abnormal tokens is <1%, and the latency increase is <3%.

[0057] Joint optimization of hyperparameters (S4)

[0058] Unified loss: ,in For the meta-gradient error feedback loss, the MAML framework is used to update... .

[0059] DAQS Search: Differentiable Architecture-Quantization Joint Search. DAQS encodes the pruning ratio, distillation temperature, quantization bits, and QLoRA rank into a continuous vector, achieving convergence in less than 24 hours on a single A100 sheet.

[0060] Output and Deployment (S5)

[0061] Generates an ONNX format compressed model, containing:

[0062] The SparseConv operator accepts a CSR descriptor {val, ptr, idx, mask}.

[0063] Operators: Runtime Fusion ;

[0064] Operator: based on Dynamically select the FP16 / INT4 path.

[0065] The final file size is less than 6% of the original model, and it can be directly loaded into TensorRT, ONNXRuntime, or OpenVINO, achieving deployment with zero modifications.

[0066] VII. Experimental Results and Advantages

[0067] On ImageNet-1K, ViT-G after processing with this invention:

[0068] The weight was compressed from 3.6GB to 216MB (16.7×), and the Top-1 accuracy decreased by only 0.48%;

[0069] The Jetson Orin NX reduces single-frame latency from 590ms to 34ms (17.4×) and power consumption by 62%.

[0070] TensorRT achieves 92% core utilization in INT4 and zero additional memory copy for sparse convolution;

[0071] The entire process is automated and completed 24 hours a day, requiring no manual parameter adjustment or secondary calibration.

[0072] In summary, this invention, through a "three-stage cascaded, dual-ring optimization" architecture, achieves for the first time on a public dataset a compression ratio of >15×, an accuracy loss of <0.5%, and an inference acceleration of >15×, and also has one-click edge deployment capability, which is significantly superior to existing technologies.

[0073] This embodiment uses "compressing ViT-G / 14 on ImageNet-1K and deploying it to NVIDIA Jetson Orin NX" as a typical scenario to fully demonstrate the entire process of implementing this invention, from the original model with billions of parameters to the edge-side executable engine. All operations are completed on a general-purpose Linux server and Jetson edge device, requiring no dedicated hardware or deeply customized framework, making it valuable for direct reproduction and batch replication in engineering.

[0074] I. Environment and Data Preparation

[0075] The training side utilizes an 8×A100-80G GPU server running Ubuntu 22.04, with CUDA 12.2, cuDNN 8.9, PyTorch 2.1, and TensorRT 8.6 all installed via the official wheel. The dataset is the original ImageNet-1K training / val directory structure, containing 1.28 million training images and 50,000 validation images. The input resolution is uniformly 224×224. Preprocessing employs a standard combination of AutoAugment and RandAugment, maintaining normalized mean and variance. The total batch size during training is set to 1024, completed in four steps using mixed precision and gradient accumulation; the validation batch size is 256 to ensure statistical stability.

[0076] II. Pruning Stage of the Access Road

[0077] The first step is to load the official ViT-G / 14 pre-trained weights, with a total parameter count of 1.03B and approximately 330G FLOPs. To perform differentiable sparse gated pruning, a learnable scaling factor is added to the output channel dimension of each Linear or Conv layer. The default initial value is 1.0, and the temperature is... Initially 0.1, and The values ​​are set to 1e-4 and 1e-5 respectively. Training uses AdamW with a learning rate of 2e-3, cosine annealing for 30 epochs, warmup for 5 epochs, and gradient clipping. Every 500 steps, a lightweight Hessian trace estimator is used to compute on 2k random samples. Real-time updates of the pruning ratio for each layer The range is locked at 0.15–0.75. (Gating threshold) The Bayesian optimization was used to determine the optimal value in 30 iterations on the validation set, ultimately set to 0.012. This corresponds to pruning 64% of the channels, generating a structured sparse skeleton S, reducing FLOPs to 66G. The weights were derived using CSR triples plus a mask, reducing disk usage from 3.6GB to 1.4GB, achieving a sparsity of 78%. This allows direct use in the sparse convolution kernel of TensorRT 8.6 without additional transposition or rearrangement.

[0078] III. In the knowledge distillation stage, the teacher network retains the original ViT-G, while the student network directly loads the skeleton S and inserts an Adapter-Bypass branch in each Transformer Block. The branch is constructed by first reducing and then increasing the dimensionality of a 1×1 convolution and then adding SE attention; the number of parameters is less than 2% of the original block. The total training epochs are 60, the batch size is 2048 (gradient accumulation 4), the optimizer is replaced with Lamb, and the learning rate is 3e-3. The distillation loss includes four terms: logits KL, attention matrix MSE, hidden state cosine similarity, and channel-residual distillation. The margin is set to 0.2. Temperature T decays using a course-based method, starting at 6 and multiplied by exp(-0.005) every 1000 steps. Structural reparameterization is performed starting at epoch 55: when BatchNormrunning_var < 1e-5 and SEscale falls within [0.98, 1.02], the weight of that branch is merged into the main branch, achieving zero inference overhead. After merging, the student network has 0.63 bytes of parameters, and the Top-1 accuracy on the validation set is 90.05%, only 0.07% lower than the teacher's.

[0079] IV. Quantitative Compensation Phase

[0080] Post-training quantization employs mixed precision: 78% channel INT4 weights and 22% INT8 weights, with token-wise dynamic INT8 activations. Bit width, QLoRA rank, and compensation threshold are jointly searched using reinforcement learning PPO, resulting in 64 possible state space combinations and a reward function. It converges in 800 steps. Quantization residual LoRA introduces anomaly channels in each layer. , Stored in symmetric INT4 format. Maintaining FP16, the rank r is searched within {4, 8, 16, 32}. The online quantization error compensator OQEC is computed per batch. Dynamic threshold It decays linearly with step. , Abnormal tokens trigger local dequantization and microcorrection, with a rollback rate of <1% and a latency increase of <3%. After 20k steps of post-training, the model size is 216MB, and the validation set Top-189.64% is only 0.48% lower than the original teacher.

[0081] V. Hyperparameter Joint Optimization

[0082] A three-stage end-to-end hyperparameter update is implemented using the MAML framework. The inner loop has 5 steps, and the outer loop has 1 step, with learning rates of 1e-3 and 1e-4 respectively. The outer loop is executed every 10 steps. DAQS encodes the pruning ratio, distillation temperature, quantization bits, and QLoRA rank into continuous vectors. After differentiable relaxation, convergence is achieved within 24 hours on a single GPU. Visualization of the search trajectory shows that the Pareto front is significantly better than manual tuning.

[0083] VI. End-side Deployment and Online Monitoring

[0084] The final model is exported as ONNX, including SparseConv, , Three custom operators, file size 216MB. Compile the TensorRT engine using trtexec:

[0085] trtexec--onnx=compressed_vit_g.onnx--saveEngine=compressed_vit_g.trt--fp16--int4--sparsity=enable--workspace=8000

[0086] JetsonOrinNX real-world testing: batch=1, 224×224 input, single-frame latency 34ms, peak memory usage 1.3GB, power consumption 12W, temperature 48℃ (without fan). After deployment, latency, power consumption, and abnormal token ratio are reported every 1 minute via gRPC; if the abnormal token ratio > 5%, the background automatically performs OTA updates. This enables adaptive compensation.

[0087] Performance Comparison: On ImageNet-1K, the uncompressed ViT-G / 14 weights reached 3.6GB, with a Top-1 accuracy of 90.12%, a single-frame latency of 590ms on the JetsonOrinNX edge, and a peak power consumption of 31W. After the full-process compression of this invention, the weights were reduced to 216MB (16.7 times), the accuracy dropped to 89.64% (a loss of 0.48%), the edge latency was shortened to 34ms (17.4 times), and the power consumption dropped to 12W (2.6 times energy saving). Moreover, deployment can be completed with one click within 24 hours without manual parameter tuning, and all indicators have broken public records.

[0088] Through this implementation method, any technical team with a foundation in PyTorch, ONNX, and TensorRT can complete the entire process of reproducing a large visual model with hundreds of billions of parameters into an executable engine on the edge within 3 days, fully verifying the significant advantages of this invention in terms of compression ratio, accuracy preservation, and ease of deployment.

[0089] The foregoing has shown and described the basic principles, main features, and advantages of the present invention. Those skilled in the art should understand that the present invention is not limited to the above embodiments. The embodiments and descriptions in the specification are merely illustrative of the principles of the invention. Various changes and modifications can be made to the invention without departing from its spirit and scope, and all such changes and modifications fall within the scope of protection claimed by the present invention. The scope of protection of the present invention is defined by the appended claims and their equivalents.

Claims

1. A visual large model channel pruning-distillation-quantization compensation cascaded compression method, characterized in that, Comprising the following steps: S1 Channel Pruning Phase: S101 for each volume or linear layer weight of the input visual large model introducing trainable scaling factors and differentiable sparse gating , is a temperature hyperparameter; S102 constructing joint loss wherein is a task loss, is an entropy regularizer; S103calculating the dynamic sparse budget of each layer based on the Hessian trace estimation and the pruning ratio is constrained accordingly; S104 The training phase uses Straight-Through Estimator updates , the inference phase permanently removes the channel of , generates a structured sparse skeleton S and saves it in CSR format {val, ptr, idx, mask}; S2 knowledge distillation stage: S201 takes the original large model as the teacher T, and the sparse skeleton is the student , and a foldable Adapter-Bypass branch is inserted in each Transformer Block, which is composed of 1x1 convolution, SE module and residual connection. During training, the branch is activated, and during inference, it is folded through structural reparameterization. S202 constructing distillation loss where P is a learnable projection; S203 employs a course-style temperature schedule , ; S3 Quantization Compensation Phase: S301 Perform mixed-precision post-training quantization on the distilled student network: search INT4 / INT8 mixed bits layer by layer for weights, and dynamically INT8 for activations per token; S302 introduces foldable quantization residual LoRA for each layer of the abnormal channel: , , , the inference stage is fused according to ; S303 Deploying Online Quantization Error Compensator OQEC: per forward Batch computation , if then trigger Partial Dequantization-Micro Correction PDC, fallback to FP16 only for the k token channels with the largest errors; S304 employs reinforcement learning to search for bit-widths per layer and , reward function ; S4 super parameter joint optimization phase: S401 construct unified loss wherein is the meta-gradient error feedback loss, updated through the MAML framework ; S402 Use differentiable architecture-quantization joint search DAQS on a single GPU to complete optimal hyperparameter determination within ≤1000 iterations; S5 Output and deployment phase: S501 Generate a compressed model file containing structured sparse descriptors, quantization scale factors, QLoRA residuals, and OQEC threshold tables, which can be directly loaded into TensorRT, ONNXRuntime, or OpenVINO for execution.

2. The visual large model channel pruning-distillation-quantization compensation cascaded compression method according to claim 1, characterized in that: In step S104 The value range of is [0.005, 0.05], and is determined on the validation set by grid search or Bayesian optimization.

3. The visual large model channel pruning-distillation-quantization compensation cascaded compression method according to claim 1, characterized in that: The Adapter-Bypass branch folding condition in step S201 is: when the running_var of BatchNorm < 1e-5 and the scale parameter of the SE module.

4. The visual large model channel pruning-distillation-quantization compensation cascaded compression method according to claim 1, characterized in that: The MarginLoss in step S202 is specifically defined as wherein .

5. The visual large model channel pruning-distillation-quantization compensation cascaded compression method according to claim 1, characterized in that: The quantized residual LoRA rank r in step S302 is searched in the set {4, 8, 16, 32} by reinforcement learning, and With symmetric quantization, the scaling factor .

6. The visual large model channel pruning-distillation-quantization compensation cascaded compression method according to claim 1, characterized in that: OQEC threshold in step S303 is a dynamic value, , , .

7. The visual supermodel channel pruning-distillation-quantization compensation cascaded compression method according to claim 1, characterized in that: MAML inner loop learning rate in step S401 , outer loop learning rate , and outer loop update is performed every 10 steps.

8. The visual supermodel channel pruning-distillation-quantization compensation cascaded compression method according to claim 1, characterized in that: The compressed model file format output in step S501 is ONNX, and the additional custom operators include: SparseConv, used to perform CSR sparse convolution; , used to perform matrix multiplication with quantized residual LoRA; , used to trigger local dequantization-micro correction during inference.

Images