Dynamic mixed precision quantization method and system based on input perception and error propagation control

Abstract

The application discloses a dynamic mixed precision quantization method and system based on input perception and error propagation control, belongs to the field of artificial intelligence model compression and efficient calculation, and realizes dynamic adaptation of quantization precision by introducing an input perception mechanism, and simultaneously establishes a quantization error propagation model to guarantee numerical stability in the dynamic adjustment process. The quantization method comprises the following steps: constructing a multi-precision configurable basic model, so that each configurable layer of the neural network supports multiple bit precisions; designing a dynamic precision decision module of input perception, dynamically deciding the quantization precision of each layer according to input data characteristics; establishing a quantization error propagation and control model, modeling the propagation of quantization error in the network and designing a feedback control mechanism; constructing a multi-objective joint optimization loss function; performing two-stage collaborative training; online dynamic reasoning, dynamically adjusting the precision according to the input characteristics and monitoring the error propagation; and system deployment, deploying the complete system to a target hardware platform.

Description

Technical Field

[0001] This invention belongs to the field of artificial intelligence model compression and efficient computing, specifically involving a dynamic hybrid precision quantization method and system based on input perception and error propagation control. In particular, it is an intelligent quantization method and system that can adaptively adjust the quantization precision of each layer inside the model according to the content complexity of real-time input data, and achieve stability control through modeling quantization error propagation. Background Technology

[0002] With the widespread application of deep learning on edge computing devices, model quantization technology has become a key means to reduce model size and computational overhead. Hybrid precision quantization, by assigning different bit widths to different layers of the neural network, achieves a better balance between accuracy and efficiency compared to uniform precision quantization.

[0003] However, existing mixed-precision quantization methods have two main drawbacks: First, they are typically static, meaning they search for and fix a single precision configuration for the entire model, making it impossible to dynamically adjust based on the actual complexity of the input data. For example, in image recognition tasks, images with simple backgrounds and images with complex scenes have different precision requirements, and static configurations struggle to achieve optimal resource allocation. Second, existing methods lack systematic modeling and control of the propagation of quantization errors in neural networks. When dynamically adjusting the quantization precision of different layers, quantization errors generated in low-precision layers may accumulate and amplify in subsequent layers, leading to unstable output results.

[0004] While some current research focuses on dynamic neural networks, such as skipping layers or adjusting network width based on input, these techniques primarily address pruning or width adjustment and have not yet been applied to the specific scenario of mixed-precision quantization. Therefore, there is an urgent need in this field for a method that can dynamically adjust quantization precision based on input content and effectively control the propagation of quantization errors, in order to achieve smarter and more efficient edge model deployment. Summary of the Invention

[0005] To overcome the shortcomings of existing technologies, this invention provides a dynamic hybrid precision quantization method and system based on input sensing and error propagation control. By introducing an input sensing mechanism, the quantization precision is dynamically adapted, while a quantization error propagation model is established to ensure numerical stability during the dynamic adjustment process.

[0006] To achieve the above objectives, the specific technical solution adopted by the present invention is as follows: On the one hand, this invention provides a dynamic hybrid precision quantization method based on input sensing and error propagation control, comprising the following steps: Step S100: Construct a multi-precision configurable base model so that each configurable layer of the neural network supports multiple bit precisions; Step S200: Design a dynamic precision decision module for input perception to dynamically determine the quantization precision of each layer based on the characteristics of the input data; Step S300: Establish a quantization error propagation and control model, model the propagation of quantization error in the network, and design a feedback control mechanism. Step S400: Construct a multi-objective joint optimization loss function to simultaneously optimize task accuracy, computational efficiency, and error control; Step S500: Perform two-stage collaborative training, first training the basic model and then jointly optimizing the decision and control modules; Step S600: Online dynamic inference, dynamically adjusting accuracy based on input features and monitoring error propagation; Step S700, System Deployment: Deploy the complete system to the target hardware platform.

[0007] Furthermore, in step S100, a parameter sharing mechanism is adopted, whereby each layer maintains a set of basic weights and adaptation parameters for different precisions. The adaptation parameters are implemented through low-rank decomposition to reduce storage overhead.

[0008] Furthermore, the dynamic precision decision module in step S200 includes a lightweight feature extractor and a decision network. The feature extractor extracts feature vectors from the input data, and the decision network generates precision selection probabilities for each layer based on these features.

[0009] Furthermore, the error propagation model in step S300 quantizes the propagation process of quantization noise in the neural network, calculates the error propagation gain through the chain product of Jacobian matrices, and designs a feedback control mechanism based on this. The feedback control mechanism monitors the quantization noise level of each layer during the inference process, and triggers accuracy enhancement or error compensation operations when the noise exceeds a preset threshold.

[0010] Furthermore, the loss function in step S400 includes task loss, efficiency reward, error propagation penalty, and decision smoothness regularization, which are weighted to balance the various optimization objectives. The total loss function is defined as: , in, For mission losses, Rewards for efficiency This is a penalty term for error propagation. This is a decision smoothness regularization term to avoid frequent switching. , , This represents the regularization strength.

[0011] Furthermore, step S500 employs a two-stage training strategy: the first stage fixes the decision module, trains the multi-precision base model, and calibrates the noise gain coefficients of each layer. and estimation The second stage fixes the base model parameters, jointly optimizes the decision network and feedback controller parameters, and uses the reinforcement learning policy gradient method: , in, This is the baseline value, used to reduce variance.

[0012] Furthermore, in step S600, firstly, features are extracted for each input. Precision configuration is obtained through decision networks. Then, inference is performed using a feedforward-feedback control method, with the feedforward control following the established procedures. Configure the precision of each layer for forward calculation, monitor the noise of each layer, and trigger the correction mechanism if the noise exceeds the threshold; finally, output the final result.

[0013] On the other hand, the present invention provides a dynamic mixed-precision quantization system based on input sensing and error propagation control, which executes the above-described dynamic mixed-precision quantization method based on input sensing and error propagation control, including: The model loading and initialization module is used to load and initialize neural network models that support multi-precision switching; The input feature extraction module is used to extract feature vectors from the input data; The dynamic precision decision module is used to generate precision configurations based on input features; The multi-precision computing scheduling module is used to schedule computing cores of different precisions according to the precision configuration. The error monitoring and feedback control module is used to monitor quantization error and perform feedback control. The dynamic inference engine module is used to perform online dynamic inference; The results output module is used to output the final reasoning results.

[0014] Furthermore, the error monitoring and feedback control module includes a noise estimation unit and a feedback control unit. The noise estimation unit calculates the quantized noise level of each layer in real time, and the feedback control unit triggers control actions based on the noise level.

[0015] The present invention also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the above-described dynamic hybrid precision quantization method based on input sensing and error propagation control.

[0016] Compared with the prior art, the present invention has the following advantages: This invention dynamically adjusts the quantization precision of each layer based on input data characteristics, while ensuring numerical stability through error propagation control, enabling smarter and more efficient model deployment on edge devices. It is the first to implement a hybrid precision quantization method that dynamically adjusts quantization precision based on input content, better adapting to diverse real-world inputs. By modeling and controlling quantization error propagation, numerical stability is ensured during the dynamic adjustment process. The dynamic precision decision module of this invention is extremely lightweight, introducing minimal overhead, making it suitable for edge device deployment. Under the same precision requirements, it can further improve efficiency compared to static hybrid precision quantization, or achieve higher precision at the same efficiency. Attached Figure Description

[0017] Figure 1 This is a flowchart of the method of the present invention; Figure 2 This is a system module diagram of the present invention. Detailed Implementation

[0018] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and specific embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention.

[0019] Example 1 like Figure 1 As shown, this embodiment provides a dynamic hybrid precision quantization method based on input sensing and error propagation control, including the following steps: Step S100: Construct a multi-precision configurable base model so that each configurable layer of the neural network supports multiple bit precisions.

[0020] Perform quantization-aware training on the pre-trained floating-point neural network model, making each configurable layer... Supports a predefined set of candidate bit precision To reduce storage overhead, a parameter sharing mechanism is adopted: each layer maintains a set of basic weights. And for different levels of precision Adaptation parameters The actual weights of the layers are represented as follows: , Among them, the adaptation parameters This is achieved through low-rank decomposition. ; and As a shared factor, A scaling vector specific to the required precision.

[0021] Step S200: Design a dynamic precision decision module for input perception, which dynamically determines the quantization precision of each layer based on the characteristics of the input data.

[0022] The dynamic precision decision module consists of a lightweight feature extractor and a decision network. The feature extractor extracts data from the input data... Extract compact feature vectors : , For image input, It can be the first few layers of a network or a dedicated lightweight network, or a decision network. As input, output the precision selection probability for each layer: , in, The temperature parameter is gradually decreased during training to make the distribution more discrete.

[0023] Step S300: Establish a quantization error propagation and control model, model the propagation of quantization error in the network, and design a feedback control mechanism.

[0024] Quantitative Operation Introduced error It will spread across the network. (Definition of the first...) Layer in precision The quantization noise variance is: , in, For the first Variance of layer activation values The noise gain coefficient, which is related to accuracy, can be obtained through offline calibration.

[0025] Error propagation is modeled using the network's Jacobian matrix. From the... layer to the first layer( > The error propagation gain of ) is: , in, For the first The Jacobian matrix of the layer. The final total noise variance of the output is approximately: , in, For the first The actual precision of the layer selection This represents the total number of floors.

[0026] Based on this model, a feedback control mechanism is designed: during the inference process, the activation variance of each layer is monitored, and if the estimated noise exceeds a threshold... This will trigger precision enhancement or error compensation: , in, and The parameter is learnable, and the operation is differentiable during backpropagation.

[0027] Step S400: Construct a multi-objective joint optimization loss function to simultaneously optimize task accuracy, computational efficiency, and error control.

[0028] Define the total loss function as: , in, For mission losses, Rewards for efficiency This is a penalty term for error propagation. This is a decision smoothness regularization term to avoid frequent switching. , , This represents the regularization strength.

[0029] Step S500: Perform two-stage collaborative training, first training the basic model and then jointly optimizing the decision and control modules.

[0030] Phase 1: Fix the decision-making module, train the multi-precision base model, and calibrate the noise gain coefficients of each layer. and estimation .

[0031] Phase Two: Fix the base model parameters and jointly optimize the parameters of the decision network and feedback controller. This is done using the reinforcement learning policy gradient method. , in, This is the baseline value, used to reduce variance.

[0032] Step S600: Online dynamic inference, dynamically adjusts the accuracy based on input features and monitors error propagation.

[0033] For each input The steps are as follows: ① Feature extraction ; ② Precision configuration is obtained through decision networks ; ③ Inference is executed using a feedforward-feedback control method, with the feedforward control following the established procedures. Configure the accuracy of each layer for forward calculation, monitor the noise of each layer, and trigger the correction mechanism if the threshold is exceeded. ④ Output the final result.

[0034] Step S700, System Deployment: Deploy the complete system to the target hardware platform.

[0035] The trained model, decision network, and feedback controller are deployed to the target hardware platform. The hardware's multi-precision computing capabilities are leveraged to achieve dynamic switching of runtime precision.

[0036] Example 2 The dynamic hybrid precision quantization method based on input perception and error propagation control provided in this embodiment takes the MobileNetV2 image classification model deployed on a mobile device as an example.

[0037] Step S100: Construct a multi-precision configurable base model. Select MobileNetV2 as the base network, and set all its inverse residual blocks as configurable layers, totaling 16 configurable units. Define the candidate precision set. , A parameter-sharing mechanism is adopted: each convolutional layer maintains basic weights. The system also includes adaptation parameters for three different precision levels. These adaptation parameters are implemented using a low-rank decomposition with rank 2. Quantization-aware training is performed for 120 rounds on the ImageNet dataset. In each round, a precision level is randomly selected for each layer, and the corresponding weights are used for calculation and gradient updates.

[0038] Step S200: Design the input-aware dynamic accuracy decision module: First, the feature extractor uses the first two convolutional layers (stem) of MobileNetV2 as the feature extractor. After the input image passes through these two layers, the channel mean and standard deviation of the output feature map are calculated to obtain a 64-dimensional feature vector. Next, a decision network is established, employing a two-layer fully connected network with 64-dimensional input, 32-dimensional hidden layers, and a 16×3-dimensional output (16 layers, each with 3 levels of logits). Differentiable sampling is achieved using the Gumbel-Softmax technique, with the initial temperature... After each round of training, the product is multiplied by 0.95 and then annealed.

[0039] Step S300: Establish a quantization error propagation and control model: Offline calibration: On the ImageNet validation set, calculate the noise gain coefficient for each layer and for each level of accuracy. Simultaneously, the error propagation gain between layers is calculated through automatic differentiation. Approximate value.

[0040] Feedback controller design: Setting noise thresholds for each layer During inference, the activation variance of each layer is estimated in real time. If so, the accuracy of that layer will be increased by one level or error compensation will be applied.

[0041] Step S400, Construct a multi-objective joint optimization loss function: Set the weights of the loss function: (Efficiency Rewards) (Error penalty) (Smoothness regularization). The efficiency reward score is set as follows: , , This reflects the relative speed of different precision levels on the target hardware.

[0042] Step S500: Perform two-stage collaborative training. First stage: Train the base model for 50 rounds, fixing the decision network as uniform selection, and labeling all... and Phase Two: Fix the base model weights and train the decision network and feedback controller for 30 epochs. Update the decision network parameters using the PPO algorithm, with a learning rate of [missing information]. .

[0043] Step S600, Online Dynamic Inference: Deployed on the Snapdragon 865 mobile platform. For each input image: Features are extracted through the first two convolutional layers (approximately 1.2ms). The decision network calculates the precision configuration (approximately 0.3ms). Dynamic inference is performed according to the configuration, while monitoring noise in each layer. If feedback control is triggered, the affected layers are recalculated.

[0044] Experimental results show that, compared with static 8-bit quantization, the method of this invention improves the average inference speed by about 40% while maintaining the same accuracy (Top-1 accuracy decrease of <0.5%). Compared with static mixed-precision quantization (fixed configuration), the speed is improved by about 15% at the same accuracy.

[0045] Step S700, System Implementation: The system includes the following modules: the first is the model loading and initialization module, the second is the input feature extraction module, the third is the dynamic precision decision module, the fourth is the multi-precision calculation scheduling module, the fifth is the dynamic inference engine module, the sixth is the error monitoring and feedback control module, and the last is the result output module.

[0046] Example 3

[0047] like Figure 2 As shown, this embodiment provides a dynamic hybrid precision quantization system based on input sensing and error propagation control, which executes the quantization methods in embodiments 1 and 2. It includes the following modules: Model loading and initialization module: Responsible for executing step S100, loading the pre-trained floating-point neural network model, and establishing a multi-precision parameter structure (including basic weights) for each configurable layer. and shared factors ), and initialize the candidate bit precision set. and related quantitative parameters.

[0048] Input feature extraction module: Responsible for performing the feature extraction part in step S200, using a lightweight feature extractor. From input data Real-time computation of compact feature vectors This provides a basis for dynamic and precise decision-making.

[0049] Dynamic precision decision module: responsible for executing the decision-making part in step S200 and the online decision-making in step S600, using a pre-trained decision network. Based on input features Generate each layer Precision selection probability distribution It outputs a discretized precision configuration vector. And the decision is optimized by combining the efficiency reward mechanism in step S400.

[0050] Multi-precision computation scheduling module: responsible for executing the dynamic inference scheduling of step S600, based on the precision configuration. Dynamically load the corresponding precision adaptation parameters at runtime. The system schedules the corresponding precision computing cores on the target hardware to perform calculations at each level, enabling real-time switching of precision configurations.

[0051] Dynamic Inference Engine Module: Responsible for executing online dynamic inference in step S600, integrating all parameters and models trained in step S500, and coordinating the collaborative work of various modules.

[0052] Error monitoring and feedback control module: Responsible for executing the error control parts in steps S300 and S600, and monitoring the activation variance and quantization noise level of each layer in real time during inference. When it exceeds the threshold At that time, a precision enhancement command is triggered or a correction operation is activated to ensure numerical stability.

[0053] The result output module is responsible for executing the final output of step S600, integrating the processing results of each module, outputting the final prediction result of the network, and optionally caching the accuracy configuration decision for reuse by subsequent similar inputs.

[0054] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A dynamic hybrid precision quantization method based on input sensing and error propagation control, characterized in that, Includes the following steps: Step S100: Construct a multi-precision configurable base model so that each configurable layer of the neural network supports multiple bit precisions; Step S200: Design a dynamic precision decision module for input perception to dynamically determine the quantization precision of each layer based on the characteristics of the input data; Step S300: Establish a quantization error propagation and control model, model the propagation of quantization error in the network, and design a feedback control mechanism. Step S400: Construct a multi-objective joint optimization loss function to simultaneously optimize task accuracy, computational efficiency, and error control; Step S500: Perform two-stage collaborative training, first training the basic model and then jointly optimizing the decision and control modules; Step S600: Online dynamic inference, dynamically adjusting accuracy based on input features and monitoring error propagation; Step S700, System Deployment: Deploy the complete system to the target hardware platform.

2. The dynamic hybrid precision quantization method based on input sensing and error propagation control according to claim 1, characterized in that, In step S100, a parameter sharing mechanism is adopted. Each layer maintains a set of basic weights and adaptation parameters for different precisions. The adaptation parameters are implemented through low-rank decomposition to reduce storage overhead.

3. The dynamic hybrid precision quantization method based on input sensing and error propagation control according to claim 1, characterized in that, The dynamic precision decision module in step S200 includes a lightweight feature extractor and a decision network. The feature extractor extracts feature vectors from the input data, and the decision network generates precision selection probabilities for each layer based on these features.

4. The dynamic hybrid precision quantization method based on input sensing and error propagation control according to claim 1, characterized in that, The error propagation model in step S300 quantizes the propagation process of quantization noise in the neural network, calculates the error propagation gain through the chain product of Jacobian matrices, and designs a feedback control mechanism based on this. The feedback control mechanism monitors the quantization noise level of each layer during the inference process. When the noise exceeds a preset threshold, it triggers an accuracy enhancement or error compensation operation.

5. The dynamic hybrid precision quantization method based on input sensing and error propagation control according to claim 1, characterized in that, The loss function in step S400 includes task loss, efficiency reward, error propagation penalty, and decision smoothness regularization, which balances the optimization objectives through weight coefficients.

6. The dynamic hybrid precision quantization method based on input sensing and error propagation control according to claim 1, characterized in that, Step S500 adopts a two-stage training strategy: the first stage fixes the decision module, trains the multi-precision basic model and calibrates and estimates the noise gain coefficients of each layer; The second stage involves fixing the basic model parameters and jointly optimizing the decision network and feedback controller parameters.

7. The dynamic hybrid precision quantization method based on input sensing and error propagation control according to claim 1, characterized in that, In step S600, firstly, features are extracted for each input, and the accuracy configuration is obtained through a decision network; then, inference is performed using a feedforward-feedback control method, with the feedforward performing forward calculations according to the configured accuracy of each layer, and the feedback monitoring of noise in each layer. If the noise exceeds the threshold, a correction mechanism is triggered; finally, the final result is output.

8. A dynamic hybrid precision quantization system based on input sensing and error propagation control, executing the dynamic hybrid precision quantization method based on input sensing and error propagation control as described in any one of claims 1-7, characterized in that, include: The model loading and initialization module is used to load and initialize neural network models that support multi-precision switching; The input feature extraction module is used to extract feature vectors from the input data; The dynamic precision decision module is used to generate precision configurations based on input features; The multi-precision computing scheduling module is used to schedule computing cores of different precisions according to the precision configuration. The error monitoring and feedback control module is used to monitor quantization error and perform feedback control. The dynamic inference engine module is used to perform online dynamic inference; The results output module is used to output the final reasoning results.

9. The dynamic hybrid precision quantization system based on input sensing and error propagation control according to claim 8, characterized in that, The error monitoring and feedback control module includes a noise estimation unit and a feedback control unit. The noise estimation unit calculates the quantized noise level of each layer in real time, and the feedback control unit triggers control actions based on the noise level.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the dynamic hybrid precision quantization method based on input sensing and error propagation control as described in any one of claims 1-7.

Images