Bit width self-adaptive mixing precision quantification method and system

Abstract

The invention provides a bit-width-adaptive mixed precision quantification method and system. The method comprises the steps that quantizers with variable bit widths are configured for all modules in a model respectively; setting a loss function and an upper bound and a lower bound of each module; according to the comparison condition of the numerical value of the loss function and the upper bound and the lower bound, the bit width of the corresponding quantizer is automatically adjusted module by module, and a mixed precision model is obtained. According to the method, a self-adaptive precision allocation technical scheme based on module characteristic difference is adopted, adaptive quantization bit widths are accurately allocated for different modules by dynamically sensing the importance, numerical distribution and quantization disturbance resistance of each sub-module in a network structure, and the adaptive quantization bit widths are accurately allocated for different modules on the premise that the overall computing resource overhead is not remarkably increased. And the compression potential of the model is fully released.

Description

Technical Field

[0001] The present application relates to the field of neural network compression and acceleration technology, and in particular to a mixed-precision quantization method and system with adaptive bit width. Background Art

[0002] With the rapid development of artificial intelligence (AI) technology, the need to deploy deep learning in resource-constrained environments, such as mobile terminals and edge computing devices, is becoming increasingly urgent. These devices generally suffer from limited computing resources, small memory capacity, and energy constraints, making it difficult to directly run traditional deep learning models. Therefore, technologies such as post-training quantization (PTQ) and quantization-aware training (QAT) have become key solutions. They significantly reduce the hardware resource requirements of low-bit quantized neural networks while maintaining model accuracy, making them suitable for deployment on resource-constrained devices. However, in practical applications, several key bottlenecks remain. Among them, the varying quantization complexity of various neural network modules is a core issue that needs to be addressed urgently. Due to the significant differences in the structural characteristics and numerical distribution of modules such as convolutional layers, attention layers, and normalization layers, a uniform bit width setting makes it difficult to balance overall model accuracy and hardware efficiency. Some key modules are highly sensitive to accuracy, and low-bit quantization can easily lead to significant performance degradation.

[0003] On this basis, other issues have further constrained the development of low-bit quantized neural networks. For example, most current mixed-precision quantization methods focus only on weight quantization and lack a joint optimization mechanism for the simultaneous quantization of weights and activations, resulting in the inability to fully unleash the model compression potential. For another example, under a multi-layer neural network structure, the mixed-precision search space grows exponentially, while existing methods mostly rely on heuristic strategies or fixed templates, lack efficient and general automated search solutions, and are difficult to adapt to complex networks and task requirements. In addition, in practical applications, the optimization effect of mixed-precision quantization is not significant, and in some scenarios it is even inferior to the unified precision quantization solution, which hinders the widespread application of this technology.

[0004] Upon inquiry, patent publication number WO2013010437A1 discloses a data compression technology that involves quantizing an input signal based on adaptive quantization information at the transmitter and compressing the quantized input information. However, this quantization model has not yet achieved ideal levels of accuracy and performance. Summary of the Invention

[0005] In view of one of the defects in the prior art, the purpose of this application is to provide a mixed-precision quantization method and system with adaptive bit width.

[0006] In a first aspect of the present application, a mixed-precision quantization method with adaptive bit width is provided, comprising:

[0007] Configuring a quantizer with a variable bit width for each module in the model; wherein the model is any neural network model;

[0008] Set the loss function and its upper and lower bounds for each module;

[0009] According to the comparison between the value of the loss function and the upper bound and the lower bound, the bit width of the corresponding quantizer is automatically adjusted module by module to obtain a mixed precision model.

[0010] Optionally, the quantizer is provided with a default initial bit width;

[0011] The default initial bit width of the quantizer corresponding to all modules of the entire model is the same;

[0012] The default initial bit width is an expected quantization bit width, or a quantization bit width corresponding to an expected compression ratio or speedup ratio.

[0013] Optionally, setting the loss function of each module and its upper and lower bounds includes:

[0014] The loss function is designed to be a combined loss function consisting of the output loss of the current module and the corresponding full-precision module, the overall output of the final quantization module, and the loss of the full-precision output;

[0015] Specify upper and lower bounds on the loss function based on the expected number of quantization bits.

[0016] Optionally, automatically adjusting the bit width of the corresponding quantizer module by module based on the comparison between the value of the loss function and the upper bound and the lower bound to obtain a mixed precision model includes:

[0017] All modules are operated one by one, and the following comparisons and adjustments are made to each module:

[0018] If the loss function stabilizes between the upper and lower bounds within the specified maximum number of steps, save the current bit width setting and continue training the module parameters.

[0019] If the loss function stabilizes below the lower bound within the specified maximum number of steps, the current bit width is increased and the calibration is restarted after returning to the initial state.

[0020] If the loss function stabilizes above the upper bound within the specified maximum number of steps, the current bit width is lowered and the calibration is restarted after returning to the initial state.

[0021] Optionally, the continued training module parameters include:

[0022] After determining the current bit width, specify a training step number as the condition for training end;

[0023] The number of training steps is 5 to 10 times the number of steps used to determine the current bit width.

[0024] Optionally, the module-by-module method refers to performing quantitative calibration in an order opposite to that of the reasoning process.

[0025] Optionally, an expansion phase with a certain number of steps is added after the quantization calibration of the last module is completed, that is, all quantization modules are updated using the overall loss function.

[0026] A second aspect of the present application provides a mixed-precision quantization system with adaptive bit width, comprising:

[0027] Quantizer module: configures a quantizer with variable bit width for each module in the model;

[0028] Loss setting module: sets the loss function and its upper and lower bounds for each module;

[0029] Quantization calibration module: According to the comparison between the value of the loss function and the upper and lower bounds, the bit width of the corresponding quantizer is automatically adjusted module by module to obtain a mixed precision model.

[0030] The third method of the present application provides an image reconstruction method, comprising:

[0031] Acquiring images to be reconstructed;

[0032] Inputting the image to be reconstructed into an image reconstruction model, wherein the image reconstruction model is a mixed precision model quantized by any one of the methods or systems;

[0033] Obtain the reconstructed image.

[0034] A fourth aspect of the present application provides an edge computing method, including:

[0035] The edge device extracts the edge signal;

[0036] Extracting features of the edge signal using a feature extraction model, wherein the feature extraction model is a mixed precision model quantized by any one of the methods or systems;

[0037] The extracted features are passed to the cloud for federated training.

[0038] The adaptive bit-width hybrid precision quantization method provided in this application adopts an adaptive precision allocation technical solution based on module characteristic differences. It dynamically perceives the importance, numerical distribution and anti-quantization disturbance capability of each sub-module in the network structure through the loss function, and accurately allocates adaptive quantization bit widths to different modules, bringing about the technical effect of fully releasing the model compression potential without significantly increasing the overall computing resource overhead.

[0039] Other technical effects brought about by the additional features will be further explained in the corresponding embodiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] Other features, objects and advantages of the present application will become more apparent upon reading the detailed description of non-limiting embodiments with reference to the following drawings:

[0041] Figure 1 4 is a flowchart of a mixed-precision quantization method with adaptive bit width according to an exemplary embodiment;

[0042] Figure 2 1 is a schematic diagram of a framework structure of a mixed-precision quantization method with adaptive bit width according to an exemplary embodiment;

[0043] Figure 3 The figure is a structural diagram of a mixed-precision quantization system with adaptive bit width according to an exemplary embodiment. DETAILED DESCRIPTION

[0044] The present application is described in detail below with reference to specific embodiments. The following examples will help those skilled in the art to further understand the present application, but are not intended to limit the present application in any form. It should be noted that, without departing from the concept of the present application, a number of variations and improvements may be made by those skilled in the art, and these all fall within the scope of protection of the present application. Parts not described in detail in the following examples may be implemented using existing technologies.

[0045] In the prior art, different modules in a neural network (such as convolutional layers, attention layers, normalization layers, etc.) have significant differences in structural characteristics and numerical distribution, which makes it difficult for a uniformly set bit width to take into account both the overall accuracy of the model and hardware efficiency. Some key modules are more sensitive to precision, and low-bit quantization may cause significant performance degradation. Based on the above problems, an embodiment of the present application provides a mixed-precision quantization method with adaptive bit width to solve the above problems.

[0046] Reference Figure 1 and Figure 2 As shown, the present application provides an adaptive bit width mixed precision quantization method, comprising:

[0047] Step 1: Configure a quantizer with variable bit width for each module in the model. Wherein, the model is any neural network model;

[0048] Specifically, the quantizer is an adaptive mixed-precision quantizer based on module-specific differences. It adaptively adjusts the quantization bit width of each module based on its varying sensitivity to precision, achieving more refined quantization control. This quantizer enables the entire method to maintain the overall average bit rate while minimizing precision loss, effectively overcoming the difficulty of existing technologies adapting to differences in quantization complexity between modules due to the use of fixed bit widths.

[0049] Step 2: Set the loss function of each module, as well as its upper and lower bounds;

[0050] Step 3: Based on the comparison between the value of the loss function and the upper and lower bounds, the bit width of the corresponding quantizer is automatically adjusted module by module to obtain a mixed precision model.

[0051] Specifically, during the quantization calibration process, the loss function acts as a core "regulator," dynamically adjusting the quantization bit width based on the performance gap between each module after quantization and the full-precision model. Specifically, this dynamic adjustment mechanism closely revolves around the importance of network structure, numerical distribution, and resistance to quantization perturbations.

[0052] In terms of importance measurement, after integrating the initialized current quantization module into the overall model, the loss function value delta between the overall output of the partial quantization model and the overall output of the full-precision model is calculated. The increase in delta compared to before the addition intuitively reflects the impact of the module on the overall model performance. The greater the increase, the more critical the module is, and more caution is required during quantization to prevent excessive quantization from causing a significant performance degradation.

[0053] In terms of the ability to resist quantization disturbances, on the one hand, the loss function value of the initialized quantization module and the corresponding full-precision module directly reflects the sensitivity of the module to quantization operations in the initial state; on the other hand, under the current bit width setting, the decrease in the loss function after a specified number of training steps further reveals the module's ability to adapt to quantization during training. The greater the decrease, the stronger the module's ability to resist quantization disturbances, and the greater the flexibility in adjusting the quantization bit width.

[0054] Numerical distribution plays a key role in both initialization and training. During initialization, quantization parameters are determined based on the maximum and minimum values ​​of the data distribution, laying the foundation for the quantization process. During training, quantization parameters are continuously captured to adapt to the current model weight and activation distribution, ensuring that the quantization scheme matches the actual data characteristics of the model. This allows quantization bit width adjustments to meet compression and acceleration requirements while maximizing model performance.

[0055] The above-mentioned embodiment of the present application proposes an adaptive precision allocation mechanism based on module characteristic differences. By dynamically sensing the importance, numerical distribution and anti-quantization disturbance capability of each sub-module in the network structure, the most suitable quantization bit width is allocated to it, thereby fully releasing the model compression potential without significantly increasing the overall computing resource overhead.

[0056] In the early stages of training, a default initial bit width needs to be set for the quantizer. In some specific implementations of the present application, the default bit width of the entire model is set to the same, which can be set to the expected quantization bit width or the quantization bit width corresponding to the expected compression rate or speedup ratio.

[0057] Specifically, there are three ways to set the default initial bit width: one is to directly set it to the expected quantization bit width, that is, the target bit width predetermined before the quantization operation. For example, if the model weights and activation values ​​are quantized to 8 bits, 8 bits is the default initial bit width; the second is to determine it based on the expected compression rate. Since different quantization bit widths correspond to different compression rates, if the expected compression rate reaches 90%, 4-bit quantization can meet the requirements after analysis, and 4 bits becomes the default initial bit width; the third is to set it based on the expected speedup ratio. Taking into account the different acceleration effects brought by different quantization bit widths on specific hardware, if quantizing the model parameters from 32 bits to 16 bits on a certain GPU can increase the inference speed by 2 times, 16 bits will be used as the default initial bit width corresponding to the expected speedup ratio.

[0058] The above-mentioned embodiment of the present application sets a default initial bit width for the quantizer, provides a unified and clear starting point for model quantization, greatly simplifies the initial setting process, and avoids confusion.

[0059] In order to enable the calibration of the model to achieve global optimization, in some specific implementations of the present application, the loss function is designed as a combined loss function consisting of the output loss of the current module and the corresponding full-precision module, the overall output of the final quantization module, and the loss of the full-precision output.

[0060] In other words, this loss function adopts an accuracy-preserving training strategy that integrates knowledge distillation. By introducing the full-precision teacher model as the learning target, it guides the quantized student model to approach the teacher output during training, effectively alleviating the performance degradation caused by quantization, and further improving the model's expressiveness and generalization capabilities, thereby solving the problem of "difficulty recovering model performance degradation after quantization."

[0061] In addition, the upper and lower bounds of the loss function are also set accordingly. The upper and lower bounds of the loss function are specified based on the expected number of quantization bits.

[0062] Specifically, during the mixed-precision quantization process, the model will be compressed. The average quantization bit width corresponding to the single-precision quantization condition can be calculated based on the compression ratio. The compression ratio of the quantized model can be set in advance, so the expected quantization bit width required can be inferred. During the initialization phase, the expected quantization bit width is used for quantization initialization for each module (i.e., the activation distribution and model weight distribution are obtained using the input data of the calibration set). After a complete model inference is completed using all the inputs of the calibration set, the initialization of all modules can be completed. After initialization, each quantization module has a quantization error value z relative to its corresponding full-precision module under a specific loss function. Based on this quantization error z, the upper and lower bounds of the expected loss function can be set for each module. Therefore, although the upper and lower bounds of each module are different, they follow a unified setting rule.

[0063] For example, the loss function of each module can be uniformly set to the interval [0.1z, 0.2z].

[0064] In an embodiment of the present application, the upper and lower bounds of the loss function are uniformly set for the adaptable quantizer to define its adaptability range. Among them, the role of the lower bound is to limit the module from excessively using high quantization bit widths to avoid unnecessary waste of resources; the upper bound can prevent the module from using too low a quantization bit width, while ensuring that the overall performance of the model is not damaged. It is worth mentioning that the loss function used here is a merging function, which cleverly combines the considerations of the effects of the current module and the overall model. It not only focuses on the difference between the quantized output of a single module and the output of the corresponding full-precision module, but also takes into account the gap between the overall output of the last quantized module and the overall output of the full-precision model, so that the quantization calibration process is more focused on the overall optimal solution, so that the model can still maintain good performance under resource constraints.

[0065] In order to obtain a mixed-precision model that ensures efficiency and compression rate, in some specific embodiments of this application, step 3 automatically adjusts the bit width of the corresponding quantizer module by module based on the comparison between the value of the loss function and the upper and lower bounds to obtain a mixed-precision model. This can be achieved by using the quantizer to automatically adjust the quantization bit width based on the decrease in the loss function during the quantization calibration process. There are three cases:

[0066] If the loss function stabilizes between the upper and lower bounds within the specified maximum number of steps, the current bit width setting is saved and training continues.

[0067] Specifically, the maximum number of steps specified above is used to determine the bit width. To ensure sufficient training, continue training after the bit width is determined to ensure full optimization of module parameters. Training can be terminated by specifying a new maximum number of steps, which can be 5 to 10 times the number of steps used to determine the bit width. These maximum number of steps are hyperparameters and can be customized.

[0068] If the loss function stabilizes below the lower bound within the specified maximum number of steps, the current bit width is increased and the calibration is restarted after returning to the initial state.

[0069] Specifically, the initial state refers to the initialization phase mentioned above, in which each module is initialized using the expected quantization bit width (i.e., the activation distribution and model weight distribution are obtained using the input data of the calibration set).

[0070] If the loss function stabilizes above the upper bound within the specified maximum number of steps, the current bit width is reduced and the calibration is restarted after returning to the initial state.

[0071] It's worth noting that even after specifying the maximum number of steps, each module still needs to be fully calibrated to ensure overall model performance. In other words, after the calibration training of the previous module is completed, the gradient used to update the training of the next module must also be used to update all previously quantized modules to ensure overall model performance.

[0072] Unlike the existing module-by-module calibration strategy, in some specific implementations of the present application, the same sequential quantization calibration as the inference order is not adopted. Instead, a reverse quantization calibration is used to ensure that the image generated by the quantization model can best represent the performance of the quantization model, making the back propagation of the loss function more effective.

[0073] The above-mentioned embodiment of the present application adopts a reverse module-by-module quantization calibration and adaptive precision search mechanism, which can gradually reverse the quantization process from the output end and dynamically adjust the precision based on the impact of the module output on the final performance. This makes the quantization process closer to the global optimal solution, enhances the model's robustness to quantization errors, and solves the problems of "quantization error accumulation and module-by-module calibration deviating from the global optimal solution" in existing methods;

[0074] Due to the module-by-module quantization and calibration method, the modules that are quantized later are at the end of the calibration order, resulting in significantly fewer calibration steps than the modules that are quantized first. This puts the risk of insufficient calibration of the post-quantization modules. Once insufficiently calibrated, these modules may have problems such as inaccurate image feature extraction and recognition logic judgment deviation when processing data, thereby affecting the recognition accuracy and stability of the entire model. To this end, in some specific implementation methods of the present application, an extended quantization calibration stage is added after the module-by-module quantization stage. When the last module completes the quantization calibration, an additional calibration cycle will be started, a certain number of calibration steps will be set, and the overall loss function will be used to update and optimize all quantized modules.

[0075] The implementation of this application adds an additional extended quantization calibration phase, allowing post-quantization modules more calibration opportunities, compensating for the lack of early calibration steps and ensuring that each module is deeply adapted to quantization operations. This allows the entire model to collaborate more smoothly and efficiently when processing image data, ultimately significantly improving the model's recognition accuracy and operational stability on devices such as mobile phones, providing users with a better user experience.

[0076] Based on the same technical concept, in some specific implementations of the present application, such as Figure 3 As shown, an adaptive bit-width mixed-precision quantization system 100 includes:

[0077] Quantizer module 110: configures a quantizer with variable bit width for each module in the model;

[0078] Loss setting module 120: sets the upper and lower bounds of the loss function of each module;

[0079] Quantization calibration module 130: According to the comparison between the value of the loss function and the upper and lower bounds, the bit width of the corresponding quantizer is automatically adjusted module by module to obtain a mixed precision model.

[0080] The specific implementation techniques of each module / unit in the above examples of this application can refer to the corresponding steps of the mixed precision quantization method with adaptive bit width in the above embodiments, which will not be repeated here.

[0081] The preferred features of the above embodiments can be used alone in any embodiment, or in any combination without conflict. In addition, parts not described in detail in the embodiments can be implemented using existing technologies.

[0082] The above-mentioned embodiment of the present application provides a hybrid precision low-bit quantization scheme that is both flexible and practical. This scheme effectively enhances the representation ability and task performance of the model while maintaining the overall average quantization accuracy. In particular, it significantly improves the deployment efficiency of the model on resource-constrained devices, while effectively overcoming the comprehensive problems faced by the existing technology in terms of module difference adaptation, precision control and compression effect. It has important engineering application value and broad promotion prospects, such as deep learning deployment in resource-constrained environments such as mobile terminals and edge computing devices.

[0083] In one embodiment of the present application, an image reconstruction method is installed on a mobile terminal, comprising:

[0084] Acquiring images to be reconstructed;

[0085] Inputting the image to be reconstructed into the image reconstruction model, where the image reconstruction model is a mixed precision model quantized by the above method or system;

[0086] Obtain the reconstructed image.

[0087] The image reconstruction method installed in the mobile terminal of the present application uses a quantized mixed precision model to bring significant convenience to users. After the image is collected and input into the model, the reconstruction speed is greatly improved due to the optimization of the mixed precision quantization technology. The task that originally took tens of seconds can now be completed in a few seconds, which greatly saves user time. While increasing the speed, the quantization technology also guarantees the performance of the model. During reconstruction, the model can accurately restore the details of the image, whether it is blurred texture or missing parts, it can be processed very finely. The reconstructed image is clear and the colors are realistic. The quality is comparable to the original high-quality image and even exceeds expectations. Moreover, the optimized model occupies less resources of the mobile terminal, which not only saves power and extends battery life, but also allows the device to run smoothly without affecting the use of other applications. It is extremely practical in scenarios such as photography and document repair.

[0088] In one embodiment of the present application, an edge computing method includes:

[0089] The edge device extracts the edge signal;

[0090] Extract the features of the edge signal using a feature extraction model, where the feature extraction model is a mixed precision model quantized by the above method or system;

[0091] The extracted features are passed to the cloud for federated training.

[0092] The edge computing technology solution of the above-mentioned embodiment of the present application adopts a quantized mixed precision model to extract edge signal features, which can process data efficiently: the model takes up less resources and runs fast. In industrial production, the device signals collected by sensors can quickly complete feature extraction to avoid data processing delays affecting production decisions. It can reduce transmission costs: only feature data is transmitted, which greatly reduces the amount of data transmission. Like in smart city traffic systems, after feature extraction of camera videos, the transmission pressure is small and the cost is low, and the cloud can receive data training faster. It can improve the quality of training: the mixed precision model is quantized and optimized, and the extracted features are accurate. Providing high-quality data for cloud-based federated training makes the model more realistic, enhances the adaptability and prediction capabilities in complex scenarios, and makes the entire edge computing system more reliable and practical.

[0093] The above describes some specific embodiments of the present application. It should be understood that the present application is not limited to the specific embodiments described above, and those skilled in the art may make various variations or modifications within the scope of the claims, which do not affect the substantive content of the present application. The above preferred features may be used in any combination as long as they do not conflict with each other.

Claims

1. A mixed precision quantization method with adaptive bit width, characterized in that: include: Configuring a quantizer with a variable bit width for each module in the model, wherein the model is any neural network model; Set the loss function and its upper and lower bounds for each module; According to the comparison between the value of the loss function and the upper bound and the lower bound, the bit width of the corresponding quantizer is automatically adjusted module by module to obtain a mixed precision model.

2. The adaptive bit-width mixed precision quantization method according to claim 1, characterized in that: The quantizer is provided with a default initial bit width; The default initial bit width of the quantizer corresponding to all modules of the entire model is the same; The default initial bit width is an expected quantization bit width, or a quantization bit width corresponding to an expected compression ratio or speedup ratio.

3. The adaptive bit width mixed precision quantization method according to claim 1, characterized in that: The setting of the loss function of each module and its upper and lower bounds includes: The loss function is designed to be a combined loss function consisting of the output loss of the current module and the corresponding full-precision module, the overall output of the final quantization module, and the loss of the full-precision output; Specify upper and lower bounds on the loss function based on the expected number of quantization bits.

4. The adaptive bit width mixed precision quantization method according to claim 1, characterized in that: The method automatically adjusts the bit width of the corresponding quantizer module by module based on the comparison between the value of the loss function and the upper bound and the lower bound to obtain a mixed precision model, including: All modules are operated one by one, and the following comparisons and adjustments are made to each module: If the loss function stabilizes between the upper and lower bounds within the specified maximum number of steps, save the current bit width setting and continue training the module parameters. If the loss function stabilizes below the lower bound within the specified maximum number of steps, the current bit width is increased and the calibration is restarted after returning to the initial state. If the loss function stabilizes above the upper bound within the specified maximum number of steps, the current bit width is lowered and the calibration is restarted after returning to the initial state.

5. The adaptive bit width mixed precision quantization method according to claim 4, characterized in that: The parameters of the continued training module include: After determining the current bit width, specify a training step number as the condition for ending training; The number of training steps is 5 to 10 times the number of steps used to determine the current bit width.

6. The adaptive bit width mixed precision quantization method according to claim 1, characterized in that: The module-by-module method refers to performing quantitative calibration in the reverse order of the inference process.

7. The adaptive bit-width mixed precision quantization method according to claim 6, characterized in that: After the quantization calibration of the last module is completed, an expansion phase of a certain number of steps is added, that is, all quantization modules are updated using the overall loss function.

8. An adaptive bit width mixed precision quantization system, characterized in that: include: Quantizer module: configures a quantizer with variable bit width for each module in the model; Loss setting module: sets the loss function and its upper and lower bounds for each module; Quantization calibration module: According to the comparison between the value of the loss function and the upper and lower bounds, the bit width of the corresponding quantizer is automatically adjusted module by module to obtain a mixed precision model.

9. An image reconstruction method, characterized in that: include: Acquiring images to be reconstructed; Inputting the image to be reconstructed into an image reconstruction model, wherein the image reconstruction model is a mixed precision model quantized by the method according to any one of claims 1 to 7 or the system according to claim 8; Obtain the reconstructed image.

10. An edge computing method, characterized in that: include: The edge device extracts the edge signal; Extracting features of the edge signal using a feature extraction model, wherein the feature extraction model is a mixed precision model quantized by the method according to any one of claims 1 to 7 or the system according to claim 8; The extracted features are passed to the cloud for federated training.

Images