Network model quantization method, device, equipment, medium, program product and chip

Abstract

The present disclosure relates to a network model quantization method, device, equipment, medium, program product and chip. The network model quantization method comprises: performing a quantization process on a first network model to obtain a second network model; determining a quantization gradient of each target configuration parameter, and determining an influence degree of each target configuration parameter on a model quantization loss value based on the quantization gradient; determining a priority ranking of each target configuration parameter based on the influence degree, and sequentially increasing the bit width of at least part of the target configuration parameters according to the priority ranking until a preset quantization condition is met to obtain a third network model. The quantization gradient is used as the basis for determining the influence degree, and the influence degree is used as the basis for determining the priority ranking, which can fully consider the inter-layer relationship of the network model, so that the final bit width of each target configuration parameter can be matched with the influence degree of the target configuration parameter on the model performance, and is beneficial to reducing the resource consumption and dependence on sample size of mixed precision quantization, and improving the efficiency and stability of mixed precision quantization.

Description

Technical Field

[0001] This disclosure relates to the field of network model quantization, specifically to a network model quantization method, apparatus, device, medium, program product, and chip. Background Technology

[0002] In recent years, with the rapid development of artificial intelligence technology, more and more network models have been widely used in people's daily lives and work. Network model quantization technology, as a key technology for deploying network models on terminals, can reduce the computational load and storage consumption of network models by compressing them, and improve the response speed of network models. Summary of the Invention

[0003] To overcome the problems existing in related technologies, this disclosure provides a method, apparatus, device, medium, program product and chip for network model quantization.

[0004] According to a first aspect of the present disclosure, a network model quantization method is provided, the network model quantization method comprising: A quantization process is performed on the first network model to reduce each target configuration parameter of the first network model from the first bit width to the second bit width, thereby obtaining a second network model. The target configuration parameters include at least some of the model parameters and / or model features of the first network model. Determine the quantization gradient of each of the target configuration parameters with respect to the model quantization loss value between the first network model and the second network model, and based on the quantization gradient, determine the degree of influence of each of the target configuration parameters on the model quantization loss value; Based on the degree of influence, the priority order of each target configuration parameter is determined, and at least some of the target configuration parameters are increased from the second bit width to the third bit width in sequence according to the priority order until the preset quantization condition is met, to obtain a third network model, wherein the third bit width of each target configuration parameter is less than or equal to the first bit width.

[0005] In this embodiment, a second network model is obtained by performing a quantization process on the first network model to reduce the bit width corresponding to each target configuration parameter. The quantization gradient, the degree of influence on the model's quantization loss value, and the priority ranking of each target configuration parameter are determined sequentially. The bit width corresponding to at least some target configuration parameters is increased according to the priority ranking to obtain a third network model. This achieves mixed-precision quantization of the first network model by simultaneously decreasing the bit width of the target configuration parameters in batches and selectively increasing them sequentially, thus ensuring a balance between quantization degree and model performance. Using the quantization gradient as the basis for determining the degree of influence and the degree of influence as the basis for determining the priority ranking fully considers the inter-layer relationships of the network model, ensuring that the final bit width corresponding to each target configuration parameter matches the degree of influence of that target configuration parameter on model performance. This guarantees the rationality of each target configuration parameter in the third network model and helps reduce the resource consumption and dependence on sample size in mixed-precision quantization, improving its efficiency and stability.

[0006] In some embodiments of this disclosure, the quantization process performed on the first network model includes: The quantization process is performed on the first network model based on the nearest even rounding strategy and the pass-through estimator.

[0007] In this embodiment, the quantization process of the first network model is performed according to the nearest even rounding strategy and the pass-through estimator, which can improve the quantization accuracy of the quantization process and avoid the gradient interruption problem. This makes the discrete quantization operation of the quantization process have the possibility of gradient propagation, so as to solve the problem of the quantization operation being non-differentiable, and provides a guarantee for the subsequent determination of the quantization gradient corresponding to each target configuration parameter.

[0008] In some embodiments of this disclosure, determining the quantization gradient of each of the target configuration parameters with respect to the model quantization loss value between the first network model and the second network model includes: Preset samples are input into the first network model and the second network model respectively to obtain the first output result and the second output result; Based on the first output result, the second output result, and the preset loss function, the model quantization loss value between the first network model and the second network model is determined; Based on the model quantization loss value and the second network model, the quantization gradient of each of the target configuration parameters under the second bit width is determined.

[0009] In this embodiment, by inputting preset samples into the first network model and the second network model respectively, the corresponding first output result and second output result are obtained. Based on the first output result, the second output result and the preset loss function, the model quantization loss value between the first network model and the second network model is determined. Then, based on the model quantization loss value and the second network model, the quantization gradient corresponding to each target configuration parameter under the second bit width is determined. The quantization gradient can be determined with only a small number of preset samples and two forward passes and one back pass, which provides a basis for determining the degree of influence, reduces the resource consumption of mixed precision quantization and the dependence on the number of samples, and improves the efficiency and stability of mixed precision quantization.

[0010] In some embodiments of this disclosure, determining the degree of influence of each of the target configuration parameters on the model quantization loss value based on the quantization gradient includes: Based on the quantization gradient of each of the target configuration parameters, an approximate value of the quantization error of each of the target configuration parameters relative to the model quantization loss value is determined; Based on the approximate quantization error values ​​of each of the target configuration parameters, the degree of influence of each of the target configuration parameters on the model quantization loss value is determined.

[0011] In this embodiment, based on the quantization gradient corresponding to each target configuration parameter, an approximate quantization error value relative to the model quantization loss value is determined. Then, based on this approximate quantization error value, the degree of influence of each target configuration parameter on the model quantization loss value is determined, providing a basis for priority ranking. Using the approximate quantization error value relative to the model quantization loss value as the basis for determining the degree of influence ensures that the influence of each target configuration parameter on the quantization loss value matches the portion of the model quantization loss value corresponding to that target configuration parameter. This facilitates a full consideration of the inter-layer relationships in the network model and helps to further improve the rationality and accuracy of sensitivity information and priority ranking.

[0012] In some embodiments of this disclosure, determining an approximate quantization error value for each of the target configuration parameters relative to the model quantization loss value based on the quantization gradient of each of the target configuration parameters includes: For each target configuration parameter, an approximate value of the quantization error is determined based on the quantization gradient of the target configuration parameter and the Taylor expansion of the quantization error of the target configuration parameter relative to the model quantization loss value.

[0013] In this embodiment, for each target configuration parameter, the approximate value of the quantization error corresponding to the target configuration parameter is determined based on the quantization gradient corresponding to the target configuration parameter and the Taylor expansion of the quantization error of the target configuration parameter relative to the model quantization loss value. This allows the first-order approximation of the quantization error based on the Taylor expansion to be used as the approximate value of the quantization error. This utilizes the characteristics of the Taylor expansion to determine the approximate value of the quantization error, providing a basis for determining the degree of influence. It can fully consider the inter-layer relationships of the network model, thereby further improving the rationality and accuracy of the degree of influence and priority ranking, and improving the efficiency and stability of mixed precision quantization.

[0014] In some embodiments of this disclosure, determining the degree of influence of each of the target configuration parameters on the model quantization loss value based on the approximate quantization error value of each of the target configuration parameters includes: The target configuration parameter is divided into multiple configuration parameter sets, each configuration parameter set including at least one target configuration parameter, and the target configuration parameter in the same configuration parameter set corresponds to the same hardware supported bit width; For each set of configuration parameters, the degree of influence of the set of configuration parameters on the quantization loss value of the model is determined based on the approximate quantization error value of each target configuration parameter in the set of configuration parameters.

[0015] In this embodiment, the target configuration parameters are divided into multiple configuration parameter sets, each containing at least one target configuration parameter. Target configuration parameters within the same set correspond to the same hardware-supported bit width. For each configuration parameter set, the degree of influence is determined based on the approximate quantization error value corresponding to each target configuration parameter within that set, providing a basis for priority ranking. This method simplifies the determination of influence by grouping multiple target configuration parameters according to their hardware-supported bit width and ensures that the degree of influence corresponds to the hardware support granularity of the target configuration parameter, thereby improving the rationality and accuracy of the influence determination.

[0016] In some embodiments of this disclosure, determining the priority order of each of the target configuration parameters based on the degree of influence includes: The priority order of each set of configuration parameters is determined based on the degree of influence of each set of configuration parameters on the quantization loss value of the model.

[0017] In this embodiment, the priority order of each set of configuration parameters is determined according to the degree of influence of each set of configuration parameters on the model quantization loss value. This serves as the priority order of each target configuration parameter, providing a basis for improving the bit width of at least some target configuration parameters. This simplifies the process of determining the priority order and ensures that the priority order and final bit width of the target configuration parameters match the hardware support granularity of the target configuration parameters. This is beneficial for further improving the rationality and accuracy of the degree of influence and the efficiency of mixed precision quantization.

[0018] In some embodiments of this disclosure, the preset quantization condition includes: the computational complexity of the second network model reaches a preset computational complexity threshold.

[0019] In this embodiment, the computational complexity corresponding to the second network model reaches a preset computational complexity threshold as a preset quantization condition, so that the number of target configuration parameters whose bit width is increased, and the size of the third bit width after the target configuration parameters are increased, can match the preset computational complexity threshold, thereby controlling the computational complexity of the final third network model, which is beneficial to the end-side deployment of the network model.

[0020] In some embodiments of this disclosure, the network model quantization method further includes: A quantization-aware training process is performed on the third network model to obtain the fourth network model.

[0021] In this embodiment, a fourth network model is obtained by performing a quantization-aware training process on the third network model, thereby achieving further optimization of the third network model and further ensuring the balance between the degree of quantization and model performance, which is conducive to the deployment of the network model on the edge.

[0022] In some embodiments of this disclosure, the first network model includes at least one of a neural network model and a diffusion model.

[0023] In this embodiment, at least one of the neural network model and the diffusion model is used as the first network model, which enables mixed precision quantization of different types of network models, achieves a balance between quantization degree and model performance, reduces the resource consumption and dependence on sample size of mixed precision quantization, improves the efficiency and stability of mixed precision quantization, and is conducive to the edge deployment of various types of network models.

[0024] According to a second aspect of the present disclosure, a network model quantization apparatus is provided, the network model quantization apparatus comprising: A quantization module is used to perform a quantization process on a first network model to reduce each target configuration parameter of the first network model from a first bit width to a second bit width to obtain a second network model. The target configuration parameters include at least some model parameters and / or model features of the first network model. The determining module is used to determine the quantization gradient of each of the target configuration parameters with respect to the model quantization loss value between the first network model and the second network model, and based on the quantization gradient, determine the degree of influence of each of the target configuration parameters on the model quantization loss value; An enhancement module is used to determine the priority order of each target configuration parameter based on the degree of influence, and to increase at least some of the target configuration parameters from the second bit width to the third bit width in sequence according to the priority order, until a preset quantization condition is met to obtain a third network model, wherein the third bit width of each target configuration parameter is less than or equal to the first bit width.

[0025] In some embodiments of this disclosure, the determining module is further configured to: Preset samples are input into the first network model and the second network model respectively to obtain the first output result and the second output result; Based on the first output result, the second output result, and the preset loss function, the model quantization loss value between the first network model and the second network model is determined; Based on the model quantization loss value and the second network model, the quantization gradient of each of the target configuration parameters under the second bit width is determined.

[0026] In some embodiments of this disclosure, the determining module is further configured to: Based on the quantization gradient of each of the target configuration parameters, an approximate value of the quantization error of each of the target configuration parameters relative to the model quantization loss value is determined; Based on the approximate quantization error values ​​of each of the target configuration parameters, the degree of influence of each of the target configuration parameters on the model quantization loss value is determined.

[0027] In some embodiments of this disclosure, the determining module is further configured to: The target configuration parameter is divided into multiple configuration parameter sets, each configuration parameter set including at least one target configuration parameter, and the target configuration parameter in the same configuration parameter set corresponds to the same hardware supported bit width; For each set of configuration parameters, the degree of influence of the set of configuration parameters on the quantization loss value of the model is determined based on the approximate quantization error value of each target configuration parameter in the set of configuration parameters.

[0028] According to a third aspect of the present disclosure, an electronic device is provided, the electronic device comprising: processor; Memory used to store processor-executable instructions; The processor is configured to execute the network model quantization method as described in the first aspect.

[0029] According to a fourth aspect of the present disclosure, a non-transitory computer-readable storage medium is provided, wherein when instructions in the storage medium are executed by a processor of an electronic device, the electronic device is enabled to perform the network model quantization method as described in the first aspect.

[0030] According to a fifth aspect of the present disclosure, a computer program product is provided, including a computer program that, when executed by a processor, implements the network model quantization method as described in the first aspect.

[0031] According to a sixth aspect of the present disclosure, a chip is provided, the chip being configured to perform the network model quantization method as described in the first aspect.

[0032] The technical solutions provided by the embodiments of this disclosure may include the following beneficial effects: using the quantization gradient as the basis for determining the degree of influence and using the degree of influence as the basis for determining the priority ranking can fully consider the inter-layer relationship of the network model, so that the bit width corresponding to each target configuration parameter can match the degree of influence of the target configuration parameter on the model performance, thereby ensuring the rationality of each target configuration parameter of the third network model, and helping to reduce the resource consumption and dependence on the sample size of mixed precision quantization, and improve the efficiency and stability of mixed precision quantization.

[0033] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and are not intended to limit this disclosure. Attached Figure Description

[0034] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with the invention and, together with the description, serve to explain the principles of the invention.

[0035] Figure 1 This is a flowchart illustrating a network model quantization method according to an exemplary embodiment.

[0036] Figure 2 This is a flowchart illustrating, according to an exemplary embodiment, the determination of the quantization gradient of each target configuration parameter with respect to the model quantization loss value between a first network model and a second network model.

[0037] Figure 3This is a flowchart illustrating, according to an exemplary embodiment, the degree of influence of each target configuration parameter on the model quantization loss value based on the quantization gradient.

[0038] Figure 4 This is a flowchart illustrating, according to an exemplary embodiment, the degree of influence of each target configuration parameter on the model quantization loss value based on the approximate quantization error value corresponding to each target configuration parameter.

[0039] Figure 5 This is a block diagram illustrating a network model quantization apparatus according to an exemplary embodiment.

[0040] Figure 6 This is a block diagram of an electronic device according to an exemplary embodiment.

[0041] In the picture: 10-Quantization module; 20-Determination module; 30-Boosting module; 400-Electronic device; 402-Processing component; 404-Memory; 406-Power supply component; 408-Multimedia component; 410-Audio component; 412-Input / output interface; 414-Sensor component; 416-Communication component; 420-Processor. Detailed Implementation

[0042] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numerals in different drawings denote the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present invention. Rather, they are merely examples of apparatuses and methods consistent with some aspects of the invention as detailed in the appended claims.

[0043] In recent years, with the rapid development of artificial intelligence technology and the continuous iteration of network models, more and more network models have been widely used in people's daily lives and work. Network model quantization technology can reduce the computational load and storage consumption of network models by compressing them, and improve the inference speed of network models. It has gradually become a key technology for realizing the deployment of network models on the edge.

[0044] Quantization of network models inevitably leads to a decrease in model performance, while mixed-precision quantization can identify key layers and increase their bit width after quantization. Related techniques include distributively sampling quantization layers and selecting the quantization scheme for the network model based on Hessian matrix information and quantization error. Alternatively, the model's network layers can be replaced layer by layer with lower-precision layers, selecting the quantization scheme based on the distance between the output features before and after each replacement. Furthermore, training can be performed to implement a supernetwork with multiple precision configurations for the network layers, allowing the selection of the network model's quantization scheme based on the differences between each network layer before and after quantization within the supernetwork.

[0045] However, the network model quantization method using related technologies is complex and overly dependent on training samples, storage resources, and computing resources, which seriously affects the efficiency and stability of the overall quantization process. Furthermore, it does not consider the inter-layer relationships of the network model, making it difficult to obtain the optimal quantization scheme to achieve a balance between the degree of quantization and model performance.

[0046] Based on this, this disclosure provides a network model quantization method. A second network model is obtained by performing a quantization process on a first network model to reduce the bit width corresponding to each target configuration parameter. The quantization gradient, the degree of influence on the model's quantization loss value, and the priority ranking of each target configuration parameter are determined sequentially. The bit width corresponding to at least some target configuration parameters is increased according to the priority ranking to obtain a third network model. This achieves mixed-precision quantization of the first network model by simultaneously reducing the bit width of the target configuration parameters in batches and selectively increasing them sequentially, thus ensuring a balance between quantization degree and model performance. Using the quantization gradient as the basis for determining the degree of influence and the degree of influence as the basis for determining the priority ranking fully considers the inter-layer relationships of the network model, ensuring that the final bit width corresponding to each target configuration parameter matches the degree of influence of that target configuration parameter on model performance. This guarantees the rationality of each target configuration parameter in the third network model and helps reduce the resource consumption and dependence on sample size in mixed-precision quantization, improving its efficiency and stability.

[0047] In one exemplary embodiment, a network model quantization method is provided, applied to an electronic device or chip. The electronic device may include, for example, a terminal device with network model deployment requirements, such as a mobile phone or tablet computer. The chip may include, for example, an inference chip, an accelerator chip, or an integrated chip. (Reference) Figure 1 As shown, network model quantization methods include: S100. Perform a quantization process on the first network model to reduce each target configuration parameter of the first network model from the first bit width to the second bit width to obtain the second network model. The target configuration parameters include at least some of the model parameters and / or model features of the first network model. In step S100, the first network model is a network model with quantization requirements. The first network model may include at least one of a neural network model and a diffusion model. The first network model is configured with a large number of model parameters and model features. Model parameters may include, for example, weight parameters or bias parameters, and model features may include, for example, activation values. At least some of the model parameters and at least some of the model features of the first network model can be used as target configuration parameters for the first network model, serving as the actual adjustment object in the network model quantization process.

[0048] A quantization process is performed on the first network model to reduce each target configuration parameter of the first network model from its corresponding first bit width to its second bit width, thereby obtaining the second network model. This enables batch synchronous quantization of the target configuration parameters. The first bit width corresponding to each target configuration parameter is determined based on the initial accuracy of the first network model, and the second bit width corresponding to each target configuration parameter can be set according to the quantization requirements.

[0049] S200. Determine the quantization gradient of each target configuration parameter with respect to the model quantization loss value between the first network model and the second network model, and based on the quantization gradient, determine the degree of influence of each target configuration parameter on the model quantization loss value.

[0050] In step S200, the first network model and the second network model have a model quantization loss value generated by the quantization process, which represents the difference in output between the first network model and the second network model for the same input, and can reflect the degree of influence of the overall quantization process on the model performance.

[0051] Determine the quantization gradient of each target configuration parameter with respect to the model quantization loss value. For each target configuration parameter, the quantization gradient corresponding to that parameter is the loss function corresponding to the model quantization loss value. Relative to the quantized target configuration parameters The partial derivatives, i.e. Based on the quantization gradient corresponding to each target configuration parameter, the influence of each target configuration parameter on the model quantization loss value can be determined, that is, the influence of the quantization of the target configuration parameter on the model performance, so as to measure the importance of the target configuration parameter to the model performance.

[0052] S300. Based on the degree of influence, determine the priority order of each target configuration parameter, and increase at least some target configuration parameters from the second bit width to the third bit width in sequence according to the priority order until the preset quantization condition is met, to obtain the third network model, wherein the third bit width of each target configuration parameter is less than or equal to the first bit width.

[0053] In step S300, the higher the degree of influence, the greater the requirement for increasing the bit width of the target configuration parameter to ensure model performance. Therefore, the priority ranking of each target configuration parameter can be determined according to the degree of influence, so that the target configuration parameters with higher influence can be assigned higher priority rankings. At least some target configuration parameters are sequentially increased from the quantized second bit width to the corresponding third bit width according to the priority ranking from high to low, until the preset quantization conditions are met. This achieves mixed-precision quantization of the first network model by first batch synchronously reducing the bit width of the target configuration parameters and then selectively increasing it sequentially, resulting in the third network model. This ensures that the third network model can achieve a balance between reducing computational load and storage consumption and improving response speed.

[0054] It should be noted that the preset quantization conditions can be set according to the quantization target or resource limit. When the preset quantization conditions are met, the bit width of the target configuration parameters will stop increasing. At this point, the bit width of only a portion of the target configuration parameters may be increased, or the bit width of all target configuration parameters may be increased. If only the bit width of a portion of the target configuration parameters is increased, the bit width of the remaining target configuration parameters will remain at the second bit width. The third bit width corresponding to each target configuration parameter can be set according to the quantization target. The third bit width is less than or equal to the first bit width to ensure that the third network model has a quantization effect compared to the first network model. The third bit width corresponding to different target configuration parameters can be the same or different.

[0055] For example, the first bit width can be 32, the second bit width can be 8, and the third bit width can be 16. The first network model can be a 32-bit floating-point (FP32) model. The weights and activation values ​​of the first network model can be used as target configuration parameters. By performing a quantization process on the first network model, the bit width of the target configuration parameters of the first network model is reduced from 32 to 8, resulting in a second network model. The second network model can be an 8-bit integer (INT8) model. The quantization gradient, the degree of influence on the quantization loss value of each target configuration parameter, and the priority ranking are determined sequentially. The bit width of the target configuration parameters with the highest priority is increased from 8 to 16, resulting in a third network model.

[0056] In this embodiment, a second network model is obtained by performing a quantization process on the first network model to reduce the bit width corresponding to each target configuration parameter. The quantization gradient, the degree of influence on the model's quantization loss value, and the priority ranking of each target configuration parameter are determined sequentially. The bit width corresponding to at least some target configuration parameters is increased according to the priority ranking to obtain a third network model. This achieves mixed-precision quantization of the first network model by simultaneously decreasing the bit width of the target configuration parameters in batches and selectively increasing them sequentially, thus ensuring a balance between quantization degree and model performance. Using the quantization gradient as the basis for determining the degree of influence and the degree of influence as the basis for determining the priority ranking fully considers the inter-layer relationships of the network model, ensuring that the final bit width corresponding to each target configuration parameter matches the degree of influence of that target configuration parameter on model performance. This guarantees the rationality of each target configuration parameter in the third network model and helps reduce the resource consumption and dependence on sample size in mixed-precision quantization, improving its efficiency and stability.

[0057] In some embodiments, performing a quantization process on a first network model includes: performing a quantization process on the first network model based on a recent even-number rounding strategy and a pass-through estimator.

[0058] When performing quantization on the first network model to reduce the bit width of each target configuration parameter, a Round Nearest Even (RNE) strategy is used, encapsulated by a Straight Through Estimator (STE). The RNE strategy quantizes values ​​to the nearest even integer, improving quantization accuracy and reducing quantization bias. The STE converts continuous inputs into discrete outputs through forward propagation and propagates the gradient back to the input intact through backpropagation, overcoming the limitation of non-differentiability of discrete functions and ensuring effective gradient backpropagation. The synergy of these two approaches enables gradient propagation in discrete quantization operations, solving the problem of non-differentiability in quantization.

[0059] In this embodiment, the quantization process of the first network model is performed according to the nearest even rounding strategy and the pass-through estimator, which can improve the quantization accuracy of the quantization process and avoid the gradient interruption problem. This makes the discrete quantization operation of the quantization process have the possibility of gradient propagation, so as to solve the problem of the quantization operation being non-differentiable, and provides a guarantee for the subsequent determination of the quantization gradient corresponding to each target configuration parameter.

[0060] In some embodiments, reference Figure 2 As shown, the quantization gradient of each target configuration parameter with respect to the model quantization loss value between the first network model and the second network model is determined, including: S210. Input the preset samples into the first network model and the second network model respectively to obtain the first output result and the second output result.

[0061] In step S210, the preset samples are used as the model input, i.e., the calibration set, and are input to the first network model and the second network model respectively, to obtain the first output result of the first network model and the second output result of the second network model. The first output result and the second output result are the forepass results of the first network model and the second network model performing one forepass on the preset samples, respectively. It should be noted that if the first network model and the second network model are classification models, the first output result and the second output result to be obtained are the original output vector logits of the model, rather than the one-hot true target label of the task, in order to avoid the loss of fine-grained information affecting the subsequent calculation of quantization gradient.

[0062] S220. Based on the first output result, the second output result, and the preset loss function, determine the model quantization loss value between the first network model and the second network model.

[0063] In step S220, since the first output and the second output are respectively the outputs of the first and second network models on the same model input, the model quantization loss value between the first and second network models can be determined based on the first output, the second output, and the preset loss function. This allows the model quantization loss value to represent the difference in output between the first and second network models for the same input, reflecting the impact of the overall quantization process on model performance. The preset loss function can be selected according to requirements; for example, it may include a cross-entropy loss function or a mean squared error loss function.

[0064] S230. Based on the model quantization loss value and the second network model, determine the quantization gradient of each target configuration parameter under the second bit width.

[0065] In step S230, based on the model quantization loss value and the second network model, the quantization gradient of each target configuration parameter of the second network model under the second bit width is determined. That is, the model quantization loss value is backpropagated through the second network model. The model quantization loss value is backpropagated to calculate the partial derivative of the preset loss function with respect to the target configuration parameter. The quantization gradient of each target configuration parameter is the backpropagation result of the second network model, so that the quantization gradient corresponding to each target configuration parameter can reflect the degree of contribution of the target configuration parameter to the model quantization loss value.

[0066] In this embodiment, by inputting preset samples into the first network model and the second network model respectively, the corresponding first output result and second output result are obtained. Based on the first output result, the second output result and the preset loss function, the model quantization loss value between the first network model and the second network model is determined. Then, based on the model quantization loss value and the second network model, the quantization gradient corresponding to each target configuration parameter under the second bit width is determined. The quantization gradient can be determined with only a small number of preset samples and two forward passes and one back pass, which provides a basis for determining the degree of influence, reduces the resource consumption of mixed precision quantization and the dependence on the number of samples, and improves the efficiency and stability of mixed precision quantization.

[0067] In some embodiments, reference Figure 3 As shown, based on the quantization gradient, the influence of each objective configuration parameter on the model quantization loss value is determined, including: S240. Based on the quantization gradient of each target configuration parameter, determine the approximate quantization error value of each target configuration parameter relative to the model quantization loss value.

[0068] In step S240, since the quantization gradient corresponding to each target configuration parameter is the partial derivative of the loss function with respect to the target configuration parameter, representing the contribution of the target configuration parameter to the model's quantization loss value, the approximate quantization error value of each target configuration parameter relative to the model's quantization loss value can be determined based on the quantization gradient corresponding to each target configuration parameter. The approximate quantization error value corresponding to each target configuration parameter is an approximate result of the loss value in the model's quantization loss value caused by the quantization of that target configuration parameter; that is, the part of the overall model loss corresponding to that target configuration parameter. The sum of the approximate quantization error values ​​corresponding to each target configuration parameter is approximately the same as the model's quantization loss value.

[0069] It should be noted that since the quantization error of each target configuration parameter relative to the model quantization loss value is difficult to obtain a completely accurate result through precise and simple calculation, it is necessary to approximate the quantization error based on the target configuration parameter, that is, to approximate the quantization error by using the approximate value of the quantization error.

[0070] S250. Based on the approximate quantization error values ​​of each target configuration parameter, determine the degree of influence of each target configuration parameter on the model quantization loss value.

[0071] In step S250, since the approximate quantization error value corresponding to each target configuration parameter can represent the part of the model quantization loss value corresponding to that target configuration parameter, it can reflect the degree of influence of the target configuration parameter on the model quantization loss value and model performance to a certain extent. Based on the approximate quantization error value corresponding to each target configuration parameter, the degree of influence of each target configuration parameter on the model quantization loss value can be determined, providing a basis for priority ranking. For example, the approximate quantization error value corresponding to each target configuration parameter can be directly used as the degree of influence of each target configuration parameter on the model quantization loss value. Alternatively, a unified normalization process can be performed on the approximate quantization error values ​​of each target configuration parameter, and the processing result can be used as the degree of influence of each target configuration parameter on the model quantization loss value.

[0072] In this embodiment, based on the quantization gradient corresponding to each target configuration parameter, an approximate quantization error value relative to the model quantization loss value is determined. Then, based on this approximate quantization error value, the degree of influence of each target configuration parameter on the model quantization loss value is determined, providing a basis for priority ranking. Using the approximate quantization error value relative to the model quantization loss value as the basis for determining the degree of influence ensures that the influence of each target configuration parameter on the quantization loss value matches the portion of the model quantization loss value corresponding to that target configuration parameter. This facilitates a full consideration of the inter-layer relationships in the network model and helps to further improve the rationality and accuracy of sensitivity information and priority ranking.

[0073] In some embodiments, determining the approximate quantization error of each target configuration parameter relative to the model quantization loss value based on the quantization gradient of each target configuration parameter includes: for each target configuration parameter, determining the approximate quantization error corresponding to the target configuration parameter based on the quantization gradient of the target configuration parameter and the Taylor expansion of the quantization error of the target configuration parameter relative to the model quantization loss value.

[0074] As mentioned earlier, the quantization error of each target parameter relative to the model quantization loss value is difficult to obtain a completely accurate result through precise and simple calculation. Therefore, the quantization error is approximated based on the target configuration parameters, i.e., the quantization error is approximated by the approximate value. Taylor expansion, as a function expansion method, can represent the function through an infinite number of polynomials, providing a rigorous mathematical expression for the quantization error. Furthermore, through local linearization and gradient correlation, the complex quantization error analysis is transformed into a computable problem.

[0075] Therefore, when determining the approximate quantization error of each target configuration parameter relative to the model quantization loss value, for each target configuration parameter, the approximate quantization error can be determined based on the quantization gradient corresponding to the target configuration parameter and the Taylor expansion of the quantization error of the target configuration parameter relative to the model quantization loss value, so as to provide a basis for determining the degree of influence of each target configuration parameter on the model quantization loss value.

[0076] For example, the first-order component of the Taylor expansion of the quantization error relative to the quantization loss value of the target configuration parameter can be used as an approximation of the quantization error, i.e., the quantization error based on a first-order approximation of the Taylor expansion. It can be determined by the following formula (1): (1) in, These are the target configuration parameters for the first network model, i.e., the first width. The target configuration parameters are for the second network model, i.e., the second bit width. Configure the quantization gradient corresponding to the target parameters. The scaling factor is a quantization parameter that maps the original parameters to a lower precision range.

[0077] because The error is caused by rounding during quantization, especially when the quantization process is performed based on the nearest even-number rounding strategy. A uniform distribution U(0, 0.5) can be used for simulation, and the quantization error approximation value can be obtained. It can be further determined by formula (2) as shown below: (2) In this embodiment, for each target configuration parameter, the approximate value of the quantization error corresponding to the target configuration parameter is determined based on the quantization gradient corresponding to the target configuration parameter and the Taylor expansion of the quantization error of the target configuration parameter relative to the model quantization loss value. This allows the first-order approximation of the quantization error based on the Taylor expansion to be used as the approximate value of the quantization error. This utilizes the characteristics of the Taylor expansion to determine the approximate value of the quantization error, providing a basis for determining the degree of influence. It can fully consider the inter-layer relationships of the network model, thereby further improving the rationality and accuracy of the degree of influence and priority ranking, and improving the efficiency and stability of mixed precision quantization.

[0078] In some embodiments, for each target configuration parameter, the degree of influence of that target configuration parameter on the model quantization loss value can be directly determined based on the approximate quantization error value corresponding to that target configuration parameter, thus providing a basis for priority ranking. In this case, calculating the degree of influence of each target configuration parameter on the model quantization loss value individually leads to a complex calculation process, and the parallel computing characteristics cannot be utilized in the parallel computing of the chip.

[0079] In some embodiments, reference Figure 4 As shown, based on the approximate quantization error values ​​of each target configuration parameter, the influence of each target configuration parameter on the model quantization loss value is determined, including: S251. Divide the target configuration parameters into multiple configuration parameter sets. Each configuration parameter set includes at least one target configuration parameter. Target configuration parameters in the same configuration parameter set correspond to the same hardware-supported bit width.

[0080] In step S251, the target configuration parameters are divided into multiple configuration parameter sets, such that each configuration parameter set includes one or more target configuration parameters, and the target configuration parameters in the same configuration parameter set correspond to the same hardware supported bit width.

[0081] S252. For each set of configuration parameters, based on the approximate quantization error of each target configuration parameter in the set, determine the degree of influence of the set of configuration parameters on the quantization loss value of the model.

[0082] In step S252, for each set of configuration parameters, the influence of the set of configuration parameters on the model quantization loss value is determined based on the approximate quantization error value corresponding to each target configuration parameter in the set of configuration parameters. This is used as the influence of each target configuration parameter in the set of configuration parameters on the model quantization loss value, so that the target configuration parameters in the same set of configuration parameters can correspond to the same degree of influence.

[0083] For example, for each set of configuration parameters, the approximate quantization error of the set of configuration parameters can be aggregated using the L2 norm based on the approximate quantization error of each target configuration parameter in the set of configuration parameters, so as to obtain the degree of influence of the set of configuration parameters on the quantization loss value of the model.

[0084] In this embodiment, the target configuration parameters are divided into multiple configuration parameter sets, each containing at least one target configuration parameter. Target configuration parameters within the same set correspond to the same hardware-supported bit width. For each configuration parameter set, the degree of influence is determined based on the approximate quantization error value corresponding to each target configuration parameter within that set, providing a basis for priority ranking. This method simplifies the determination of influence by grouping multiple target configuration parameters according to their hardware-supported bit width and ensures that the degree of influence corresponds to the hardware support granularity of the target configuration parameter, thereby improving the rationality and accuracy of the influence determination.

[0085] In some embodiments, determining the priority order of each target configuration parameter based on the degree of influence includes: determining the priority order of each configuration parameter set based on the degree of influence of each configuration parameter set on the model quantization loss value.

[0086] By dividing the target configuration parameters into multiple sets of configuration parameters and determining the degree of influence of each set of configuration parameters on the model quantization loss value, the priority order of each set of configuration parameters can be determined according to the degree of influence of each set of configuration parameters on the model quantization loss value. This ensures that target configuration parameters in the same set of target configuration parameters, that is, those corresponding to the same hardware-supported bit width, can have the same priority order, and thus correspond to the same bit width in the third network model.

[0087] In this embodiment, the priority order of each set of configuration parameters is determined according to the degree of influence of each set of configuration parameters on the model quantization loss value. This serves as the priority order of each target configuration parameter, providing a basis for improving the bit width of at least some target configuration parameters. This simplifies the process of determining the priority order and ensures that the priority order and final bit width of the target configuration parameters match the hardware support granularity of the target configuration parameters. This is beneficial for further improving the rationality and accuracy of the degree of influence and the efficiency of mixed precision quantization.

[0088] In some embodiments, the preset quantization condition includes: the computational load of the second network model reaches a preset computational load threshold.

[0089] As mentioned earlier, at least some of the target configuration parameters are sequentially increased from the quantized second bit width to the corresponding third bit width in order of priority from high to low, until the bit width of the target configuration parameters is stopped when the preset quantization condition is met, thus obtaining the third network model. This allows the preset quantization condition to affect the number of target configuration parameters whose bit width is increased, as well as the size of the increased third bit width.

[0090] The computational cost of the second network model can be set to a preset computational cost threshold as a preset quantization condition. When the computational cost reaches the preset computational cost threshold, the improvement of the second network model can be stopped, so that the preset computational cost threshold can be used as a given computational cost budget to control the computational cost of the final third network model.

[0091] In this embodiment, the computational complexity corresponding to the second network model reaches a preset computational complexity threshold as a preset quantization condition, so that the number of target configuration parameters whose bit width is increased, and the size of the third bit width after the target configuration parameters are increased, can match the preset computational complexity threshold, thereby controlling the computational complexity of the final third network model, which is beneficial to the end-side deployment of the network model.

[0092] In some embodiments, the network model quantization method further includes: performing a quantization-aware training process on the third network model to obtain a fourth network model.

[0093] After obtaining the third network model, Quantization Aware Training (QAT) can be performed on it to integrate quantization operations into the iterative training process of the network model, optimize the third network model, and further achieve a balance between the degree of quantization and model performance.

[0094] In this embodiment, a fourth network model is obtained by performing a quantization-aware training process on the third network model, thereby achieving further optimization of the third network model and further ensuring the balance between the degree of quantization and model performance, which is conducive to the deployment of the network model on the edge.

[0095] In some embodiments, the first network model includes at least one of a neural network model and a diffusion model.

[0096] At least one of the neural network model and the diffusion model can be used as the first network model, and mixed precision quantization of different types of network models can be achieved by using the same network model quantization method.

[0097] In this embodiment, at least one of the neural network model and the diffusion model is used as the first network model, which enables mixed precision quantization of different types of network models, achieves a balance between quantization degree and model performance, reduces the resource consumption and dependence on sample size of mixed precision quantization, improves the efficiency and stability of mixed precision quantization, and is conducive to the edge deployment of various types of network models.

[0098] In one exemplary embodiment, a network model quantization apparatus is provided, referencing Figure 5As shown, the network model quantization device includes a quantization module 10, a determination module 20, and an enhancement module 30. The quantization module 10 performs a quantization process on a first network model to reduce the target configuration parameters of the first network model from a first bit width to a second bit width, thereby obtaining a second network model. The target configuration parameters include at least some model parameters and / or model features of the first network model. The determination module 20 determines the quantization gradient of each target configuration parameter with respect to the model quantization loss value between the first and second network models, and determines the degree of influence of each target configuration parameter on the model quantization loss value based on the quantization gradient. The enhancement module 30 determines the priority order of each target configuration parameter based on the degree of influence, and sequentially increases at least some target configuration parameters from the second bit width to the third bit width according to the priority order until a preset quantization condition is met, thereby obtaining a third network model, wherein the third bit width of each target configuration parameter is less than or equal to the first bit width.

[0099] In this embodiment, the quantization module 10 performs a quantization process on the first network model to reduce the bit width corresponding to each target configuration parameter, resulting in a second network model. The determination module 20 determines the quantization gradient corresponding to each target configuration parameter and its impact on the model's quantization loss value. The enhancement module 30 determines the priority order of each target configuration parameter and increases the bit width corresponding to at least some of the target configuration parameters according to the priority order, resulting in a third network model. This achieves mixed-precision quantization of the first network model by simultaneously reducing the bit width of the target configuration parameters in batches and selectively increasing them sequentially, thus ensuring a balance between quantization degree and model performance. Using the quantization gradient as the basis for determining the degree of influence and the degree of influence as the basis for determining the priority order fully considers the inter-layer relationships of the network model, ensuring that the final bit width corresponding to each target configuration parameter matches the degree of influence of that target configuration parameter on the model performance. This guarantees the rationality of each target configuration parameter in the third network model and helps reduce the resource consumption and dependence on sample size of mixed-precision quantization, improving the efficiency and stability of mixed-precision quantization.

[0100] In one embodiment, the quantization module 10 is further configured to: perform a quantization process on the first network model based on the nearest even rounding strategy and the pass-through estimator.

[0101] In one embodiment, the determining module 20 is further configured to: input preset samples into the first network model and the second network model respectively to obtain a first output result and a second output result; determine the model quantization loss value between the first network model and the second network model based on the first output result, the second output result and the preset loss function; and determine the quantization gradient of each target configuration parameter under the second bit width based on the model quantization loss value and the second network model.

[0102] In one embodiment, the determining module 20 is further configured to: determine an approximate value of the quantization error of each target configuration parameter relative to the model quantization loss value based on the quantization gradient of each target configuration parameter; and determine the degree of influence of each target configuration parameter on the model quantization loss value based on the approximate value of the quantization error of each target configuration parameter.

[0103] In one embodiment, the determining module 20 is further configured to: for each target configuration parameter, determine an approximate value of the quantization error corresponding to the target configuration parameter based on the quantization gradient of the target configuration parameter and the Taylor expansion of the quantization error of the target configuration parameter relative to the model quantization loss value.

[0104] In one embodiment, the determining module 20 is further configured to: divide the target configuration parameters into multiple configuration parameter sets, each configuration parameter set including at least one target configuration parameter, and the target configuration parameters in the same configuration parameter set correspond to the same hardware supported bit width; for each configuration parameter set, determine the degree of influence of the configuration parameter set on the model quantization loss value based on the approximate quantization error value of each target configuration parameter in the configuration parameter set.

[0105] In one embodiment, the enhancement module 30 is further configured to: determine the priority order of each set of configuration parameters based on the degree of influence of each set of configuration parameters on the model quantization loss value.

[0106] In one embodiment, the preset quantization condition includes: the computational load of the second network model reaches a preset computational load threshold.

[0107] In one embodiment, the network model quantization device further includes a training module, which is used to perform a quantization-aware training process on the third network model to obtain a fourth network model.

[0108] In one embodiment, the first network model includes at least one of a neural network model and a diffusion model.

[0109] In one exemplary embodiment, an electronic device is provided, which may include, for example, a terminal device such as a mobile phone or tablet computer that has network model deployment requirements, and the electronic device is capable of performing the network model quantization method as described above.

[0110] refer to Figure 6 As shown, the electronic device 400 may include one or more of the following components: processing component 402, memory 404, power supply component 406, multimedia component 408, audio component 410, input / output (I / O) interface 412, sensor component 414, and communication component 416.

[0111] Processing component 402 typically controls the overall operation of electronic device 400, such as operations associated with display, telephone calls, data communication, camera operation, and recording. Processing component 402 may include one or more processors 420 to execute instructions to perform all or part of the steps of the methods described above. Furthermore, processing component 402 may include one or more modules to facilitate interaction between processing component 402 and other components. For example, processing component 402 may include a multimedia module to facilitate interaction between multimedia component 408 and processing component 402.

[0112] Memory 404 is configured to store various types of data to support the operation of electronic device 400. Examples of such data include instructions for any application or method operating on electronic device 400, contact data, phonebook data, messages, pictures, videos, etc. Memory 404 can be implemented by any type of volatile or non-volatile storage terminal or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic storage, flash memory, magnetic disk, or optical disk.

[0113] Power supply component 406 provides power to various components of electronic device 400. Power supply component 406 may include a power management system, one or more power supplies, and other components associated with generating, managing, and distributing power to electronic device 400.

[0114] Multimedia component 408 includes a screen that provides an output interface between electronic device 400 and user. In some embodiments, the screen may include a liquid crystal display (LCD) and a touch panel (TP). If the screen includes a touch panel, the screen may be implemented as a touchscreen to receive input signals from the user. The touch panel includes one or more touch sensors to sense touches, swipes, and gestures on the touch panel. The touch sensors may sense not only the boundaries of touch or swipe actions but also the duration and pressure associated with the touch or swipe operation. In some embodiments, multimedia component 408 includes a front-facing camera module and / or a rear-facing camera module. When electronic device 400 is in an operating mode, such as shooting mode or video mode, the front-facing camera module and / or rear-facing camera module may receive external multimedia data. Each front-facing camera module and rear-facing camera module may be a fixed optical lens system or have focal length and optical zoom capabilities.

[0115] Audio component 410 is configured to output and / or input audio signals. For example, audio component 410 includes a microphone (MIC) configured to receive external audio signals when electronic device 400 is in an operating mode, such as call mode, recording mode, and voice recognition mode. The received audio signals may be further stored in memory 404 or transmitted via communication component 416. In some embodiments, audio component 410 also includes a speaker for outputting audio signals.

[0116] I / O interface 412 provides an interface between processing component 402 and peripheral interface modules, such as keyboards, click wheels, buttons, etc. These buttons may include, but are not limited to, home buttons, volume buttons, power buttons, and lock buttons.

[0117] Sensor assembly 414 includes one or more sensors for providing state assessments of various aspects of electronic device 400. For example, sensor assembly 414 may receive the on / off state of electronic device 400, the relative positioning of components such as the display and keypad of electronic device 400, changes in position of electronic device 400 or a component of electronic device 400, the presence or absence of user contact with electronic device 400, orientation or acceleration / deceleration of electronic device 400, and temperature changes of electronic device 400. Sensor assembly 414 may include a proximity sensor configured to detect the presence of nearby objects without any physical contact. Sensor assembly 414 may also include a light sensor, such as a CMOS or CCD image sensor, for use in imaging applications. In some embodiments, sensor assembly 414 may also include an accelerometer, gyroscope, magnetometer, pressure sensor, or temperature sensor.

[0118] Communication component 416 is configured to facilitate wired or wireless communication between electronic device 400 and other terminals. Electronic device 400 can access wireless networks based on communication standards, such as WiFi, 2G, 3G, 4G, 5G, or combinations thereof. In one exemplary embodiment, communication component 416 receives broadcast signals or broadcast-related information from an external broadcast management system via a broadcast channel. In one exemplary embodiment, communication component 416 also includes a near-field communication (NFC) module to facilitate short-range communication. For example, the NFC module may be implemented based on radio frequency identification (RFID) technology, Infrared Data Association (IrDA) technology, ultra-wideband (UWB) technology, Bluetooth (BT) technology, and other technologies.

[0119] In an exemplary embodiment, the electronic device 400 may be implemented by one or more application-specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing terminals (DSPDs), programmable logic devices (PLDs), field-programmable gate arrays (FPGAs), controllers, microcontrollers, microprocessors, or other electronic components to perform the methods shown in the above embodiments or combinations thereof.

[0120] In one exemplary embodiment, a non-transitory computer-readable storage medium including instructions is also provided, such as a memory 404 including instructions, which can be executed by a processor 420 of an electronic device 400 to perform the methods shown in the above embodiments or combinations thereof. For example, the non-transitory computer-readable storage medium may be a ROM, random access memory (RAM), CD-ROM, magnetic tape, floppy disk, and optical data storage terminal, etc. When the instructions in the storage medium are executed by the processor of the terminal, the terminal is able to perform the methods shown in the above embodiments or combinations thereof.

[0121] In one exemplary embodiment, a computer program product is provided, including a computer program or instructions that, when executed by a processor, implement the methods shown in the above embodiments or combinations thereof.

[0122] In one exemplary embodiment, a chip is provided, which is configured to perform the methods shown in the above embodiments or combinations thereof.

[0123] Other embodiments of this disclosure will readily occur to those skilled in the art upon consideration of the specification and practice of the invention disclosed herein. This disclosure is intended to cover any variations, uses, or adaptations of this disclosure that follow the general principles of this disclosure and include common knowledge or customary techniques in the art not disclosed herein. The specification and examples are to be considered exemplary only, and the true scope and spirit of this disclosure are indicated by the claims.

[0124] In the description of this specification, the references to terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., refer to specific features, structures, materials, or characteristics described in connection with that embodiment or example, which are included in at least one embodiment or example of this disclosure. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples. Moreover, without contradiction, those skilled in the art can combine and integrate the different embodiments or examples described in this specification, as well as the features of different embodiments or examples.

[0125] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one of that feature. In the description of this disclosure, "a plurality of" means at least two, such as two, three, etc., unless otherwise explicitly specified.

[0126] It should be understood that the various forms of processes shown above can be used to rearrange, add, or delete steps. For example, the steps described in this disclosure can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution disclosed in this disclosure can be achieved, and this is not limited herein.

[0127] The specific embodiments described above do not constitute a limitation on the scope of protection of this disclosure. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this disclosure should be included within the scope of protection of this disclosure.

[0128] The examples in this document may involve user data, data acquisition, and / or use. All of these aspects comply with relevant laws, regulations, and rules. In the examples, all data collection, acquisition, processing, manipulation, forwarding, and use are conducted with the user's knowledge and confirmation. Accordingly, when implementing each example, the type, scope of use, and usage scenarios of any data or information that may be involved should be communicated to the user and their authorization obtained through appropriate means, in accordance with relevant laws and regulations. The specific methods of notification and / or authorization can vary depending on the actual situation and application scenario; the scope of the solution is not limited in this regard.

[0129] In this manual and the sample solutions, any processing of personal information will be conducted only under legal grounds (such as obtaining the consent of the data subject or being necessary for the performance of a contract) and will only be carried out within the scope stipulated or agreed upon. A user's refusal to process personal information beyond what is necessary for basic functions will not affect the user's use of basic functions.

Claims

1. A method for quantizing network models, characterized in that, The network model quantization method includes: A quantization process is performed on the first network model to reduce each target configuration parameter of the first network model from the first bit width to the second bit width, thereby obtaining a second network model. The target configuration parameters include at least some of the model parameters and / or model features of the first network model. Determine the quantization gradient of each of the target configuration parameters with respect to the model quantization loss value between the first network model and the second network model, and based on the quantization gradient, determine the degree of influence of each of the target configuration parameters on the model quantization loss value; Based on the degree of influence, the priority order of each target configuration parameter is determined, and at least some of the target configuration parameters are increased from the second bit width to the third bit width in sequence according to the priority order until the preset quantization condition is met, to obtain a third network model, wherein the third bit width of each target configuration parameter is less than or equal to the first bit width.

2. The network model quantization method according to claim 1, characterized in that, The quantization process performed on the first network model includes: The quantization process is performed on the first network model based on the nearest even rounding strategy and the pass-through estimator.

3. The network model quantization method according to claim 1, characterized in that, Determining the quantization gradient of each of the target configuration parameters for the model quantization loss value between the first network model and the second network model includes: Preset samples are input into the first network model and the second network model respectively to obtain the first output result and the second output result; Based on the first output result, the second output result, and the preset loss function, the model quantization loss value between the first network model and the second network model is determined; Based on the model quantization loss value and the second network model, the quantization gradient of each of the target configuration parameters under the second bit width is determined.

4. The network model quantization method according to claim 1, characterized in that, The step of determining the degree of influence of each of the target configuration parameters on the model quantization loss value based on the quantization gradient includes: Based on the quantization gradient of each of the target configuration parameters, an approximate value of the quantization error of each of the target configuration parameters relative to the model quantization loss value is determined; Based on the approximate quantization error values ​​of each of the target configuration parameters, the degree of influence of each of the target configuration parameters on the model quantization loss value is determined.

5. The network model quantization method according to claim 4, characterized in that, Determining an approximate quantization error value for each of the target configuration parameters relative to the model quantization loss value based on the quantization gradient of each of the target configuration parameters includes: For each target configuration parameter, an approximate value of the quantization error is determined based on the quantization gradient of the target configuration parameter and the Taylor expansion of the quantization error of the target configuration parameter relative to the model quantization loss value.

6. The network model quantization method according to claim 4, characterized in that, The determination of the influence of each target configuration parameter on the model quantization loss value based on the approximate quantization error value of each target configuration parameter includes: The target configuration parameter is divided into multiple configuration parameter sets, each configuration parameter set including at least one target configuration parameter, and the target configuration parameter in the same configuration parameter set corresponds to the same hardware supported bit width; For each set of configuration parameters, the degree of influence of the set of configuration parameters on the quantization loss value of the model is determined based on the approximate quantization error value of each target configuration parameter in the set of configuration parameters.

7. The network model quantization method according to claim 6, characterized in that, The step of determining the priority order of each target configuration parameter based on the degree of influence includes: The priority order of each set of configuration parameters is determined based on the degree of influence of each set of configuration parameters on the quantization loss value of the model.

8. The network model quantization method according to any one of claims 1 to 7, characterized in that, The preset quantization conditions include: the computational load of the second network model reaches a preset computational load threshold.

9. The network model quantization method according to any one of claims 1 to 7, characterized in that, The network model quantization method also includes: A quantization-aware training process is performed on the third network model to obtain the fourth network model.

10. The network model quantization method according to any one of claims 1 to 7, characterized in that, The first network model includes at least one of a neural network model and a diffusion model.

11. A network model quantization device, characterized in that, The network model quantization device includes: A quantization module is used to perform a quantization process on a first network model to reduce each target configuration parameter of the first network model from a first bit width to a second bit width to obtain a second network model. The target configuration parameters include at least some model parameters and / or model features of the first network model. The determining module is used to determine the quantization gradient of each of the target configuration parameters with respect to the model quantization loss value between the first network model and the second network model, and based on the quantization gradient, determine the degree of influence of each of the target configuration parameters on the model quantization loss value; An enhancement module is used to determine the priority order of each target configuration parameter based on the degree of influence, and to increase at least some of the target configuration parameters from the second bit width to the third bit width in sequence according to the priority order, until a preset quantization condition is met to obtain a third network model, wherein the third bit width of each target configuration parameter is less than or equal to the first bit width.

12. The network model quantization device according to claim 11, characterized in that, The determining module is also used for: Preset samples are input into the first network model and the second network model respectively to obtain the first output result and the second output result; Based on the first output result, the second output result, and the preset loss function, the model quantization loss value between the first network model and the second network model is determined; Based on the model quantization loss value and the second network model, the quantization gradient of each of the target configuration parameters under the second bit width is determined.

13. The network model quantization device according to claim 11, characterized in that, The determining module is also used for: Based on the quantization gradient of each of the target configuration parameters, an approximate value of the quantization error of each of the target configuration parameters relative to the model quantization loss value is determined; Based on the approximate quantization error values ​​of each of the target configuration parameters, the degree of influence of each of the target configuration parameters on the model quantization loss value is determined.

14. The model quantization device according to claim 13, characterized in that, The determining module is also used for: The target configuration parameter is divided into multiple configuration parameter sets, each configuration parameter set including at least one target configuration parameter, and the target configuration parameter in the same configuration parameter set corresponds to the same hardware supported bit width; For each set of configuration parameters, the degree of influence of the set of configuration parameters on the quantization loss value of the model is determined based on the approximate quantization error value of each target configuration parameter in the set of configuration parameters.

15. An electronic device, characterized in that, The electronic device includes: processor; Memory used to store processor-executable instructions; The processor is configured to perform the network model quantization method as described in any one of claims 1 to 10.

16. A non-transitory computer-readable storage medium, characterized in that, When the instructions in the storage medium are executed by the processor of the electronic device, the electronic device is able to perform the network model quantization method as described in any one of claims 1 to 10.

17. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the network model quantization method as described in any one of claims 1 to 10.

18. A chip, characterized in that, The chip is configured to perform the network model quantization method as described in any one of claims 1 to 10.

Images