Self-adaptive model quantification method, device, equipment, medium and program product
Through the adaptive optimization model quantization method, combined with the target quantization formula and the search space to optimize the initial hyperparameters, the performance loss and hardware adaptability problems in complex tasks are solved, and more efficient model quantization effect is achieved.
Patent Information
- Application Number
- CN202411332642.1
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-09-24
- Publication Date
- 2025-02-28
AI Technical Summary
In more complex tasks such as object detection and semantic segmentation, existing model quantization techniques lead to large performance losses, and it is difficult to organically combine different types of model compression algorithms and adapt to different hardware limitations.
By establishing the target quantization formula of the target model and determining the search space, the initial single hyperparameter is optimized, the weights are cropped to obtain the cropped weights, and the quantization error is determined based on these weights to achieve adaptive model quantization.
It achieves better accuracy and performance balance in complex tasks, and adapts to different hardware platforms through adaptive adjustments, enhancing the universality and practicality of the model.
Smart Images
Figure CN119539114A_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the field of model quantization technology, and in particular to an adaptive model quantization method, device, equipment, medium and program product. Background Art
[0002] Model quantization aims to compress the model size by reducing the number of bits of model weights, thereby improving the model's inference speed.
[0003] In the related technology, the binary weight recoding method can be used to introduce binary weights and activation calculation parameter gradients during network training, and bit operations can be used instead of arithmetic operations in the forward transmission process; the XOR network can be introduced to binary quantize the input and filter weights; the 32-bit floating point number can be quantized into 3 bits based on the retrained fixed-point algorithm; the Hessian approximation that directly minimizes the loss of the binary weight can be optimized through the diagonal approximation Newton algorithm; the sparse coding idea can be used to construct multiple sparse codebooks to achieve sparse coding; the bit width controller can also be analogized to the action of reinforcement learning based on the dynamic network quantization framework of reinforcement learning, and then the policy gradient training can be used to balance the accuracy and compression ratio.
[0004] However, in related technologies, performance loss is large in more complex tasks such as target detection and semantic segmentation. In addition, for a specific model structure and compression target, it is difficult to organically combine different types of model compression algorithms. Moreover, due to the different existing hardware constraints, it is difficult to adapt to hardware requirements and improvement is urgently needed. Summary of the invention
[0005] The present application provides an adaptive model quantization method, apparatus, device, medium and program product to solve the problems in related technologies, such as large performance loss in more complex tasks such as target detection and semantic segmentation. In addition, for a specific model structure and compression target, it is difficult to organically combine different types of model compression algorithms, and it is difficult to adapt to hardware requirements due to different existing hardware restrictions.
[0006] The first aspect of the present application provides an adaptive model quantization method, comprising the following steps: establishing a target quantization formula for weights in the target model based on the target model; determining a search space for the target model according to a scaling factor of the target model; optimizing an initial single hyperparameter of the search space in combination with the search space and the target quantization formula to obtain a final single hyperparameter; based on the final single hyperparameter, clipping the weights to obtain clipped weights, and determining a quantization error of the target model based on the clipped weights.
[0007] Optionally, in one embodiment of the present application, before optimizing the initial single hyperparameter of the search space in combination with the search space and the target quantization formula to obtain the final single hyperparameter, it also includes: establishing a loss function of the weight in combination with the search space and the target quantization formula; obtaining a first quantization error of the weight based on the loss function; determining whether the first quantization error is greater than a set threshold; if the first quantization error is greater than or equal to the set threshold, allowing the initial single hyperparameter to be optimized, otherwise the initial single hyperparameter is not allowed to be optimized.
[0008] Optionally, in one embodiment of the present application, the combination of the search space and the target quantization formula optimizes the initial single hyperparameter of the search space to obtain a final single hyperparameter, including: obtaining a training set of the loss function based on a training set of the target model; using the training set to optimize the initial single hyperparameter, iteratively optimizing until the first quantization error is less than the set threshold to obtain the final single hyperparameter.
[0009] Optionally, in one embodiment of the present application, before the weight is clipped based on the final single hyperparameter to obtain the clipped weight, it also includes: obtaining a second quantization error of the weight based on the final single hyperparameter; judging whether the second quantization error reaches a preset minimum value; if the second quantization error reaches the preset minimum value, clipping of the weight is allowed, otherwise clipping of the weight is not allowed.
[0010] Optionally, in one embodiment of the present application, the expression of the final single hyperparameter may be, but is not limited to:
[0011]
[0012] Among them, S X is the average activation of each channel, α is a single hyperparameter that balances the protection of important and non-important weights, and the loss function L(S) = ||S·Q(S -1 w)x-wx||2 represents the quantization error of the output after quantization.
[0013] The second aspect of the present application provides an adaptive model quantization device, including: a first establishment module, used to establish a target quantization formula for weights in the target model based on the target model; a determination module, used to determine the search space of the target model according to the scaling factor of the target model; a first generation module, used to optimize the initial single hyperparameter of the search space in combination with the search space and the target quantization formula to obtain a final single hyperparameter; a first clipping module, used to clip the weights based on the final single hyperparameter to obtain clipped weights, and determine the quantization error of the target model based on the clipped weights.
[0014] Optionally, in one embodiment of the present application, it also includes: a second establishment module, which is used to establish a loss function of the weight in combination with the search space and the target quantization formula before optimizing the initial single hyperparameter of the search space in combination with the search space and the target quantization formula to obtain the final single hyperparameter; a second generation module, which is used to obtain a first quantization error of the weight based on the loss function; a judgment module, which is used to judge whether the first quantization error is greater than a set threshold; and an optimization module, which is used to allow the initial single hyperparameter to be optimized when the first quantization error is greater than or equal to the set threshold, otherwise the initial single hyperparameter is not allowed to be optimized.
[0015] Optionally, in one embodiment of the present application, the second establishment module includes: an acquisition unit, used to obtain a training set of the loss function based on the training set of the target model; an optimization unit, used to optimize the initial single hyperparameter using the training set, and iteratively optimize until the first quantization error is less than the set threshold to obtain the final single hyperparameter.
[0016] Optionally, in one embodiment of the present application, it also includes: a third generation module, used to obtain a second quantization error of the weight based on the final single hyperparameter before clipping the weight based on the final single hyperparameter to obtain the clipped weight; a judgment module, used to judge whether the second quantization error reaches a preset minimum value; a second clipping module, used to allow clipping of the weight when the second quantization error reaches the preset minimum value, otherwise not allowing clipping of the weight.
[0017] Optionally, in one embodiment of the present application, the expression of the final single hyperparameter may be, but is not limited to:
[0018]
[0019] Among them, S Xis the average activation of each channel, α is a single hyperparameter that balances the protection of important and non-important weights, and the loss function L(S) = ||S·Q(S -1 w)x-wx||2 represents the quantization error of the output after quantization.
[0020] The third aspect of the present application provides an electronic device, comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the adaptive model quantization method as described in the above embodiment.
[0021] The fourth aspect of the present application provides a computer-readable storage medium, which stores a computer program, and when the program is executed by a processor, it implements the above-mentioned adaptive model quantization method.
[0022] The fifth aspect of the present application provides a computer program product, including a computer program, which implements the above-mentioned adaptive model quantization method when executed.
[0023] The embodiment of the present application can determine the search space of the target model in combination with the target quantization formula and the scaling factor, and then optimize the initial single hyperparameter of the search space, trim the weights, and then determine the quantization error of the target model. By analyzing the error caused by weight quantization, the initial single hyperparameter is adaptively adjusted to meet the needs of different models and tasks, thereby achieving a better balance between accuracy and performance. In addition, the embodiment of the present application can also be adaptively adjusted according to the characteristics of different hardware platforms, so that the quantized model can be efficiently run on a variety of hardware, enhancing the versatility and practicality of the model. Thus, the problem in the related art that in more complex tasks such as target detection and semantic segmentation, the performance loss is large. In addition, for a specific model structure and compression target, it is difficult to organically combine different types of model compression algorithms, and it is difficult to adapt to hardware requirements due to the different existing hardware restrictions.
[0024] Additional aspects and advantages of the present application will be given in part in the description below, and in part will become apparent from the description below, or will be learned through the practice of the present application. BRIEF DESCRIPTION OF THE DRAWINGS
[0025] The above and / or additional aspects and advantages of the present application will become apparent and easily understood from the following description of the embodiments in conjunction with the accompanying drawings, in which:
[0026] Figure 1 A flowchart of an adaptive model quantization method provided according to an embodiment of the present application;
[0027] Figure 2A flowchart of the working principle of an adaptive model quantization method provided according to an embodiment of the present application;
[0028] Figure 3 A block diagram of an adaptive model quantization device provided according to an embodiment of the present application;
[0029] Figure 4 It is a schematic diagram of the structure of an electronic device provided according to an embodiment of the present application. DETAILED DESCRIPTION
[0030] Embodiments of the present application are described in detail below, and examples of the embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are intended to be used to explain the present application, and should not be construed as limiting the present application.
[0031] The following describes the adaptive model quantization method, device, equipment, medium and program product of the embodiment of the present application with reference to the accompanying drawings. In view of the fact that the performance loss is large in more complex tasks such as target detection and semantic segmentation mentioned in the above background technology, in addition, for a specific model structure and compression target, it is difficult to organically combine different types of model compression algorithms, and due to the different existing hardware restrictions, it is difficult to adapt to the problem of hardware requirements, the present application provides an adaptive model quantization method, in which the target quantization formula and the scaling factor can be combined to determine the search space of the target model, and then optimize the initial single hyperparameter of the search space, and the weight is clipped to determine the quantization error of the target model, and by analyzing the error caused by the weight quantization, the initial single hyperparameter is adaptively adjusted to meet the needs of different models and tasks, so as to achieve a better balance of accuracy and performance. In addition, the embodiment of the present application can also be adaptively adjusted according to the characteristics of different hardware platforms, so that the quantized model can be efficiently operated on a variety of hardware, enhancing the versatility and practicality of the model. This solves the problem in related technologies that in more complex tasks such as object detection and semantic segmentation, there is a large performance loss. In addition, for a specific model structure and compression target, it is difficult to organically combine different types of model compression algorithms, and it is difficult to adapt to hardware requirements due to different existing hardware constraints.
[0032] Specifically, Figure 1 The present invention provides a flowchart of an adaptive model quantization method according to an embodiment of the present application.
[0033] like Figure 1 As shown, the adaptive model quantization method includes the following steps:
[0034] In step S101, a target quantization formula of weights in the target model is established based on the target model.
[0035] It can be understood by those skilled in the art that the quantization methods, quantization parameters, hardware constraints, etc. corresponding to different target model structures in the embodiments of the present application are different. Therefore, the embodiments of the present application can establish target quantization formulas for weights according to different target models.
[0036] For example, the embodiment of the present application may assume that a set of weights in the target model is w, and the corresponding linear operation may be, but is not limited to, expressed as y=wx, and the corresponding quantization form is y=Q(w)x. The specific target quantization formula may be, but is not limited to, expressed as:
[0037]
[0038] in, N is the number of quantization bits, and Δ is the quantization scaling factor determined by the absolute maximum value.
[0039] In step S102, a search space of the target model is determined according to the scaling factor of the target model.
[0040] It can be understood that the principle of activation perception in the embodiment of the present application can be understood as reducing the quantization error of important weights by scaling each channel in the target model. This principle shows that the importance of the weight channel is determined by the activation scale.
[0041] Exemplarily, the embodiment of the present application may define a simple search space according to factors affecting the scaling factor. In this case, the search space defined in the embodiment of the present application may be, but is not limited to, expressed as:
[0042]
[0043] Among them, S X is the mean activation of each channel and α is a single hyperparameter that balances the protection of important and non-important weights.
[0044] Optionally, in one embodiment of the present application, before optimizing the initial single hyperparameter of the search space in combination with the search space and the target quantization formula to obtain the final single hyperparameter, it also includes: establishing a loss function of the weight in combination with the search space and the target quantization formula; obtaining a first quantization error of the weight based on the loss function; determining whether the first quantization error is greater than a set threshold; if the first quantization error is greater than or equal to the set threshold, allowing the initial single hyperparameter to be optimized, otherwise the initial single hyperparameter is not allowed to be optimized.
[0045] As a possible implementation method, the embodiment of the present application can first establish a loss function for the weight before combining the search space and the target quantization formula to optimize the initial single hyperparameter of the search space to obtain the final single hyperparameter, and then calculate the first quantization error of the weight based on the loss function, and determine whether the first quantization error is greater than a set threshold. If the first quantization error is greater than or equal to the set threshold, optimization of the initial single hyperparameter is allowed, otherwise optimization of the initial single hyperparameter is not allowed.
[0046] It can be understood that in the embodiment of the present application, if the first quantization error is greater than or equal to the set threshold, the initial single hyperparameter is allowed to be adaptively optimized for quantization-aware training, that is, the initial single hyperparameter α is set as a learnable parameter, a selected part of the training set is used, and the loss function L(S) is used to perform back propagation optimization on the training set to optimize the learnable parameter, otherwise the learnable parameter is not allowed to be optimized. Among them, the set threshold can be set by a person skilled in the art according to actual conditions, and this application does not impose specific restrictions.
[0047] In addition, in the embodiment of the present application, the expression of the loss function may be, but is not limited to,:
[0048] L(S)=||S·Q(S -1 w)x-wx||2,
[0049] In step S103, the initial single hyperparameter of the search space is optimized by combining the search space and the target quantization formula to obtain a final single hyperparameter. The expression of the final single hyperparameter may be, but is not limited to,:
[0050]
[0051] Among them, S X is the average activation of each channel, α is a single hyperparameter that balances the protection of important and non-important weights, and the loss function L(S) = ||S·Q(S -1 w)x-wx||2 represents the quantization error of the output after quantization.
[0052] In the actual implementation process, the embodiment of the present application can combine the search space and the target quantization formula to optimize the initial single hyperparameter of the search space, and then obtain the final single hyperparameter. The expression of the final single hyperparameter can be but is not limited to:
[0053]
[0054] Among them, S X is the average activation of each channel, α is a single hyperparameter that balances the protection of important and non-important weights, and the loss function L(S) = ||S·Q(S -1w)x-wx||2 represents the quantization error of the output after quantization.
[0055] Optionally, in one embodiment of the present application, the initial single hyperparameter of the search space is optimized in combination with the target quantization formula to obtain the final single hyperparameter, including: obtaining a training set of the loss function based on the training set of the target model; using the training set to optimize the initial single hyperparameter, iteratively optimizing until the first quantization error is less than a set threshold to obtain the final single hyperparameter.
[0056] It can be understood by those skilled in the art that the embodiment of the present application can select a part of the training set of the target model as the training set of the loss function, use the training set obtained above to optimize the initial single hyperparameter, iterate the optimization until the first quantization error is less than the set threshold, and then obtain the final single hyperparameter.
[0057] Exemplarily, the embodiment of the present application can perform adaptive optimization of quantization-aware training on the initial single hyperparameter, that is, set the initial single hyperparameter α as a learnable parameter, use a selected part of the training set, and use the loss function L(S) to perform back propagation optimization on the training set to minimize the target: To get the final single hyperparameter.
[0058] Among them, in the embodiment of the present application, α is a single hyperparameter that balances the protection of important and non-important weights, and the importance evaluation of weights is mainly determined by the activation scale rather than the weight scale. Input features with larger activation values are usually more important, and retaining the corresponding weights can maintain these features, thereby improving the performance of the target model.
[0059] Optionally, in one embodiment of the present application, before clipping the weights based on the final single hyperparameter to obtain the clipped weights, it also includes: obtaining a second quantization error of the weights based on the final single hyperparameter; determining whether the second quantization error reaches a preset minimum value; if the second quantization error reaches the preset minimum value, clipping of the weights is allowed, otherwise clipping of the weights is not allowed.
[0060] In some embodiments, the embodiments of the present application can calculate the second quantization error of the target model weight based on the final single hyperparameter, and determine whether the second quantization error reaches a certain minimum value. If it does not reach the minimum value, the weight is trimmed.
[0061] In some embodiments, the embodiments of the present application can calculate the second quantization error of the target model weight based on the final single hyperparameter, and determine whether the second quantization error reaches a certain minimum value. If it reaches the minimum value, the weight is not clipped.
[0062] Among them, the certain minimum value can be set by technicians in this field according to actual conditions, and this application does not impose any specific restrictions.
[0063] In step S104, the weights are pruned based on the final single hyperparameter to obtain pruned weights, and the quantization error of the target model is determined based on the pruned weights.
[0064] In the actual implementation process, after obtaining the final single hyperparameter, the embodiment of the present application can further trim the weights to minimize the quantization error L(S) of the target model and determine the quantization error of the target model.
[0065] The following is a flowchart of the working principle of the adaptive model quantization method proposed in the embodiment of the present application using a specific embodiment.
[0066] Figure 2 The present invention is a flowchart of the working principle of an adaptive model quantization method provided according to an embodiment of the present application.
[0067] Step S201: establishing a target quantization formula for the target model weight to analyze the quantization error of the target model.
[0068] It is understandable that the embodiments of the present application can establish a corresponding target quantization formula according to the weight of the target model.
[0069] Step S202: Evaluate weight importance and retain important weights.
[0070] It can be understood that the embodiment of the present application can divide weights into important weights and unimportant weights, and their importance evaluation is mainly determined by the activation scale rather than the weight scale.
[0071] Step S203: Activate perceptual scaling to reduce the first quantization error of important weights by per-channel scaling of the target model.
[0072] It can be understood that the embodiments of the present application can calculate the first quantization error of the weight by establishing a loss function based on the activation-aware scaling principle.
[0073] Step S204: determining whether the first quantization error is greater than or equal to a set threshold.
[0074] It can be understood that the embodiment of the present application can execute step S205 to optimize the initial single hyperparameter to obtain the final single hyperparameter when the first quantization error is greater than or equal to the set threshold. Otherwise, the initial single hyperparameter is not optimized and step S206 is executed.
[0075] Step S205: Optimize the initial single hyperparameter to obtain the final single hyperparameter.
[0076] It can be understood that the embodiment of the present application can perform adaptive optimization of quantization-aware training on the initial single hyperparameter, that is, set the initial single hyperparameter α as a learnable parameter, use a selected part of the training set, and use the loss function L(S) to perform back propagation optimization on the training set to minimize the target: To get the final single hyperparameter.
[0077] Step S206: trimming the weights.
[0078] It is understandable that, after obtaining the final single hyperparameter, the embodiment of the present application can further trim the weights to minimize the quantization error L(S) of the target model.
[0079] According to the adaptive model quantization method proposed in the embodiment of the present application, the search space of the target model can be determined by combining the target quantization formula and the scaling factor, and then the initial single hyperparameter of the search space can be optimized, the weights can be clipped, and then the quantization error of the target model can be determined. By analyzing the error caused by weight quantization, the initial single hyperparameter can be adaptively adjusted to meet the needs of different models and tasks, thereby achieving a better balance between accuracy and performance. In addition, the embodiment of the present application can also be adaptively adjusted according to the characteristics of different hardware platforms, so that the quantized model can be efficiently run on a variety of hardware, thereby enhancing the versatility and practicality of the model. Thus, the problem in the related art that in more complex tasks such as target detection and semantic segmentation, the performance loss is large. In addition, for a specific model structure and compression target, it is difficult to organically combine different types of model compression algorithms, and it is difficult to adapt to hardware requirements due to the different existing hardware restrictions.
[0080] Next, an adaptive model quantization device proposed according to an embodiment of the present application is described with reference to the accompanying drawings.
[0081] Figure 3 A block diagram of an adaptive model quantization device provided according to an embodiment of the present application.
[0082] like Figure 3 As shown, the adaptive model quantization device 10 includes: a first establishing module 100 , a determining module 200 , a first generating module 300 and a first cutting module 400 .
[0083] The first establishing module 100 is used to establish a target quantization formula of weights in the target model based on the target model.
[0084] The determination module 200 is used to determine the search space of the target model according to the scaling factor of the target model.
[0085] The first generation module 300 is used to optimize the initial single hyperparameter of the search space by combining the search space and the target quantization formula to obtain the final single hyperparameter.
[0086] The first clipping module 400 is used to clip the weights based on the final single hyperparameter to obtain clipped weights, and determine the quantization error of the target model based on the clipped weights.
[0087] Optionally, in one embodiment of the present application, it also includes: a second establishing module, a second generating module, a judging module and an optimizing module.
[0088] Among them, the second establishment module is used to establish a weight loss function by combining the search space and the target quantization formula before optimizing the initial single hyperparameter of the search space to obtain the final single hyperparameter.
[0089] The second generating module is used to obtain a first quantization error of the weight based on the loss function.
[0090] The judging module is used to judge whether the first quantization error is greater than a set threshold.
[0091] The optimization module is used to allow the optimization of the initial single hyperparameter when the first quantization error is greater than or equal to a set threshold, and otherwise does not allow the optimization of the initial single hyperparameter.
[0092] Optionally, in one embodiment of the present application, the second establishing module includes: an acquisition unit and an optimization unit.
[0093] The acquisition unit is used to obtain a training set of the loss function based on the training set of the target model.
[0094] The optimization unit is used to optimize the initial single hyperparameter using the training set, and iteratively optimize until the first quantization error is less than a set threshold to obtain the final single hyperparameter.
[0095] Optionally, in one embodiment of the present application, it further includes: a third generating module, a judging module and a second cutting module.
[0096] The third generation module is used to obtain a second quantization error of the weight based on the final single hyperparameter before the weight is pruned based on the final single hyperparameter to obtain the pruned weight.
[0097] The judging module is used to judge whether the second quantization error reaches a preset minimum value.
[0098] The second clipping module is used to allow the weight to be clipped when the second quantization error reaches a preset minimum value, and otherwise does not allow the weight to be clipped.
[0099] Optionally, in one embodiment of the present application, the final expression of a single hyperparameter may be, but is not limited to:
[0100]
[0101] Among them, S X is the average activation of each channel, α is a single hyperparameter that balances the protection of important and non-important weights, and the loss function L(S) = ||S·Q(S -1 w)x-wx||2 represents the quantization error of the output after quantization.
[0102] It should be noted that the above explanation of the embodiment of the adaptive model quantization method is also applicable to the adaptive model quantization device of this embodiment, and will not be repeated here.
[0103] According to the adaptive model quantization device proposed in the embodiment of the present application, the search space of the target model can be determined by combining the target quantization formula and the scaling factor, and then the initial single hyperparameter of the search space can be optimized, the weights can be clipped, and then the quantization error of the target model can be determined. By analyzing the error caused by weight quantization, the initial single hyperparameter can be adaptively adjusted to meet the needs of different models and tasks, thereby achieving a better balance between accuracy and performance. In addition, the embodiment of the present application can also be adaptively adjusted according to the characteristics of different hardware platforms, so that the quantized model can be efficiently run on a variety of hardware, thereby enhancing the versatility and practicality of the model. Thus, the problem in the related art that in more complex tasks such as target detection and semantic segmentation, the performance loss is large, and in addition, for a specific model structure and compression target, it is difficult to organically combine different types of model compression algorithms, and due to the different existing hardware restrictions, it is difficult to adapt to hardware requirements.
[0104] Figure 4 This is a schematic diagram of the structure of an electronic device provided according to an embodiment of the present application. The electronic device may include:
[0105] Memory 401 , processor 402 , and a computer program stored in the memory 401 and executable on the processor 402 .
[0106] When the processor 402 executes the program, the adaptive model quantization method provided in the above embodiment is implemented.
[0107] Furthermore, the electronic device further comprises:
[0108] The communication interface 403 is used for communication between the memory 401 and the processor 402 .
[0109] The memory 401 is used to store computer programs that can be executed on the processor 402 .
[0110] The memory 401 may include a high-speed RAM memory, and may also include a non-volatile memory (non-volatile memory), such as at least one disk memory.
[0111] If the memory 401, the processor 402 and the communication interface 403 are implemented independently, the communication interface 403, the memory 401 and the processor 402 can be connected to each other through a bus and communicate with each other. The bus can be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus. The bus can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 4 Only one thick line is used in the diagram, but this does not mean that there is only one bus or only one type of bus.
[0112] Optionally, in a specific implementation, if the memory 401, the processor 402 and the communication interface 403 are integrated on a chip, the memory 401, the processor 402 and the communication interface 403 can communicate with each other through an internal interface.
[0113] The processor 402 may be a central processing unit (CPU), or an application specific integrated circuit (ASIC), or one or more integrated circuits configured to implement the embodiments of the present application.
[0114] An embodiment of the present application also provides a computer-readable storage medium having a computer program stored thereon, which implements the above-mentioned adaptive model quantization method when executed by a processor.
[0115] An embodiment of the present application also provides a computer program product, including a computer program, which implements the above-mentioned adaptive model quantization method when executed.
[0116] In the description of this specification, the description with reference to the terms "one embodiment", "some embodiments", "example", "specific example", or "some examples" etc. means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present application. In this specification, the schematic representations of the above terms do not necessarily refer to the same embodiment or example. And the specific features, structures, materials or characteristics described may be combined in any one or N embodiments or examples in a suitable manner. In addition, those skilled in the art may combine and combine the different embodiments or examples described in this specification and the features of the different embodiments or examples, unless they contradict each other.
[0117] In addition, the terms "first" and "second" are used for descriptive purposes only and should not be understood as indicating or implying relative importance or implicitly indicating the number of technical features indicated. Therefore, a feature defined as "first" or "second" may explicitly or implicitly include at least one of the features. In the description of this application, "N" means at least two, such as two, three, etc., unless otherwise clearly and specifically defined.
[0118] Any process or method description in a flowchart or otherwise described herein may be understood to represent a module, fragment or portion of code comprising one or N executable instructions for implementing the steps of a custom logical function or process, and the scope of the preferred embodiments of the present application includes alternative implementations in which functions may not be performed in the order shown or discussed, including performing functions in a substantially simultaneous manner or in reverse order depending on the functions involved, which should be understood by technicians in the technical field to which the embodiments of the present application belong.
[0119] The logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as an ordered list of executable instructions for implementing logical functions, and can be embodied in any computer-readable medium for use by an instruction execution system, device or apparatus (such as a computer-based system, a system including a processor, or other system that can fetch instructions from an instruction execution system, device or apparatus and execute instructions), or in combination with these instruction execution systems, devices or apparatuses. For the purpose of this specification, "computer-readable medium" can be any device that can contain, store, communicate, propagate or transmit a program for use by an instruction execution system, device or apparatus, or in combination with these instruction execution systems, devices or apparatuses. More specific examples of computer-readable media (a non-exhaustive list) include the following: an electrical connection with one or N wirings (electronic devices), a portable computer disk box (magnetic device), a random access memory (RAM), a read-only memory (ROM), an erasable and programmable read-only memory (EPROM or flash memory), a fiber optic device, and a portable compact disk read-only memory (CDROM). In addition, the computer-readable medium may even be paper or other suitable medium on which the program is printed, since the program may be obtained electronically by optically scanning the paper or other medium and then editing, interpreting or processing in other suitable ways as necessary and then storing it in a computer memory.
[0120] It should be understood that the various parts of the present application can be implemented by hardware, software, firmware or a combination thereof. In the above embodiment, the N steps or methods can be implemented by software or firmware stored in a memory and executed by a suitable instruction execution system. If implemented by hardware, as in another embodiment, it can be implemented by any one or a combination of multiple of the following technologies known in the art: a discrete logic circuit having a logic gate circuit for implementing a logic function for a data signal, a dedicated integrated circuit having a suitable combination of logic gate circuits, a programmable gate array (PGA), a field programmable gate array (FPGA), etc.
[0121] A person skilled in the art may understand that all or part of the steps in the method for implementing the above-mentioned embodiment may be completed by instructing related hardware through a program, and the program may be stored in a computer-readable storage medium, which, when executed, includes one or a combination of the steps of the method embodiment.
[0122] In addition, each functional unit in each embodiment of the present application may be integrated into a processing module, or each unit may exist physically separately, or two or more units may be integrated into one module. The above-mentioned integrated module may be implemented in the form of hardware or in the form of a software functional module. If the integrated module is implemented in the form of a software functional module and sold or used as an independent product, it may also be stored in a computer-readable storage medium.
[0123] The storage medium mentioned above may be a read-only memory, a magnetic disk or an optical disk, etc. Although the embodiments of the present application have been shown and described above, it can be understood that the above embodiments are exemplary and cannot be understood as limiting the present application. A person of ordinary skill in the art may change, modify, replace and modify the above embodiments within the scope of the present application.
Claims
1. An adaptive model quantization method, characterized in that: The following steps are involved: Establishing a target quantization formula for weights in the target model based on the target model; Determining a search space of the target model according to a scaling factor of the target model; Optimizing an initial single hyperparameter of the search space in combination with the target quantization formula to obtain a final single hyperparameter; Based on the final single hyperparameter, the weight is clipped to obtain a clipped weight, and a quantization error of the target model is determined based on the clipped weight.
2. The method according to claim 1, characterized in that Before optimizing the initial single hyperparameter of the search space by combining the search space and the target quantization formula to obtain a final single hyperparameter, the method further includes: Establishing a loss function of the weight by combining the search space and the target quantization formula; Obtaining a first quantization error of the weight based on the loss function; Determining whether the first quantization error is greater than a set threshold; If the first quantization error is greater than or equal to the set threshold, the initial single hyperparameter is allowed to be optimized; otherwise, the initial single hyperparameter is not allowed to be optimized.
3. The method according to claim 2, characterized in that The step of optimizing an initial single hyperparameter of the search space by combining the search space and the target quantization formula to obtain a final single hyperparameter includes: Obtaining a training set of the loss function based on the training set of the target model; The initial single hyperparameter is optimized using the training set, and the optimization is iteratively performed until the first quantization error is less than the set threshold, so as to obtain the final single hyperparameter.
4. The method according to claim 1, characterized in that: Before clipping the weight based on the final single hyperparameter to obtain the clipped weight, the method further includes: Based on the final single hyperparameter, obtaining a second quantization error of the weight; Determining whether the second quantization error reaches a preset minimum value; If the second quantization error reaches the preset minimum value, the weight is allowed to be clipped; otherwise, the weight is not allowed to be clipped.
5. The method according to claim 1, characterized in that The expression of the final single hyperparameter is: Among them, S X is the average activation of each channel, α is a single hyperparameter that balances the protection of important and non-important weights, and the loss function L(S) = ||S·Q(S -1 w)x-wx||2 represents the quantization error of the output after quantization.
6. An adaptive model quantization device, characterized in that: include: A first establishing module is used to establish a target quantization formula of weights in the target model based on the target model; A determination module, configured to determine a search space of the target model according to a scaling factor of the target model; A first generating module, configured to optimize an initial single hyperparameter of the search space by combining the search space and the target quantization formula to obtain a final single hyperparameter; The first clipping module is used to clip the weight based on the final single hyperparameter to obtain the clipped weight, and determine the quantization error of the target model based on the clipped weight.
7. The device according to claim 6, characterized in that Also includes: A second establishment module is used to establish a loss function of the weight by combining the search space and the target quantization formula before optimizing the initial single hyperparameter of the search space to obtain a final single hyperparameter; A second generating module, used for obtaining a first quantization error of the weight based on the loss function; A judging module, used to judge whether the first quantization error is greater than a set threshold; The optimization module is used to allow the optimization of the initial single hyperparameter when the first quantization error is greater than or equal to the set threshold, and otherwise not allow the optimization of the initial single hyperparameter.
8. The device according to claim 7, characterized in that The second establishing module comprises: An acquisition unit, configured to obtain a training set of the loss function based on a training set of the target model; An optimization unit is used to optimize the initial single hyperparameter using the training set, and iteratively optimize until the first quantization error is less than the set threshold, so as to obtain the final single hyperparameter.
9. The device according to claim 6, characterized in that Also includes: A third generating module is used for obtaining a second quantization error of the weight based on the final single hyperparameter before clipping the weight based on the final single hyperparameter to obtain the clipped weight; A judging module, used for judging whether the second quantization error reaches a preset minimum value; The second clipping module is used to allow the weight to be clipped when the second quantization error reaches the preset minimum value, and otherwise does not allow the weight to be clipped.
10. The device according to claim 6, characterized in that The expression of the final single hyperparameter is: Among them, S X is the average activation of each channel, α is a single hyperparameter that balances the protection of important and non-important weights, and the loss function L(S) = ||S·Q(S -1 w)x-wx||2 represents the quantization error of the output after quantization.
11. An electronic device, characterized in that: include: A memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the adaptive model quantization method according to any one of claims 1 to 5.
12. A computer-readable storage medium having a computer program stored thereon, characterized in that: The program is executed by a processor to implement the adaptive model quantization method as described in any one of claims 1 to 5.
13. A computer program product, characterized in that It comprises a computer program, which, when executed, is used to implement the adaptive model quantization method as described in any one of claims 1 to 5.