Model quantification method and device, equipment and storage medium
By obtaining the performance deviation of the model layer, the layered quantization sequence is constructed and layered quantization process is performed, which solves the problem of model quantization performance loss under low bits, and achieves performance optimization and inter-device consistency in the decoder.
Patent Information
- Application Number
- CN202310871579.8
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2023-07-14
- Publication Date
- 2025-01-14
AI Technical Summary
The prior art cannot effectively reduce the performance loss caused by quantization of the model when using low bits, especially in neural networks in decoders. Each layer has inconsistent sensitivity to quantization, and some layers are difficult to ensure at lower bits. Quantization performance, and some devices do not support high-bit quantization operations.
By obtaining the performance deviations of each model layer in the decoded model to be quantized, a layered quantization sequence is constructed, and the model layer is layered based on this sequence. The model parameters are adjusted using the gradient derivative method to reduce the performance deviation and ensure that at low bits Minimize performance losses as much as possible.
This achieves a reduced performance loss when quantizing the model at low bits, ensuring decoding consistency between different devices and optimization of computing resources.
Smart Images
Figure CN119312864A_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the field of model quantization technology, and in particular to a model quantization method, device, equipment and storage medium. Background Art
[0002] In the field of image coding and decoding, AI-based methods can achieve higher compression rates than traditional methods, and have therefore become an important trend in the field of coding and decoding. This is due to the fact that networks trained through deep learning can better compress and restore images, but it also brings more computational and resource consumption than traditional methods, which undoubtedly brings higher requirements to decoding devices.
[0003] Image decoders are generally deployed on devices that decode bitstreams. The neural network in the decoder has a large number of parameters and computational complexity, which requires more storage and computing resources for its forward calculation. At the same time, under floating-point parameters, due to differences in calculation accuracy between different devices, the decoding results between different devices may be inconsistent.
[0004] Model quantization is a common method to solve the above problems. Quantizing the floating-point parameters in the model into fixed-point types can reduce the storage consumption of the model while reducing the bandwidth of the model's forward calculation. In addition, converting the floating-point parameters in the model into fixed-point parameters can reduce the amount of calculation of the model, avoid the impact of calculation accuracy, and ensure decoding consistency between different devices.
[0005] However, the sensitivity of each layer of the neural network in the decoder to quantization is inconsistent, and it is difficult to guarantee the quantization performance of some layers at lower bits. Since some devices do not support high-bit quantization operations, in order to adapt to this scenario, how to reduce the performance loss caused by quantizing the model at lower bits is a difficult problem.
[0006] The above contents are only used to assist in understanding the technical solution of the present application and do not constitute an admission that the above contents are prior art. Summary of the invention
[0007] The main purpose of this application is to provide a model quantization method, device, equipment and storage medium, aiming to solve the technical problem that the existing technology cannot reduce the performance loss caused by quantizing the model using low bits.
[0008] To achieve the above objectives, the present application provides a model quantization method, the method comprising:
[0009] Obtain the performance deviation corresponding to each model layer in the decoding model to be quantized;
[0010] Constructing a hierarchical quantization sequence according to the performance deviations corresponding to each model layer;
[0011] Based on the hierarchical quantization sequence, a hierarchical quantization process is performed on the model layer in the decoding model to be quantized.
[0012] In a possible implementation manner of the present application, constructing a hierarchical quantization sequence according to the performance deviations corresponding to the model layers includes:
[0013] The model layers in the decoding model to be quantized are sorted from large to small according to the corresponding performance deviations to obtain a hierarchical quantization sequence.
[0014] In a possible implementation manner of the present application, the step of sorting the model layers in the decoding model to be quantized from large to small according to the corresponding performance deviations to obtain a hierarchical quantization sequence includes:
[0015] Sorting the model layers in the decoding model to be quantized from large to small according to the corresponding performance deviations to obtain a model layer sequence;
[0016] The model layer sequence is cut from front to back according to a preset length to obtain a layered quantization sequence.
[0017] In a possible implementation of the present application, the performance deviation corresponding to the model layer is the deviation of the model output before and after the model layer in the decoding model to be quantized is individually quantized.
[0018] In a possible implementation manner of the present application, performing hierarchical quantization processing on the model layer in the decoding model to be quantized based on the hierarchical quantization sequence includes:
[0019] Traversing the hierarchical quantization sequence, and taking the traversed model layer as the current model layer;
[0020] Performing quantization processing on the current model layer;
[0021] With the goal of reducing the performance deviation corresponding to the current model layer, adjusting at least one model parameter in the decoding model to be quantized;
[0022] Detecting whether there are any model layers that have not been traversed in the hierarchical quantization sequence;
[0023] If there is a model layer that has not been traversed, the step of traversing the hierarchical quantization sequence is returned, and the traversed model layer is used as the current model layer.
[0024] In a possible implementation manner of the present application, adjusting at least one model parameter in the decoding model to be quantized with the goal of reducing the performance deviation corresponding to the current model layer includes:
[0025] A gradient derivation method is adopted to adjust at least one model parameter in the decoding model to be quantized with the goal of reducing the performance deviation corresponding to the current model layer.
[0026] In a possible implementation manner of the present application, the adjusting at least one model parameter in the decoding model to be quantized includes at least one of the following:
[0027] Adjusting at least one model parameter of the current model layer;
[0028] Adjusting at least one model parameter of the current model layer and a model layer adjacent to the current model layer;
[0029] At least one model parameter of each model layer in the decoding model to be quantized is adjusted.
[0030] In a possible implementation manner of the present application, after detecting whether there are any model layers that have not been traversed in the hierarchical quantization sequence, the method further includes:
[0031] If there is no model layer that has not been traversed, then detecting whether there is still an unquantized model layer in the decoding model to be quantized;
[0032] If there is no unquantized model layer, the traversal of the hierarchical quantization sequence is terminated.
[0033] In a possible implementation manner of the present application, if there is no model layer that has not been traversed, after detecting whether there is still an unquantized model layer in the decoding model to be quantized, the method further includes:
[0034] If there is an unquantized model layer, quantizing the unquantized model layer in the decoding model to be quantized;
[0035] When the quantization process is completed, the traversal of the hierarchical quantization sequence is terminated.
[0036] In addition, to achieve the above purpose, the present application also proposes a model quantization device, the model quantization device comprising:
[0037] An acquisition module, used to obtain the performance deviation corresponding to each model layer in the decoding model to be quantized;
[0038] A construction module, used to construct a hierarchical quantization sequence according to the performance deviations corresponding to each model layer;
[0039] A quantization module is used to perform hierarchical quantization processing on the model layer in the decoding model to be quantized based on the hierarchical quantization sequence.
[0040] In addition, to achieve the above-mentioned purpose, the present application also proposes a model quantization device, which includes: a processor, a memory, and a model quantization program stored in the memory and executable on the processor, and the model quantization program implements the steps of the model quantization method described above when executed by the processor.
[0041] In addition, to achieve the above-mentioned purpose, the present application also proposes a coding device, which includes: a processor, a memory, and a coding program stored in the memory and runnable on the processor, and when the coding program is executed by the processor, the steps of the model quantization method as described above are implemented.
[0042] In addition, to achieve the above-mentioned purpose, the present application also proposes a decoding device, which includes: a processor, a memory, and a decoding program stored in the memory and executable on the processor, and when the decoding program is executed by the processor, the steps of the model quantization method described above are implemented.
[0043] In addition, to achieve the above-mentioned purpose, the present application also proposes a storage medium, on which a model quantization program is stored, and when the model quantization program is executed, the steps of the model quantization method described above are implemented.
[0044] The present application obtains the performance deviation corresponding to each model layer in the decoding model to be quantized; constructs a hierarchical quantization sequence according to the performance deviation corresponding to each model layer; and performs hierarchical quantization processing on the model layers in the decoding model to be quantized based on the hierarchical quantization sequence. Since a hierarchical quantization sequence is constructed according to the performance deviation corresponding to each model layer in the decoding model to be quantized, and hierarchical quantization processing is performed on each model layer in the decoding model to be quantized according to the sequence order of the hierarchical quantization sequence, parameter adjustment correction is used after quantization during the processing process to minimize the performance loss of the model caused by low-bit quantization. BRIEF DESCRIPTION OF THE DRAWINGS
[0045] Figure 1 It is a structural diagram of an electronic device of a hardware operating environment involved in an embodiment of the present application;
[0046] Figure 2 This is a flow chart of the first embodiment of the model quantization method of the present application;
[0047] Figure 3 A schematic diagram of a coding and decoding architecture according to an embodiment of the present application;
[0048] Figure 4 This is a schematic diagram of the model layer structure of an embodiment of the present application;
[0049] Figure 5 This is a flow chart of the second embodiment of the model quantization method of the present application;
[0050] Figure 6 A schematic diagram of the correlation between model layer parameters according to an embodiment of the present application;
[0051] Figure 7 This is a flow chart of the second embodiment of the model quantization method of the present application;
[0052] Figure 8 This is a structural block diagram of the first embodiment of the model quantization device of the present application.
[0053] The realization of the purpose, functional features and advantages of this application will be further explained in conjunction with embodiments and with reference to the accompanying drawings. DETAILED DESCRIPTION
[0054] It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.
[0055] Reference Figure 1 , Figure 1 A schematic diagram of the structure of a model quantization device for a hardware operating environment involved in an embodiment of the present application.
[0056] like Figure 1 As shown, the electronic device may include: a processor 1001, such as a central processing unit (CPU), a communication bus 1002, a user interface 1003, a network interface 1004, and a memory 1005. Among them, the communication bus 1002 is used to realize the connection and communication between these components. The user interface 1003 may include a display screen (Display), an input unit such as a keyboard (Keyboard), and the optional user interface 1003 may also include a standard wired interface and a wireless interface. The network interface 1004 may optionally include a standard wired interface and a wireless interface (such as a wireless fidelity (WIreless-FIdelity, WI-FI) interface). The memory 1005 may be a high-speed random access memory (Random Access Memory, RAM), or a stable non-volatile memory (Non-Volatile Memory, NVM), such as a disk storage. The memory 1005 may also be a storage device independent of the aforementioned processor 1001.
[0057] Those skilled in the art will understand that Figure 1 The structure shown in the figure does not constitute a limitation on the electronic device, and may include more or less components than shown in the figure, or combine certain components, or arrange the components differently.
[0058] like Figure 1As shown, the memory 1005 as a storage medium may include an operating system, a network communication module, a user interface module, and a model quantization program.
[0059] exist Figure 1 In the electronic device shown, the network interface 1004 is mainly used for data communication with a network server; the user interface 1003 is mainly used for data interaction with a user; the processor 1001 and the memory 1005 in the electronic device of the present application can be set in a model quantization device, and the electronic device calls the model quantization program stored in the memory 1005 through the processor 1001, and executes the model quantization method provided in the embodiment of the present application.
[0060] The present application embodiment provides a model quantization method, referring to Figure 2 , Figure 2 This is a flow chart of a first embodiment of a model quantization method of the present application.
[0061] In this embodiment, the model quantization method includes the following steps:
[0062] Step S10: Obtain the performance deviation corresponding to each model layer in the decoding model to be quantized.
[0063] It should be noted that the executor of this embodiment can be the model quantization device, or it can be an encoding device or decoding device when encoding / decoding image data. The model quantization device can be an electronic device such as a personal computer or a server. Of course, it can also be other devices that can achieve the same or similar functions. This embodiment does not limit this. In this embodiment and the following embodiments, the model quantization method of this application is explained by taking the model quantization device as an example.
[0064] It should be noted that the decoding model to be quantized can be a neural network model contained in a decoder deployed in a device that decodes a bitstream. There can be multiple decoding models to be quantized. When there are multiple decoding models to be quantized, each decoding model to be quantized can be independently quantized using a model quantization method. The performance deviation corresponding to the model layer is the deviation of the model output before and after the model layer in the decoding model to be quantized is individually quantized.
[0065] For ease of understanding, now combined Figure 3 This is for illustration only, but not for limitation. Figure 3 Schematic diagram of the coding and decoding architecture of this embodiment. Figure 3 As shown in Figure 2, the image encoding and decoding process based on artificial intelligence is as follows: Figure 3As shown in the figure, the network models involved in the overall encoding and decoding process mainly include: Analysis Transform Net, Prediction Fusion Net, ContextModel Net, Hyper Decoder Net, Hyper Scale Decoder Net, Hyper Encoder Net, Synthesis Transform Net and ICCI filter. Among them, Figure 3 The lower middle part is the image decoder, which is generally deployed on the device that decodes the bitstream. The neural network model involved in the decoder (including SynthesisTransformNet and ICCI filter, etc.) can be used as a decoding model to be quantized.
[0066] In a specific implementation, obtaining the performance deviation corresponding to each model layer in the decoding model to be quantized can be to first input a preset sample into the decoding model to be quantized, and obtain the model output, marked as the initial output, and then quantize each model layer network in the decoding model to be quantized separately, and record the model output for the preset sample after quantization, so as to obtain the quantized output corresponding to each model layer, and calculate the difference between the initial output and the quantized output corresponding to each model layer, so as to obtain the performance deviation corresponding to each model layer in the decoding model to be quantized. Among them, since this embodiment is to reduce the performance loss caused by using low bits to quantize the model, when quantizing the model layer network in this embodiment, low bits can be used for quantization (for example, 8 bits are used for quantization).
[0067] For ease of understanding, now combined Figure 4 This is for illustration only, but not for limitation. Figure 4 is a schematic diagram of the model layer structure of this embodiment, such as Figure 4 As shown, for a model containing L layers (each model layer can be represented by w 1, w 2 ,......,w L ) is a decoding model to be quantized. At this time, the preset sample x is input into the decoding model to be quantized, and its model output can be expressed as y=f(w 1, w 2 ,......,w L ; x), each model layer is quantized separately. In this process, when the lth (1≤l≤L) model layer is quantized separately, each model layer can be expressed as in, represents the quantized l-th model layer, and the quantized output corresponding to the l-th model layer can be expressed as At this time, the performance deviation of the lth model layer
[0068] Step S20: constructing a hierarchical quantization sequence according to the performance deviations corresponding to each model layer.
[0069] It should be noted that constructing a hierarchical quantization sequence according to the performance deviation corresponding to each model layer may be to sort each model layer according to the performance deviation corresponding to each model layer, thereby constructing a hierarchical quantization sequence.
[0070] In actual use, in order to reduce the superposition of the noise gradient introduced by STE in each layer in the reverse derivation during the model quantization operation, it is necessary to start from the most difficult to quantize layer (the model layer with the largest performance deviation), and quantize the model layer by layer step by step. Since the larger the performance deviation, the greater the performance loss when quantizing the model layer, and in actual processing, it is necessary to first process the model layer with a larger performance loss. At this time, step S20 described in this embodiment may include:
[0071] The model layers in the decoding model to be quantized are sorted from large to small according to the corresponding performance deviations to obtain a hierarchical quantization sequence.
[0072] It is understandable that because the model layer that produces a large performance loss needs to be processed first, when constructing a hierarchical quantization sequence, it is necessary to ensure that the model layer with a large performance loss is located at the front of the sequence. At this time, the model layers in the quantization decoding model can be sorted from large to small according to the corresponding performance deviation to obtain a hierarchical quantization sequence.
[0073] For example: Assume that the model has three layers, A, B, and C, and their corresponding performance deviations are a, b, and c, respectively. If c>a>b exists, the hierarchical quantization sequence constructed at this time is CAB.
[0074] In a possible implementation of this embodiment, in order to speed up the quantization rate as much as possible, the step of sorting the model layers in the decoding model to be quantized from large to small according to the corresponding performance deviation in this embodiment to obtain a hierarchical quantization sequence may include:
[0075] Sorting the model layers in the decoding model to be quantized from large to small according to the corresponding performance deviations to obtain a model layer sequence;
[0076] The model layer sequence is cut from front to back according to a preset length to obtain a layered quantization sequence.
[0077] It should be noted that the preset length can be set in advance by the administrator of the model quantization device according to actual needs, for example: setting the preset length to 3.
[0078] In actual use, since the performance loss caused by quantizing the model using low bits may be caused by a very small part of all model layers, only a part of them can be processed when processing them. Therefore, the manager of the model quantization equipment can pre-set a preset length, and then sort the model layers in the decoding model to be quantized from large to small according to the corresponding performance deviation to obtain the model layer sequence, and then intercept the model layer sequence from front to back according to the preset length to obtain a layered quantization sequence.
[0079] For example, assuming that the preset length is 3 and the model layer sequence is ABCDE, then the model layer sequence is intercepted from front to back according to the preset length, and the obtained hierarchical quantization sequence is ABC.
[0080] Of course, since the performance deviation can be used to represent the performance loss when the model layer is quantized, it is also possible to not truncate the model layer sequence from front to back according to a preset length, but to preset a performance deviation threshold, and then sort the model layers in the decoding model to be quantized from large to small according to the corresponding performance deviation to obtain a model layer sequence, and truncate the part of the model layer sequence whose corresponding performance deviation is greater than the performance deviation threshold, and use the truncation part as a layered quantization sequence.
[0081] For example: assuming that the model layer sequence is ABCDE, and its corresponding performance deviations are a, b, c, d, and e, respectively, where a>b>c>d>e. At this time, if the performance deviation threshold is x, and d<x<c, then the hierarchical quantization sequence intercepted at this time is ABC.
[0082] Step S30: performing hierarchical quantization processing on the model layers in the decoding model to be quantized based on the hierarchical quantization sequence.
[0083] It should be noted that the hierarchical quantization processing of the model layers in the decoding model to be quantized based on the hierarchical quantization sequence can be based on the sequence order in the hierarchical quantization sequence, and the model layers in the decoding model to be quantized are sequentially subjected to hierarchical quantization processing. The hierarchical quantization processing can include steps such as model quantization and parameter adjustment.
[0084] This embodiment obtains the performance deviation corresponding to each model layer in the decoding model to be quantized; constructs a hierarchical quantization sequence according to the performance deviation corresponding to each model layer; and performs hierarchical quantization processing on the model layers in the decoding model to be quantized based on the hierarchical quantization sequence. Since the hierarchical quantization sequence is constructed according to the performance deviation corresponding to each model layer in the decoding model to be quantized, and the hierarchical quantization processing is performed on each model layer in the decoding model to be quantized according to the sequence order of the hierarchical quantization sequence, parameter adjustment correction is used after quantization in the processing process to minimize the performance loss of the model caused by low-bit quantization.
[0085] refer to Figure 5 , Figure 5 This is a flow chart of a second embodiment of a model quantization method of the present application.
[0086] Based on the above first embodiment, step S30 of the model quantization method of this embodiment includes:
[0087] Step S301: traverse the hierarchical quantization sequence, and use the traversed model layer as the current model layer.
[0088] It should be noted that traversing the hierarchical quantization sequence may be traversing the hierarchical quantization sequence according to the sequence order of the hierarchical quantization sequence.
[0089] Step S302: performing quantization processing on the current model layer.
[0090] It should be noted that quantizing the current model layer may be to use low bits (such as 8 bits) to quantize the model parameters in the current model layer, and to approximate the model parameters and input and output features represented by floating-point values using fixed-point values.
[0091] Step S303: with the goal of reducing the performance deviation corresponding to the current model layer, adjusting at least one model parameter in the decoding model to be quantized.
[0092] It should be noted that, with the goal of reducing the performance deviation corresponding to the current model layer, adjusting at least one model parameter in the decoding model to be quantized may be adjusting at least one model parameter in the decoding model to be quantized, thereby attempting to reduce the performance deviation corresponding to the current model layer. After each parameter adjustment, the performance deviation corresponding to the current model layer (i.e., the performance deviation generated after the current model layer is quantized) may be recalculated.
[0093] In actual use, the gradient derivation method can be used to adjust at least one model parameter in the quantized decoding model with the goal of reducing the performance deviation corresponding to the current model layer. For example, a direct gradient evaluation algorithm can be used to adjust at least one model parameter in the quantized decoding model with the goal of reducing the performance deviation corresponding to the current model layer. Of course, other similar algorithms can also be used.
[0094] When adjusting parameters using an algorithm, the algorithm module provided by the model framework can be called for adjustment. For example, if a straight-through gradient evaluation algorithm is used, the straight-through estimator (STE) in the model framework can be directly called. Of course, calculations can also be performed by oneself, and this embodiment does not limit this.
[0095] In practical applications, the parameters in each model layer are correlated. That is, if the parameters of one model layer are changed, the input and output of other model layers will be affected. The relationship between the model layers is as follows: Figure 6 As shown, quantizing a parameter in a layer causes a large deviation in the model output, resulting in a large performance loss in the model. The correlation between other parameters between the layers and the parameter can be used to adjust other parameters using the method of reconstructing the output to correct the deviation caused by quantization. In addition, the cross-layer correlation can be used to further correct the feature deviation caused by quantization. Therefore, when adjusting at least one model parameter in the decoding model to be quantized, at least one of the following can be included:
[0096] Adjusting at least one model parameter of the current model layer;
[0097] Adjusting at least one model parameter of the current model layer and a model layer adjacent to the current model layer;
[0098] At least one model parameter of each model layer in the decoding model to be quantized is adjusted.
[0099] Assuming that the current model layer is the lth model layer, at least one model parameter of the current model layer is adjusted to reduce the performance deviation P of the current model layer. l To optimize the target, the gradient derivation method is used to optimize the model parameters in the l-th model layer (since the l-th layer has been quantized, its model parameters can be expressed as ) Update with gradient;
[0100] Adjusting at least one model parameter of the current model layer and the model layer adjacent to the current model layer may be used to reduce the performance deviation P of the current model layer. l To optimize the model parameters in the l-th model layer and the l-1 and l+1 model layers Make updates;
[0101] Adjusting at least one model parameter of each model layer in the decoding model to be quantized can be used to reduce the performance deviation P of the current model layer. l The model parameters of all model layers in the decoding model to be quantized are updated for the optimization objective.
[0102] Step S304: Detect whether there are any model layers that have not been traversed in the hierarchical quantization sequence.
[0103] It should be noted that, in the process of traversing the hierarchical quantization sequence, the index value during traversal can actually be recorded, and it can be judged based on the index value whether the traversal of the hierarchical quantization sequence is completed. If the traversal is not completed, it can be determined that there are model layers that have not been traversed.
[0104] Of course, it is also possible to record the layer numbers of the traversed model layers during traversal, and mark the traversed model layers in the quantization sequence according to the layer numbers, and then determine whether there are any model layers that have not been traversed based on the marks. This embodiment does not limit this.
[0105] Step S305: If there is a model layer that has not been traversed, return to the step of traversing the hierarchical quantization sequence and taking the traversed model layer as the current model layer.
[0106] It is understandable that if there are model layers that have not been traversed, the layered quantization sequence can be continuously stacked for traversal. Therefore, the process can return to step S301 and continue to stack the layered quantization sequence for traversal.
[0107] This embodiment traverses the hierarchical quantization sequence and quantizes the current model layer traversed. After the processing is completed, at least one model parameter in the quantized decoding model is adjusted with the goal of reducing the performance deviation corresponding to the current model layer. The embodiment attempts to correct the model output through parameter adjustment to reduce the performance loss of the model caused by low-bit quantization, so as to minimize the performance loss of the overall model caused by low-bit quantization.
[0108] refer to Figure 7 , Figure 7 This is a flow chart of a third embodiment of a model quantization method of the present application.
[0109] Based on the above second embodiment, after step S304, the model quantization method of this embodiment further includes:
[0110] Step S305': if there is no model layer that has not been traversed, then detecting whether there is any model layer that has not been quantized in the decoding model to be quantized;
[0111] Step S306 ′: if there is no unquantized model layer, the traversal of the hierarchical quantization sequence is terminated.
[0112] It should be noted that since the hierarchical quantization sequence may be obtained by truncation, it may not contain all the model layers in the decoding model to be quantized. Therefore, when there are no untraversed model layers in the hierarchical quantization sequence, some model layers in the decoding model to be quantized may not have been quantized. Therefore, when there are no untraversed model layers in the hierarchical quantization sequence, it is also necessary to detect whether there are any unquantized model layers in the decoding model to be quantized.
[0113] Among them, detecting whether there are still unquantized model layers in the decoding model to be quantized can be detecting whether there are still model parameters or input-output features expressed in floating point type in the decoding model to be quantized. If there are model parameters or input-output features expressed in floating point type, it is determined that there are still unquantized model layers; if there are no model parameters or input-output features expressed in floating point type, it can be determined that there is no unquantized model layer.
[0114] It can be understood that if there is no unquantized model layer, it means that all model layers in the decoding model to be quantized have been quantized at this time, so the traversal of the hierarchical quantization sequence can be terminated directly.
[0115] In order to ensure that all model layers are quantized, if there is no model layer that has not been traversed, the step of detecting whether there is still an unquantized model layer in the decoding model to be quantized may further include:
[0116] If there is an unquantized model layer, quantizing the unquantized model layer in the decoding model to be quantized;
[0117] When the quantization process is completed, the traversal of the hierarchical quantization sequence is terminated.
[0118] It should be noted that if there are no untraversed model layers in the hierarchical quantization sequence, but there are unquantized model layers in the decoding model to be quantized, it means that the hierarchical quantization sequence is truncated, which does not include all model layers in the decoding model to be quantized. When truncating, the model layers that are generally not added to the hierarchical quantization sequence are model layers that produce very little performance loss during quantization, and have very little impact on the overall performance. Therefore, the unquantized model layers in the decoding model to be quantized can be directly quantized. Among them, quantizing the unquantized model layers in the decoding model to be quantized can be quantizing the model parameters and input and output features in the unquantized model layers in the decoding model to be quantized using low bits.
[0119] It can be understood that when the quantization process of the unquantized model layers in the decoding model to be quantized is completed, all model layers in the decoding model to be quantized have been quantized, and therefore, the traversal of the hierarchical quantization sequence can be terminated.
[0120] This embodiment detects whether there are any unquantized model layers in the decoding model to be quantized when there are no untraversed model layers in the hierarchical quantization sequence, and quantizes the unquantized model layers if any, thereby ensuring that all model layers in the decoding model to be quantized have been quantized when the processing is completed.
[0121] In addition, an embodiment of the present application further proposes a storage medium, on which a model quantization program is stored. When the model quantization program is executed by a processor, the steps of the model quantization method described above are implemented.
[0122] Reference Figure 8 , Figure 8 This is a structural block diagram of the first embodiment of the model quantization device of the present application.
[0123] like Figure 8 As shown, the model quantization device proposed in the embodiment of the present application includes:
[0124] An acquisition module 10 is used to obtain the performance deviation corresponding to each model layer in the decoding model to be quantized;
[0125] A construction module 20, configured to construct a hierarchical quantization sequence according to the performance deviations corresponding to each model layer;
[0126] The quantization module 30 is used to perform hierarchical quantization processing on the model layers in the decoding model to be quantized based on the hierarchical quantization sequence.
[0127] This embodiment obtains the performance deviation corresponding to each model layer in the decoding model to be quantized; constructs a hierarchical quantization sequence according to the performance deviation corresponding to each model layer; and performs hierarchical quantization processing on the model layers in the decoding model to be quantized based on the hierarchical quantization sequence. Since the hierarchical quantization sequence is constructed according to the performance deviation corresponding to each model layer in the decoding model to be quantized, and the hierarchical quantization processing is performed on each model layer in the decoding model to be quantized according to the sequence order of the hierarchical quantization sequence, parameter adjustment correction is used after quantization in the processing process to minimize the performance loss of the model caused by low-bit quantization.
[0128] In a possible implementation of this embodiment, the construction module 20 is further used to sort the model layers in the decoding model to be quantized from large to small according to the corresponding performance deviations to obtain a hierarchical quantization sequence.
[0129] In a possible implementation of this embodiment, the construction module 20 is also used to sort the model layers in the decoding model to be quantized from large to small according to the corresponding performance deviation to obtain a model layer sequence; and to intercept the model layer sequence from front to back according to a preset length to obtain a layered quantization sequence.
[0130] In a possible implementation of this embodiment, the performance deviation corresponding to the model layer is the deviation of the model output before and after the model layer in the decoding model to be quantized is individually quantized.
[0131] In a possible implementation of the present embodiment, the quantization module 30 is also used to traverse the hierarchical quantization sequence and use the traversed model layer as the current model layer; perform quantization processing on the current model layer; adjust at least one model parameter in the decoding model to be quantized with the goal of reducing the performance deviation corresponding to the current model layer; detect whether there are any model layers that have not been traversed in the hierarchical quantization sequence; if there are any model layers that have not been traversed, return to the step of traversing the hierarchical quantization sequence and using the traversed model layer as the current model layer.
[0132] In a possible implementation of this embodiment, the quantization module 30 is further used to use a gradient derivation method to adjust at least one model parameter in the decoding model to be quantized with the goal of reducing the performance deviation corresponding to the current model layer.
[0133] In a possible implementation manner of this embodiment, the adjusting at least one model parameter in the decoding model to be quantized includes at least one of the following:
[0134] Adjusting at least one model parameter of the current model layer;
[0135] Adjusting at least one model parameter of the current model layer and a model layer adjacent to the current model layer;
[0136] At least one model parameter of each model layer in the decoding model to be quantized is adjusted.
[0137] In a possible implementation of the present embodiment, the quantization module 30 is further used to detect whether there are any unquantized model layers in the decoding model to be quantized if there are no unquantized model layers; if there are no unquantized model layers, the traversal of the hierarchical quantization sequence is terminated.
[0138] In a possible implementation of this embodiment, the quantization module 30 is also used to perform quantization processing on the unquantized model layer in the decoding model to be quantized if there is an unquantized model layer; when the quantization processing is completed, terminate the traversal of the hierarchical quantization sequence.
[0139] It should be understood that the above is only an example and does not constitute any limitation on the technical solution of the present application. In specific applications, technicians in this field can make settings as needed, and the present application does not impose any limitation on this.
[0140] It should be noted that the workflow described above is merely illustrative and does not limit the scope of protection of the present application. In practical applications, technicians in this field can select part or all of it according to actual needs to achieve the purpose of the present embodiment, and no limitation is made here.
[0141] In addition, for technical details not described in detail in this embodiment, please refer to the model quantization method provided in any embodiment of the present application, which will not be repeated here.
[0142] In addition, it should be noted that, in this article, the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, article or system including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or system. In the absence of further restrictions, an element defined by the sentence "comprises a ..." does not exclude the existence of other identical elements in the process, method, article or system including the element.
[0143] The serial numbers of the above-mentioned embodiments of the present application are for description only and do not represent the advantages or disadvantages of the embodiments.
[0144] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus a necessary general hardware platform, and of course by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present application, or the part that contributes to the prior art, can be embodied in the form of a software product, which is stored in a storage medium (such as a read-only memory (ROM) / RAM, a magnetic disk, or an optical disk), and includes a number of instructions for a terminal device (which can be a mobile phone, a computer, a server, or a network device, etc.) to execute the methods described in each embodiment of the present application.
[0145] The above are only preferred embodiments of the present application, and are not intended to limit the patent scope of the present application. Any equivalent structure or equivalent process transformation made using the contents of the present application specification and drawings, or directly or indirectly applied in other related technical fields, are also included in the patent protection scope of the present application.
Claims
1. A model quantization method, characterized in that: The model quantization method comprises: Obtain the performance deviation corresponding to each model layer in the decoding model to be quantized; Constructing a hierarchical quantization sequence according to the performance deviations corresponding to each model layer; Based on the hierarchical quantization sequence, a hierarchical quantization process is performed on the model layer in the decoding model to be quantized.
2. The model quantization method according to claim 1, characterized in that: The step of constructing a hierarchical quantization sequence according to the performance deviations corresponding to each model layer includes: The model layers in the decoding model to be quantized are sorted from large to small according to the corresponding performance deviations to obtain a hierarchical quantization sequence.
3. The model quantization method according to claim 2, characterized in that: The step of sorting the model layers in the decoding model to be quantized from large to small according to the corresponding performance deviations to obtain a hierarchical quantization sequence includes: Sorting the model layers in the decoding model to be quantized from large to small according to the corresponding performance deviations to obtain a model layer sequence; The model layer sequence is cut from front to back according to a preset length to obtain a layered quantization sequence.
4. The model quantization method according to claim 1, characterized in that: The performance deviation corresponding to the model layer is the deviation of the model output before and after the model layer in the decoding model to be quantized is individually quantized.
5. The model quantization method according to any one of claims 1 to 4, characterized in that: The step of performing hierarchical quantization processing on the model layer in the decoding model to be quantized based on the hierarchical quantization sequence includes: Traversing the hierarchical quantization sequence, and taking the traversed model layer as the current model layer; Performing quantization processing on the current model layer; With the goal of reducing the performance deviation corresponding to the current model layer, adjusting at least one model parameter in the decoding model to be quantized; Detecting whether there are any model layers that have not been traversed in the hierarchical quantization sequence; If there is a model layer that has not been traversed, the step of traversing the hierarchical quantization sequence is returned, and the traversed model layer is used as the current model layer.
6. The model quantization method according to claim 5, characterized in that: The step of adjusting at least one model parameter in the decoding model to be quantized with the goal of reducing the performance deviation corresponding to the current model layer includes: A gradient derivation method is adopted to adjust at least one model parameter in the decoding model to be quantized with the goal of reducing the performance deviation corresponding to the current model layer.
7. The model quantization method according to claim 5, characterized in that: The adjusting of at least one model parameter in the decoding model to be quantized includes at least one of the following: Adjusting at least one model parameter of the current model layer; Adjusting at least one model parameter of the current model layer and a model layer adjacent to the current model layer; At least one model parameter of each model layer in the decoding model to be quantized is adjusted.
8. The model quantization method according to claim 5, characterized in that: After detecting whether there are any model layers that have not been traversed in the hierarchical quantization sequence, the method further includes: If there is no model layer that has not been traversed, then detecting whether there is still an unquantized model layer in the decoding model to be quantized; If there is no unquantized model layer, the traversal of the hierarchical quantization sequence is terminated.
9. The model quantization method according to claim 8, characterized in that: If there is no model layer that has not been traversed, after detecting whether there is still an unquantized model layer in the decoding model to be quantized, the method further includes: If there is an unquantized model layer, quantizing the unquantized model layer in the decoding model to be quantized; When the quantization process is completed, the traversal of the hierarchical quantization sequence is terminated.
10. A model quantization device, characterized in that: The model quantization device comprises: An acquisition module, used to obtain the performance deviation corresponding to each model layer in the decoding model to be quantized; A construction module, used to construct a hierarchical quantization sequence according to the performance deviations corresponding to each model layer; A quantization module is used to perform hierarchical quantization processing on the model layer in the decoding model to be quantized based on the hierarchical quantization sequence.
11. A model quantization device, characterized in that: The model quantization device includes: a processor, a memory, and a model quantization program stored in the memory and executable on the processor. When the model quantization program is executed by the processor, the steps of the model quantization method according to any one of claims 1 to 9 are implemented.
12. A coding device, characterized in that: The encoding device comprises: a processor, a memory, and an encoding program stored in the memory and executable on the processor, wherein when the encoding program is executed by the processor, the steps of the model quantization method according to any one of claims 1 to 9 are implemented.
13. A decoding device, characterized in that: The decoding device comprises: a processor, a memory, and a decoding program stored in the memory and executable on the processor, wherein the decoding program implements the steps of the model quantization method according to any one of claims 1 to 9 when executed by the processor.
14. A storage medium, characterized in that: The storage medium stores a model quantization program, which, when executed, implements the steps of the model quantization method according to any one of claims 1 to 9.