Model quantification method and device and medium

Abstract

The invention discloses a model quantification method and device and a medium, and the method comprises the steps: carrying out the quantification of a pre-training model, obtaining an initial quantification model, and determining the target number of lifting modules inserted into the initial quantification model; dividing the initial quantification model into a target number of backbone network modules; inserting a lifting module into each backbone network module in parallel to obtain a to-be-optimized model comprising a plurality of target units; the target unit comprises a lifting module and a backbone network module; optimizing each target unit in the to-be-optimized model in sequence to obtain an intermediate optimization model; and performing global optimization on the intermediate optimization model through a preset loss function to obtain a target optimization model. Therefore, the lifting module is inserted into the initial quantization model to obtain the to-be-optimized model, that is, the calculation complexity is increased in the initial quantization model to improve the calculation precision. Furthermore, local and global optimization is carried out on the to-be-optimized model in sequence to obtain a target optimization model with high operation speed and high calculation precision.

Description

Technical Field

[0001] The present application relates to the field of computer technology, and in particular to a model quantization method, device and medium. Background Art

[0002] In recent years, with the continuous development and maturity of deep learning technology, it has been widely used in more and more fields such as autonomous driving, medical health and computer vision. However, as the model complexity increases, the computing efficiency decreases, which gradually limits the application expansion of the model.

[0003] At present, in order to improve the efficiency of model calculation, the parameters in the model calculation can be expressed as low-bit fixed-point numbers through model quantization, so that floating-point operations can be converted into fixed-point operations, thereby achieving the purpose of improving calculation efficiency. However, this method will inevitably lead to precision loss, that is, reduce the accuracy of model calculation.

[0004] It can be seen that how to improve the efficiency of model calculation while ensuring the accuracy of model calculation is an urgent problem to be solved by technical personnel in this field. Summary of the invention

[0005] In view of this, one aspect of the present application provides a model quantization method, the method comprising: Quantize the pre-trained model to obtain the initial quantized model; Determining a target number of boosting modules to be inserted into the initial quantization model according to a preset insertion rule; Dividing the initial quantization model into the target number of backbone network modules; After inserting one of the boosting modules in parallel into each of the backbone network modules in the initial quantization model, a model to be optimized including a plurality of target units is obtained; wherein the target unit includes the boosting module and the backbone network module; Based on the order from the input end to the output end, each of the target units in the model to be optimized is optimized in turn to obtain an intermediate optimization model; The intermediate optimization model is globally optimized through a preset loss function to obtain a target optimization model.

[0006] Optionally, determining a target number of boosting modules to be inserted into the initial quantization model according to a preset insertion rule includes: Determine a first parameter value for characterizing spatial complexity and a second parameter value for characterizing computational complexity of the initial quantization model after inserting an initial number of the lifting modules; Determine whether the first parameter value is not greater than a first constraint value, and whether the second parameter value is not greater than a second constraint value; If yes, increase the number of the inserted lifting modules, and calculate the first parameter value and the second parameter value; and return to the step of determining whether the first parameter value is not greater than the first constraint value, and whether the second parameter value is not greater than the second constraint value; If not, the insertion quantity of the lifting modules corresponding to the last judgment result is taken as the target quantity.

[0007] Optionally, dividing the initial quantization model into the target number of backbone network modules includes: When the total number of network layers of the initial quantization model is an integer multiple of the target number, each integer multiple of the network layers in the total number of network layers is divided into one backbone network module; When the total number of network layers of the initial quantization model is not an integer multiple of the target number, among the target number of backbone network modules, the number of network layers of one backbone network module is the total number of network layers divided by the target number and rounded up, and the number of network layers of the other backbone network modules is the total number of network layers divided by the target number and rounded down.

[0008] Optionally, optimizing each of the target units in the model to be optimized in sequence based on the order from the input end to the output end includes: Dividing the pre-trained model and the initial quantization model into the same structure to obtain a target pre-trained model including multiple sub-modules; Inputting the specified sample into the target pre-trained model and the model to be optimized; Each of the submodules and each of the target units is traversed in turn, and the output result of each of the submodules is used as the optimization target to iteratively optimize the corresponding target unit.

[0009] Optionally, taking the output results of each of the submodules as the optimization target, iteratively optimizing the corresponding target unit includes: Obtaining a current output result of a current target unit, and determining whether a difference between the current output result and a previous iteration output result is within a preset range; If yes, enter the next iterative optimization of the target unit; If not, perform the following steps; Under the condition of fixing the quantization parameter of the current backbone network module in the current target unit, determining the target improvement parameter according to the first difference between the current output result and the corresponding optimization target; Under the condition of fixing the lifting parameter of the current lifting module in the current target unit, determining the target quantization parameter according to the second difference between the current output result and the corresponding optimization target; Optimizing the current boosting module by the target boosting parameter, and optimizing the current backbone network module by the target quantization parameter; Return to the step of obtaining the current output result of the current target unit and determining whether the difference between the current output result and the output result of the previous iteration is within a preset range.

[0010] Optionally, the preset loss function is composed of an output loss function, an activation regularization loss function, and an improvement model regularization loss function; The output loss function is used to optimize the difference between the output of the intermediate optimization model and the output of the pre-training model; The activation regularization loss function is used to optimize the difference between the output of each target unit in the intermediate optimization model and the output of the corresponding submodule; The lifting model regularization loss function is used to limit the distance between the lifting parameters of the lifting module and the intermediate optimization model.

[0011] Optionally, globally optimizing the intermediate optimization model by using a preset loss function includes: Inputting a specified sample into the pre-trained model and the intermediate optimization model; Obtaining total outputs of the pre-trained model and the intermediate optimization model respectively; Obtaining target activation output, wherein the target activation output includes activation output of each target unit in the intermediate optimization model and activation output of the corresponding submodule; Respectively obtaining the boosting parameters of the boosting modules in the model to be optimized and the intermediate optimization model; The intermediate optimization model is optimized by the preset loss function according to the total output, the target activation output and the boost parameter.

[0012] Another aspect of the present application provides a model quantization device, the device comprising: A quantization unit, used to quantize the pre-trained model to obtain an initial quantized model; A target number determination unit, used to determine a target number of boosting modules to be inserted into the initial quantization model according to a preset insertion rule; A division unit, used for dividing the initial quantization model into the target number of backbone network modules; A lifting module inserting unit, used for inserting one lifting module in parallel into each backbone network module in the initial quantization model, so as to obtain a model to be optimized including a plurality of target units; wherein the target unit includes the lifting module and the backbone network module; An intermediate optimization unit, used to optimize each of the target units in the model to be optimized in sequence based on the order from the input end to the output end, to obtain an intermediate optimization model; The target optimization unit is used to globally optimize the intermediate optimization model through a preset loss function to obtain a target optimization model.

[0013] Another aspect of the present application provides a model quantization device, including a memory and a processor, wherein the memory stores a computer program that can be executed on the processor, and the processor implements the steps of the model quantization method when executing the program.

[0014] Another aspect of the present application provides a computer-readable storage medium having a computer program stored thereon, and when the program is executed by a processor, the steps of the model quantization method are implemented.

[0015] The present application provides a model quantization method, device and medium, which have the following beneficial effects: by quantizing the pre-trained model to improve the model calculation efficiency, and inserting the improvement module into the initial quantization model to obtain the model to be optimized, that is, increasing the calculation complexity in the initial quantization model, thereby improving the calculation accuracy of the model. Furthermore, the model to be optimized is locally optimized and globally optimized in turn, and the target optimization model with high running speed and high calculation accuracy is obtained through two optimizations. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] Figure 1 A schematic diagram of a flow chart of a model quantization method provided in an embodiment of the present application; Figure 2 A schematic diagram of the structure of a model to be optimized provided in an embodiment of the present application; Figure 3 A schematic diagram of the principle of model optimization provided in an embodiment of the present application; Figure 4 A schematic diagram of a flow chart of a model quantization method provided in another embodiment of the present application; Figure 5 A schematic diagram of the structure of a model quantization device provided in an embodiment of the present application; Figure 6 A schematic diagram of the structure of a model quantization device provided in another embodiment of the present application.

[0017] The figure numbers are as follows: 2 is the model to be optimized, 20 is the target unit, 200 is the backbone network module, 201 is the upgrading module, 60 is the memory, 61 is the processor, 62 is the display screen, 63 is the input and output interface, 64 is the communication interface, 65 is the power supply, 66 is the communication bus, 601 is the computer program, 602 is the operating system, and 603 is the data. DETAILED DESCRIPTION

[0018] The terms used in this application are for the purpose of describing specific embodiments only and are not intended to limit this application. The singular forms of "a", "said" and "the" used in this application and the appended claims are also intended to include plural forms unless the context clearly indicates other meanings. It should also be understood that the term "and / or" used herein refers to and includes any or all possible combinations of one or more associated listed items.

[0019] It should be understood that although the terms first, second, third, etc. may be used in the present application to describe various information, these information should not be limited to these terms. These terms are only used to distinguish the same type of information from each other. For example, without departing from the scope of the present application, the first information may also be referred to as the second information, and similarly, the second information may also be referred to as the first information. Depending on the context, the word "if" as used herein may be interpreted as "at the time of" or "when" or "in response to determining".

[0020] Figure 1 A flow chart of a model quantization method provided in an embodiment of the present application is shown in FIG. Figure 1 As shown, the method includes: S10: quantize the pre-trained model to obtain an initial quantized model; In a specific embodiment, a pre-trained model with higher computational accuracy is read. In an optional embodiment, the pre-trained model may be a fully trained 32-bit floating-point model with high accuracy. However, the computational efficiency of complex pre-trained models is often low. In order to improve the computational speed of the pre-trained model, the read pre-trained model needs to be quantized.

[0021] Specifically, an observer and a quantizer are added to the pre-trained model. Further, after a small amount of unlabeled data is input into the pre-trained model, the weight amplitude range and activation amplitude range of each layer of the network in the pre-trained model are obtained through the observer. Among them, the observer may include but is not limited to a minimum and maximum value observer and an optimal mean square error observer.

[0022] It is worth noting that the weight assignment range can be obtained as long as the model is loaded, without the need for input data. However, the activation assignment range generally refers to the output of the middle layer of the network, which can only be obtained by inputting data to perform the inference process. Therefore, a small amount of unlabeled data needs to be input into the pre-trained model to obtain the activation amplitude range.

[0023] Furthermore, the quantizer quantizes the weight parameters and activation parameters of the pre-trained model according to the weight amplitude range and activation amplitude range obtained by the observer, thereby obtaining an initial quantized model. The quantizer includes but is not limited to a linear quantizer.

[0024] It should be noted that model quantization refers to converting the weights in a trained model (i.e., pre-trained model) from high precision (e.g., 32-bit floating point numbers) to low precision (e.g., 8-bit integers). The purpose of model quantization is to reduce the storage requirements of the model, speed up inference, and reduce computing resource consumption, while maintaining the model's prediction accuracy as much as possible.

[0025] S11: Determine the target number of boosting modules to be inserted into the initial quantization model according to a preset insertion rule; S12: Divide the initial quantization model into a target number of backbone network modules; S13: inserting a boosting module in parallel into each backbone network module in the initial quantization model, thereby obtaining a model to be optimized including a plurality of target units; wherein the target unit includes a boosting module and a backbone network module; Figure 2 A structural schematic diagram of a model to be optimized provided in an embodiment of the present application. In an optional embodiment, the initial quantization model after quantization will result in a decrease in calculation accuracy due to the improvement in calculation efficiency. Therefore, in order to solve this technical problem, an enhancement module that can improve the calculation complexity can be inserted into the initial quantization model.

[0026] Specifically, firstly, according to the preset insertion rule, the target number of boosting modules that can be inserted into the initial quantization model is calculated, that is, the maximum number that can be inserted. At the same time, in order to insert the boosting modules, the initial quantization model needs to be divided into the target number of backbone network modules.

[0027] Furthermore, a boosting module is inserted in parallel into the input and output ends of each backbone network module of the initial quantization model, thereby converting the initial quantization model into a model to be optimized including multiple target units, wherein each target unit includes a backbone network module and a boosting module.

[0028] like Figure 2 As shown, in the model 2 to be optimized, every three network layers are divided into a backbone network module 200, and a lifting module 201 is inserted into the input and output ends of each backbone network module 200 to form a target unit 20. Thus, the obtained model 2 to be optimized includes multiple target units 20.

[0029] In an optional embodiment, the boosting module may be composed of linear operators. Specifically, the boosting module includes but is not limited to a fully connected layer and a convolutional layer. It is understandable that the fully connected layer can be directly represented as a linear operator. The convolutional layer can greatly reduce the number of parameters by sharing parameters, and the convolutional layer can meet the linearity after being expanded, wherein the convolution type includes conventional convolution and grouped convolution.

[0030] In an optional embodiment, the expression of the linear operator of the lifting module can be: (1) in, , , , To improve module output, To improve the module input, To improve the module weight parameters, It is the bias parameter of the improvement module.

[0031] It should be noted that the boost module is essentially used to compensate for the difference between the quantized model and the pre-trained model output, and the boost module can also be understood as a module for improving the calculation accuracy of the initial quantized model. In a specific embodiment, the boost module can improve the calculation accuracy of the initial quantized model with a small amount of resource consumption by increasing the calculation complexity.

[0032] It should also be noted that, in an optional embodiment, the calculation accuracy of the enhancement module should be higher than the calculation accuracy of the initial quantization model. Therefore, when the accuracy of the pre-trained model is a 32-bit floating point model, the enhancement module may include but is not limited to 32-bit floating point and 16-bit floating point.

[0033] S14: Based on the order from the input end to the output end, each target unit in the optimization model is optimized in turn to obtain an intermediate optimization model; S15: Globally optimize the intermediate optimization model through a preset loss function to obtain a target optimization model.

[0034] After obtaining the model to be optimized, in order to further improve the calculation accuracy of the model, the module to be optimized is optimized through step S14. Specifically, from the input end to the output end of the model to be optimized, each target unit is traversed in turn, so that each target unit is optimized independently, thereby obtaining an intermediate optimization model.

[0035] Furthermore, the intermediate optimization model is globally optimized through a preset loss function, that is, the intermediate optimization model is optimized as a whole to obtain the final target optimization model.

[0036] It can be understood that the technical solution provided in the embodiment of the present application divides the optimization of the model to be optimized after the insertion of the enhancement module into two stages. The first stage is to independently optimize each target unit, and then the second stage is to perform global optimization, so as to obtain a high-precision target optimization model.

[0037] Therefore, the model quantization method provided in the embodiment of the present application quantizes the pre-trained model to improve the model calculation efficiency, and inserts the improvement module into the initial quantization model to obtain the model to be optimized, that is, the calculation complexity is increased in the initial quantization model, thereby improving the calculation accuracy of the model. Further, the model to be optimized is locally optimized and globally optimized in turn, and the target optimization model with high running speed and high calculation accuracy is obtained through two optimizations.

[0038] As an optional embodiment, determining the target number of boosting modules to be inserted into the initial quantization model according to a preset insertion rule includes: Determine a first parameter value for characterizing spatial complexity and a second parameter value for characterizing computational complexity of the initial quantization model after inserting an initial number of lifting modules; Determine whether the first parameter value is not greater than the first constraint value, and whether the second parameter value is not greater than the second constraint value; If yes, increase the number of inserted lifting modules, and calculate the first parameter value and the second parameter value; and return to the step of determining whether the first parameter value is not greater than the first constraint value, and whether the second parameter value is not greater than the second constraint value; If not, the insertion quantity of the lifting modules corresponding to the last judgment result is taken as the target quantity.

[0039] In a specific embodiment, the number of inserted lifting modules needs to meet the space complexity constraints and computational complexity constraints of the initial quantization model. It is understandable that after the lifting modules are inserted, the space complexity and computational complexity of the initial quantization model will increase. In a specific embodiment, if too many lifting modules are inserted, the space complexity and computational complexity will be too high, resulting in reduced computational efficiency. Therefore, it is necessary to determine the maximum number of lifting modules that can be inserted under the constraints of the space complexity and computational complexity of the initial quantization model, so as to maximize the computational accuracy and computational efficiency of the lifting model.

[0040] Specifically, an initial number of boosting modules are first inserted into the initial quantization model, and the first parameter value and the second parameter value in the initial quantization model after the boosting modules are inserted are calculated, wherein the first parameter value is a parameter value used to characterize spatial complexity, and the second parameter value is a parameter value used to characterize computational complexity.

[0041] When calculating the first parameter value, since the spatial complexity of the model is proportional to the model weight storage amount (i.e., the model size), in an optional embodiment, the spatial complexity of the model can be measured by calculating the model weight storage amount, that is, the model weight storage amount is used as the first parameter value. The specific calculation formula is: (2) in, is the weight storage of the initial quantization module, that is, the first parameter value, is the number of network layers of the initial quantization module, For the The number of weights of the layer, For the The weight bit width of the layer.

[0042] When determining the computational complexity, it can be measured by BOPS (Bit Operations Per Second). The specific calculation formula is: (3) in, is the second parameter value used to characterize the computational complexity, It is a floating point operand, usually calculated based on the MAC (multiplication-addition operand) of each layer of the network. For the The layer activation quantization bit width, i.e., The input of the layer.

[0043] It is worth noting that the number of the initial quantity increasing modules initially inserted can be one or more, which is not limited in this application. When one is inserted, it is inserted in parallel based on the input and output of the initial quantization model.

[0044] The first parameter in Dedao and the second parameter After that, further, it is determined whether the first parameter value is not greater than the first constraint value, and whether the second parameter value is not greater than the second constraint value. In other words, it is determined whether the space complexity and computational complexity of the current initial quantization model are within the constraint conditions after the quantization model is inserted, that is, the first parameter value cannot exceed the first constraint value, and the second parameter value cannot exceed the second constraint value.

[0045] If it is within the constraints, the number of inserted boosting modules is increased, and the first parameter value and the second parameter value of the initial quantization model after adding the boosting unit are calculated, and the constraints are judged cyclically until the maximum number of boosting modules that can be inserted under the constraints is determined, and this maximum number is used as the target number.

[0046] It should be noted that the number of lifting modules may be increased one by one each time, for example, if 3 modules were inserted last time, 4 modules may be inserted next time.

[0047] In any loop judgment, when it is determined that the first parameter value is not greater than the first constraint value and the second parameter value is not greater than the second constraint value cannot be satisfied at the same time, the number of lifting modules inserted corresponding to the conditions met last time can be used as the target number, that is, the number of lifting modules inserted corresponding to the last judgment result is used as the target number.

[0048] In a specific embodiment, the pre-trained model is a ResNet50 model, and the weight storage capacity of the model, that is, the model size is 97.8MB, the computational complexity is 3951GBOPS, and the model accuracy is 76.6%. Through the method provided in this application, 4-bit quantization is used for weights and activations to obtain an initial quantized model with a model size of 13.1MB, a computational complexity of 67GBOPS, and a model accuracy of 63.7%. Thus, while reducing the model size and computational complexity, high computational accuracy can also be guaranteed.

[0049] As an optional embodiment, the initial quantization model is divided into a target number of backbone network modules, including: When the total number of network layers of the initial quantization model is an integer multiple of the target number, each integer multiple of the total number of network layers is divided into a backbone network module; When the total number of network layers of the initial quantization model is not an integer multiple of the target number, among the target number of backbone network modules, the number of network layers of one backbone network module is the total number of network layers divided by the target number and rounded up, and the number of network layers of other backbone network modules is the total number of network layers divided by the target number and rounded down.

[0050] In an optional embodiment, when the initial quantization model is divided, if the target number of the inserted lifting modules is an integer multiple of the total number of network layers of the initial quantization model, the initial quantization model can be evenly divided, that is, each integer multiple of the network layers can be divided into a backbone network module. Figure 2 As shown, every three network layers of the initial quantization model can be divided into a backbone network module.

[0051] In another optional embodiment, if the initial quantization model cannot be evenly divided, that is, the total number of network layers of the initial quantization model is not an integer multiple of the target number, the difference in the number of network layers between the backbone network modules should be minimized as much as possible.

[0052] Specifically, among the target number of backbone network modules, the number of network layers of one backbone network module is the total number of network layers divided by the target number and rounded up, and the number of network layers of the remaining backbone network modules is the total number of network layers divided by the target number and rounded down.

[0053] For example, when the total number of network layers of the initial quantization model is 7 and the target number is 3, 7 divided by 3 is rounded up to 3 and rounded down to 2. Therefore, when the initial quantization model is divided into 3 backbone network modules, one backbone network module includes 3 network layers, and the remaining 2 backbone network modules each include 2 network layers.

[0054] It should be noted that when the backbone network module is inserted into the upgrading module, it is necessary to ensure that the input and output parameters of the upgrading module are consistent with the input and output parameters of the backbone network module, where the input and output parameters may include but are not limited to input data size and format.

[0055] Figure 3 A schematic diagram of a model optimization principle provided in an embodiment of the present application. In an optional embodiment, each target unit in the optimization model is optimized in sequence based on the order from the input end to the output end, including: Divide the pre-trained model and the initial quantization model into the same structure to obtain a target pre-trained model including multiple sub-modules; Input the specified samples into the target pre-trained model and the model to be optimized; Each submodule and each target unit is traversed in turn, and the output result of each submodule is used as the optimization target, and the corresponding target unit is iteratively optimized.

[0056] In a specific embodiment, in order to optimize the model to be optimized, the pre-trained model needs to be divided into the same structure as the initial quantization model, that is, the pre-trained model is divided into a target number of sub-modules, each of which corresponds to the backbone network module in the initial quantization model. Figure 3 As shown, submodule 1 corresponds to the backbone network module in target unit 1 and has the same number of network layers.

[0057] Furthermore, the target pre-trained model after the designated sample input is divided into multiple sub-modules, and the model to be optimized after the boosting module is inserted. Then, from the input end to the output end, each module and each target unit are traversed in turn, and the output result of the sub-module is used as the optimization target of the target unit, and the corresponding target unit is iteratively optimized.

[0058] Specifically, Figure 3 As shown, in submodule 1, the output result A1 of submodule 1 is used as the optimization target of target unit 1, and target unit 1 is iteratively trained until the preset conditions are met, and then target unit 2 is optimized. This cycle is repeated until target unit n is optimized and an intermediate optimization model is obtained, that is, after each target unit of the optimization model is independently optimized, an intermediate optimization module is obtained.

[0059] It is understandable that the computational accuracy of the pre-trained model is higher than that of the initial quantized model. After the boost module is inserted, the accuracy is improved, but it is still far lower than that of the pre-trained model. Therefore, in order to improve the computational accuracy of the model to be optimized, the output of the model to be optimized should be as close as possible to the output of the pre-trained model, that is, the output result of each submodule is the optimization target, and the target unit is iteratively trained so that the computational accuracy of each target module is close to the computational accuracy of the submodule, thereby achieving independent optimization of each target unit in the first stage, thereby improving the overall computational accuracy of the quantized model.

[0060] Figure 4 A flow chart of a model quantization method provided in another embodiment of the present application is provided. Based on the above embodiment, as an optional embodiment, Figure 4 As shown, the output results of each submodule are used as the optimization target, and the corresponding target unit is iteratively optimized, including: S40: Obtain the current output result of the current target unit, and determine whether the difference between the current output result and the output result of the previous iteration is within a preset range; if so, proceed to step S41; if not, execute steps S42 to S44; S41: Enter the iterative optimization of the next target unit; In a specific embodiment, the selected unit is used as the current target unit in the traversal process, and it is determined whether the difference between the current output result of the current target unit and the output result of the previous iteration is within a preset range. If so, it indicates that the training of the current target unit has achieved the expected result, and the next target unit can be iteratively trained in step S41. If not, it indicates that the training of the current target unit has not achieved the expected result, and the current target unit needs to be further optimized. Specifically, steps S42 to S44 are executed.

[0061] It should be noted that the termination condition of the iterative training is that the difference between the current output result of the current target unit and the output result of the previous iteration is within a preset range. Therefore, each target unit is iteratively trained at least twice, and after the second training, enter step S40 and execute subsequent steps.

[0062] S42: Under the condition of fixing the quantization parameter of the current backbone network module in the current target unit, determining the target improvement parameter according to the first difference between the current output result and the corresponding optimization target; In an optional embodiment, the quantization parameters of the current backbone network module in the current target unit are fixed, wherein the quantization parameters include a scale factor and a zero point. Under the condition that the quantization parameters are fixed, the target boosting parameters are determined according to the difference between the current output result and the corresponding optimization target, that is, in the next round of training, the boosting module in the current target unit should be adjusted to the target boosting parameters, wherein the boosting parameters include a weight parameter and a bias parameter.

[0063] When calculating the target improvement parameters, the bias parameters can be Fusion to weight parameters In the above equation, the linear operator expression of the fused lifting module is: (4) in, For the paranoid parameter Fusion to weight parameters The fusion parameters after , It is the input of the boosting module under the fusion parameters.

[0064] The current output result of the current target unit in the model to be optimized Includes backbone network module output and boost module output ,Right now: (5) During the optimization process, the goal is to make the current output result of the current target unit Optimization goals of corresponding submodules Therefore, under the condition that the quantization parameter is fixed, Fixed, in order to make the current output result and optimization goals Consistent, it is necessary to calculate the target improvement parameters for the next iterative training, thus obtaining the fusion parameters The closed form solution is: (6) S43: Under the condition of fixing the lifting parameter of the current lifting module in the current target unit, determining the target quantization parameter according to the second difference between the current output result and the corresponding optimization target; In another optional embodiment, the boosting parameters of the current boosting module in the current target unit are fixed, wherein the boosting parameters include weight parameters and bias parameters. Under the condition that the boosting parameters are fixed, the boosting unit outputs Similarly, in order to make the current output and optimization goals Therefore, it is necessary to determine the target quantization parameter for the next iterative training. The specific calculation formula is: (7) in, and Constitute the quantitative parameters, specifically, is the scale factor, It is zero point.

[0065] Therefore, in a specific embodiment, the target quantization parameter may be solved by using a gradient descent method. In an optional embodiment, the optimizer Adam may be used to perform a gradient descent solution.

[0066] S44: Optimize the current boosting module by the target boosting parameter, and optimize the current backbone network module by the target quantization parameter; and return to step S40.

[0067] Thus, the calculated fusion parameters As the target improvement parameter, when the current target unit is optimized for the next time, the improvement parameter is adjusted to the target improvement parameter. At the same time, when the next optimization is performed, the quantization parameter of the backbone network module is adjusted to the target quantization parameter. The training is iterated in this way until the difference between the current output result and the output result of the previous iteration is within the preset range.

[0068] In an optional embodiment, the preset loss function is composed of an output loss function, an activation regularization loss function, and an improvement model regularization loss function, and the specific calculation formula is: (8) in, is the preset loss function, is the output loss function, To activate the regularized loss function, To improve the model regularization loss function, and is the weighting parameter, is the total output of the pre-trained model, is the total output of the intermediate optimization model, The target number of boost modules to be inserted, For the The activation output of the submodule in the pre-trained model corresponding to the inserted lifting unit, For the The activation output of the insertion lifting unit, For the intermediate optimization model The lifting parameters of the inserted lifting unit, The model to be optimized Insert the lifting parameters of the lifting unit.

[0069] Among them, the output loss function is used to optimize the difference between the output of the intermediate optimization model and the output of the pre-trained model. The activation regularization loss function is used to optimize the difference between the output of each target unit in the intermediate optimization model and the output of the corresponding submodule. The boosting model regularization loss function is used to limit the distance between the boosting parameters of the boosting module and the intermediate optimization model.

[0070] It should be noted that, in an optional embodiment, the output loss function , activation regularization loss function And improve the model regularization loss function You can choose the square error loss function.

[0071] Based on the above embodiment, as an optional embodiment, the intermediate optimization model is globally optimized by using a preset loss function, including: Input the specified samples into the pre-trained model and the intermediate optimization model; Get the total output of the pre-trained model and the intermediate optimized model respectively; Obtaining target activation output, which includes activation output of each target unit in the intermediate optimization model and activation output of the corresponding submodule; Respectively obtain the boosting parameters of the boosting modules in the model to be optimized and the intermediate optimization model; According to the total output, target activation output and improvement parameters, the intermediate optimization model is optimized by the preset loss function.

[0072] In a specific embodiment, after obtaining the intermediate optimization model, in order to further improve the calculation accuracy of the model, the model can be globally optimized in the second stage, that is, the model is optimized as a whole. Specifically, a small amount of specified samples are input into the pre-training model and the intermediate optimization model.

[0073] Furthermore, the total output of the pre-trained model is obtained separately , total output of the intermediate optimization model , the activation output of the submodule , the activation output of the lifting unit , the improvement parameters corresponding to the intermediate optimization model The improved parameters corresponding to the model to be optimized .

[0074] Therefore, based on the data obtained above, the difference between the output of the intermediate optimization model and the output of the pre-trained model, as well as the difference between the output of each target unit in the intermediate optimization model and the output of the corresponding sub-module can be determined. Then, the intermediate optimization model can be globally optimized through the above formula (8), that is, through the preset loss function to obtain a high-precision and high-efficiency target optimization model.

[0075] It should be noted that the optimization of the intermediate optimization model can be iterative training until a preset number of iterations is reached. In some optional embodiments, 500 specified input samples of most tasks can complete the entire optimization process. After adding the boost module, the model size is 14.5MB, the computational complexity is 75GBOPS, and the model accuracy can reach 75.9%, which is only 0.7% lower than the pre-trained model. That is, a target optimization model with small model weight storage, high computational complexity, high computational accuracy and high computational efficiency can be obtained.

[0076] In the above embodiments, the model quantization method is described in detail. The present application also provides an embodiment corresponding to a model quantization device.

[0077] Figure 5 A schematic diagram of the structure of a model quantization device provided in an embodiment of the present application is shown in FIG. Figure 5 As shown, the device comprises: A quantization unit 50, used to quantize the pre-trained model to obtain an initial quantized model; A target number determination unit 51 is used to determine the target number of boosting modules to be inserted into the initial quantization model according to a preset insertion rule; A division unit 52, used for dividing the initial quantization model into a target number of backbone network modules; The lifting module inserting unit 53 is used to insert a lifting module in parallel into each backbone network module in the initial quantization model to obtain a model to be optimized including multiple target units; wherein the target unit includes a lifting module and a backbone network module; The intermediate optimization unit 54 is used to optimize each target unit in the optimization model in sequence based on the order from the input end to the output end to obtain the intermediate optimization model; The target optimization unit 55 is used to perform global optimization on the intermediate optimization model through a preset loss function to obtain a target optimization model.

[0078] In addition, the model quantization device provided in the embodiment of the present application also includes: A parameter determination unit, used to determine a first parameter value for characterizing spatial complexity and a second parameter value for characterizing computational complexity of the initial quantization model after inserting an initial number of lifting modules; A judgment unit is used to judge whether the first parameter value is not greater than the first constraint value, and whether the second parameter value is not greater than the second constraint value; if so, increase the number of inserted upgrading modules, and calculate the first parameter value and the second parameter value; and return to the step of judging whether the first parameter value is not greater than the first constraint value, and whether the second parameter value is not greater than the second constraint value; if not, take the number of inserted upgrading modules corresponding to the last judgment result as the target number.

[0079] A first division subunit is used for dividing each integer multiple of the total number of network layers into a backbone network module when the total number of network layers of the initial quantization model is an integer multiple of the target number; The second division subunit is used for when the total number of network layers of the initial quantization model is not an integer multiple of the target number. Among the target number of backbone network modules, the number of network layers of one backbone network module is the total number of network layers divided by the target number and rounded up, and the number of network layers of other backbone network modules is the total number of network layers divided by the target number and rounded down.

[0080] A third division subunit is used to divide the pre-trained model and the initial quantization model into the same structure to obtain a target pre-trained model including multiple sub-modules; A first sample input unit, used to input a specified sample into a target pre-trained model and a model to be optimized; The intermediate optimization unit is also used to traverse each submodule and each target unit in turn, take the output result of each submodule as the optimization target, and iteratively optimize the corresponding target unit.

[0081] The intermediate optimization unit is also used to obtain the current output result of the current target unit and determine whether the difference between the current output result and the output result of the previous iteration is within a preset range; If yes, enter the iterative optimization of the next target unit; If not, perform the following steps; Under the condition of fixing the quantization parameter of the current backbone network module in the current target unit, determining the target improvement parameter according to the first difference between the current output result and the corresponding optimization target; Under the condition of fixing the lifting parameter of the current lifting module in the current target unit, determining the target quantization parameter according to the second difference between the current output result and the corresponding optimization target; The current boosting module is optimized through the target boosting parameters, and the current backbone network module is optimized through the target quantization parameters; Return to the step of obtaining the current output result of the current target unit and determining whether the difference between the current output result and the output result of the previous iteration is within a preset range.

[0082] A second sample input unit, used to input a specified sample into the pre-training model and the intermediate optimization model; A total output acquisition unit, used to respectively acquire the total outputs of the pre-trained model and the intermediate optimization model; An activation output acquisition unit, used to acquire a target activation output, wherein the target activation output includes the activation output of each target unit in the intermediate optimization model and the activation output of the corresponding submodule; An improvement parameter acquisition unit, used to respectively obtain the improvement parameters of the improvement modules in the model to be optimized and the intermediate optimization model; The target optimization unit is also used to optimize the intermediate optimization model through a preset loss function according to the total output, the target activation output and the improvement parameters.

[0083] Figure 6 This is a schematic diagram of the structure of a model quantization device provided in another embodiment of the present application, such as Figure 6 As shown, the model quantization device includes: a memory 60 for storing a computer program; The processor 61 is used to implement the steps of the model quantization method mentioned in the above embodiment when executing the computer program.

[0084] The model quantization device provided in this embodiment may include but is not limited to a laptop computer or a desktop computer.

[0085] Among them, the processor 61 may include one or more processing cores, such as a 4-core processor, an 8-core processor, etc. The processor 61 can be implemented in at least one hardware form of a digital signal processor (Digital Signal Processor, referred to as DSP), a field programmable gate array (Field-Programmable Gate Array, referred to as FPGA), and a programmable logic array (Programmable Logic Array, referred to as PLA). The processor 61 may also include a main processor and a coprocessor. The main processor is a processor for processing data in the awake state, also known as a central processing unit (Central Processing Unit, referred to as CPU); the coprocessor is a low-power processor for processing data in the standby state. In some embodiments, the processor 61 may be integrated with a graphics processing unit (Graphics Processing Unit, referred to as GPU), and the GPU is responsible for rendering and drawing the content to be displayed on the display screen. In some embodiments, the processor 61 may also include an artificial intelligence (Artificial Intelligence, referred to as AI) processor, which is used to process computing operations related to machine learning.

[0086] The memory 60 may include one or more computer-readable storage media, which may be non-transitory. The memory 60 may also include a high-speed random access memory, and a non-volatile memory, such as one or more disk storage devices, flash memory storage devices. In this embodiment, the memory 60 is at least used to store the following computer program 601, wherein the computer program can implement the relevant steps of the model quantization method disclosed in any of the aforementioned embodiments after being loaded and executed by the processor 61. In addition, the resources stored in the memory 60 may also include an operating system 602 and data 603, etc., and the storage method may be temporary storage or permanent storage. Among them, the operating system 602 may include Windows, Unix, Linux, etc. Data 603 may include, but is not limited to, relevant data involved in the model quantization method, etc.

[0087] In some embodiments, the model quantization device may further include a display screen 62 , an input / output interface 63 , a communication interface 64 , a power supply 65 , and a communication bus 66 .

[0088] Those skilled in the art will understand that Figure 6 The structure shown in the figure does not constitute a limitation on the model quantization device, and may include more or fewer components than those shown in the figure.

[0089] The model quantization device provided in the embodiment of the present application includes a memory and a processor. When the processor executes the program stored in the memory, it can implement the model quantization method in the above embodiment.

[0090] It should be noted that, although the operations are depicted in a specific order in the accompanying drawings, this should not be understood as requiring these operations to be performed in the specific order shown or to be performed sequentially, or requiring all illustrated operations to be performed to achieve the desired results. In some cases, multitasking and parallel processing may be advantageous. In addition, the separation of various system modules and components in the above-described embodiments should not be understood as requiring such separation in all embodiments, and it should be understood that the described program components and systems can generally be integrated together in a single software product, or packaged into multiple software products.

Claims

1. A model quantization method, characterized in that: The method comprises: Quantize the pre-trained model to obtain the initial quantized model; Determining a target number of boosting modules to be inserted into the initial quantization model according to a preset insertion rule; Dividing the initial quantization model into the target number of backbone network modules; After inserting one of the boosting modules in parallel into each of the backbone network modules in the initial quantization model, a model to be optimized including a plurality of target units is obtained; wherein the target unit includes the boosting module and the backbone network module; Based on the order from the input end to the output end, each of the target units in the model to be optimized is optimized in turn to obtain an intermediate optimization model; The intermediate optimization model is globally optimized through a preset loss function to obtain a target optimization model.

2. The model quantization method according to claim 1, characterized in that: Determining a target number of boosting modules to be inserted into the initial quantization model according to a preset insertion rule includes: Determine a first parameter value for characterizing spatial complexity and a second parameter value for characterizing computational complexity of the initial quantization model after inserting an initial number of the lifting modules; Determine whether the first parameter value is not greater than a first constraint value, and whether the second parameter value is not greater than a second constraint value; If yes, increase the number of the inserted lifting modules, and calculate the first parameter value and the second parameter value; and return to the step of determining whether the first parameter value is not greater than the first constraint value, and whether the second parameter value is not greater than the second constraint value; If not, the insertion quantity of the lifting modules corresponding to the last judgment result is taken as the target quantity.

3. The model quantization method according to claim 1, characterized in that: Dividing the initial quantization model into the target number of backbone network modules includes: When the total number of network layers of the initial quantization model is an integer multiple of the target number, each integer multiple of the network layers in the total number of network layers is divided into one backbone network module; When the total number of network layers of the initial quantization model is not an integer multiple of the target number, among the target number of backbone network modules, the number of network layers of one backbone network module is the total number of network layers divided by the target number and rounded up, and the number of network layers of the other backbone network modules is the total number of network layers divided by the target number and rounded down.

4. The model quantization method according to claim 1, characterized in that: Based on the order from the input end to the output end, each target unit in the model to be optimized is optimized in turn, including: Dividing the pre-trained model and the initial quantization model into the same structure to obtain a target pre-trained model including multiple sub-modules; Inputting the specified sample into the target pre-trained model and the model to be optimized; Each of the submodules and each of the target units is traversed in turn, and the output result of each of the submodules is used as the optimization target to iteratively optimize the corresponding target unit.

5. The model quantization method according to claim 4, characterized in that: Taking the output results of each submodule as the optimization target, the corresponding target unit is iteratively optimized, including: Obtaining a current output result of a current target unit, and determining whether a difference between the current output result and a previous iteration output result is within a preset range; If yes, enter the next iterative optimization of the target unit; If not, perform the following steps; Under the condition of fixing the quantization parameter of the current backbone network module in the current target unit, determining the target improvement parameter according to the first difference between the current output result and the corresponding optimization target; Under the condition of fixing the lifting parameter of the current lifting module in the current target unit, determining the target quantization parameter according to the second difference between the current output result and the corresponding optimization target; Optimizing the current boosting module by the target boosting parameter, and optimizing the current backbone network module by the target quantization parameter; Return to the step of obtaining the current output result of the current target unit and determining whether the difference between the current output result and the output result of the previous iteration is within a preset range.

6. The model quantization method according to claim 4, characterized in that: The preset loss function is composed of an output loss function, an activation regularization loss function and an improvement model regularization loss function; The output loss function is used to optimize the difference between the output of the intermediate optimization model and the output of the pre-training model; The activation regularization loss function is used to optimize the difference between the output of each target unit in the intermediate optimization model and the output of the corresponding submodule; The lifting model regularization loss function is used to limit the distance between the lifting parameters of the lifting module and the intermediate optimization model.

7. The model quantization method according to claim 4, characterized in that: The intermediate optimization model is globally optimized by using a preset loss function, including: Inputting a specified sample into the pre-trained model and the intermediate optimization model; Obtaining total outputs of the pre-trained model and the intermediate optimization model respectively; Obtaining target activation output, wherein the target activation output includes activation output of each target unit in the intermediate optimization model and activation output of the corresponding submodule; Respectively obtaining the boosting parameters of the boosting modules in the model to be optimized and the intermediate optimization model; The intermediate optimization model is optimized by the preset loss function according to the total output, the target activation output and the boost parameter.

8. A model quantization device, characterized in that: The device comprises: A quantization unit, used to quantize the pre-trained model to obtain an initial quantized model; A target number determination unit, used to determine a target number of boosting modules to be inserted into the initial quantization model according to a preset insertion rule; A division unit, used for dividing the initial quantization model into the target number of backbone network modules; A lifting module inserting unit, used for inserting one lifting module in parallel into each backbone network module in the initial quantization model, so as to obtain a model to be optimized including a plurality of target units; wherein the target unit includes the lifting module and the backbone network module; An intermediate optimization unit, used to optimize each of the target units in the model to be optimized in sequence based on the order from the input end to the output end, to obtain an intermediate optimization model; The target optimization unit is used to globally optimize the intermediate optimization model through a preset loss function to obtain a target optimization model.

9. A model quantization device, comprising a memory and a processor, wherein the memory stores a computer program that can be run on the processor, characterized in that: When the processor executes the program, the steps of the model quantization method described in any one of claims 1 to 7 are implemented.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the steps of the model quantization method described in any one of claims 1 to 7 are implemented.

Images