Quantitative reasoning method and device of model, electronic equipment and storage medium

Abstract

The invention provides a quantitative reasoning method and device of a model, electronic equipment and a storage medium, and relates to the technical field of computers. The quantitative reasoning method of the model comprises the following steps: acquiring a calibration data set; the calibration data set comprises a plurality of pieces of input data used for being input into a to-be-quantized model, and each piece of input data is a vector comprising at least one scalar parameter; inputting the input data in the calibration data set into a to-be-quantized model in sequence; after inputting the input data in the calibration data set into the to-be-quantized model, for each layer of neural network structure in the to-be-quantized model, respectively calculating an input quantization parameter, a neural network quantization parameter and an output quantization parameter corresponding to the layer of neural network structure; and based on the input quantization parameter, the neural network quantization parameter and the output quantization parameter corresponding to each layer of neural network structure, performing quantization reasoning on the to-be-quantized model.

Description

Technical Field

[0001] The present application relates to the technical field of computers, and in particular to a quantitative reasoning method, device, electronic device and storage medium for a model. Background Art

[0002] With the development of technology, artificial intelligence models have been widely used. However, when artificial intelligence models are used in devices with limited computing and storage resources, the models are often required to have efficient reasoning capabilities.

[0003] At present, the problem is generally solved by artificial intelligence model quantization technology, which mainly includes the following steps / modules: obtaining calibration data and floating-point models; calculating the quantization parameters of the model through the maximum and minimum values ​​or sliding average algorithm; quantizing the input and output data of each layer of the model and the parameters of the model itself. However, the quantization parameters obtained by these methods are not accurate enough. Summary of the invention

[0004] The present application provides a model quantization inference method, device, electronic device and storage medium to solve the problem that the quantization parameters obtained in the prior art are not accurate enough.

[0005] In a first aspect, the present application provides a method for quantitative reasoning of a model, comprising: obtaining a calibration data set; the calibration data set comprises a plurality of input data for inputting into the model to be quantized, each of the input data being a vector comprising at least one scalar parameter; the input data in the calibration data set is input into the model to be quantized in sequence; after all the input data in the calibration data set are input into the model to be quantized, for each layer of neural network structure in the model to be quantized, the input quantization parameter, neural network quantization parameter, and output quantization parameter corresponding to the layer of neural network structure are calculated respectively; based on the input quantization parameter, neural network quantization parameter, and output quantization parameter corresponding to each layer of the neural network structure, quantitative reasoning is performed on the model to be quantized.

[0006] In an embodiment of the present application, the input data in the calibration data set is sequentially input into the model to be quantized. After the input data in the calibration data set are input into the model to be quantized, the input quantization parameters, neural network quantization parameters, and output quantization parameters corresponding to each layer of the neural network structure in the model to be quantized can be calculated. Since the input quantization parameters, neural network quantization parameters, and output quantization parameters of each layer of the neural network structure are calculated separately, the calculated input quantization parameters, neural network quantization parameters, and output quantization parameters can more accurately fit the actual situation of the neural network structure of this layer, so that the quantization parameters are more accurate, thereby making the effect of quantitative reasoning better.

[0007] In combination with the technical solution provided in the first aspect above, in some possible implementations, calculating the input quantization parameter corresponding to the layer of neural network structure includes: when the layer of neural network structure receives input data, updating a first preset set and a second preset set based on the input data; wherein the parameters in the first preset set are greater than the parameters in the second preset set; after all calibration data in the calibration data set are input, calculating the difference between the largest X parameters in the first preset set and the smallest X parameters in the second preset set to obtain X differences; calculating the standard deviation of the X differences; calculating a first ratio and a second ratio according to the standard deviation and a first preset constant, respectively; determining a first parameter matching the first ratio from the first preset set; determining a second parameter matching the second ratio from the second preset set; and calculating the input quantization parameter according to the first parameter, the second parameter, and the quantization bit width.

[0008] In the embodiment of the present application, the parameters included in the first preset set and the second preset set are determined by the input data, so that the first preset set and the second preset set can accurately represent the maximum and minimum values ​​of the input data. The first ratio and the second ratio determined by the standard deviation can represent the dispersion of the data, thereby avoiding outlier data, thereby improving the accuracy of the subsequent quantization parameters and improving the precision of quantization reasoning.

[0009] In combination with the technical solution provided in the first aspect above, in some possible implementations, when the neural network structure at this layer receives input data, the first preset set and the second preset set are updated based on the input data, including: when the neural network structure at this layer receives input data, a maximum parameter is determined from the first preset set and the input data, and the maximum parameter is added to the first preset set; a minimum parameter is determined from the second preset set and the input data, and the minimum parameter is added to the second preset set.

[0010] In the embodiment of the present application, since the first preset set contains maximum parameters and the second preset set contains minimum parameters, the quantization interval can be accurately determined by the first preset set and the second preset set.

[0011] In combination with the technical solution provided in the first aspect above, in some possible implementations, a maximum parameter is determined from a first preset set and input data, and the parameter is added to the first preset set, including: determining the maximum parameter in the first preset set and the input data; if the maximum parameter is a parameter in the first preset set, the first preset set is not updated; if the maximum parameter is a parameter in the input data, the maximum parameter is added to the first preset set.

[0012] In the embodiment of the present application, when the maximum parameter is a parameter in the first preset set, since the maximum parameter already exists in the first preset set, there is no need to add it to the first preset set again.

[0013] In combination with the technical solution provided in the first aspect above, in some possible implementations, a minimum parameter is determined from a second preset set and input data, and the parameter is added to the second preset set, including: determining the minimum parameter in the second preset set and the input data; if the minimum parameter is a parameter in the second preset set, the second preset set is not updated; if the minimum parameter is a parameter in the input data, the minimum parameter is added to the first preset set.

[0014] In the embodiment of the present application, when the minimum parameter is a parameter in the second preset set, since the minimum parameter already exists in the second preset set, there is no need to add it to the second preset set again.

[0015] In combination with the technical solution provided in the first aspect above, in some possible implementations, the first ratio and the second ratio are calculated according to the standard deviation and the first preset constant, respectively, including: according to the formula Calculate the first ratio; according to the formula The second ratio is calculated; wherein std1 represents the standard deviation of the X differences, c represents the first ratio, d represents the second ratio, and β1 is a first preset constant.

[0016] In combination with the technical solution provided in the first aspect above, in some possible implementations, calculating the output quantization parameter corresponding to the layer of neural network structure includes: when the layer of neural network structure generates output data, updating the third preset set and the fourth preset set based on the output data; wherein the parameters in the third preset set are greater than the parameters in the fourth preset set; after all calibration data in the calibration data set are input, calculating the difference between the largest Y parameters in the third preset set and the smallest Y parameters in the fourth preset set to obtain Y differences; calculating the standard deviation of the Y differences; calculating the third ratio and the fourth ratio respectively according to the standard deviation and a second preset constant; determining a third parameter matching the third ratio from the third preset set; determining a fourth parameter matching the fourth ratio from the fourth preset set; and calculating the output quantization parameter according to the third parameter, the fourth parameter, and the quantization bit width.

[0017] In the embodiment of the present application, the third ratio and the fourth ratio determined by the standard deviation can represent the dispersion of the data, thereby avoiding outlier data, thereby improving the accuracy of the subsequently obtained quantization parameters and improving the precision of quantization reasoning.

[0018] In combination with the technical solution provided in the first aspect above, in some possible implementations, when the neural network structure of this layer generates output data, the third preset set and the fourth preset set are updated based on the output data, including: when the neural network structure of this layer generates output data, determining the maximum parameter from the third preset set and the output data, and adding the maximum parameter to the third preset set; determining the minimum parameter from the fourth preset set and the output data, and adding the minimum parameter to the fourth preset set.

[0019] In the embodiment of the present application, since the third preset set contains maximum parameters and the fourth preset set contains minimum parameters, the quantization interval can be accurately determined by the third preset set and the fourth preset set.

[0020] In combination with the technical solution provided in the first aspect above, in some possible implementations, a maximum parameter is determined from a third preset set and output data, and the parameter is added to the third preset set, including: determining the maximum parameter in the third preset set and the output data; if the maximum parameter is a parameter in the third preset set, the third preset set is not updated; if the maximum parameter is a parameter in the output data, the maximum parameter is added to the third preset set.

[0021] In the embodiment of the present application, when the maximum parameter is a parameter in the third preset set, since the maximum parameter already exists in the third preset set, there is no need to add it to the third preset set again.

[0022] In combination with the technical solution provided in the first aspect above, in some possible implementations, a minimum parameter is determined from a fourth preset set and the output data, and the parameter is added to the fourth preset set, including: determining the minimum parameter in the fourth preset set and the output data; if the minimum parameter is a parameter in the fourth preset set, the fourth preset set is not updated; if the minimum parameter is a parameter in the output data, the minimum parameter is added to the fourth preset set.

[0023] In the embodiment of the present application, when the minimum parameter is a parameter in the fourth preset set, since the minimum parameter already exists in the fourth preset set, there is no need to add it to the fourth preset set again.

[0024] In combination with the technical solution provided in the first aspect above, in some possible implementations, calculating the neural network quantization parameters corresponding to the neural network structure of this layer includes: obtaining a fifth preset set and a sixth preset set corresponding to the neural network structure of this layer; wherein the fifth preset set and the sixth preset set are obtained based on the parameter matrix corresponding to the neural network structure of this layer; the fifth preset set includes the largest Z parameters in the parameter matrix, and the sixth preset set includes the smallest Z parameters in the parameter matrix; calculating the difference between the Z parameters in the fifth preset set and the Z parameters in the sixth preset set to obtain Z differences; calculating the standard deviation of the Z differences; calculating the fifth ratio and the sixth ratio respectively according to the standard deviation and the third preset constant; determining the fifth parameter matching the fifth ratio from the fifth preset set; determining the sixth parameter matching the sixth ratio from the sixth preset set; and calculating the neural network quantization parameters according to the fifth parameter, the sixth parameter, and the quantization bit width.

[0025] In the embodiment of the present application, the fifth ratio and the sixth ratio determined by the standard deviation can represent the dispersion of the data, thereby avoiding outlier data, thereby improving the accuracy of the subsequently obtained quantization parameters and improving the precision of quantization reasoning.

[0026] In a second aspect, the present application provides a quantitative reasoning device for a model, comprising: an acquisition module and a processing module, the acquisition module is used to acquire a calibration data set; the processing module is used to input the calibration data set into the model to be quantized in sequence; after all the input data in the calibration data set are input into the model to be quantized, for each layer of neural network structure in the model to be quantized, the input quantization parameters, neural network quantization parameters, and output quantization parameters corresponding to the layer of neural network structure are respectively calculated; based on the input quantization parameters, neural network quantization parameters, and output quantization parameters corresponding to each layer of the neural network structure, quantitative reasoning is performed on the model to be quantized.

[0027] In a third aspect, the present application provides an electronic device, comprising: a memory and a processor, the memory and the processor being connected; the memory being used to store programs; and the processor being used to call the programs stored in the memory to execute the method described in the first aspect above and / or in combination with any possible implementation of the first aspect above.

[0028] In a fourth aspect, the present application provides a computer-readable storage medium having a computer program stored thereon. When the computer program is executed by a computer, the computer program executes the method described in the first aspect and / or in combination with any possible implementation of the first aspect. BRIEF DESCRIPTION OF THE DRAWINGS

[0029] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required for use in the embodiments will be briefly introduced below. It should be understood that the following drawings only show certain embodiments of the present application and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without paying creative work.

[0030] Figure 1 A flowchart of a quantitative reasoning method for a model shown in an embodiment of the present application;

[0031] Figure 2 A histogram of random error and test error shown in an embodiment of the present application;

[0032] Figure 3 A structural block diagram of a quantitative reasoning device for a model shown in an embodiment of the present application;

[0033] Figure 4 A structural block diagram of an electronic device shown in an embodiment of the present application. DETAILED DESCRIPTION

[0034] The technical solutions in the embodiments of the present application will be described below in conjunction with the drawings in the embodiments of the present application.

[0035] It should be noted that similar numbers and letters represent similar items in the following figures, so once an item is defined in one figure, it does not need to be further defined and explained in subsequent figures. At the same time, in the description of this application, relational terms such as "first", "second", etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the term "comprise" or any other variant thereof is intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device.

[0036] The technical solution of the present application will be described in detail below with reference to the accompanying drawings.

[0037] See also Figure 1 , Figure 1 A flowchart of a quantitative reasoning method provided in an embodiment of the present application is shown below. Figure 1 Describe the steps it involves.

[0038] S100: Acquire a calibration data set.

[0039] The calibration data set includes a plurality of input data for inputting into the model to be quantized, and each input data is a vector including at least one scalar parameter.

[0040] The calibration data set may be acquired in advance and stored in a local storage medium, and can be directly acquired when needed. Alternatively, the calibration data set may be acquired from a third party when needed.

[0041] Optionally, the calibration data set can be obtained by first obtaining the original data that the model to be quantized needs to process, such as text data, etc. Then, each piece of original data is converted into a vector to obtain the calibration data set. The specific type of the original data is determined according to the actual application scenario of the model to be quantized, and its specific type is not limited here.

[0042] In one implementation, the model to be quantized may be a model for processing image data, such as a face recognition model, an object recognition model, etc. In this case, each piece of input data in the calibration data set is image data represented by a vector.

[0043] In this implementation, the model to be quantized can be used in an image processing device, such as a computer, server, mobile phone, etc. with image processing capabilities.

[0044] Optionally, the model to be quantized may also be a model for processing audio data, such as a speech recognition model, etc. In this case, each piece of input data in the calibration data set is audio data represented by a vector.

[0045] In this implementation, the model to be quantized can be used in an audio processing device, such as a computer, server, mobile phone, headset, smart bracelet, etc. with audio processing capabilities.

[0046] Optionally, the model to be quantized may also be a model for processing text data, such as an inference model, a text classification model, etc. In this case, each piece of input data in the calibration data set is text data represented by a vector.

[0047] In this implementation, the model to be quantized can be used in an audio processing device, such as a computer, server, mobile phone, headset, smart bracelet, etc. with audio processing capabilities.

[0048] S200: Inputting the input data in the calibration data set into the model to be quantized in sequence.

[0049] Optionally, each piece of input data in the calibration data set may be input into the model to be quantized in sequence according to the arrangement order of each piece of input data in the calibration data set.

[0050] Alternatively, a piece of input data may be randomly determined from the calibration data set and input into the quantization model, and then a piece of input data that has not been input is randomly determined from the calibration data set and input into the quantization model, until every piece of input data in the calibration data set is input into the model to be quantized.

[0051] S300: After all input data in the calibration data set are input into the model to be quantized, for each layer of the neural network structure in the model to be quantized, the input quantization parameter, the neural network quantization parameter, and the output quantization parameter corresponding to the neural network structure are calculated respectively.

[0052] The neural network structure included in the model to be quantized may include an input layer, a convolutional layer, a fully connected layer, a pooling layer, a function activation layer, a batch normalization layer, etc. The neural network structure included in the model to be quantized may not be limited to the types listed here.

[0053] In one implementation, the input quantization parameters corresponding to each layer of the neural network structure may be calculated by first updating the first preset set and the second preset set based on the input data when the neural network structure of the layer receives the input data, wherein the parameters in the first preset set are greater than the parameters in the second preset set.

[0054] Then, after all calibration data in the calibration data set are input, the difference between the largest X parameters in the first preset set and the smallest X parameters in the second preset set is calculated to obtain X difference values.

[0055] Afterwards, the standard deviation of the X differences is calculated, and the first ratio (dispersion) and the second ratio (dispersion) are respectively calculated based on the standard deviation and the first preset constant.

[0056] Then, a first parameter matching the first ratio is determined from the first preset set, and a second parameter matching the second ratio is determined from the second preset set.

[0057] Finally, the input quantization parameter is calculated according to the first parameter, the second parameter, and the quantization bit width.

[0058] Optionally, all parameters in the first preset set may be arranged from large to small, and all parameters in the second preset set may be arranged from small to large.

[0059] In one implementation, when the neural network structure at this layer receives input data, the first preset set and the second preset set are updated based on the input data. The specific method may be: when the neural network structure at this layer receives input data, a maximum parameter is determined from the first preset set and the input data, and the maximum parameter is added to the first preset set. And a minimum parameter is determined from the second preset set and the input data, and the minimum parameter is added to the second preset set.

[0060] When the determined maximum parameter is a parameter in the first preset set, since the parameter already exists in the first preset set, there is no need to add the maximum parameter to the first preset set again.

[0061] When the determined minimum parameter is a parameter in the second preset set, since the parameter already exists in the second preset set, there is no need to add the minimum parameter to the second preset set again.

[0062] For ease of understanding, the example of input data including three scalar parameters of 11, 14, and 9 is used for explanation. If the largest parameter in the first preset set is 13, the smallest parameter in the second preset set is 7. Since 14 is greater than 13, 14 needs to be added to the first preset set. However, there is no scalar parameter less than 7 in the input data, so there is no need to add a new parameter to the second preset set. The examples here are only for ease of understanding and should not be used as limitations on the present application.

[0063] Optionally, a specific method of determining a maximum parameter from the first preset set and the input data and adding the parameter to the first preset set may also be: firstly determining a maximum parameter in the first preset set and the input data.

[0064] If the maximum parameter is a parameter in the first preset set, the first preset set is not updated.

[0065] If the maximum parameter is a parameter in the input data, the maximum parameter is added to the first preset set.

[0066] Optionally, determining a minimum parameter from the second preset set and the input data, and adding the parameter to the second preset set may be: determining a minimum parameter from the second preset set and the input data.

[0067] If the minimum parameter is a parameter in the second preset set, the second preset set is not updated.

[0068] If the minimum parameter is a parameter in the input data, the minimum parameter is added to the first preset set.

[0069] In one implementation, when the neural network structure at this layer receives input data, the first preset set and the second preset set are updated based on the input data. The specific method may be: when the neural network structure at this layer receives input data, all scalar parameters in the input data that are greater than the largest parameter in the first preset set are added to the first preset set. And all scalar parameters in the input data that are less than the smallest parameter in the second preset set are added to the second preset set.

[0070] For ease of understanding, the example of input data including six scalar parameters of 11, 14, 9, 4, 6, and 18 is used for explanation. If the largest parameter in the first preset set is 13, the smallest parameter in the second preset set is 7. Since 14 and 18 are both greater than 13, 14 and 18 need to be added to the first preset set. In the input data, 4 and 6 are both less than 7, so 4 and 6 need to be added to the second preset set. The examples here are only for ease of understanding and should not be used as limitations on the present application.

[0071] Optionally, when the neural network structure of this layer receives input data, the first preset set and the second preset set are updated based on the input data. The specific method may be: adding the largest K1 scalar parameters in the input data to the first preset set, and adding the smallest K2 scalar parameters in the input data to the second preset set.

[0072] Wherein, K1 and K2 are both positive integers, and K1 and K2 may be equal.

[0073] For example, if K1 and K2 are both 1, and the input data includes six scalar parameters, 11, 14, 9, 4, 6, and 18, then 18 needs to be added to the first preset set, and 4 needs to be added to the second preset set. The examples here are only for ease of understanding and should not be used as limitations on the present application.

[0074] To facilitate understanding of the specific method of calculating the difference between the largest X parameters in the first preset set and the smallest X parameters in the second preset set to obtain X differences after all calibration data in the calibration data set are input, an example will be given below to illustrate.

[0075] If the first preset set includes 10 parameters, namely 50, 49, 48, 47, 46, 45, 44, 43, 42, and 41, and the second preset set includes 8 parameters, namely 5, 6, 7, 8, 9, 10, 11, and 12, and X=3, then the largest X parameters in the first preset set are 50, 49, and 48, respectively, and the smallest X parameters in the second preset set are 5, 6, and 7, respectively.

[0076] Therefore, the difference between the first parameter (i.e., 50) in the first preset set and the first parameter (i.e., 5) in the second preset set is calculated to be 45. And the difference between the second parameter (i.e., 49) in the first preset set and the first parameter (i.e., 6) in the second preset set is calculated to be 43. The difference between the third parameter (i.e., 48) in the first preset set and the third parameter (i.e., 7) in the second preset set is calculated to be 41. That is, the three calculated differences are 45, 43, and 41, respectively. The examples here are only for ease of understanding and should not be used as limitations on the present application.

[0077] Optionally, the standard deviation of the X differences is calculated. The specific method of calculating the first ratio and the second ratio according to the standard deviation and the first preset constant can be: firstly calculate the standard deviation of the X differences, and then calculate the first ratio according to the formula The first ratio is calculated. According to the formula The second ratio is calculated. Wherein, std1 represents the standard deviation of the X differences, c represents the first ratio, d represents the second ratio, and β1 is a first preset constant.

[0078] β1 may be obtained in advance and used directly, or β1 may be obtained through testing.

[0079] For example, multiple β1 values ​​may be preconfigured, and then quantitative reasoning may be performed using the solution based on each β1 value, and finally the β value corresponding to the model with the best actual use effect obtained after quantitative reasoning may be determined as the final β1 value.

[0080] Optionally, the specific manner of determining the first parameter matching the first ratio from the first preset set and determining the second parameter matching the second ratio from the second preset set may be: determining the parameter whose ranking position accounts for the ratio of the total number of the first preset set closest to the first ratio from the first preset set as the first parameter, and determining the parameter whose ranking position accounts for the ratio of the total number of the second preset set closest to the second ratio from the second preset set as the second parameter.

[0081] In order to facilitate understanding of the specific manner of determining the first parameter matching the first ratio from the first preset set and determining the second parameter matching the second ratio from the second preset set, the following is an example in which the first parameter is 0.3, the second parameter is 0.5, the first preset set includes 10 parameters including 50, 49, 48, 47, 46, 45, 44, 43, 42, and 41, and the second preset set includes 8 parameters including 5, 6, 7, 8, 9, 10, 11, and 12.

[0082] For the first parameter, the parameter ranked at the 30% (ie, 10×0.3) in the first preset set is used as the first parameter, that is, 48 ​​ranked at the third position is used as the first parameter.

[0083] For the second parameter, the parameter ranked at the 50th percentile (ie, 8×0.5) in the second preset set is used as the second parameter, that is, the parameter 8 ranked at the 4th position is used as the second parameter.

[0084] Optionally, a specific calculation formula for obtaining the input quantization parameter according to the first parameter, the second parameter, and the quantization bit width is: Among them, S x represents the input quantization parameter, C represents the first parameter, D represents the second parameter, and N represents the quantization bit width.

[0085] In one implementation, the specific method for calculating the neural network quantization parameters corresponding to each layer of the neural network structure may be: first obtain the fifth preset set and the sixth preset set corresponding to the neural network structure of the layer. The fifth preset set and the sixth preset set are obtained based on the parameter matrix corresponding to the neural network structure of the layer. The fifth preset set includes the largest Z parameters in the parameter matrix, the sixth preset set includes the smallest Z parameters in the parameter matrix, and all parameters in the fifth preset set are arranged from large to small; all parameters in the sixth preset set are arranged from small to large.

[0086] Then, the difference between the Ith parameter in the fifth preset set and the Ith parameter in the sixth preset set is calculated to obtain Z differences; wherein I is 1 to Z respectively.

[0087] Then, the standard deviation of the Z differences is calculated, and the fifth ratio and the sixth ratio are calculated according to the standard deviation and the third preset constant.

[0088] A fifth parameter matching the fifth ratio is determined from the fifth preset set. A sixth parameter matching the sixth ratio is determined from the sixth preset set. Finally, the neural network quantization parameter is calculated based on the fifth parameter, the sixth parameter, and the quantization bit width.

[0089] Optionally, in order to facilitate understanding of the method of calculating the difference between the Ith parameter in the fifth preset set and the Ith parameter in the sixth preset set to obtain Z differences, the fifth preset set includes four parameters, 50, 49, 48, and 47, and the sixth preset set includes four parameters, 5, 6, 7, and 8.

[0090] Then the difference between the first parameter in the fifth preset set (i.e., 50) and the first parameter in the sixth preset set (i.e., 5) is calculated to be 45. And the difference between the second parameter in the fifth preset set (i.e., 49) and the first parameter in the sixth preset set (i.e., 6) is calculated to be 43. The difference between the third parameter in the fifth preset set (i.e., 48) and the third parameter in the sixth preset set (i.e., 7) is calculated to be 41. The difference between the fourth parameter in the fifth preset set (i.e., 47) and the fourth parameter in the sixth preset set (i.e., 8) is calculated to be 39. That is, the four calculated differences are 45, 43, 41, and 39, respectively. The examples here are only for ease of understanding and should not be regarded as limitations on the present application.

[0091] Optionally, the specific method of calculating the fifth ratio and the sixth ratio according to the standard deviation and the third preset constant can be: firstly calculate the standard deviation of X differences, and then calculate the standard deviation according to the formula The fifth ratio is calculated. According to the formula The sixth ratio is calculated. Wherein, std2 represents the standard deviation of the Z differences, i represents the fifth ratio, j represents the sixth ratio, and β2 is a third preset constant.

[0092] β2 may be obtained in advance and used directly, or β2 may be obtained through testing.

[0093] For example, multiple β2 values ​​may be preconfigured, and then quantitative reasoning may be performed using the solution based on each β2 value, and finally the β value corresponding to the model with the best actual use effect obtained after quantitative reasoning may be determined as the final β2 value.

[0094] Optionally, the specific manner of determining the fifth parameter matching the fifth ratio from the fifth preset set and the specific manner of determining the sixth parameter matching the sixth ratio from the sixth preset set is the same as the specific manner of determining the first parameter matching the first ratio from the first preset set and the specific manner of determining the second parameter matching the second ratio from the second preset set, which will not be described in detail for the sake of brevity.

[0095] Optionally, the specific calculation formula for obtaining the neural network quantization parameter according to the fifth parameter, the sixth parameter, and the quantization bit width is: Among them, S p Represents the neural network quantization parameter, I represents the first parameter, J represents the second parameter, and N represents the quantization bit width.

[0096] In one implementation, a specific method for calculating the output quantization parameters corresponding to the neural network structure of this layer may be: first, when the neural network structure of this layer generates output data, the third preset set and the fourth preset set are updated based on the output data; wherein the parameters in the third preset set are greater than the parameters in the fourth preset set.

[0097] After all calibration data in the calibration data set are input, the difference between the largest Y parameters in the third preset set and the smallest Y parameters in the fourth preset set is calculated to obtain Y difference values.

[0098] Then the standard deviation of the Y differences is calculated, and the third ratio and the fourth ratio are calculated based on the standard deviation and the second preset constant.

[0099] Then, a third parameter matching the third ratio is determined from the third preset set, and a fourth parameter matching the fourth ratio is determined from the fourth preset set.

[0100] Finally, the output quantization parameter is calculated according to the third parameter, the fourth parameter and the quantization bit width.

[0101] Optionally, all parameters in the third preset set may be arranged from large to small, and all parameters in the fourth preset set may be arranged from small to large.

[0102] In one implementation, when the neural network structure at this layer generates output data, the specific method of updating the third preset set and the fourth preset set based on the output data may be that when the neural network structure at this layer receives the output data, a maximum parameter is determined from the third preset set and the output data, and the maximum parameter is added to the third preset set. And a minimum parameter is determined from the fourth preset set and the output data, and the minimum parameter is added to the fourth preset set.

[0103] When the determined maximum parameter is a parameter in the third preset set, since the parameter already exists in the third preset set, there is no need to add the maximum parameter to the third preset set again.

[0104] When the determined minimum parameter is a parameter in the fourth preset set, since the parameter already exists in the fourth preset set, there is no need to add the minimum parameter to the fourth preset set again.

[0105] Optionally, a specific method of determining a maximum parameter from the third preset set and the output data and adding the parameter to the third preset set may be: firstly determining a maximum parameter in the third preset set and the output data.

[0106] If the maximum parameter is a parameter in the third preset set, the third preset set is not updated.

[0107] If the maximum parameter is a parameter in the output data, the maximum parameter is added to the third preset set.

[0108] Optionally, a specific method of determining a minimum parameter from the fourth preset set and the output data and adding the parameter to the fourth preset set may be: firstly determining the minimum parameter in the fourth preset set and the output data.

[0109] If the minimum parameter is a parameter in the fourth preset set, the fourth preset set is not updated.

[0110] If the minimum parameter is a parameter in the output data, the minimum parameter is added to the fourth preset set.

[0111] In one implementation, when the neural network structure of this layer generates output data, the specific method of updating the third preset set and the fourth preset set based on the output data may be to add all scalar parameters in the output data that are greater than the largest parameter in the third preset set to the third preset set. And all scalar parameters in the output data that are less than the smallest parameter in the fourth preset set are added to the fourth preset set.

[0112] In one implementation, when this layer of neural network structure generates output data, the specific method of updating the third preset set and the fourth preset set based on the output data can be to add the largest K3 scalar parameters in the output data to the third preset set, and to add the smallest K4 scalar parameters in the output data to the fourth preset set.

[0113] Wherein, K3 and K4 are both positive integers, and K3 and K4 may be equal.

[0114] Among them, the specific method of updating the third preset set and the fourth preset set based on the output data is similar to the specific method of updating the first preset set and the second preset set based on the input data. For the sake of brief description, the content not described in this embodiment can refer to the corresponding content mentioned above and will not be repeated here.

[0115] To facilitate understanding of the specific method of calculating the difference between the largest Y parameters in the third preset set and the smallest Y parameters in the fourth preset set to obtain Y differences after all calibration data in the calibration data set are input, an example will be given below to illustrate.

[0116] If the third preset set includes 10 parameters, namely, 50, 49, 48, 47, 46, 45, 44, 43, 42, and 41, and the fourth preset set includes 8 parameters, namely, 5, 6, 7, 8, 9, 10, 11, and 12, and Y=3, then the largest Y parameters in the third preset set are 50, 49, and 48, respectively, and the smallest Y parameters in the fourth preset set are 5, 6, and 7, respectively.

[0117] Therefore, the difference between the first parameter in the third preset set (i.e., 50) and the first parameter in the fourth preset set (i.e., 5) is calculated to be 45. And the difference between the second parameter in the third preset set (i.e., 49) and the third parameter in the fourth preset set (i.e., 6) is calculated to be 43. The difference between the third parameter in the third preset set (i.e., 48) and the third parameter in the fourth preset set (i.e., 7) is calculated to be 41. That is, the three calculated differences are 45, 43, and 41, respectively. The examples here are only for ease of understanding and should not be used as limitations on the present application.

[0118] In one implementation, the standard deviation of Y differences is calculated. The specific implementation method of calculating the third ratio and the fourth ratio based on the standard deviation and the second preset constant can be: first calculate the standard deviation of Y differences. Then calculate the third ratio based on the formula The third ratio is calculated. According to the formula A fourth ratio is calculated.

[0119] Among them, std3 represents the standard deviation of Y differences, r represents the third ratio, t represents the fourth ratio, and β3 is a second preset constant.

[0120] β3 may be obtained in advance and used directly, or β3 may be obtained through testing.

[0121] For example, multiple β3 values ​​may be preconfigured, and then quantitative reasoning may be performed using the solution based on each β3 value, and finally the β value corresponding to the model with the best actual use effect obtained after quantitative reasoning may be determined as the final β3 value.

[0122] Optionally, the specific manner of determining the third parameter matching the third ratio from the third preset set and the specific manner of determining the fourth parameter matching the fourth ratio from the fourth preset set is the same as the specific manner of determining the first parameter matching the first ratio from the first preset set and determining the second parameter matching the second ratio from the second preset set, which will not be described in detail for the sake of brevity.

[0123] Optionally, a specific calculation formula for calculating the output quantization parameter according to the third parameter, the fourth parameter, and the quantization bit width is: Among them, S y represents the output quantization parameter, R represents the third parameter, T represents the fourth parameter, and N represents the quantization bit width.

[0124] S400: Perform quantitative reasoning on the model to be quantized based on the input quantization parameters, the neural network quantization parameters, and the output quantization parameters corresponding to each layer of the neural network structure.

[0125] After obtaining the input quantization parameters, the neural network quantization parameters, and the output quantization parameters, the specific method of performing quantitative reasoning on the model based on the quantization parameters is well known to those skilled in the art and will not be described here for the sake of brevity.

[0126] To understand the effect of this solution, please refer to Figure 2 .

[0127] like Figure 2 As shown in the figure, the random error and test error of the model obtained by using this method for quantitative reasoning of the model are both smaller than the random error and test error of the model using the maximum and minimum value method and the sliding average method for quantitative reasoning. It can be seen that this scheme can improve the accuracy of quantitative reasoning of artificial intelligence models.

[0128] Based on the same technical concept, the present application also provides a quantitative reasoning device for a model, such as Figure 3 As shown, the quantitative reasoning device 100 of the model includes an acquisition module 110 and a processing module 120 .

[0129] The acquisition module 110 is used to acquire a calibration data set.

[0130] The processing module 120 is used to input the calibration data set into the model to be quantized in sequence; after all the input data in the calibration data set are input into the model to be quantized, for each layer of the neural network structure in the model to be quantized, the input quantization parameter, the neural network quantization parameter, and the output quantization parameter corresponding to the layer of the neural network structure are calculated; based on the input quantization parameter, the neural network quantization parameter, and the output quantization parameter corresponding to each layer of the neural network structure, the model to be quantized is subjected to quantitative reasoning.

[0131] The processing module 120 is specifically used to update the first preset set and the second preset set based on the input data when the neural network structure of this layer receives the input data; wherein the parameters in the first preset set are greater than the parameters in the second preset set; after all the calibration data in the calibration data set are input, the difference between the largest X parameters in the first preset set and the smallest X parameters in the second preset set is calculated to obtain X differences; the standard deviation of the X differences is calculated; the first ratio and the second ratio are calculated according to the standard deviation and the first preset constant respectively; the first parameter matching the first ratio is determined from the first preset set; the second parameter matching the second ratio is determined from the second preset set; the input quantization parameter is calculated according to the first parameter, the second parameter, and the quantization bit width.

[0132] The processing module 120 is specifically used to determine the maximum parameter from the first preset set and the input data when the neural network structure of this layer receives input data, and add the maximum parameter to the first preset set; determine the minimum parameter from the second preset set and the input data, and add the minimum parameter to the second preset set.

[0133] The processing module 120 is specifically used to determine the maximum parameter in the first preset set and the input data; if the maximum parameter is a parameter in the first preset set, the first preset set is not updated; if the maximum parameter is a parameter in the input data, the maximum parameter is added to the first preset set.

[0134] The processing module 120 is specifically used to determine the minimum parameter in the second preset set and the input data; if the minimum parameter is a parameter in the second preset set, the second preset set is not updated; if the minimum parameter is a parameter in the input data, the minimum parameter is added to the first preset set.

[0135] The processing module 120 is specifically configured to: Calculate the first ratio; according to the formula The second ratio is calculated; wherein std1 represents the standard deviation of the X differences, c represents the first ratio, d represents the second ratio, and β1 is a first preset constant.

[0136] The processing module 120 is specifically used to update the third preset set and the fourth preset set based on the output data when the neural network structure of this layer generates output data; wherein the parameters in the third preset set are greater than the parameters in the fourth preset set; after all the calibration data in the calibration data set are input, the difference between the largest Y parameters in the third preset set and the smallest Y parameters in the fourth preset set is calculated to obtain Y differences; the standard deviation of the Y differences is calculated; the third ratio and the fourth ratio are calculated according to the standard deviation and the second preset constant respectively; a third parameter matching the third ratio is determined from the third preset set; a fourth parameter matching the fourth ratio is determined from the fourth preset set; and an output quantization parameter is calculated according to the third parameter, the fourth parameter, and the quantization bit width.

[0137] The processing module 120 is specifically used to determine the maximum parameter from the third preset set and the output data, and add the maximum parameter to the third preset set when the neural network structure of this layer generates output data; determine the minimum parameter from the fourth preset set and the output data, and add the minimum parameter to the fourth preset set.

[0138] The processing module 120 is specifically used to determine the maximum parameter in the third preset set and the output data; if the maximum parameter is a parameter in the third preset set, the third preset set is not updated; if the maximum parameter is a parameter in the output data, the maximum parameter is added to the third preset set.

[0139] The processing module 120 is specifically used to determine the minimum parameter in the fourth preset set and the output data; if the minimum parameter is a parameter in the fourth preset set, the fourth preset set is not updated; if the minimum parameter is a parameter in the output data, the minimum parameter is added to the fourth preset set.

[0140] The processing module 120 is specifically used to obtain the fifth preset set and the sixth preset set corresponding to the neural network structure of this layer; wherein the fifth preset set and the sixth preset set are obtained based on the parameter matrix corresponding to the neural network structure of this layer; the fifth preset set includes the largest Z parameters in the parameter matrix, and the sixth preset set includes the smallest Z parameters in the parameter matrix; the difference between the Z parameters in the fifth preset set and the Z parameters in the sixth preset set is calculated to obtain Z differences; the standard deviation of the Z differences is calculated; the fifth ratio and the sixth ratio are respectively calculated according to the standard deviation and the third preset constant; the fifth parameter matching the fifth ratio is determined from the fifth preset set; the sixth parameter matching the sixth ratio is determined from the sixth preset set; the neural network quantization parameter is calculated according to the fifth parameter, the sixth parameter, and the quantization bit width.

[0141] The quantitative reasoning device 100 of the model provided in the embodiment of the present application has the same implementation principle and technical effects as those of the quantitative reasoning method embodiment of the aforementioned model. For the sake of brief description, for matters not mentioned in the device embodiment, reference may be made to the corresponding contents in the quantitative reasoning method embodiment of the aforementioned model.

[0142] See also Figure 4 , which is an electronic device 200 provided in an embodiment of the present application. The electronic device 200 includes: a processor 210 and a memory 220.

[0143] The memory 220 and the processor 210 are electrically connected to each other directly or indirectly to achieve data transmission or interaction. For example, these components can be electrically connected to each other via one or more communication buses or signal lines. The memory 220 is used to store computer programs, such as storing Figure 3The software function module shown in is the quantitative reasoning device 100 of the model. Among them, the quantitative reasoning device 100 of the model includes at least one software function module that can be stored in the memory 220 in the form of software or firmware or solidified in the operating system (OS) of the electronic device 200. The processor 210 is used to execute the executable module stored in the memory 220, such as the software function module or computer program included in the quantitative reasoning device 100 of the model. At this time, the processor 210 is used to obtain a calibration data set; the calibration data set includes a plurality of input data for inputting into the model to be quantized, and each of the input data is a vector including at least one scalar parameter; the input data in the calibration data set is sequentially input into the model to be quantized; after the input data in the calibration data set are all input into the model to be quantized, for each layer of the neural network structure in the model to be quantized, the input quantization parameter, the neural network quantization parameter, and the output quantization parameter corresponding to the neural network structure of the layer are calculated respectively; based on the input quantization parameter, the neural network quantization parameter, and the output quantization parameter corresponding to each layer of the neural network structure, the model to be quantized is quantitatively reasoned.

[0144] Among them, the memory 220 can be, but is not limited to, RAM (Random Access Memory), ROM (Read Only Memory), PROM (Programmable Read-Only Memory), EPROM (Erasable Programmable Read-Only Memory), EEPROM (Electric Erasable Programmable Read-Only Memory), etc.

[0145] The processor 210 may be an integrated circuit chip with signal processing capabilities. The above-mentioned processor may be a general-purpose processor, including a CPU (Central Processing Unit), an NP (Network Processor), etc.; it may also be a DSP (Digital Signal Processor), an ASIC (Application Specific Integrated Circuit), an FPGA (Field Programmable Gate Array) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components. The various methods, steps and logic block diagrams disclosed in the embodiments of the present application may be implemented or executed. A general-purpose processor may be a microprocessor or the processor 210 may also be any conventional processor, etc.

[0146] The electronic device 200 mentioned above includes but is not limited to a personal computer, a server, etc.

[0147] The embodiment of the present application also provides a computer-readable storage medium (hereinafter referred to as storage medium), on which a computer program is stored, and when the computer program is run by a computer such as the above-mentioned electronic device 200, the quantitative reasoning method of the model shown above is executed. The computer-readable storage medium includes: a U disk, a mobile hard disk, a read-only memory, a random access memory, a magnetic disk or an optical disk, and other media that can store program codes.

[0148] The above description is only the preferred embodiment of the present application and is not intended to limit the present application. For those skilled in the art, the present application may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application shall be included in the protection scope of the present application.

Claims

1. A quantitative reasoning method for a model, characterized in that: include: Acquire a calibration data set; the calibration data set includes a plurality of input data for inputting into the model to be quantized, each of the input data is a vector including at least one scalar parameter; Inputting the input data in the calibration data set into the model to be quantized in sequence; After all input data in the calibration data set are input into the model to be quantized, for each layer of the neural network structure in the model to be quantized, the input quantization parameter, the neural network quantization parameter, and the output quantization parameter corresponding to the neural network structure are calculated respectively; Based on the input quantization parameters, neural network quantization parameters, and output quantization parameters corresponding to each layer of the neural network structure, quantization reasoning is performed on the model to be quantized.

2. The method according to claim 1, characterized in that Calculate the input quantization parameters corresponding to the neural network structure of this layer, including: When the neural network structure at this layer receives input data, a first preset set and a second preset set are updated based on the input data; wherein the parameters in the first preset set are greater than the parameters in the second preset set; After all calibration data in the calibration data set are input, calculating the difference between the largest X parameters in the first preset set and the smallest X parameters in the second preset set to obtain X difference values; Calculate the standard deviation of the X differences; A first ratio and a second ratio are calculated respectively according to the standard deviation and a first preset constant; Determining a first parameter matching a first ratio from the first preset set; Determine a second parameter matching the second ratio from the second preset set; An input quantization parameter is calculated according to the first parameter, the second parameter, and the quantization bit width.

3. The method according to claim 2, characterized in that When the neural network structure at this layer receives input data, updating the first preset set and the second preset set based on the input data includes: When the neural network structure at this layer receives input data, a maximum parameter is determined from the first preset set and the input data, and the maximum parameter is added to the first preset set; A minimum parameter is determined from the second preset set and the input data, and the minimum parameter is added to the second preset set.

4. The method according to claim 3, characterized in that Determining a maximum parameter from a first preset set and input data, and adding the parameter to the first preset set, comprises: Determining a maximum parameter among the first preset set and the input data; If the maximum parameter is a parameter in the first preset set, the first preset set is not updated; If the maximum parameter is a parameter in the input data, the maximum parameter is added to the first preset set.

5. The method according to claim 3, characterized in that: Determining a minimum parameter from the second preset set and the input data, and adding the parameter to the second preset set, comprises: Determining a minimum parameter among the second preset set and the input data; If the minimum parameter is a parameter in the second preset set, the second preset set is not updated; If the minimum parameter is a parameter in the input data, the minimum parameter is added to the first preset set.

6. The method according to claim 2, characterized in that The first ratio and the second ratio are calculated according to the standard deviation and the first preset constant, respectively, including: According to the formula A first ratio is calculated; According to the formula The second ratio is calculated; Wherein, std1 represents the standard deviation of the X differences, c represents the first ratio, d represents the second ratio, and β1 is a first preset constant.

7. The method according to claim 1, characterized in that Calculate the output quantization parameters corresponding to the neural network structure of this layer, including: When the neural network structure of this layer generates output data, the third preset set and the fourth preset set are updated based on the output data; wherein the parameters in the third preset set are greater than the parameters in the fourth preset set; After all calibration data in the calibration data set are input, the difference between the largest Y parameters in the third preset set and the smallest Y parameters in the fourth preset set is calculated to obtain Y difference values; Calculate the standard deviation of the Y differences; A third ratio and a fourth ratio are calculated according to the standard deviation and the second preset constant respectively; Determining a third parameter matching a third ratio from the third preset set; Determining a fourth parameter matching a fourth ratio from the fourth preset set; An output quantization parameter is calculated according to the third parameter, the fourth parameter, and the quantization bit width.

8. The method according to claim 7, characterized in that When the neural network structure of this layer generates output data, updating the third preset set and the fourth preset set based on the output data includes: When the neural network structure of this layer generates output data, determining a maximum parameter from the third preset set and the output data, and adding the maximum parameter to the third preset set; A minimum parameter is determined from the fourth preset set and the output data, and the minimum parameter is added to the fourth preset set.

9. The method according to claim 8, characterized in that Determining a maximum parameter from the third preset set and the output data, and adding the parameter to the third preset set, comprises: Determining a maximum parameter among the third preset set and the output data; If the maximum parameter is a parameter in the third preset set, the third preset set is not updated; If the maximum parameter is a parameter in the output data, the maximum parameter is added to the third preset set.

10. The method according to claim 8, characterized in that Determining a minimum parameter from the fourth preset set and the output data, and adding the parameter to the fourth preset set, comprises: Determining a minimum parameter among the fourth preset set and the output data; If the minimum parameter is a parameter in the fourth preset set, the fourth preset set is not updated; If the minimum parameter is a parameter in the output data, the minimum parameter is added to the fourth preset set.

11. The method according to claim 1, characterized in that: Calculate the neural network quantization parameters corresponding to the neural network structure of this layer, including: Obtain a fifth preset set and a sixth preset set corresponding to the neural network structure of this layer; wherein the fifth preset set and the sixth preset set are obtained based on the parameter matrix corresponding to the neural network structure of this layer; the fifth preset set includes the largest Z parameters in the parameter matrix, and the sixth preset set includes the smallest Z parameters in the parameter matrix; Calculating differences between Z parameters in the fifth preset set and Z parameters in the sixth preset set to obtain Z differences; Calculate the standard deviation of the Z differences; A fifth ratio and a sixth ratio are calculated respectively according to the standard deviation and the third preset constant; Determining a fifth parameter matching a fifth ratio from the fifth preset set; Determining a sixth parameter matching a sixth ratio from the sixth preset set; The neural network quantization parameter is calculated according to the fifth parameter, the sixth parameter, and the quantization bit width.

12. A quantitative reasoning device for a model, characterized in that: include: An acquisition module, used to acquire a calibration data set; A processing module, used for inputting the calibration data set into the model to be quantized in sequence; after all the input data in the calibration data set are input into the model to be quantized, for each layer of neural network structure in the model to be quantized, respectively calculating the input quantization parameter, neural network quantization parameter, and output quantization parameter corresponding to the neural network structure of this layer; Based on the input quantization parameters, neural network quantization parameters, and output quantization parameters corresponding to each layer of the neural network structure, quantization reasoning is performed on the model to be quantized.

13. An electronic device, characterized in that: include: A memory and a processor, wherein the memory and the processor are connected; The memory is used to store programs; The processor is used to call the program stored in the memory to execute the method according to any one of claims 1-11.

14. A computer-readable storage medium, characterized in that: A computer program is stored thereon, and when the computer program is executed by a computer, the method according to any one of claims 1 to 11 is executed.

Images