Data processing method and device, electronic equipment and storage medium

Abstract

The invention discloses a data processing method and device, electronic equipment and a storage medium, and relates to the fields of image processing, text processing, voice processing and the like, and the method comprises the steps: obtaining a configuration parameter according to a quantized data type, an original weight, an original bias parameter and a sample input value; and performing quantization processing on the current input value according to the input value quantization parameter and the input value zero parameter, performing product operation on a quantization processing result according to the quantization weight, and performing bias operation on a product operation result according to the scaling parameter and the fusion bias parameter. According to the technical scheme, the input value zero point parameter and the original bias parameter form the fusion bias parameter, the number of parameters participating in operation in the quantization process is reduced, the data calculation step in the quantization process is simplified, and the data processing efficiency of a linear layer is greatly improved.

Description

Technical Field

[0001] The present invention relates to the technical fields of image processing, text processing and speech processing, and in particular to a data processing method, device, electronic device and storage medium. Background Art

[0002] With the continuous development of artificial intelligence large model technology, the number of model parameters and the amount of calculation have increased dramatically, among which the calculation of the linear layer has become a key factor restricting the reasoning speed.

[0003] As an effective means of data compression, quantization technology can reduce the amount of data calculation and the required memory space to a certain extent. Therefore, in the existing technology, the linear layer usually adopts quantization technology as a data calculation method, thereby reducing the complexity of data calculation by quantizing the input activation values ​​and original weights.

[0004] However, in the traditional quantization method, the intermediate parameters generated by the quantization process and the bias parameters involved in the calculation process still greatly restrict the data processing efficiency of the linear layer, and it still has the problem of low data calculation efficiency. Summary of the invention

[0005] The present invention provides a data processing method, device, electronic device and storage medium to solve the problem of low efficiency of linear layer quantization calculation.

[0006] According to one aspect of the present invention, there is provided a data processing method, comprising:

[0007] According to the quantization data type, the original weight, the original bias parameter and the sample input value, the configuration parameters are obtained; wherein the configuration parameters include an input value quantization parameter, an input value zero point parameter, a quantization weight, a scaling parameter and a fusion bias parameter;

[0008] quantizing the current input value according to the input value quantization parameter and the input value zero point parameter, performing a product operation on the quantization processing result according to the quantization weight, and performing a bias operation on the product operation result according to the scaling parameter and the fusion bias parameter;

[0009] The bias operation result is sent to a downstream functional module, so that the downstream functional module obtains a recognition result of the current input value according to the bias operation result.

[0010] The configuration parameters are obtained according to the quantization data type, the original weight, the original bias parameter and the sample input value, including: according to the quantization data type, asymmetric quantization of the sample input value to obtain the input value quantization parameter and the input value zero point parameter, and according to the quantization data type, symmetric quantization of the original weight to obtain the quantization weight.

[0011] The data precision of the quantized data type of the sample input value is greater than the data precision of the quantized data type of the original weight.

[0012] After quantizing the current input value according to the input value quantization parameter and the input value zero point parameter, multiplying the quantization result according to the quantization weight, and biasing the product result according to the scaling parameter and the fusion bias parameter, the method further includes: obtaining a first error value between the data calculation accuracy and a first accuracy threshold according to the data calculation accuracy of the current data calculation task; if it is determined that the first error value is greater than the first error threshold, updating the first target parameter, and updating the configuration parameter according to the updated first target parameter; wherein the first target parameter includes at least one of a quantization data type, an original weight, an original bias parameter, and a sample input value.

[0013] After quantizing the current input value according to the input value quantization parameter and the input value zero point parameter, multiplying the quantization result according to the quantization weight, and biasing the product result according to the scaling parameter and the fusion bias parameter, the method further includes: obtaining a second error value between the bias operation result of the current input data and the expected operation result of the current input data; if it is determined that the second error value is greater than a second error threshold, updating the second target parameter, and updating the configuration parameter according to the updated second target parameter; wherein the second target parameter includes at least one of a quantization data type and a sample input value.

[0014] The method of obtaining the second error value between the bias operation result of the current input data and the expected operation result of the current input data also includes: if it is determined that the maximum element value of the first preset number of input values ​​is greater than the maximum element value of the sample input value, or the minimum element value of the second preset number of input values ​​is less than the minimum element value of the sample input value, obtaining the second error value between the bias operation result of the current input data and the expected operation result of the current input data.

[0015] According to one aspect of the present invention, there is provided a data processing device, comprising:

[0016] A configuration parameter acquisition module, used to acquire configuration parameters according to the quantization data type, the original weight, the original bias parameter and the sample input value; wherein the configuration parameters include input value quantization parameters, input value zero point parameters, quantization weights, scaling parameters and fusion bias parameters;

[0017] a quantization operation execution module, configured to perform quantization processing on the current input value according to the input value quantization parameter and the input value zero point parameter, perform a product operation on the quantization processing result according to the quantization weight, and perform a bias operation on the product operation result according to the scaling parameter and the fusion bias parameter;

[0018] The operation result sending module is used to send the bias operation result to the downstream functional module, so that the downstream functional module obtains the recognition result of the current input value according to the bias operation result.

[0019] According to another aspect of the present invention, an electronic device is provided, comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores a computer program executable by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor can execute the data processing method described in any embodiment of the present invention.

[0020] According to another aspect of the present invention, a computer-readable storage medium is provided, wherein the computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a processor to implement the data processing method described in any embodiment of the present invention when executed.

[0021] According to another aspect of the present invention, a computer program product is provided, comprising a computer program, wherein when the computer program is executed by a processor, the computer program implements the data processing method according to any embodiment of the present invention.

[0022] The technical solution of the embodiment of the present invention, after obtaining the configuration parameters according to the quantization data type, the original weight, the original bias parameter and the sample input value, quantizes the current input value according to the input value quantization parameter and the input value zero point parameter, and performs a product operation on the quantization processing result according to the quantization weight, and performs a bias operation on the product operation result according to the scaling parameter and the fusion bias parameter, and finally sends the bias operation result to the downstream functional module, so that the downstream functional module obtains the recognition result of the current input value according to the bias operation result; thereby, by forming the input value zero point parameter and the original bias parameter into a fusion bias parameter, the number of parameters involved in the operation in the quantization process is reduced, the data calculation steps of the quantization process are simplified, and the data processing efficiency of the linear layer is greatly improved.

[0023] It should be understood that the contents described in this section are not intended to identify the key or important features of the embodiments of the present invention, nor are they intended to limit the scope of the present invention. Other features of the present invention will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0025] Figure 1 is a flow chart of a data processing method provided according to Embodiment 1 of the present invention;

[0026] Figure 2 is a schematic diagram of a linear layer quantization process provided according to Embodiment 1 of the present invention;

[0027] Figure 3 is a flow chart of a data processing method provided according to Embodiment 2 of the present invention;

[0028] Figure 4 is a flow chart of a data processing method provided according to Embodiment 3 of the present invention;

[0029] Figure 5 is a structural schematic diagram of a data processing device provided according to a fourth embodiment of the present invention;

[0030] Figure 6 It is a structural schematic diagram of an electronic device for implementing the data processing method of an embodiment of the present invention. DETAILED DESCRIPTION

[0031] In order to enable those skilled in the art to better understand the scheme of the present invention, the technical scheme in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present invention.

[0032] It should be noted that the terms "first", "second", etc. in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0033] Embodiment 1

[0034] Figure 1 This is a flowchart of a data processing method provided in the first embodiment of the present invention. The method is applicable to the linear layer or other functional layer of the data processing model that performs quantitative calculations. The method can be performed by the data processing device in any embodiment of the present invention. The data processing device can be implemented in the form of hardware and / or software, and the data processing device can be configured in an electronic device. Figure 1 As shown, the method includes:

[0035] S101. Acquire configuration parameters according to quantization data type, original weight, original bias parameter and sample input value; wherein the configuration parameters include input value quantization parameter, input value zero point parameter, quantization weight, scaling parameter and fusion bias parameter.

[0036] Take the linear layer in the data processing model as an example; the original weight refers to the weight of the linear layer before the quantization operation is performed. As an inherent parameter of the linear layer, it has been pre-configured when the data processing model is created; the original bias parameter refers to the bias parameter of the linear layer before the quantization operation is performed. As an inherent parameter of the linear layer, it has been pre-configured when the data processing model is created; the sample input value refers to the value of the training sample after the data processing operation is performed by each functional layer (that is, the upstream functional module) located before the current linear layer in the data processing model, and the output result is input into the linear layer as the activation value.

[0037] The quantization data type is the numerical type of the quantization result when the linear layer quantizes the current input value and the original weight. For example, the result after the input value is quantized can be configured as an unsigned integer data, that is, the "UINT" type, expressed as "UINT_A"; the result after the original weight is quantized can be configured as a signed integer data, that is, the "INT" type, expressed as "INT_A"; where "A" represents the quantization accuracy, that is, the number of bits of the quantization result, for example, "A" can be 8 bits or 4 bits; the quantization data type and quantization data accuracy can be pre-configured according to the quantization requirements.

[0038] The data processing model can be constructed based on deep learning (DL) methods and artificial neural network (ANN) technology, and is used to process input data such as image data, text data, and voice data. That is, the sample input value can be image data, text data, or voice data, etc., and the recognition result of the input data is obtained according to the data processing result; for example, the prediction result and classification result of the input data are obtained; optionally, in the embodiment of the present invention, the structure and function of the data processing model are not specifically limited.

[0039] The input value of the linear layer is expressed in the form of a matrix or vector. The first element with the largest value in each sample input value is obtained, and then the one with the largest value among all the first elements is taken as the maximum input value x max ; Similarly, get the second element with the smallest value in each sample input value, and then take the one with the smallest value among all the second elements as the minimum input value x min ; Then obtain the input value quantization parameter S according to the following equation x and the input value zero parameter Z p :

[0040]

[0041] Among them: the parameter θ is determined by the quantized data type A, which can be obtained by the following equation:

[0042] θ=2 A-1 -1(Formula 3);

[0043] For example, when the quantization data type A is configured as 8 bits, the corresponding parameter θ is 127.

[0044] The original weights W in the linear layer k×n In, W i W k×n The weight of the i-th channel in W. Since each column in the weight represents a data processing channel, W i In fact, it also represents the weight W k×n The weight of the i-th column in ; where i = 1, 2, 3...n. The weight quantization parameter S of each channel can be obtained by the following equation Wi , and the quantization weight W of each channel i INT_A :

[0045]

[0046] Among them, abs is the absolute value function; clip is the truncation function.

[0047] The quantization parameters S of each channel Wi According to the channel order combination, the weight quantization parameter S of the linear layer can be obtained W ; The quantization weight W of each channel i INT_A By combining the channels in order, we can get the quantized weight W of the linear layer. INT_A .

[0048] Then the parameter S can be quantized according to the input value obtained above. x and weight quantization parameter S W , the scaling factor is obtained by the following equation:

[0049] scaling = S x ×S W (Formula 6);

[0050] In addition, the compensation parameters are constructed by the following equation

[0051]

[0052] Among them, 1 m×k Represents a matrix with all elements set to 1, m rows, and k columns.

[0053] Number of compensation parameters The elements of any two rows in are the same, and the elements of any row are recorded as Will With the original bias parameter C 1×n After merging, the fused bias parameters can be obtained through the following equation, that is, the fused bias parameters

[0054]

[0055] In traditional quantization operations, both the original bias parameter and the input value zero-point parameter need to participate in the quantization operation. In an embodiment of the present invention, the original bias parameter and the input value zero-point parameter are fused to form a new bias parameter, i.e., a fused bias parameter. In the process of performing the quantization operation, the linear layer only needs to calculate the fused bias parameter, and no longer needs to process the input value zero-point parameter separately. Since the calculation amount of the fused bias parameter is the same as the calculation amount of the original bias parameter, it is equivalent to no longer needing to perform calculations related to the input value zero-point parameter, thereby reducing the calculation amount of the linear layer quantization process and improving the quantization efficiency.

[0056] Optionally, in an embodiment of the present invention, the configuration parameters are obtained according to the quantization data type, the original weights, the original bias parameters and the sample input values, including: performing asymmetric quantization on the sample input values ​​according to the quantization data type to obtain input value quantization parameters and input value zero point parameters, and performing symmetric quantization on the original weights according to the quantization data type to obtain quantization weights.

[0057] Specifically, if the input value quantization adopts the symmetric quantization method, unnecessary additional calculations need to be introduced to improve the calculation accuracy. The amount of computation of the above additional calculations is extremely large, which will greatly reduce the data processing efficiency of the linear layer. Therefore, in order to reduce the amount of computation in the input value quantization process and ensure the high accuracy of the input value quantization process, the input value needs to be asymmetric quantized to avoid the quantization error of the input value causing a large error in the output result.

[0058] As for the quantization of the original weights, if asymmetric quantization is used, the zero-point parameters of the input value cannot be hidden in the subsequent bias calculation process, and more time and computing resources must be spent on processing the related operations of the zero-point parameters of the input value. At the same time, in the quantization process of the input value and the original weight, the calculation error is mainly reflected in the quantization process of the input value, and the error of the original weight quantization process is smaller. Therefore, the original weights are symmetrically quantized to reduce the quantization calculation amount of the linear layer and improve the quantization calculation efficiency.

[0059] Optionally, in an embodiment of the present invention, the data precision of the quantization data type of the sample input value is greater than the data precision of the quantization data type of the original weight. Specifically, as described in the above technical solution, since in the quantization process of the input value and the original weight, the calculation error is mainly reflected in the quantization process of the input value, the data precision of the quantization data type of the sample input value can be configured to a larger value (for example, 8 bits) to improve the quantization precision of the input value quantization process; at the same time, since the error of the original weight quantization process is small, the data precision of the quantization data type of the original weight can be configured to a smaller value (for example, 4 bits) to improve the quantization efficiency of the original weight quantization process; in addition, after configuring the above-mentioned type of configuration parameters, the linear layer can also improve the generalization of the data processing model.

[0060] In particular, if the data precision of the quantization data type of the sample input value is different from the data precision of the quantization data type corresponding to the original weight, the two need to correspond to different A values ​​respectively. In the calculation process of Formula 1, the parameter θ needs to be determined by the data precision of the quantization data type of the sample input value; in the calculation process of Formula 5, the parameter θ needs to be determined by the data precision of the quantization data type of the original weight.

[0061] S102. quantize the current input value according to the input value quantization parameter and the input value zero point parameter, multiply the quantization result according to the quantization weight, and bias the product result according to the scaling parameter and the fusion bias parameter.

[0062] The above process is equivalent to completing the sample training of the linear layer, and thereby obtaining the configuration parameters of the training, so as to quantize the input value based on the configuration parameters of the training; in order to facilitate the execution of the quantization process, three types of calculation modules can be pre-generated in the linear layer, namely, the input value quantization module, the weight quantization module and the fusion bias module, the calculation logic of each calculation module is pre-generated, and the configuration parameters obtained above are configured to each calculation module to complete the quantization calculation of the input value through the above calculation module.

[0063] by Figure 2 As an example, the input value quantization parameter and the input value zero point parameter are configured to the input value quantization module, and the input value quantization module is used to quantize the current input value according to the input value quantization parameter and the input value zero point parameter; first, after the input value X enters the input value quantization module, the input value quantization module compares the current input value X with the input value quantization parameter S x Perform division operation and obtain the division result; then, compare the division result with the input value zero point parameter Z p Perform an addition operation and obtain the result of the addition operation; then round the result of the addition operation through the Round() function and obtain the rounded result; finally, truncate the rounded result through the Clip() function.

[0064] As described in the above technical solution, in an embodiment of the present invention, it is necessary to keep the quantization result of the current input value within a range greater than or equal to 0 and less than or equal to 2θ+1, that is, the Clip() function is configured as Clip(0, 2θ+1); in addition, if the sample input value and the current input value are both floating-point 32-bit data (i.e., FP32), and the quantization data type of the input value is UINT8, then the data types of the corresponding division operation results and addition operation results are both FP32, and the data type of the quantization processing result output by the input value quantization module is UINT8.

[0065] The quantization weight is configured to the weight quantization module, and the weight quantization module is used to perform a product operation on the quantization processing result output by the input value quantization module according to the quantization weight; first, the weight quantization module multiplies the output result of the input value quantization module by the quantization weight; since the output result of the input value quantization module and the quantization weight are multiplication operations between matrices, it is necessary to record each data row in the output result of the input value quantization module in a cumulative manner, and the product result between each data column of the quantization weight, respectively, as the output result of the weight quantization module; wherein, the data output by the weight quantization module are all signed 32-bit data, namely INT32.

[0066] The scaling parameters and fusion bias parameters are configured to the fusion bias module, and the fusion bias module is used to perform bias operation on the product operation result according to the scaling parameters and the fusion bias parameters; first, the fusion bias module multiplies the output result of the weight quantization module with the scaling parameters; then, the product operation result is added to the fusion bias parameter, and the addition result is the output result of the linear layer; wherein, the data type of the product operation result of the output result of the weight quantization module and the scaling parameter is FP32, and the data type of the product operation result and the addition result of the fusion bias parameter is also FP32.

[0067] S103: Send the bias operation result to the downstream functional module, so that the downstream functional module obtains the recognition result of the current input value according to the bias operation result.

[0068] The linear layer sends the final output quantization result (i.e., the bias operation result) to other functional layers (i.e., downstream functional modules) located after itself in the data processing model. The downstream functional modules continue to perform subsequent data processing according to the quantization result until the recognition result of the current input value is obtained, such as a text recognition result, an image recognition result, or a speech recognition result. Optionally, in an embodiment of the present invention, the type and number of functional layers before the current linear layer (i.e., upstream functional modules) are not specifically limited; the type and number of functional layers after the current linear layer (i.e., downstream functional modules) are also not specifically limited.

[0069] The technical solution of the embodiment of the present invention, after obtaining the configuration parameters according to the quantization data type, the original weight, the original bias parameter and the sample input value, quantizes the current input value according to the input value quantization parameter and the input value zero point parameter, and performs a product operation on the quantization processing result according to the quantization weight, and performs a bias operation on the product operation result according to the scaling parameter and the fusion bias parameter, and finally sends the bias operation result to the downstream functional module, so that the downstream functional module obtains the recognition result of the current input value according to the bias operation result; thereby, by forming the input value zero point parameter and the original bias parameter into a fusion bias parameter, the number of parameters involved in the operation in the quantization process is reduced, the data calculation steps of the quantization process are simplified, and the data processing efficiency of the linear layer is greatly improved.

[0070] Embodiment 2

[0071] Figure 3 This is a flow chart of a data processing method provided by Embodiment 2 of the present invention. The relationship between this embodiment and the above embodiment is that it determines whether to update the configuration parameters according to the data calculation accuracy of the current data calculation task. Figure 3 As shown, the method includes:

[0072] S201. Acquire configuration parameters according to the quantization data type, original weight, original bias parameter and sample input value; wherein the configuration parameters include input value quantization parameter, input value zero point parameter, quantization weight, scaling parameter and fusion bias parameter.

[0073] S202. quantize the current input value according to the input value quantization parameter and the input value zero point parameter, multiply the quantization result according to the quantization weight, and bias the product result according to the scaling parameter and the fusion bias parameter.

[0074] S203. According to the data calculation accuracy of the current data calculation task, obtain a first error value between the data calculation accuracy and a first accuracy threshold.

[0075] For each data calculation task, while the data processing model is processing data, the third-party system or the data processing model itself will synchronously monitor the calculation accuracy of the final calculation result of the current calculation task; wherein, the final calculation result refers to the result finally output by the data processing model after the input data to be identified is input into the data processing model and the data is processed by each functional module in the data processing model; for the linear layer, the calculation accuracy of the data processing model can be used as the basis for updating to obtain the error value between the data calculation accuracy and a pre-set first accuracy threshold, that is, the first error value.

[0076] S204. If it is determined that the first error value is greater than a first error threshold, update the first target parameter, and update the configuration parameter according to the updated first target parameter; wherein the first target parameter includes at least one of a quantization data type, an original weight, an original bias parameter, and a sample input value.

[0077] If the first error value is greater than the first error threshold, it means that the data calculation accuracy of the current data calculation task is low. At this time, the first target parameter can be updated, and then the configuration parameters can be retrained according to the updated first target parameter. Among them, the data calculation accuracy of the current data calculation task is low, which may be due to the error of the sample input value, which does not conform to the data characteristics of the actual input value. For example, the maximum element value in the sample input value is too small, or the minimum element value in the sample input value is too large, which will cause the error of the obtained configuration parameters to be large and cannot be applied to the current data calculation task, thereby generating calculation errors in the quantization process. At the same time, it may also be that the accuracy of the quantized data type is insufficient, resulting in calculation errors in the quantization process. In addition, if the original weights and original bias parameters of the linear layer are set incorrectly, it will also cause the first error value to be too large.

[0078] In particular, when errors exist in different types of first target parameters, the magnitude of the first error values ​​caused is also different; among them, the quantized data type makes the data precision different, which will only lead to a smaller calculation error, that is, a first error value with a smaller value. Therefore, if the first error value is in the first numerical interval with a smaller value, it means that the data precision of the quantized data type is too low. At this time, the data precision of the quantized data type can be improved, for example, from 4 bits to 8 bits. Incorrect sample input values ​​will lead to larger calculation errors, that is, a first error value with a larger value. Therefore, if the first error value is in the second numerical interval with a larger value, it means that there is a numerical error in the sample input value. At this time, the maximum input value x max Increase, and / or minimize input value x min The left endpoint of the second numerical interval is greater than the right endpoint of the first numerical interval.

[0079] In addition, incorrect original weights and original bias parameters will lead to huge calculation errors, that is, to the appearance of a first error value with a very large value. Therefore, if the first error value is in a third numerical interval with a very large value, it means that the original weights and original bias parameters are incorrectly set. At this time, according to the configuration mapping table, by querying the configuration mapping table, the matching original weights and original bias parameters can be obtained according to the first error value query; wherein the left endpoint of the third numerical interval is greater than the right endpoint of the second numerical interval; based on this, according to the obtained new first target parameters, the configuration parameters are retrained to obtain the configuration parameters, thereby realizing the update of the configuration parameters and ensuring the high calculation accuracy of the data processing model.

[0080] The technical solution of the embodiment of the present invention, after completing the quantization processing of the current input value, obtains the first error value between the data calculation accuracy and the first accuracy threshold according to the data calculation accuracy of the current data calculation task, if it is determined that the first error value is greater than the first error threshold, updates the first target parameter, and updates the configuration parameter according to the updated first target parameter; thereby, according to the data processing model for the data calculation accuracy of the data calculation task, the configuration parameter is updated to ensure a higher calculation accuracy of the data processing model.

[0081] Embodiment 3

[0082] Figure 4 This is a flowchart of a data processing method provided by Embodiment 3 of the present invention. The relationship between this embodiment and the above embodiments is that it determines whether to update the configuration parameters according to the bias operation result and the expected operation result of the current input data. Figure 4 As shown, the method includes:

[0083] S301. Acquire configuration parameters according to quantization data type, original weight, original bias parameter and sample input value; wherein the configuration parameters include input value quantization parameter, input value zero point parameter, quantization weight, scaling parameter and fusion bias parameter.

[0084] S302. quantize the current input value according to the input value quantization parameter and the input value zero point parameter, multiply the quantization result according to the quantization weight, and bias the product result according to the scaling parameter and the fusion bias parameter.

[0085] S303: Obtain a second error value between a bias operation result of current input data and an expected operation result of current input data.

[0086] Before quantization, the linear layer itself contains two parameters, namely, the original weight and the original bias parameter. The actual operation to be performed is to multiply the current input data by the original weight, and then sum the product result with the original bias parameter. The sum result is the result that actually needs to be output when the quantization operation is not performed. The sum result is used as the expected operation result of the current input data, and the second error value between the above-mentioned quantized bias operation result and the expected operation result is obtained; in particular, the operation of obtaining the expected operation result can be configured to be periodic or irregularly obtained, so as to ensure that the bias operation result of the linear layer is of high precision through sampling detection, thereby reducing the computational pressure of the linear layer.

[0087] S304. If it is determined that the second error value is greater than a second error threshold, update the second target parameter, and update the configuration parameter according to the updated second target parameter; wherein the second target parameter includes at least one of a quantization data type and a sample input value.

[0088] If the second error value is greater than the second error threshold, it means that the computational error of the quantization operation of the linear layer is large. Since the computational accuracy of the entire data calculation task is not affected at this time, it is not a large-scale calculation process distortion caused by errors in the original weights and / or original bias parameters. The reason may be that the accuracy of the quantization data type of the linear layer is low, or the sample input value cannot cover the current input value. Therefore, the quantization data type and / or sample input value needs to be updated.

[0089] Similarly, when errors exist in different types of second target parameters, the magnitude of the second error values ​​will also be different; among them, the quantized data type makes the data precision different, which will only lead to a smaller calculation error, that is, a smaller second error value. Therefore, if the second error value is in the fourth numerical interval with a smaller value, it means that the data precision of the quantized data is too low. At this time, the data precision of the quantized data type can be improved, for example, from 4 bits to 8 bits. Incorrect sample input values ​​will lead to larger calculation errors, that is, a larger second error value. Therefore, if the second error value is in the fifth numerical interval with a larger value, it means that there is a numerical error in the sample input value. At this time, the maximum input value x max To increase, the minimum input value x min The left endpoint of the fifth numerical interval is greater than the right endpoint of the fourth numerical interval.

[0090] After updating the second target parameter, the linear layer is continuously trained according to the updated second target parameter to obtain the retrained configuration parameter, that is, the updated configuration parameter, and then the subsequent input value is quantized according to the updated configuration parameter. Thus, the calculation result of the linear layer itself is used as the evaluation basis, and the accuracy of the quantization result of the linear layer is ensured by comparing the bias operation result of the current input data with the expected operation result.

[0091] Optionally, in an embodiment of the present invention, the method of obtaining a second error value between the bias operation result of the current input data and the expected operation result of the current input data further includes: if it is determined that the maximum element value in the first preset number of input values ​​is greater than the maximum element value of the sample input value, or the minimum element value in the second preset number of input values ​​is less than the minimum element value of the sample input value, obtaining the second error value between the bias operation result of the current input data and the expected operation result of the current input data.

[0092] Specifically, usually, when a calculation error occurs in the linear layer, it is because the error between the current input value and the sample input value is large, and the configuration parameters obtained based on the sample input value training cannot meet the quantization calculation of the current input value. Therefore, if there is no numerical error between the values ​​of the currently obtained input values ​​and the sample input value, or there is a numerical error between a small number of input values ​​and the sample input value, then there is actually no need to update the configuration parameters at this time. However, if the number of input values ​​with large numerical errors with the sample input value is large, that is, the maximum element value of the first preset number of input values ​​is greater than the maximum element value of the sample input value, or the minimum element value of the second preset number of input values ​​is less than the minimum element value of the sample input value, it indicates that the numerical error between a large number of input values ​​and the sample input value is large, which may lead to the generation of quantization calculation errors. At this time, the second error value between the bias operation result of the current input data and the expected operation result of the current input data is obtained, so that when there may be a quantization calculation error, the second error value between the bias operation result and the expected calculation result is obtained in a targeted manner, which not only ensures the timely detection of the quantization calculation error, but also realizes the targeted acquisition of the expected calculation result, thereby improving the quantization efficiency and quantization accuracy of the linear layer.

[0093] The technical solution of the embodiment of the present invention, after completing the quantization processing of the current input value, obtains the second error value between the bias operation result of the current input data and the expected operation result of the current input data; if it is determined that the second error value is greater than the second error threshold, updates the second target parameter, and updates the configuration parameter according to the updated second target parameter; thereby, the calculation result of the linear layer itself is used as the evaluation basis, and the accuracy of the quantization result of the linear layer is ensured by comparing the bias operation result of the current input data with the expected operation result.

[0094] Embodiment 4

[0095] Figure 5 : is a structural block diagram of a data processing device provided by Embodiment 4 of the present invention, and the data processing device specifically includes:

[0096] The configuration parameter acquisition module 401 is used to acquire configuration parameters according to the quantization data type, the original weight, the original bias parameter and the sample input value; wherein the configuration parameters include input value quantization parameters, input value zero point parameters, quantization weights, scaling parameters and fusion bias parameters;

[0097] A quantization operation execution module 402, configured to perform quantization processing on the current input value according to the input value quantization parameter and the input value zero point parameter, perform a product operation on the quantization processing result according to the quantization weight, and perform a bias operation on the product operation result according to the scaling parameter and the fusion bias parameter;

[0098] The operation result sending module 403 is used to send the bias operation result to the downstream functional module, so that the downstream functional module obtains the recognition result of the current input value according to the bias operation result.

[0099] The technical solution of the embodiment of the present invention, after obtaining the configuration parameters according to the quantization data type, the original weight, the original bias parameter and the sample input value, quantizes the current input value according to the input value quantization parameter and the input value zero point parameter, and performs a product operation on the quantization processing result according to the quantization weight, and performs a bias operation on the product operation result according to the scaling parameter and the fusion bias parameter, and finally sends the bias operation result to the downstream functional module, so that the downstream functional module obtains the recognition result of the current input value according to the bias operation result; thereby, by forming the input value zero point parameter and the original bias parameter into a fusion bias parameter, the number of parameters involved in the operation in the quantization process is reduced, the data calculation steps of the quantization process are simplified, and the data processing efficiency of the linear layer is greatly improved.

[0100] Optionally, a configuration parameter acquisition module 401 is specifically used to asymmetrically quantize sample input values ​​according to the quantization data type to obtain input value quantization parameters and input value zero point parameters, and to symmetrically quantize original weights according to the quantization data type to obtain quantization weights.

[0101] Optionally, the data precision of the quantized data type of the sample input value is greater than the data precision of the quantized data type of the original weight.

[0102] Optionally, the data processing device is also used to obtain a first error value between the data calculation accuracy and a first accuracy threshold based on the data calculation accuracy of the current data calculation task; if it is determined that the first error value is greater than the first error threshold, update the first target parameter, and update the configuration parameter based on the updated first target parameter; wherein the first target parameter includes at least one of a quantized data type, an original weight, an original bias parameter, and a sample input value.

[0103] Optionally, the data processing device is also used to obtain a second error value between a bias operation result of current input data and an expected operation result of the current input data; if it is determined that the second error value is greater than a second error threshold, update the second target parameter, and update the configuration parameter according to the updated second target parameter; wherein the second target parameter includes at least one of a quantized data type and a sample input value.

[0104] Optionally, the data processing device is also used to obtain a second error value between the bias operation result of the current input data and the expected operation result of the current input data if it is determined that the maximum element value among the first preset number of input values ​​is greater than the maximum element value of the sample input value, or the minimum element value among the second preset number of input values ​​is less than the minimum element value of the sample input value.

[0105] The data processing device provided by the present invention can execute the data processing method provided by any embodiment of the present invention, and has the corresponding functional modules and beneficial effects of the execution method. For technical details not described in detail in this embodiment, please refer to the data processing method provided by any embodiment of the present invention.

[0106] Embodiment 5

[0107] Figure 6 A schematic diagram of the structure of an electronic device 10 that can be used to implement an embodiment of the present invention is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices (such as helmets, glasses, watches, etc.) and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present invention described and / or required herein.

[0108] like Figure 6 As shown, the electronic device 10 includes at least one processor 11, and a memory connected to the at least one processor 11, such as a read-only memory (ROM) 12, a random access memory (RAM) 13, etc., wherein the memory stores a computer program that can be executed by at least one processor, and the processor 11 can perform various appropriate actions and processes according to the computer program stored in the read-only memory (ROM) 12 or the computer program loaded from the storage unit 18 to the random access memory (RAM) 13. In the RAM 13, various programs and data required for the operation of the electronic device 10 can also be stored. The processor 11, the ROM 12, and the RAM 13 are connected to each other through a bus 14. An input / output (I / O) interface 15 is also connected to the bus 14.

[0109] A number of components in the electronic device 10 are connected to the I / O interface 15, including: an input unit 16, such as a keyboard, a mouse, etc.; an output unit 17, such as various types of displays, speakers, etc.; a storage unit 18, such as a disk, an optical disk, etc.; and a communication unit 19, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 19 allows the electronic device 10 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks.

[0110] The processor 11 may be a variety of general and / or special processing components with processing and computing capabilities. Some examples of the processor 11 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various dedicated artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, a digital signal processor (DSP), and any appropriate processor, controller, microcontroller, etc. The processor 11 performs the various methods and processes described above, such as a data processing method.

[0111] In some embodiments, the data processing method may be implemented as a computer program, which is tangibly contained in a computer-readable storage medium, such as a storage unit. In some embodiments, part or all of the computer program may be loaded and / or installed on a heterogeneous hardware accelerator via a ROM and / or a communication unit. When the computer program is loaded into RAM and executed by a processor, one or more steps of the data processing method described above may be performed. Alternatively, in other embodiments, the processor may be configured to perform the data processing method in any other appropriate manner (e.g., by means of firmware).

[0112] Various implementations of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), load programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various implementations can include: being implemented in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0113] Computer programs for implementing the methods of the present invention may be written in any combination of one or more programming languages. These computer programs may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, so that when the computer program is executed by the processor, the functions / operations specified in the flow chart and / or block diagram are implemented. The computer program may be executed entirely on the machine, partially on the machine, partially on the machine and partially on a remote machine as a stand-alone software package, or entirely on a remote machine or server.

[0114] In the context of the present invention, a computer-readable storage medium may be a tangible medium that may contain or store a computer program for use by or in combination with an instruction execution system, device or equipment. A computer-readable storage medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices or equipment, or any suitable combination of the foregoing. Alternatively, a computer-readable storage medium may be a machine-readable signal medium. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0115] To provide interaction with a user, the systems and techniques described herein may be implemented on a heterogeneous hardware accelerator having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to a user; and a keyboard and pointing device (e.g., a mouse or trackball) through which a user can provide input to the heterogeneous hardware accelerator. Other types of devices may also be used to provide interaction with a user; for example, the feedback provided to the user may be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user may be received in any form (including acoustic input, voice input, or tactile input).

[0116] The systems and techniques described herein may be implemented in a computing system that includes backend components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes frontend components (e.g., a user computer with a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such backend components, middleware components, or frontend components. The components of the system may be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), a blockchain network, and the Internet.

[0117] A computing system may include a client and a server. The client and the server are generally remote from each other and usually interact through a communication network. The client and server relationship is generated by computer programs running on the corresponding computers and having a client-server relationship with each other. The server may be a cloud server, also known as a cloud computing server or cloud host, which is a host product in the cloud computing service system to solve the defects of difficult management and weak business scalability in traditional physical hosts and VPS services.

[0118] It should be understood that the various forms of processes shown above can be used to reorder, add or delete steps. For example, the steps described in the present invention can be executed in parallel, sequentially or in different orders, as long as the desired results of the technical solution of the present invention can be achieved, and this document does not limit this.

[0119] The above specific implementations do not constitute a limitation on the protection scope of the present invention. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions can be made according to design requirements and other factors. Any modification, equivalent substitution and improvement made within the spirit and principle of the present invention should be included in the protection scope of the present invention.

Claims

1. A data processing method, characterized in that: include: According to the quantization data type, the original weight, the original bias parameter and the sample input value, the configuration parameters are obtained; wherein the configuration parameters include an input value quantization parameter, an input value zero point parameter, a quantization weight, a scaling parameter and a fusion bias parameter; quantizing the current input value according to the input value quantization parameter and the input value zero point parameter, performing a product operation on the quantization processing result according to the quantization weight, and performing a bias operation on the product operation result according to the scaling parameter and the fusion bias parameter; The bias operation result is sent to a downstream functional module, so that the downstream functional module obtains a recognition result of the current input value according to the bias operation result.

2. The method according to claim 1, characterized in that The obtaining of configuration parameters according to the quantized data type, the original weight, the original bias parameter and the sample input value includes: According to the quantization data type, the sample input value is asymmetrically quantized to obtain an input value quantization parameter and an input value zero point parameter, and according to the quantization data type, the original weight is symmetrically quantized to obtain a quantization weight.

3. The method according to claim 1, characterized in that The data precision of the quantized data type of the sample input value is greater than the data precision of the quantized data type of the original weight.

4. The method according to claim 1, characterized in that After quantizing the current input value according to the input value quantization parameter and the input value zero point parameter, performing a product operation on the quantization processing result according to the quantization weight, and performing a bias operation on the product operation result according to the scaling parameter and the fusion bias parameter, the method further includes: According to the data calculation accuracy of the current data calculation task, a first error value between the data calculation accuracy and a first accuracy threshold is obtained; If it is determined that the first error value is greater than a first error threshold, the first target parameter is updated, and the configuration parameter is updated according to the updated first target parameter; wherein the first target parameter includes at least one of a quantized data type, an original weight, an original bias parameter, and a sample input value.

5. The method according to claim 1, characterized in that After quantizing the current input value according to the input value quantization parameter and the input value zero point parameter, performing a product operation on the quantization processing result according to the quantization weight, and performing a bias operation on the product operation result according to the scaling parameter and the fusion bias parameter, the method further includes: Obtaining a second error value between a bias operation result of current input data and an expected operation result of current input data; If it is determined that the second error value is greater than a second error threshold, the second target parameter is updated, and the configuration parameter is updated according to the updated second target parameter; wherein the second target parameter includes at least one of a quantized data type and a sample input value.

6. The method according to claim 5, characterized in that The step of obtaining a second error value between the bias operation result of the current input data and the expected operation result of the current input data further includes: If it is determined that the maximum element value among the first preset number of input values ​​is greater than the maximum element value of the sample input value, or the minimum element value among the second preset number of input values ​​is less than the minimum element value of the sample input value, a second error value between the bias operation result of the current input data and the expected operation result of the current input data is obtained.

7. A data processing device, characterized in that: include: A configuration parameter acquisition module, used to acquire configuration parameters according to the quantization data type, the original weight, the original bias parameter and the sample input value; wherein the configuration parameters include input value quantization parameters, input value zero point parameters, quantization weights, scaling parameters and fusion bias parameters; a quantization operation execution module, configured to perform quantization processing on the current input value according to the input value quantization parameter and the input value zero point parameter, perform a product operation on the quantization processing result according to the quantization weight, and perform a bias operation on the product operation result according to the scaling parameter and the fusion bias parameter; The operation result sending module is used to send the bias operation result to the downstream functional module, so that the downstream functional module obtains the recognition result of the current input value according to the bias operation result.

8. An electronic device, characterized in that: The electronic device comprises: at least one processor; and a memory communicatively connected to the at least one processor; wherein, The memory stores a computer program executable by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor can perform the data processing method according to any one of claims 1 to 6.

9. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a processor to implement the data processing method according to any one of claims 1 to 6 when executed.

10. A computer program product, comprising a computer program, wherein when the computer program is executed by a processor, the computer program implements the data processing method according to any one of claims 1 to 6.

Images