Data processing method, electronic device, medium and program product

Abstract

The invention relates to the field of data processing, and discloses a data processing method, electronic equipment, a medium and a program product. The method comprises the following steps: acquiring a plurality of weight parameters of a first model, wherein the plurality of weight parameters are quantized based on 16-bit floating-point numbers; performing inverse quantization on the plurality of weight parameters based on the first quantization coefficient and the second quantization coefficient to obtain target weight parameters of a plurality of 8-bit floating-point numbers, the first quantization coefficient and the second quantization coefficient being related to a third quantization coefficient used for inversely quantizing the plurality of weight parameters into 16-bit floating-point numbers; determining a second activation value based on the first activation value of the 8-bit floating-point number input into the ith calculation layer and the target weight parameter of the ith calculation layer; and calibrating the second activation value based on the fourth quantization coefficient to obtain a target activation value. The first quantization coefficient and the second quantization coefficient are related to the third quantization coefficient, and quantization precision is not affected. The weight parameter and the activation value are 8-bit floating-point numbers, so that the power consumption can be reduced, and the operation efficiency of the model is improved.

Description

Technical Field

[0001] This application relates to the field of data processing technology, and in particular to a data processing method, electronic device, medium, and program product. Background Technology

[0002] With the development of artificial intelligence (AI) technology, large models are being deployed more and more widely on the edge. In order to reduce the storage space and bandwidth requirements of the model, the weight parameters of the model are usually quantized. For example, the weight parameters of the model are quantized into 4-bit integers (int4) through W4A16 type models (such as gptq model, awq model), while the activation values ​​of the model are kept as 16-bit floating-point numbers (fp16).

[0003] When running the model, the weight parameters are dequantized into 16-bit floating-point numbers, and matrix multiplication of the 16-bit floating-point weight parameters and the 16-bit floating-point activation values ​​is performed.

[0004] However, the high power consumption of 16-bit floating-point multiplication (i.e., fp16*fp16) may affect the efficiency of model operation. Summary of the Invention

[0005] The purpose of this application is to provide a data processing method, a readable medium, and an electronic device.

[0006] The first aspect of this application provides a data processing method applied to an electronic device. The method includes: acquiring multiple weight parameters of a first model, wherein the multiple weight parameters of the first model are determined based on 16-bit floating-point quantization, the first model includes N computational layers, where N is an integer greater than 2; performing dequantization processing on the multiple weight parameters based on a first quantization coefficient and a second quantization coefficient to obtain multiple target weight parameters, wherein the target weight parameters are 8-bit floating-point numbers, the first quantization coefficient and the second quantization coefficient are related to a third quantization coefficient, the first quantization coefficient is used to adjust the multiple weight parameters to 8-bit floating-point, the second quantization coefficient is used to calibrate the target weight parameters, and the third quantization coefficient is used to adjust the multiple weight parameters of the first model to 8-bit floating-point, the first quantization coefficient is used to calibrate the target weight parameters, and the third quantization coefficient is used to adjust the multiple weight parameters of the first model to 8-bit floating-point, the second ... The weight parameters are dequantized to 16-bit floating-point numbers; the first activation value of the i-th computational layer of the first model is obtained, which is an 8-bit floating-point number, where the first activation value is related to the input data of the first model, and i is an integer greater than 0 and less than or equal to N; based on the first activation value of the i-th computational layer and at least one target weight parameter of the i-th computational layer, the second activation value of the output of the i-th computational layer is determined; a fourth quantization coefficient is obtained, and the second activation value is calibrated based on the fourth quantization coefficient to obtain the target activation value, where the target activation value is an 8-bit floating-point number, and the fourth quantization coefficient is used to adjust the second activation value to 8-bit floating-point; the target activation value is used as the first activation value of the (i+1)-th computational layer of the first model.

[0007] In this embodiment, the first and second quantization coefficients are related to the third quantization coefficient used to dequantize multiple weight parameters into 16-bit floating-point numbers, which does not affect the quantization accuracy. Furthermore, since both the weight parameters and activation values ​​are 8-bit floating-point numbers, they can be calculated using 8-bit floating-point multiplication (i.e., fp8*fp8), thus reducing power consumption and improving model operating efficiency.

[0008] In one possible implementation of the first aspect above, the absolute value of the difference between the first product and the second product is less than an absolute value threshold, the first product is the product of the third quantization coefficient and multiple weight parameters, and the second product is the product of the first quantization coefficient, the second quantization coefficient and multiple weight parameters.

[0009] In this embodiment, the set of quantization coefficients with the smallest absolute value in formula (1) among multiple sets of quantization coefficients can be used as the first quantization coefficient and the second quantization coefficient. This improves the accuracy of the first and second quantization coefficients, thereby enhancing the precision of the target weight parameter quantized based on the first and second quantization coefficients.

[0010] In this embodiment of the application, the first set of quantization coefficients among multiple sets of quantization coefficients satisfying formula (1) can also be used as the first quantization coefficient and the second quantization coefficient. In this way, the time for determining the first quantization coefficient and the second quantization coefficient can be reduced, and the computing resources for determining the first quantization coefficient and the second quantization coefficient can be saved.

[0011] In one possible implementation of the first aspect above, the third quantization coefficient is a P×Q matrix, where P is an integer greater than 1 and Q is an integer greater than 1. The first and second quantization coefficients are determined as follows: The maximum and minimum values ​​among the Q elements in the j-th row of the P rows of the third quantization coefficient are determined, where j is an integer greater than 0 and less than or equal to P; L candidate second quantization coefficients are determined based on the relationship between the reciprocal of the minimum value and the maximum representation range of an 8-bit floating-point number, where L is an integer greater than 1; L quotients are determined between the third quantization coefficient and the L candidate second quantization coefficients, and the L quotients are converted to 8-bit floating-point numbers, with the converted L 8-bit floating-point numbers serving as the L candidate first quantization coefficients; L products of the L candidate first quantization coefficients, the L candidate second quantization coefficients, and multiple weight parameters are determined, respectively; and the first and second quantization coefficients are determined based on the similarity between the L products and the first product.

[0012] In one possible implementation of the first aspect described above, determining the first quantization coefficient and the second quantization coefficient based on the similarity between the L products and the first product includes: determining the L cosine similarities between the L products and the first product respectively; performing a Fourier transform on the first product to obtain a first feature, and performing a Fourier transform on the L products to obtain L second features; determining the mean square errors of the L second features and the first feature respectively; corresponding to the product of the kth mean square error among the L mean square errors being less than the minimum mean square error and the kth cosine similarity among the L cosine similarities, modifying the minimum mean square error to the kth mean square error, where k is an integer greater than 0 and less than or equal to L; using the candidate first quantization coefficient corresponding to the minimum mean square error as the first quantization coefficient, and using the candidate second quantization coefficient corresponding to the minimum mean square error as the second quantization coefficient.

[0013] In the embodiments of this application, L cosine similarities can be used to represent the time-domain similarity between L products and the first product, and L mean square errors can be used to represent the frequency-domain similarity between L products and the first product. By combining time-domain similarity and frequency-domain similarity to determine the first quantization coefficient and the second quantization coefficient, the closer the second product is to the first product, the more accurate the data processing can be improved.

[0014] In one possible implementation of the first aspect above, the first maximum value is greater than the second maximum value, and the first minimum value is greater than the second minimum value. Specifically, the first maximum value is the maximum value among L candidate second quantization coefficients when the reciprocal of the minimum value is less than the maximum representation range of an 8-bit floating-point number; the first minimum value is the minimum value among L candidate second quantization coefficients when the reciprocal of the minimum value is less than the maximum representation range of an 8-bit floating-point number; the second maximum value is the maximum value among L candidate second quantization coefficients when the reciprocal of the minimum value is greater than or equal to the maximum representation range of an 8-bit floating-point number; and the second minimum value is the minimum value among L candidate second quantization coefficients when the reciprocal of the minimum value is greater than or equal to the maximum representation range of an 8-bit floating-point number.

[0015] In the embodiments of this application, the difference between any two adjacent candidate second quantization coefficients among the L candidate second quantization coefficients can be equal, and the L candidate second quantization coefficients can also be randomly generated.

[0016] In one possible implementation of the first aspect above, when i is 1, the first activation value is an 8-bit floating-point number obtained after processing the input data of the first model; when i is not 1, the first activation value is the target activation value output by the (i-1)th computational layer of the first model.

[0017] In one possible implementation of the first aspect above, the fourth quantization coefficient is determined by the following method: obtaining the set of floating-point numbers output by each of the N computational layers of the first model, performing noise reduction processing on each set of floating-point numbers to obtain a set of denoised floating-point numbers; determining the fourth quantization coefficient based on the quantization interval of the set of denoised floating-point numbers, wherein the quantization interval of the set of denoised floating-point numbers is determined based on the maximum and minimum floating-point numbers in the set of denoised floating-point numbers.

[0018] In one possible implementation of the first aspect above, the noise reduction process includes at least one of the following processing methods: singular value decomposition processing method, discrete cosine transform processing method, and wavelet decomposition processing method.

[0019] In one possible implementation of the first aspect described above, the data type of the input data of the first model includes at least one of the following: text, image, and speech.

[0020] In one possible implementation of the first aspect above, the method further includes: using the data output by the Nth computational layer of the first model as the processing result of the input data of the first model, wherein the first model is used to implement at least one of the following functions: classification, generation, recognition, object detection, and semantic segmentation.

[0021] A second aspect of this application provides an electronic device including a processor and a memory, the memory being used to store program instructions, and the processor being used to invoke the program instructions to execute any of the methods described in the first aspect above.

[0022] A third aspect of this application provides a computer-readable storage medium storing instructions that, when executed, implement any of the methods described in the first aspect above.

[0023] The fourth aspect of this application provides a computer program product including instructions that, when executed, implement any of the methods described in the first aspect above. Attached Figure Description

[0024] To more clearly illustrate the specific embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the specific embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0025] Figure 1 A schematic diagram illustrating a calculation process for performing multiplication and addition operations using a multiply-adder is shown according to an embodiment of this application.

[0026] Figure 2 An embodiment of this application illustrates a schematic flowchart of a data processing method.

[0027] Figure 3 A logic block diagram of a data processing procedure is shown according to an embodiment of this application;

[0028] Figure 4 A schematic diagram of the structure of an electronic device 100 is shown according to some embodiments of this application;

[0029] Figure 5 A schematic diagram of a processor structure is shown according to an embodiment of this application. Detailed Implementation

[0030] The illustrative embodiments of this application include, but are not limited to, a data processing method, an electronic device, a medium, and a program product.

[0031] Multiply-accumulate (MAC) operation: MAC operation, also known as multiply-accumulate, is a technique used in digital signal processors (DSPs) and microprocessors. The computational methods used in processors such as programmed control units (MCUs).

[0032] The MAC operation involves multiplying the multiplier and multiplicand to obtain the product, and then adding the product to the addend to get the result of the MAC operation. In other words, the MAC operation is a compound operation consisting of a multiplication operation and an addition operation. The function of the MAC operation is to first perform a multiplication operation, then add the result of the multiplication operation to another number to obtain the result of the addition operation.

[0033] To improve the speed of multiplication operations, the multiplier is typically encoded first (e.g., using the Booth algorithm). The encoded result (multiple encoded signals) is then multiplied by the multiplicand to obtain multiple partial products. These partial products are then accumulated to obtain the product of the multiplier and multiplicand. The accumulation of partial products is essentially a shift-and-add operation.

[0034] For example, Figure 1 A schematic diagram of the calculation process of implementing MAC operation through a multiplier accumulator (MAC) is shown.

[0035] like Figure 1 As shown, the multiplier-adder includes an encoding module, a partial product generation and accumulation module, and an addition module. The encoding module encodes the multiplier to obtain the encoded result. The partial product generation and accumulation module obtains the product of the multiplier and multiplicand based on the encoded result and the multiplicand. The addition module adds the product and the addend to obtain the result of the MAC operation.

[0036] Since the product of each matrix element is accumulated during matrix multiplication to form an element of the resulting matrix, matrix multiplication can also be viewed as a combination of MAC operations, and can be implemented using a multiply-accumulator.

[0037] As mentioned earlier, to reduce the model's storage and bandwidth requirements, W4A16-type models (such as GPTQ and AWQ models) are typically used to quantize the model's weight parameters (or weight parameters in the weight matrix) into 4-bit integers (int4), while maintaining the model's activation values ​​as 16-bit floating-point numbers (fp16). When running the model, the weight parameters are dequantized to 16-bit floating-point numbers, and a matrix multiplication is performed between the 16-bit floating-point weight parameters and the 16-bit floating-point activation values.

[0038] However, the high power consumption of 16-bit floating-point multiplication (i.e., fp16*fp16) may affect the efficiency of model operation.

[0039] However, directly using fixed-point multiplication (such as int4*int4) may affect the accuracy of the model, and thus the accuracy of the model's data processing.

[0040] Therefore, to solve the above problems, this application provides a data processing method, which includes: obtaining multiple weight parameters of a first model, wherein the multiple weight parameters of the first model are determined based on 16-bit floating-point quantization, the first model includes N computational layers, where N is an integer greater than 2; performing dequantization processing on the multiple weight parameters based on a first quantization coefficient and a second quantization coefficient to obtain multiple target weight parameters, wherein the target weight parameters are 8-bit floating-point (fp8) numbers, the first quantization coefficient and the second quantization coefficient are related to a third quantization coefficient, the first quantization coefficient is used to adjust the multiple weight parameters to 8-bit floating-point, the second quantization coefficient is used to calibrate the target weight parameters, and the third quantization coefficient is used to adjust the multiple weight parameters of the first model ... second quantization coefficient is used to calibrate the target weight parameters, and the third quantization coefficient is used to adjust the multiple weight parameters of the first model to 8-bit floating-point, the second quantization coefficient is used to calibrate the target weight parameters, and the third quantization coefficient is used to adjust the multiple weight parameters of the first model to 8-bit floating-point, the second quantization coefficient is used to calibrate the target weight parameters, and the third quantization coefficient is used to adjust the multiple weight parameters of the first model to 8-bit The weight parameters are dequantized into 16-bit floating-point numbers; the first activation value of the i-th computational layer of the first model is obtained, and the first activation value is an 8-bit floating-point number, wherein the first activation value is related to the input data of the first model, and i is an integer greater than 0 and less than or equal to N; based on the first activation value of the i-th computational layer and at least one target weight parameter of the i-th computational layer, the second activation value of the output of the i-th computational layer is determined; the fourth quantization coefficient is obtained, and the second activation value is calibrated based on the fourth quantization coefficient to obtain the target activation value, wherein the target activation value is an 8-bit floating-point (fp8) number, and the fourth quantization coefficient is used to adjust the second activation value to 8-bit floating-point; the target activation value is used as the first activation value of the (i+1)-th computational layer of the first model.

[0041] It is understandable that the first and second quantization coefficients are related to the third quantization coefficient used to dequantize multiple weight parameters into 16-bit floating-point numbers, so they do not affect the quantization accuracy. Furthermore, since both the weight parameters and activation values ​​are 8-bit floating-point numbers, they can be calculated using 8-bit floating-point multiplication (i.e., fp8*fp8), which reduces power consumption and improves model running efficiency.

[0042] To more clearly illustrate the technical solutions provided in the embodiments of this application, the technical solutions of this application will be described below in conjunction with the accompanying drawings.

[0043] Figure 2 An implementation flowchart of a data processing method is shown according to an embodiment of this application. It can be understood that... Figure 2 The electronic device 100 is the primary entity executing each step of the process shown. For simplicity, the following description... Figure 2 The execution entity for each step in the illustrated process will not be described again. For example... Figure 2 As shown, the process includes, but is not limited to, the following steps:

[0044] S201: Obtain multiple weight parameters of the first model, wherein the multiple weight parameters of the first model are determined based on 16-bit floating-point quantization, and the first model includes N computational layers, where N is an integer greater than 2.

[0045] In some embodiments, multiple weight parameters can constitute a weight matrix. The data type of the multiple weight parameters can be a 4-digit integer (int4), or it can be a 3-digit integer, 2-digit integer, 1-digit integer, or other data type with fewer bits than a 4-digit integer. This application does not impose any restrictions on this. For ease of description, this application will use the example of multiple weight parameters being of 4-digit integer data type for illustration.

[0046] In some embodiments, the data type of the input data for the first model includes at least one of the following: text, image, and speech. The first model is used to implement at least one of the following functions: classification, generation, recognition, object detection, and semantic segmentation.

[0047] It is understood that the first model includes, but is not limited to, convolutional neural networks (CNN), feedforward neural networks (FNN), recurrent neural networks (RNN), and other neural network models and their variants.

[0048] S202: Based on the first quantization coefficient and the second quantization coefficient, multiple weight parameters are dequantized to obtain multiple target weight parameters. The target weight parameters are 8-bit floating-point numbers. The first quantization coefficient, the second quantization coefficient and the third quantization coefficient are related. The first quantization coefficient is used to adjust the multiple weight parameters to 8-bit floating-point numbers. The second quantization coefficient is used to calibrate the target weight parameters. The third quantization coefficient is used to dequantize the multiple weight parameters of the first model to 16-bit floating-point numbers.

[0049] In some embodiments, the electronic device 100 acquires the zero point (zerop) and the bias of the i-th computational layer. Based on the zero point, the bias of the i-th computational layer, the first quantization coefficient and the second quantization coefficient, and at least one target weight parameter of the i-th computational layer, it determines multiple target weight parameters of the i-th computational layer.

[0050] In some embodiments, the absolute value of the difference between the first product and the second product is less than an absolute value threshold. The first product is the product of the third quantization coefficient and multiple weight parameters, and the second product is the product of the first quantization coefficient, the second quantization coefficient, and multiple weight parameters. For example, refer to the relationship between the first product and the second product shown in formula (1).

[0051] Formula (1)

[0052] in, This is a function used to calculate the absolute value of floating-point numbers. This is a weight matrix consisting of multiple weight parameters, where each weight parameter is a 4-digit integer. This is the third quantization coefficient. The first quantization coefficient, This is the second quantization coefficient. This is the absolute value threshold.

[0053] It is understood that the absolute value threshold is greater than 0, and the value of the absolute value threshold can be configured and adjusted by the developers. This application does not limit the specific value of the absolute value threshold.

[0054] In some embodiments, the set of quantization coefficients with the smallest absolute value in formula (1) among multiple sets of quantization coefficients can be used as the first quantization coefficient and the second quantization coefficient. In this way, the accuracy of the first quantization coefficient and the second quantization coefficient can be improved, thereby improving the accuracy of the target weight parameter quantized based on the first quantization coefficient and the second quantization coefficient.

[0055] In other embodiments, the first set of quantization coefficients among multiple sets of quantization coefficients satisfying formula (1) above can be used as the first quantization coefficient and the second quantization coefficient. In this way, the time for determining the first quantization coefficient and the second quantization coefficient can be reduced, saving the computing resources required to determine the first quantization coefficient and the second quantization coefficient.

[0056] The following section uses the example of determining the first and second quantization coefficients by taking the set of quantization coefficients with the smallest absolute value in the above formula (1) as the first and second quantization coefficients.

[0057] In some embodiments, the third quantization coefficient is a P×Q matrix, where P is an integer greater than 1 and Q is an integer greater than 1. The first and second quantization coefficients are determined as follows: The maximum and minimum values ​​of the Q elements in the j-th row of the P rows of the third quantization coefficient are determined, where j is an integer greater than 0 and less than or equal to P; L candidate second quantization coefficients are determined based on the relationship between the reciprocal of the minimum value and the maximum representation range of an 8-bit floating-point number, where L is an integer greater than 1; L quotients are determined between the third quantization coefficient and the L candidate second quantization coefficients, and the L quotients are converted to 8-bit floating-point numbers, which are then used as L candidate first quantization coefficients; L products of the L candidate first quantization coefficients, the L candidate second quantization coefficients, and multiple weight parameters are determined, respectively; and the first and second quantization coefficients are determined based on the similarity between the L products and the first product.

[0058] For example, in some embodiments, the electronic device 100 determines L products and L cosine similarities of the first product; performs a Fourier transform on the first product to obtain a first feature, and performs a Fourier transform on the L products to obtain L second features; determines the mean square error of the L second features and the first feature; corresponding to the product of the kth mean square error among the L mean square errors being less than the minimum mean square error and the kth cosine similarity among the L cosine similarities, modifies the minimum mean square error to the kth mean square error, where k is an integer greater than 0 and less than or equal to L; and uses the candidate first quantization coefficient corresponding to the minimum mean square error as the first quantization coefficient, and uses the candidate second quantization coefficient corresponding to the minimum mean square error as the second quantization coefficient.

[0059] It is understandable that L cosine similarities can be used to represent the time-domain similarity between L products and the first product, and L mean square errors can be used to represent the frequency-domain similarity between L products and the first product. By combining time-domain similarity and frequency-domain similarity to determine the first and second quantization coefficients, the closer the second product is to the first product, the more accurate the data processing can be.

[0060] In some embodiments, the first maximum value is greater than the second maximum value, and the first minimum value is greater than the second minimum value. Specifically, the first maximum value is the maximum value among L candidate second quantization coefficients when the reciprocal of the minimum value is less than the maximum representation range of an 8-bit floating-point number; the first minimum value is the minimum value among L candidate second quantization coefficients when the reciprocal of the minimum value is less than the maximum representation range of an 8-bit floating-point number; the second maximum value is the maximum value among L candidate second quantization coefficients when the reciprocal of the minimum value is greater than or equal to the maximum representation range of an 8-bit floating-point number; and the second minimum value is the minimum value among L candidate second quantization coefficients when the reciprocal of the minimum value is greater than or equal to the maximum representation range of an 8-bit floating-point number.

[0061] In some embodiments, the difference between any two adjacent candidate second quantization coefficients among the L candidate second quantization coefficients is equal. In other embodiments, the L candidate second quantization coefficients may also be randomly generated, and this application does not limit this.

[0062] The following example, using the case where the difference between any two adjacent candidate second quantization coefficients among L candidate second quantization coefficients is equal and the maximum representation range of an 8-bit floating-point number is 448, combines formulas and code to introduce the process of determining the first and second quantization coefficients.

[0063] For example, if the reciprocal of the minimum value (1 / scale_min) and the maximum representation range of an 8-bit floating-point number (448) are such that 1 / scale_min is greater than 448, then the first coefficient (alpha) is determined to be 0.75 and the second coefficient (beta) to be 1.625. If the reciprocal of the minimum value (1 / scale_min) and the maximum representation range of an 8-bit floating-point number (448) are such that 1 / scale_min is less than or equal to 448, then the first coefficient (alpha) is determined to be 0.5 and the second coefficient (beta) to be 0.98. The product of the first coefficient and the maximum value (scale_max) is taken as the smallest second quantization coefficient among the L candidate second quantization coefficients (i.e., the first minimum or the second minimum), and the product of the second coefficient and the maximum value (scale_max) is taken as the largest second quantization coefficient among the L candidate second quantization coefficients (i.e., the first maximum or the second maximum).

[0064] For example, the first maximum value can be 1.625*scale_max, the first minimum value can be 0.75*scale_max, the second maximum value can be 0.98*scale_max, and the first minimum value can be 0.5*scale_max.

[0065] For example, the following code determines L candidate second quantization coefficients:

[0066] for x in (alpha, beta, stepsize=10)

[0067] scale1 = x * scale_max

[0068] Here, `for` represents a loop, and `stepsize=10` indicates a step size of (beta-alpha) / 10. That is, the L candidate quantization coefficients are: alpha*scale_max, alpha*(scale_max+(beta-alpha) / 10), alpha*(scale_max+2(beta-alpha) / 10), ..., beta*scale_max.

[0069] Determine the L quotients of the third quantization coefficient and the L candidate second quantization coefficients, and convert the L quotients into 8-bit floating-point numbers. Use the L converted 8-bit floating-point numbers as the L candidate first quantization coefficients. For example, refer to formula (2).

[0070] Formula (2)

[0071] in, The quotient of the third quantization coefficient and the candidate second quantization coefficient. This is the third quantization coefficient. As a candidate second quantization coefficient, The first candidate quantization coefficient, This indicates that the quotient of the third quantization coefficient and the candidate second quantization coefficient is converted into an 8-bit floating-point number.

[0072] Then, determine the L candidate first quantization coefficients, the L candidate second quantization coefficients, and the L products of multiple weight parameters. For example, refer to formula (3).

[0073] (Formula 3)

[0074] in, This is the product of the candidate first quantization coefficient and multiple weight parameters. The first candidate quantization coefficient, For multiple weight parameters, It is the product of the first candidate quantization coefficient, the second candidate quantization coefficient, and multiple weight parameters. This is a candidate second quantization coefficient.

[0075] It is understandable that formula (3) is only used to illustrate the determination of one of the L products. All L products can be determined by formula (3).

[0076] Perform a Fourier transform on the first product to obtain the first feature, and then perform Fourier transforms on the L products of the L candidate first quantization coefficients, the L candidate second quantization coefficients, and the L weight parameters to obtain the L second features. The method for determining the first and second features can be found in formula (4).

[0077] (Formula 4)

[0078] in, As the second feature, It is the product of the first candidate quantization coefficient, the second candidate quantization coefficient, and multiple weight parameters. Here is the function used to perform the Fourier transform. As the first feature, This is the first product.

[0079] It is understandable that formula (4) is only used to illustrate the determination of one second feature. All L second features can be determined by formula (4).

[0080] L candidate first quantization coefficients, L candidate second quantization coefficients, and L products of multiple weight parameters are determined, along with L cosine similarities of the first product. L mean square errors of the L second features and the L first features are also determined. The method for determining cosine similarity can refer to formula (5), and the method for determining mean square error can refer to formula (6).

[0081] (Formula 5)

[0082] in, The product of the candidate first quantization coefficient, the candidate second quantization coefficient, and multiple weight parameters, and the cosine similarity of the first product, are given. Let be the function used to determine cosine similarity. The first product, It is the product of the first candidate quantization coefficient, the second candidate quantization coefficient, and multiple weight parameters.

[0083] It is understandable that Formula (5) is only used to illustrate the cosine similarity between one of the L products and the first product. All L cosine similarities can be determined by Formula (5).

[0084] (Formula 6)

[0085] in, The mean squared errors of the second and first features are given. Let be the function used to determine the mean square error. As the second feature, This is the first characteristic.

[0086] It is understandable that formula (6) is only used to illustrate the mean square error between one of the L second features and the first feature. All L mean square errors can be determined by formula (6).

[0087] Finally, corresponding to the fact that the kth mean square error among the L mean square errors is less than the product of the minimum mean square error and the kth cosine similarity among the L cosine similarities, the minimum mean square error is modified to the kth mean square error; the candidate first quantization coefficient corresponding to the minimum mean square error is used as the first quantization coefficient, and the candidate second quantization coefficient corresponding to the minimum mean square error is used as the second quantization coefficient.

[0088] For example, the first and second quantization coefficients are determined using the following code:

[0089]

[0090]

[0091]

[0092]

[0093] in, Mean square error, Here, cos_sim represents the minimum mean squared error, and cos_sim represents the cosine similarity. The first candidate quantization coefficient, This is a candidate second quantization coefficient.

[0094] In other words, if a certain mean squared error (e.g., the k-th one) is less than the product of the minimum mean squared error and the corresponding cosine similarity (e.g., the k-th cosine similarity), then the minimum mean squared error is modified to this mean squared error, and the candidate first quantization coefficient and candidate second quantization coefficient corresponding to this mean squared error are recorded. Finally, after traversing L mean squared errors and L cosine similarities, the candidate first quantization coefficient corresponding to the minimum mean squared error is taken as the first quantization coefficient, and the candidate second quantization coefficient corresponding to the minimum mean squared error is taken as the second quantization coefficient.

[0095] It can be understood that the first and second quantization coefficients are vectors with a length of P. Each element in the first and second quantization coefficients corresponds to Q elements in each of the P rows. and .

[0096] S203: Obtain the first activation value of the i-th computational layer of the input first model. The first activation value is an 8-bit floating-point number, where the first activation value is related to the input data of the first model, and i is an integer greater than 0 and less than or equal to N.

[0097] In some embodiments, when i is 1, the first activation value is an 8-bit floating-point number obtained after processing the input data of the first model. When i is not 1, the first activation value is the target activation value output by the (i-1)th computational layer of the first model.

[0098] In other words, if the i-th computation layer is the first computation layer of the first model, then the first activation value is an 8-bit floating-point number after processing the first data. If the i-th computation layer is not the first computation layer of the first model, then the first activation value is an 8-bit floating-point number output by the (i-1)-th computation layer.

[0099] S204: Determine the second activation value of the output of the i-th computational layer based on the first activation value of the i-th computational layer and at least one target weight parameter of the i-th computational layer.

[0100] For example, the method for determining the second activation value can refer to formula (7).

[0101] Formula (7)

[0102] in, This is the second activation value. The first activation value, Let be the weight parameters of the i-th computational layer. Zero point The first quantization coefficient, This is the second quantization coefficient. This is the bias of the i-th computation layer. The target weight parameters.

[0103] S205: Obtain the fourth quantization coefficient and calibrate the second activation value based on the fourth quantization coefficient to obtain the target activation value, wherein the target activation value is an 8-bit floating-point number, and the fourth quantization coefficient is used to adjust the second activation value to an 8-bit floating-point number.

[0104] For example, the method for determining the target activation value can refer to formula (8).

[0105] Formula (8)

[0106] in, The target activation value, This is the second activation value. This is the fourth quantization coefficient.

[0107] For example, Figure 3 A logic block diagram of a data processing procedure is shown according to an embodiment of this application.

[0108] like Figure 3As shown, the electronic device 100 performs an addition operation on multiple weight parameters (weight(in4)) and zero point (zerop) of the i-th computation layer, and then performs a multiplication operation (mul) on the result of the addition operation and the first quantization coefficient (scale(fp8)) to obtain multiple weight parameters (weight(fp8)) of the i-th computation layer after dequantization.

[0109] Perform a multiplication (MAC) operation of fp8*fp8 on the multiple weight parameters (weight(fp8)) of the i-th computational layer after dequantization and the first activation value (activation(fp8)) of the i-th computational layer. Perform a multiplication (mul) operation on the result of the multiplication operation of fp8*fp8 and the second quantization coefficient (scale1(fp16)). Then, perform an addition operation on the result of the multiplication operation and the bias of the i-th computational layer. Perform a multiplication (mul) operation on the result of the addition operation and the fourth quantization coefficient to obtain the target activation value of the i-th computational layer.

[0110] In some embodiments, the electronic device 100 determines the fourth quantization coefficient by the following method: obtaining the set of floating-point numbers output by each of the N computational layers of the first model, performing noise reduction processing on each set of floating-point numbers to obtain a set of denoised floating-point numbers; determining the fourth quantization coefficient based on the quantization range of the set of denoised floating-point numbers, wherein the quantization range of the set of denoised floating-point numbers is determined based on the maximum and minimum floating-point numbers in the set of denoised floating-point numbers.

[0111] For example, the set of floating-point numbers X output by each of the N computational layers in the first model by electronic device 100. n Denoising is performed to obtain multiple sets of denoised floating-point numbers X. ni Determine the set X of each group of floating-point numbers. ni The corresponding maximum floating-point number X ni max and the minimum value of floating-point number X ni min And based on each floating-point set X ni The maximum value X ni max and the minimum value of floating-point number X ni min Determine the quantization coefficient scale_value for the corresponding computation layer.

[0112] In some embodiments, the noise reduction process includes at least one of the following processing methods: singular value decomposition (SVD) processing method, discrete cosine transform (DCT) processing method, and wavelet decomposition processing method.

[0113] The process of determining the fourth quantization coefficient will be explained below using singular value decomposition as an example.

[0114] First, after obtaining the floating-point number set X1, use x=U∑V T Singular value decomposition is performed on the floating-point number set X1 to obtain the singular value matrix ∑. In this matrix ∑, all elements except those on the main diagonal are zero, and each element on the main diagonal is called a singular value. The target singular value matrix ∑ is obtained by filtering out singular values ​​in matrix ∑ that are less than a preset feature threshold. de The features of the corresponding floating-point set X1 are represented by singular values ​​that are greater than or equal to the feature threshold.

[0115] It is understandable that filtering out singular values ​​in matrix ∑ that are less than the pre-characteristic threshold can filter out noise signals that may exist in the floating-point number set X1.

[0116] It is understandable that in a matrix ∑, singular values ​​are usually sorted in descending order from largest to smallest, and the singular values ​​decay relatively quickly. Typically, the sum of the first 1% or the first 10% of the sorted singular values ​​can account for 99% of the sum of all singular values. Therefore, filtering out some singular values ​​will not cause data loss. On the contrary, filtering out some of the sorted singular values ​​(e.g., the last 10% of singular values) can be used to denoise the singular value matrix of the matrix, thereby filtering out any noise signals that may exist within it.

[0117] It is understood that the feature threshold can be the maximum value among the 10% after the singular values ​​are sorted in descending order, or it can be the maximum value among the last 20%, or it can be the maximum value among the last 90%. This application does not impose any restrictions on this.

[0118] Secondly, through x=U∑ de V T Solving the matrix and restoring it yields the set of floating-point numbers X. 11 . Set the floating-point number set X 11 The corresponding maximum floating-point number X 11 max and the minimum value of floating-point number X 11 minThe maximum and minimum floating-point values ​​to be calibrated are defined as αScale. Multiple calibration quantization coefficients (i.e., the fourth quantization coefficient) are determined based on a preset quantization coefficient range. The quantization coefficient range and the calibration quantization coefficients can be represented as [αScale]. i ,βScale i [n]. Where α and β represent the interval range, and α is less than 1, β is greater than 1, Scale i This represents the initial quantization coefficient, and n is the number of calibration quantization coefficients.

[0119] For example, the initial quantization coefficient of the floating-point number set X1 is 1, α is 0.48, β is 2, and n is 50. Then the quantization coefficient range of the floating-point number set X1 is from 0.48 to 2. This range is divided into 50 equal parts, and one quantization coefficient is selected from each of these 50 parts as a calibration quantization coefficient. It can be understood that within this quantization coefficient range, some calibration quantization coefficients are less than the initial quantization coefficient, and some are greater than the initial quantization coefficient.

[0120] Then, based on multiple calibration quantization coefficients, the floating-point number set X1 is quantized to obtain multiple calibration floating-point numbers. Based on the multiple calibration floating-point numbers and their corresponding calibration quantization coefficients, dequantization is performed to obtain multiple calibration floating-point number sets. Singular value decomposition is performed on each calibration floating-point number set to obtain the singular value matrix ∑1 corresponding to each calibration floating-point number set.

[0121] Finally, the singular value matrices ∑1 and ∑1 are obtained by performing singular value decomposition on the floating-point set X1. de Compare and obtain the matrix ∑ de The calibration quantization coefficient corresponding to the closest matrix ∑1 is used to determine the maximum and minimum floating-point values ​​after calibration.

[0122] It can be understood that matrix ∑1 can represent the characteristics of the corresponding set of calibrated floating-point numbers, and matrix ∑ de It can represent the characteristics of the corresponding floating-point set X1, when matrix ∑1 and matrix ∑ de When the closest value is found, it means that the characteristics of the calibration floating-point set are most similar to the characteristics of the floating-point set X1, i.e., the error is the smallest.

[0123] It is understandable that feature comparison is performed through noise reduction during the calibration process, that is, noise reduction is performed again during the calibration process, which can further improve the accuracy of the data.

[0124] S206: Use the target activation value as the first activation value of the (i+1)th computational layer of the first model.

[0125] Understandable. Figure 2The process shown is merely an illustrative example; in other embodiments, Figure 2 The process shown may include more or fewer steps, and this application does not limit this.

[0126] For example, in some embodiments, the electronic device 100 may also use the data output by the Nth computational layer of the first model as the processing result of the input data of the first model.

[0127] For example, if the first model is used to implement a classification function, the processing result of the input data of the first model is the classification result corresponding to the input data, such as image classification, audio classification, etc. If the first model is used to implement a generation function, the processing result of the input data of the first model is the generation result corresponding to the input data, such as an image generated based on the input data, text generated based on the input data, etc. It can be understood that the specific type and form of the processing result of the input data of the first model are determined based on the function of the first model and the content of the input data. This application does not limit the specific type and form of the processing result of the input data of the first model.

[0128] It is understood that the data processing method provided in this application can be applied to any electronic device capable of running neural network models, including but not limited to mobile phones, wearable devices (such as smartwatches), tablets, desktops, laptops, handheld computers, laptops, and ultra-mobile personal computers. Mobile personal computers (UMPCs), netbooks, cellular phones, personal digital assistants (PDAs), augmented reality (AR) / virtual reality (VR) devices, etc., are included in this application. The specific types of electronic devices are not limited in this application.

[0129] To facilitate understanding of the technical solutions of the embodiments of this application, the structure of the electronic device to which the data processing method provided in the embodiments of this application is applicable will be described using electronic device 100 as an example.

[0130] For example, Figure 4 A schematic diagram of the structure of an electronic device 100 is shown according to some embodiments of this application.

[0131] like Figure 4 As shown, the electronic device 100 includes one or more processors 101, system memory 102, and non-volatile memory (non-volatile memory). The system includes a volatile memory (NVM) 103, a communication interface 104, an input / output (I / O) device 105, and a system control logic unit 106 for coupling the processor 101, system memory 102, non-volatile memory 103, communication interface 104, and input / output device 105.

[0132] Processor 101 can be used to control electronic device 100 to execute the data processing method of this application. Processor 101 may include one or more processing units, such as a central processing unit (CPU), graphics processing unit (GPU), digital signal processor, microprocessor, artificial intelligence (AI) processor, or a processing module or circuit of a field programmable gate array (FPGA). Processor 101 may include one or more single-core or multi-core processors.

[0133] In some embodiments, the processor 101 can also be used to perform addition, multiplication and other operations corresponding to each computational layer of the first model through the arithmetic units in the processor (such as the multiply-add (MAC) unit, arithmetic logic unit (ALU).

[0134] System memory 102 is volatile memory, such as random access memory (RAM). System memory includes RAM (Rapid Access Memory), double data rate synchronous dynamic random access memory (DDR SDRAM), etc. System memory is used for temporary storage of data and / or instructions.

[0135] The non-volatile memory 103 may include one or more tangible, non-transitory computer-readable media for storing data and / or instructions. In some embodiments, the non-volatile memory 103 may include any suitable non-volatile memory such as flash memory and / or any suitable non-volatile storage device, such as a hard disk drive (HDD), a compact disc (CD), a digital versatile disc (DVD), or a solid-state drive (SSD). State drive (SSD), etc. In some embodiments, the non-volatile memory 103 may also be a removable storage medium, such as a secure digital (SD) memory card, etc.

[0136] Specifically, system memory 102 and non-volatile memory 103 may each include a temporary copy and a permanent copy of instructions 107. Instructions 107 may include instructions executed by processor 101, such as instructions for implementing the data processing methods provided in the embodiments of this application.

[0137] The communication interface 104 may include a transceiver for providing a wired or wireless communication interface for the electronic device 100, thereby enabling communication with any other suitable device via one or more networks. In some embodiments, the communication interface 104 may be integrated into other components of the electronic device 100, for example, the communication interface 104 may be integrated into the processor 101.

[0138] Input / output devices 105 may include input devices such as keyboards and mice, and output devices such as monitors.

[0139] System control logic 106 may include any suitable interface controller to provide any suitable interface for other modules of electronic device 100. For example, in some embodiments, system control logic 106 may include one or more memory controllers to provide an interface to system memory 102 and non-volatile memory 103.

[0140] In some embodiments, at least one of the processors 101 may be packaged together with the logic of one or more controllers for the system control logic unit 106 to form a system in package (SiP). In other embodiments, at least one of the processors 101 may also be integrated on the same chip with the logic of one or more controllers for the system control logic unit 106 to form a system on chip (SoC).

[0141] Understandable. Figure 4 The structure of the electronic device 100 shown is merely an example. In other embodiments, the electronic device 100 may include more or fewer components than illustrated, or combine some components, or split some components, or have different component arrangements. The illustrated components may be implemented in hardware, software, or a combination of software and hardware.

[0142] Furthermore, Figure 5 A schematic diagram of a processor structure is shown according to an embodiment of this application.

[0143] like Figure 5As shown, the processor 101 includes a core (also called a physical core) 1011. In some embodiments, the core 1011 is used to execute the data processing method of the embodiments of this application.

[0144] The core 1011 includes a control unit (CU) 1012, a multiply-accumulate unit (MAC) 1013, and an arithmetic logic unit (ALU) 1014.

[0145] The control unit 1012 is used to acquire multiple weight parameters of the first model, wherein the multiple weight parameters of the first model are determined based on 16-bit floating-point quantization, and the first model includes N computational layers, where N is an integer greater than 2. It also acquires a first quantization coefficient, a second quantization coefficient, and a third quantization coefficient, and inputs the multiple weight parameters, the first quantization coefficient, the second quantization coefficient, and the third quantization coefficient to the multiply-accumulate unit 1013 and / or the arithmetic logic unit 1014.

[0146] The control unit 1012 is also used to control the multiply-add unit 1013 and the arithmetic logic unit 1014 to complete the corresponding operations, so as to complete the operations of the above formulas (7) and (8).

[0147] For example, refer to Figure 3 The various operations in the unit are as follows: the multiply-add unit 1013 is used to perform multiplication (MAC) operations, and the arithmetic logic unit 1014 is used to perform addition (add) operations and multiplication (mul) operations.

[0148] The multiply-add unit 1013 is used to perform MAC operations, such as matrix multiplication, that is, the multiplication of multiple weight parameters of the first model with the first quantization parameter.

[0149] The arithmetic logic unit 1014 is used to perform arithmetic operations such as addition, subtraction, multiplication, and division, as well as logical operations, such as performing... Figure 3 The addition and multiplication operations in the language.

[0150] Understandable. Figure 5 The description only uses the example of execution units such as the multiply-accumulate unit 1013 and the arithmetic logic unit 1014 being located inside the core 1011. In other embodiments, execution units such as the multiply-accumulate unit 1013 and the arithmetic logic unit 1014 may also be located outside the core 1011, and this application does not impose any restrictions on this.

[0151] Understandable. Figure 5The processor 101 shown is merely an example. In other embodiments, the processor 101 may include more or fewer components than illustrated, or combine some components, or split some components, or have different component arrangements. The illustrated components may be implemented in hardware, software, or a combination of software and hardware. For example, the processor 101 may also include a cache, and the core 1011 may also include registers, etc., which are not limited in this application.

[0152] In some embodiments, this application also provides a computer-readable storage medium storing at least one computer program instruction, at least one program segment, code set, or instruction set, which is loaded and executed by a model training system to implement the data processing methods provided in the above-described method embodiments.

[0153] In some embodiments, this application also provides a computer program product, which includes computer program instructions that, when executed by a model training system, enable the device to implement the data processing methods provided in the above-described method embodiments.

[0154] Various embodiments of the mechanisms disclosed in this application can be implemented in hardware, software, firmware, or combinations of these implementation methods. Embodiments of this application can be implemented as computer programs or program code executable on a programmable system, the programmable system including at least one processor, a storage system (including volatile and non-volatile memory and / or storage elements), at least one input device, and at least one output device.

[0155] Program code can be applied to input instructions to execute the functions described in this application and generate output information. The output information can be applied to one or more output devices in a known manner. For the purposes of this application, the processing system includes any system having a processor such as, for example, a digital signal processor (DSP), a microcontroller, an application-specific integrated circuit (ASIC), or a microprocessor.

[0156] The program code can be implemented using a high-level procedural language or an object-oriented programming language to communicate with the processing system. Assembly language or machine language can also be used when needed. In fact, the mechanisms described in this application are not limited to any particular programming language. In either case, the language can be a compiled language or an interpreted language.

[0157] In some cases, the disclosed embodiments may be implemented in hardware, firmware, software, or any combination thereof. The disclosed embodiments may also be implemented as instructions carried or stored thereon on one or more temporary or non-temporary machine-readable (e.g., computer-readable) storage media, which may be read and executed by one or more processors. For example, the instructions may be distributed via a network or through other computer-readable media. Therefore, machine-readable media may include any mechanism for storing or transmitting information in a machine-readable (e.g., computer-readable) form, including but not limited to floppy disks, optical disks, CD-ROMs, and read-only memories (CDs). ROMs, magneto-optical disks, read-only memory (ROM), random access memory (RAM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EPROM). Machine-readable media include any type of machine-readable medium suitable for storing or transmitting electronic instructions or information in a machine-readable form, such as an EEPROM, magnetic or optical card, flash memory, or tangible machine-readable storage for transmitting information via the Internet in the form of electrical, optical, acoustic, or other forms of propagation signals (e.g., carrier waves, infrared signals, digital signals, etc.). Therefore, machine-readable media include any type of machine-readable medium suitable for storing or transmitting electronic instructions or information in a machine-readable (e.g., computer-readable) form.

[0158] In the accompanying drawings, some structural or methodological features may be shown in a specific arrangement and / or order. However, it should be understood that such a specific arrangement and / or order may not be necessary. Rather, in some embodiments, these features may be arranged in a manner and / or order different from that shown in the illustrative drawings. Furthermore, the inclusion of structural or methodological features in a particular figure does not imply that such features are required in all embodiments, and in some embodiments, these features may be omitted or may be combined with other features.

[0159] It should be noted that all units / modules mentioned in the device embodiments of this application are logical units / modules. Physically, a logical unit / module can be a physical unit / module, a part of a physical unit / module, or a combination of multiple physical units / modules. The physical implementation of these logical units / modules themselves is not the most important factor; the combination of functions implemented by these logical units / modules is the key to solving the technical problems proposed in this application. Furthermore, to highlight the innovative aspects of this application, the above-described device embodiments of this application have not introduced units / modules that are not closely related to solving the technical problems proposed in this application. This does not mean that the above-described device embodiments do not contain other units / modules.

[0160] It should be noted that, in the examples and description of this patent, the terms "comprising," "including," or any other variations thereof are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one" does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0161] Although this application has been illustrated and described with reference to certain preferred embodiments thereof, those skilled in the art should understand that various changes in form and detail may be made thereto without departing from the spirit and scope of this application.

Claims

1. A data processing method, characterized in that, Applied to electronic devices, the method includes: Obtain multiple weight parameters of the first model, wherein the multiple weight parameters of the first model are determined based on 16-bit floating-point quantization, and the first model includes N computational layers, wherein N is an integer greater than 2; Based on the first quantization coefficient and the second quantization coefficient, the plurality of weight parameters are dequantized to obtain a plurality of target weight parameters, wherein the target weight parameters are 8-bit floating-point numbers, the first quantization coefficient and the second quantization coefficient are related to the third quantization coefficient, the first quantization coefficient is used to adjust the plurality of weight parameters to 8-bit floating-point, the second quantization coefficient is used to calibrate the target weight parameters, and the third quantization coefficient is used to dequantize the plurality of weight parameters of the first model to 16-bit floating-point numbers; Obtain the first activation value of the i-th computational layer of the first model. The first activation value is an 8-bit floating-point number, where the first activation value is related to the input data of the first model, and i is an integer greater than 0 and less than or equal to N. Based on the first activation value of the i-th computation layer and at least one target weight parameter of the i-th computation layer, determine the second activation value output by the i-th computation layer; A fourth quantization coefficient is obtained, and the second activation value is calibrated based on the fourth quantization coefficient to obtain a target activation value, wherein the target activation value is an 8-bit floating-point number, and the fourth quantization coefficient is used to adjust the second activation value to an 8-bit floating-point number; The target activation value is used as the first activation value of the (i+1)th computational layer of the first model.

2. The method according to claim 1, characterized in that, The absolute value of the difference between the first product and the second product is less than the absolute value threshold. The first product is the product of the third quantization coefficient and the plurality of weight parameters, and the second product is the product of the first quantization coefficient, the second quantization coefficient and the plurality of weight parameters.

3. The method according to claim 2, characterized in that, The third quantization coefficient is a P×Q matrix, where P is an integer greater than 1 and Q is an integer greater than 1. Furthermore, the first and second quantization coefficients are determined in the following manner: Determine the maximum and minimum values ​​of the Q elements in the j-th row of the P rows of the third quantization coefficient, where j is an integer greater than 0 and less than or equal to P; Based on the relationship between the reciprocal of the minimum value and the maximum representation range of an 8-bit floating-point number, L candidate second quantization coefficients are determined, where L is an integer greater than 1; Determine the L quotients between the third quantization coefficient and the L candidate second quantization coefficients, and convert the L quotients into 8-bit floating-point numbers. Use the L converted 8-bit floating-point numbers as the L candidate first quantization coefficients. Determine the L products of the L candidate first quantization coefficients, the L candidate second quantization coefficients, and the L weight parameters, respectively; Based on the similarity between the L products and the first product, the first quantization coefficient and the second quantization coefficient are determined.

4. The method according to claim 3, characterized in that, The step of determining the first quantization coefficient and the second quantization coefficient based on the similarity between the L products and the first product includes: Determine the L cosine similarities between the L products and the first product, respectively; Perform a Fourier transform on the first product to obtain a first feature, and perform a Fourier transform on the L products to obtain L second features; Determine the mean square error of the L second features and the first feature respectively; Corresponding to the fact that the kth mean square error among the L mean square errors is less than the product of the minimum mean square error and the kth cosine similarity among the L cosine similarities, the minimum mean square error is modified to the kth mean square error, where k is an integer greater than 0 and less than or equal to L; The candidate first quantization coefficient corresponding to the minimum mean square error is used as the first quantization coefficient, and the candidate second quantization coefficient corresponding to the minimum mean square error is used as the second quantization coefficient.

5. The method according to claim 3 or 4, characterized in that, The first maximum value is greater than the second maximum value, and the first minimum value is greater than the second minimum value. The first maximum value is the maximum value among the L candidate second quantization coefficients when the reciprocal of the minimum value is less than the maximum representation range of an 8-bit floating-point number; the first minimum value is the minimum value among the L candidate second quantization coefficients when the reciprocal of the minimum value is less than the maximum representation range of an 8-bit floating-point number; the second maximum value is the maximum value among the L candidate second quantization coefficients when the reciprocal of the minimum value is greater than or equal to the maximum representation range of an 8-bit floating-point number; and the second minimum value is the minimum value among the L candidate second quantization coefficients when the reciprocal of the minimum value is greater than or equal to the maximum representation range of an 8-bit floating-point number.

6. The method according to any one of claims 1 to 5, characterized in that, Corresponding to i being 1, the first activation value is an 8-bit floating-point number obtained after processing the input data of the first model; For any i not equal to 1, the first activation value is the target activation value output by the (i-1)th computational layer of the first model.

7. The method according to any one of claims 1 to 6, characterized in that, The fourth quantization coefficient is determined by the following method: Obtain the set of floating-point numbers output by each of the N computational layers in the first model, and perform noise reduction processing on each set of floating-point numbers to obtain the set of denoised floating-point numbers. The fourth quantization coefficient is determined based on the quantization range of the denoised floating-point number set, wherein the quantization range of the denoised floating-point number set is determined based on the maximum and minimum floating-point numbers in the denoised floating-point number set.

8. The method according to claim 7, characterized in that, The noise reduction process includes at least one of the following methods: singular value decomposition, discrete cosine transform, and wavelet decomposition.

9. The method according to any one of claims 1 to 8, characterized in that, The data type of the input data for the first model includes at least one of the following: text, image, and voice.

10. The method according to any one of claims 1 to 8, characterized in that, The method further includes: using the data output by the Nth computational layer of the first model as the processing result of the input data of the first model, wherein the first model is used to implement at least one of the following functions: classification, generation, recognition, object detection, and semantic segmentation.

11. An electronic device, characterized in that, It includes a processor and a memory, the memory being used to store program instructions, and the processor being used to invoke the program instructions to perform the method of any one of claims 1 to 10.

12. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores instructions that, when executed, implement the method of any one of claims 1 to 10.

13. A computer program product, characterized in that, The computer program product includes instructions that, when executed, implement the method of any one of claims 1 to 10.

Images