Entropy coded integer coding, coding method, decoding method and quantization method

Abstract

The present invention relates to an entropy coded integer coding, coding method, decoding method and quantization method, which belongs to the field of data processing and solves the problems of low current coding efficiency and high computational cost. The technical solution of the present invention mainly includes: an entropy coded integer coding method, used to generate the coding of the entropy coded integer; an exponent field is represented by a first numerical sequence, the first numerical sequence is unary, and the maximum number of bits of the exponent field is E; a valid number field is represented by a second numerical sequence, the second numerical sequence is binary; the first numerical sequence and the second numerical sequence are connected to form the N-bit coding.

Description

Technical Field

[0001] The present invention belongs to the field of data processing, and in particular relates to an entropy coded integer coding, a coding method, a decoding method and a quantization method. Background Art

[0002] The explosive development of deep learning technology requires more complex and larger models and data sets, and requires higher computing power. The computing power required for deep learning doubles every few months, which is also known as "Moore's Law in deep learning." However, Moore's Law in the hardware field is expected to die out in 2030, and the development of computer hardware is slowing down, so new technologies are needed to eliminate the contradiction between the two Moore's Laws.

[0003] The power consumption and area of ​​the arithmetic circuit are directly related to the word length of the operand. The power consumption and area of ​​the adder are linearly proportional to the bit width of the operand; the power consumption and area of ​​the multiplier are proportional to the square of the bit width of the operand. Therefore, in order to reduce the power consumption and area overhead of processors / accelerators for deep learning, reducing the bit width of the operand through quantization has become one of the key technologies that has received widespread attention. Traditional fixed-point quantization will introduce large network prediction errors in the case of low bit width. This is because the distribution of the model weights and activation values ​​of the neural network is approximately Gaussian, while the gradient is approximately lognormal, such as Figure 1 As shown in the figure, its data distribution is very close to a bell curve. It is difficult for traditional fixed-point numbers to take into account both its expression range and accuracy at the same time. Therefore, the information entropy is low, and there is a large information loss when describing the network model.

[0004] In the prior art, integers, floating point numbers, logarithms, table lookup methods, etc. are usually used for quantization.

[0005] Integer quantization has a high quantization error due to the large difference between its uniform numerical distribution and Gaussian distribution. After multiple layers of iterative calculations, the accumulated error causes a significant loss in model accuracy. In other words, if you want to maintain a certain accuracy, you need a higher bit width, which is contrary to the original intention of quantization.

[0006] Floating-point numbers can overcome the shortcomings of fixed-point numbers in terms of range and precision, and can reduce the bit width while maintaining precision. For example, TPUv4i uses BF16 as the data type; NVidia A100 GPU uses FP16 / BF16 / TF32 and other data types, and the latest Hopper100 has added the FP8 type. However, floating-point numbers require exponential order, normalization, rounding, exception handling, etc. in calculations, resulting in high circuit area and power consumption.

[0007] Using a logarithmic system for quantization significantly increases the operational overhead when performing addition operations.

[0008] In addition, there is also non-uniform quantization technology that uses indirect method or table lookup method, which can significantly reduce the bit width and model size. However, the model quantized by the table lookup method cannot use the quantization value for direct calculation. The model expressed by the table lookup method must first obtain the true value of each weight through table lookup (i.e., inverse quantization) for calculation, and the true value is usually represented by a high-bitwidth floating-point number. Therefore, the table lookup method can only save memory usage, but the hardware circuit cost and power consumption are also very high.

[0009] From the perspective of volume compression effect and accuracy, the coding of the quantization target value itself has a huge impact on the quantization effect. In summary, the existing technology has the problems of low coding efficiency and high computational cost. Summary of the invention

[0010] In view of the above analysis, the embodiments of the present invention aim to provide an encoding method, a decoding method, an encoding and a quantization method of an entropy coded integer, so as to solve the current problems of low encoding efficiency and high computational cost.

[0011] A first aspect of the present invention provides an entropy coded integer encoding method for generating an encoding of the entropy coded integer, comprising:

[0012] An exponent field is represented by a first numerical sequence, the first numerical sequence is unary, and a maximum number of bits of the exponent field is E;

[0013] The significant number field is represented by a second numerical sequence, wherein the second numerical sequence is binary;

[0014] Concatenate the first numerical sequence and the second numerical sequence into the N-bit code;

[0015] The corresponding relationship between the encoding and the true value is expressed as:

[0016]

[0017]

[0018] A=m×2 e

[0019] Among them, A represents the real value, ε represents the exponential field, Indicates the valid number field, Represents a binary number consisting of 1 and the valid digit field in a bit string.

[0020] In some embodiments, a sign bit is represented by 1 bit 0 or 1, where bit 0 represents that the code is a positive number, and bit 1 represents that the code is a negative number.

[0021] In some embodiments, the encoding includes, from left to right, the sign bit, the exponent field, and the significand field;

[0022] The corresponding relationship between the code and the real value is expressed as:

[0023] s=δ

[0024]

[0025]

[0026] A=(-1) δ ×m×2 e

[0027] Among them, A represents the real value, δ represents the sign bit, and ε represents the exponent field. Indicates the valid number field, Represents a binary number consisting of 1 and the valid digit field in a bit string.

[0028] In some embodiments, the exponent field includes a number of consecutive bit 1s, the number of bit 1s in the number of consecutive bit 1s represents the value of the exponent field, and the value of the exponent field is less than or equal to E.

[0029] In some embodiments, the step of concatenating the first numerical sequence and the second numerical sequence into the N-bit code comprises:

[0030] If the value of the exponent field is not equal to E, the first numerical sequence and the second numerical sequence are connected in the following manner: the first numerical sequence, the separation symbol and the second numerical sequence are concatenated according to a bit string, and the separation symbol is bit 0;

[0031] If the value of the exponent field is equal to E, the first numerical sequence and the second numerical sequence are connected in the following manner: the first numerical sequence and the second numerical sequence are concatenated according to a bit string.

[0032] A second aspect of the present invention provides an entropy coded integer decoding method for decoding an entropy coded integer into a corresponding decoded value;

[0033] Selecting hyperparameters (N, E) of the encoding, where N represents the bit width of the encoding and E represents the maximum number of bits of the exponent field;

[0034] Extracting an exponent field and a valid number field in the code according to the hyperparameter;

[0035] Decoding and calculating the exponent field to obtain a decoded value of the exponent field, and decoding and calculating the significand field to obtain a decoded value of the significand field;

[0036] The true value of the code is calculated according to the value in the valid number field and the value in the exponent field, and the calculation formula is expressed as:

[0037]

[0038]

[0039] A=m×2 e

[0040] Among them, A represents the true value, e represents the decoded value of the exponent field, m represents the decoded value of the significand field, and ε represents the unary exponent field. Represents the binary significant digit field, Represents a binary number consisting of 1 and the valid digit field in a bit string.

[0041] In some embodiments, before extracting the exponent field and the significand field, the method further includes:

[0042] The sign bit at the first position of the code is extracted, and the bit value of the sign bit is used as the decoded value of the sign bit.

[0043] In some embodiments, extracting the exponent field and the valid number field in the encoding according to the hyperparameter includes:

[0044] Extracting consecutive bits 1 that are no more than E bits after the sign bit as an exponent field, and the number of bits 1 is the value ε of the exponent field;

[0045] If the value of the exponent field is not equal to E, extract the separator after the exponent field and the valid number field after the separator, wherein the separator is a bit 0 and the valid number field is a binary number;

[0046] If the value of the exponent field is equal to E, a significant number field located after the exponent field is extracted, and the significant number field is a binary number.

[0047] The third aspect of the present invention provides an entropy coded integer code, comprising at least an exponent field and a significand field;

[0048] The exponent field uses unary to represent the exponent size;

[0049] The significant digit field uses binary to represent the significant digit size;

[0050] The encoding also has hyperparameters (N, E), where N represents the bit width of the encoding and E represents the maximum number of bits of the exponent field;

[0051] The decoding process of the encoding includes:

[0052] Extract the exponent field ε and the valid number field in the encoding according to the hyperparameter

[0053] Decoding and calculating the exponent field to obtain a decoded value of the exponent field, and decoding and calculating the significand field to obtain a decoded value of the significand field;

[0054] According to the value ε in the significand field and the value of the exponent field Calculate the true value of the code, the calculation formula is expressed as:

[0055]

[0056]

[0057] A=m×2 e

[0058] Among them, A represents the true value, e represents the decoded value of the exponent field, m represents the decoded value of the significand field, and ε represents the unary exponent field. Represents the binary significant digit field, Represents a binary number consisting of 1 and the valid digit field in a bit string.

[0059] In some embodiments, the code further includes a sign bit, and the sign bit is represented by a bit 0 or 1, where bit 0 indicates that the code is a positive number, and bit 1 indicates that the code is a negative number.

[0060] In some embodiments, the encoding includes, from left to right, the sign bit, the exponent field, and the significand field;

[0061] The corresponding relationship between the code and the real value is expressed as:

[0062] s=δ

[0063]

[0064]

[0065] A=(-1) δ ×m×2 e

[0066] Among them, A represents the real value, s represents the decoded value of the sign bit, e represents the decoded value of the exponent field, m represents the decoded value of the significand field, δ represents the sign bit, ε represents the exponent field, Indicates the valid number field, Represents a binary number consisting of 1 and the valid digit field in a bit string.

[0067] In some embodiments, the exponent field includes a number of consecutive bit 1s, the number of bit 1s in the number of consecutive bit 1s represents the value of the exponent field, and the value of the exponent field is less than or equal to E.

[0068] In some embodiments, if the decoded value of the exponent field is not equal to E, a separator is inserted between the exponent field and the significand field, and the separator is a bit 0;

[0069] If the value of the exponent field is equal to E, the significand field is directly concatenated after the exponent field.

[0070] In some embodiments, the multiplication between the two codes includes an exclusive-OR operation of the sign bit, an accumulation operation of the decoded value of the exponent field, and a product operation of the decoded value of the significand field.

[0071] A fourth aspect of the present invention provides a method for quantizing a deep neural network model, for quantizing data in the deep neural network into the entropy coded integer code described in any one of the above items, wherein the deep neural network model is a model having L network layers, comprising the following steps:

[0072] Obtaining a bit width N of the entropy coded integer code;

[0073] Determining optimal encoding hyperparameters for quantization of at least some of the network layers;

[0074] The data to be quantized of each network layer is obtained, and quantized according to the corresponding optimal coding format.

[0075] In some embodiments, the determining of the optimal coding format for each quantization of at least part of the network layers comprises the following steps:

[0076] S1. quantizing the network layer i in the deep neural network model according to the selected hyperparameters (N, E) of the entropy coded integer coding to obtain an i-layer separate quantization model, where E represents the maximum number of bits of the exponent field of the entropy coded integer coding;

[0077] S2, using the i-layer separate quantization model to perform the prediction task to obtain a prediction result;

[0078] S3, repeat steps S1-S2 until the value range of E is traversed, and the value range of E is 0, 1, 2, ..., N;

[0079] S4, comparing the prediction results of each of the i-th layer's separate quantization models to determine the optimal value of E of the i-th layer;

[0080] S5. Repeat steps S3-S4 until all or part of the value range of i is traversed, and the value range of i is 1, 2, ..., L.

[0081] The embodiments of the present invention have at least the following beneficial effects:

[0082] The encoding is performed by combining a unary exponent field and a binary valid number field, and the real value corresponding to the encoding is obtained by multiplying the decoded values ​​corresponding to the exponent field and the valid number field. Since the bit width of the exponent field can be any bit width less than or equal to E, the numerical distribution of the encoded entropy coded integer is uneven. Specifically, the numerical values ​​corresponding to the encoding of the entropy coded integer with a lower exponent field bit width are denser, and the difference between adjacent numerical values ​​(quantization interval) is smaller; conversely, when the exponent field bit width is higher, the numerical values ​​corresponding to the encoding are sparser and the difference between adjacent numerical values ​​(quantization interval) is large. Moreover, the multiplication operation between the entropy coded integers provided by the present invention can be split into integer accumulation operation of the exponent field and integer multiplication of the valid number field, avoiding the complex operations such as exponent alignment and normalization required for traditional floating-point numbers, so its decoding and operation costs are very low compared to existing numerical coding. BRIEF DESCRIPTION OF THE DRAWINGS

[0083] In order to more clearly illustrate the embodiments of this specification or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in the embodiments of this specification. For ordinary technicians in this field, other drawings can also be obtained based on these drawings.

[0084] Figure 1 It is the distribution diagram of neural network parameter data;

[0085] Figure 2 A schematic flow chart of a method for encoding an entropy coded integer provided in an embodiment of the first aspect of the present invention;

[0086] Figure 3 A schematic flow chart of a method for decoding an entropy coded integer provided in an embodiment of the second aspect of the present invention;

[0087] Figure 4 A schematic diagram of the relationship between the hyperparameters and coding value adjustment of the entropy coding integer coding provided by the embodiment of the third aspect of the present invention;

[0088] Figure 5 A schematic flow chart of a method for quantizing a deep neural network model provided in an embodiment of the fourth aspect of the present invention;

[0089] Figure 6 A schematic flow chart of a quantization method for a neural network model provided in an embodiment of the fifth aspect of the present invention;

[0090] Figure 7 A comparison chart of relative energy consumption data of computing units with different bit widths and formats provided in an embodiment of the present invention;

[0091] Figure 8 A comparison diagram of relative area data of operation units of different formats and bit widths provided in an embodiment of the present invention;

[0092] Fig. 9 A comparative data diagram of power consumption of accelerators of different formats and bit widths provided in an embodiment of the present invention;

[0093] Fig.10 A data chart comparing the areas of accelerators of different formats and bit widths provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0094] In order to make the purpose, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. It should be noted that, in the absence of conflict, the embodiments and features in the embodiments disclosed in this disclosure can be combined, separated, interchanged and / or rearranged with each other. Based on the embodiments in the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present invention.

[0095] The terms used here are for the purpose of describing specific embodiments, and are not intended to be restrictive. As used here, unless the context clearly indicates otherwise, the singular forms "one (kind, person)" and "said (the)" are also intended to include plural forms. In addition, when the terms "comprise" and / or "include" and their variations are used in this specification, it is explained that there are stated features, integral bodies, steps, operations, parts, assemblies and / or their groups, but it is not excluded that there are or add one or more other features, integral bodies, steps, operations, parts, assemblies and / or their groups. It should also be noted that, as used here, the terms "substantially", "approximately" and other similar terms are used as approximate terms and not as degree terms, so that they are used to explain the inherent deviations of the measured values, calculated values ​​and / or the values ​​provided that will be recognized by those of ordinary skill in the art.

[0096] First, the terms involved in one or more embodiments of the present invention are explained.

[0097] Unary is a number system with 1 as the base, and is the simplest number system for representing natural numbers. To represent a natural number N in unary, we first arbitrarily choose a symbol to represent "one", and then repeat it N times. For example, using 1 (a counting sign) as the symbol, the number six is ​​represented by 111111. Counting with fingers is also a unary system.

[0098] Binary refers to a number system based on 2 in mathematics and digital circuits. It is a binary system that uses 2 as the base number representation system. In this system, two different symbols 0 (representing zero) and 1 (representing one) are usually used to represent. In digital electronic circuits, the implementation of logic gates directly applies binary, so modern computers and computer-dependent devices all use binary. Each digit is called a bit (Bit, the abbreviation of Binary digit).

[0099] Bit (BIT, Binary digit), a computer term, is a unit of information volume, which is a transliteration of the English word BIT. It is also a bit in the unary and binary digits, a unit of measurement for information volume, and the smallest unit of information volume.

[0100] As deep learning has made breakthroughs in image classification, target detection, and natural language processing, the demand for applying it to real-life scenarios has become increasingly strong. At present, mobile and portable electronic devices have greatly facilitated people's lives, and deep learning will greatly improve the intelligence and entertainment of these devices. Therefore, it has become an urgent need to deploy deep learning neural network models on mobile terminals and embedded systems. However, in actual deployment, neural network models using deep learning usually face the problem of too large model size. For example, the file size of neural network models generally ranges from tens of megabytes to hundreds of megabytes. For such file sizes, for mobile terminals, the traffic consumed during downloading and the long transmission waiting time caused by the impact of bandwidth are unbearable for users; especially for some embedded systems with limited storage space, there may not be enough storage space to store such a large neural network model file. At the same time, deep learning neural network models have high requirements for computing resources and computing power; when using large neural network models for calculations, mobile terminals and embedded systems either cannot provide the required computing resources, or the calculations are slow, resulting in too high response delays and cannot meet actual application scenarios. In addition, neural network models also consume a lot of power. During the neural network calculation process, the processor needs to frequently read the parameters of the neural network model. Therefore, a larger neural network model also results in a higher number of memory accesses. Frequent memory accesses will also greatly increase power consumption. High power consumption is not conducive to deploying neural network models on mobile devices.

[0101] Therefore, in order to deploy a neural network with good performance on hardware devices with limited resources, it is necessary to compress and accelerate the neural network model. The distribution of the model weights, activation values, and gradients of the neural network is approximately Gaussian, such as Figure 1 As shown, its data distribution is very close to a bell curve, and the data distribution shows great non-uniformity. In order to reduce the loss of model accuracy caused by quantization, the present invention provides an entropy coding integer code (ECI), a coding method, a decoding method and a quantization method to solve the problem of a coding method and a coding method thereof that lacks non-uniform distribution and has low computational cost.

[0102] The first aspect of the present invention provides an entropy coded integer encoding method for generating a code of the entropy coded integer, such as Figure 2 As shown, including:

[0103] An exponent field is represented by a first numerical sequence, the first numerical sequence is unary, and a maximum number of bits of the exponent field is E;

[0104] The significant number field is represented by a second numerical sequence, wherein the second numerical sequence is binary;

[0105] Concatenate the first numerical sequence and the second numerical sequence into the N-bit code;

[0106] The corresponding relationship between the encoding and the true value is expressed as:

[0107]

[0108]

[0109] A=m×2 e

[0110] Among them, A represents the real value, ε represents the exponential field, Indicates the valid number field, It represents a binary number formed by concatenating 1 and the effective digit field according to a bit string. It should be understood that the symbols e and m here are intermediate numbers set for the convenience of representing A.

[0111] The design purpose of ECI coding is to make the numerical distribution of the code match the numerical distribution of the parameters to be quantized in, for example, a neural network, so that the distribution characteristics of the numerical values ​​can be retained during the quantization process, the data information entropy of the quantization model can be maximized, and the prediction accuracy of the quantization model can be improved.

[0112] Therefore, the design concept of ECI is very straightforward, that is, dense sampling is performed in high-density areas (i.e., the area near the center point of the bell curve), and sparse sampling is performed in low-density areas (the long tail areas extending at both ends of the bell curve), that is, values ​​with larger amplitudes have larger quantization steps.

[0113] Specifically, the ECI coding system is a coding with uneven numerical distribution, and two hyperparameters N and E are used to define the ECI number system: N is the bit width of the ECI number, that is, the number of bits of the ECI code, and E is the maximum number of bits of the exponent field. Among them, E is called the maximum number of bits, indicating that in the ECI (N, E) coding that determines the hyperparameters, the number of bits of the ECI exponent field can be changed. For example, when E=3, the number of bits of the exponent field can be 1, 2 or 3. Correspondingly, since the bit width N is fixed, the number of bits in the effective number field also changes accordingly. According to the above-mentioned conversion formula for the corresponding relationship between the ECI code and the true value, the more bits in the effective number field, the more values ​​the effective number field can take. In other words, when the number of bits in the exponent field is low, the ECI encoding has a smaller value, a larger number of values ​​and a small value interval (high value density). Conversely, when the number of bits in the exponent field is high, for example, when the maximum value of the exponent field is E, the ECI encoding has a larger value, a smaller number of values ​​and a large value interval (low value density). It can be seen that the numerical distribution of ECI is more suitable for data with a bell-shaped curve numerical distribution than general encoding.

[0114] It should be understood that application scenarios for numerical quantization using the ECI coding provided by the embodiment of the present invention include, for example, an inference process or a training process.

[0115] Preferably, in some embodiments, the sign bit is represented by 1 bit 0 or 1, bit 0 represents that the code is a positive number, and bit 1 represents that the code is a negative number.

[0116] Preferably, in some embodiments, the encoding includes, from left to right, the sign bit, the exponent field, and the valid number field;

[0117] The corresponding relationship between the code and the real value is expressed as:

[0118] s=δ

[0119]

[0120]

[0121] A=(-1) δ ×m×2 e

[0122] Among them, A represents the real value, δ represents the sign bit, and ε represents the exponent field. Indicates the valid number field, Represents a binary number consisting of 1 and the valid digit field in a bit string.

[0123] Preferably, in some embodiments, the exponent field includes a number of consecutive bit 1s, the number of bit 1s in the number of consecutive bit 1s represents the value of the exponent field, and the value of the exponent field is less than or equal to E.

[0124] Preferably, in some embodiments, said connecting the first numerical sequence and the second numerical sequence into the N-bit code comprises:

[0125] If the value of the exponent field is not equal to E, the first numerical sequence and the second numerical sequence are connected in the following manner: the first numerical sequence, the separation symbol and the second numerical sequence are concatenated according to a bit string, and the separation symbol is bit 0;

[0126] If the value of the exponent field is equal to E, the first numerical sequence and the second numerical sequence are connected in the following manner: the first numerical sequence and the second numerical sequence are concatenated according to a bit string.

[0127] Specifically, the number in ECI (N, E) format consists of three fields: the sign bit (s), the exponent field (e), and the significant digit field (m). The sign bit contains only one bit, indicating whether the number is positive or negative (there may be no sign bit in the unsigned ECI number), with bit 0 indicating positive and bit 1 indicating negative. The exponent field is followed by the sign bit. The exponent field is represented in base 1, that is, the exponent is determined by the number of consecutive bits 1 in the field, and the value of the exponent field cannot exceed E. For example, when E=4, the exponent field value extracted from the bit sequence "1110101" with a length of 7 is 3 (there are three consecutive leading bits 1); and the exponent field value extracted from the bit sequence "1111101" is 4 (although there are 5 consecutive leading bits 1, the exponent field value cannot exceed E, so the exponent field extracted is 4). The exponent field is followed by the significant digit field (m). When the value of the exponent field is not equal to E, a bit 0 is required between the exponent and the effective digit field as a separator; when the value of the exponent field is equal to E, the separator is omitted. Still taking the bit sequence "1110101" with a length of 7 as an example, when E = 4, its effective digit is "101", and the effective digit of the bit sequence "1111101" is also "101".

[0128] It should be understood that in the above coding scheme description, by swapping the 0s and 1s in some parts or fields (for example, the sign bit uses a single-bit 1 to represent a positive number and a single-bit 0 to represent a negative number; or using consecutive bits 0 instead of bit 1 to represent the exponent field) or rearranging the order of certain fields (for example, moving the sign bit from the leftmost end to the rightmost end); these bit swaps / rearrangements are merely simple bit replacements and do not substantially affect the coding method, and therefore can all be regarded as equivalent coding methods of ECI.

[0129] The second aspect of the present invention provides a method for decoding an entropy coded integer, which is used to decode the encoding of the entropy coded integer into a corresponding decoding value, such as Figure 3 As shown, including:

[0130] Selecting hyperparameters (N, E) of the encoding, where N represents the bit width of the encoding and E represents the maximum number of bits of the exponent field;

[0131] Extracting an exponent field and a valid number field in the code according to the hyperparameter;

[0132] Decoding and calculating the exponent field to obtain a decoded value of the exponent field, and decoding and calculating the significand field to obtain a decoded value of the significand field;

[0133] The true value of the code is calculated according to the value in the valid number field and the value in the exponent field, and the calculation formula is expressed as:

[0134]

[0135]

[0136] A=m×2 e

[0137] Among them, A represents the true value, e represents the decoded value of the exponent field, m represents the decoded value of the significand field, and ε represents the unary exponent field. Represents the binary significant digit field, Represents a binary number consisting of 1 and the valid digit field in a bit string.

[0138] In some embodiments, before extracting the exponent field and the significand field, the method further includes:

[0139] The sign bit at the first position of the code is extracted, and the bit value of the sign bit is used as the decoded value of the sign bit.

[0140] In some embodiments, extracting the exponent field and the valid number field in the encoding according to the hyperparameter includes:

[0141] Extracting consecutive bits 1 that are no more than E bits after the sign bit as an exponent field, and the number of bits 1 is the value ε of the exponent field;

[0142] If the value of the exponent field is not equal to E, extract the separator after the exponent field and the valid number field after the separator, wherein the separator is a bit 0 and the valid number field is a binary number;

[0143] If the value of the exponent field is equal to E, a significant number field located after the exponent field is extracted, and the significant number field is a binary number.

[0144] The third aspect of the present invention provides an entropy coded integer code, comprising at least an exponent field and a significand field;

[0145] The exponent field uses unary to represent the exponent size;

[0146] The significant digit field uses binary to represent the significant digit size;

[0147] The encoding also has hyperparameters (N, E), where N represents the bit width of the encoding and E represents the maximum number of bits of the exponent field;

[0148] The decoding process of the encoding includes:

[0149] Extracting an exponent field and a valid number field in the code according to the hyperparameter;

[0150] Decoding and calculating the exponent field to obtain a decoded value of the exponent field, and decoding and calculating the significand field to obtain a decoded value of the significand field;

[0151] The true value of the code is calculated according to the value in the valid number field and the value in the exponent field, and the calculation formula is expressed as:

[0152]

[0153]

[0154] A=m×2 e

[0155] Among them, A represents the true value, e represents the decoded value of the exponent field, m represents the decoded value of the significand field, and ε represents the unary exponent field. Represents the binary significant digit field, Represents a binary number consisting of 1 and the valid digit field in a bit string.

[0156] In some embodiments, the code further includes a sign bit, and the sign bit is represented by a bit 0 or 1, where bit 0 indicates that the code is a positive number, and bit 1 indicates that the code is a negative number.

[0157] In some embodiments, the encoding includes, from left to right, the sign bit, the exponent field, and the significand field;

[0158] The corresponding relationship between the code and the real value is expressed as:

[0159] s=δ

[0160]

[0161]

[0162] A=(-1) δ ×m×2 e

[0163] Among them, A represents the real value, s represents the decoded value of the sign bit, e represents the decoded value of the exponent field, m represents the decoded value of the significand field, δ represents the sign bit, ε represents the exponent field, Indicates the valid number field, Represents a binary number consisting of 1 and the valid digit field in a bit string.

[0164] In some embodiments, the exponent field includes a number of consecutive bit 1s, the number of bit 1s in the number of consecutive bit 1s represents the value of the exponent field, and the value of the exponent field is less than or equal to E.

[0165] In some embodiments, if the decoded value of the exponent field is not equal to E, a separator is inserted between the exponent field and the significand field, and the separator is a bit 0;

[0166] If the value of the exponent field is equal to E, the significand field is directly concatenated after the exponent field.

[0167] In some embodiments, the multiplication between the two codes includes an exclusive-OR operation of the sign bit, an accumulation operation of the decoded value of the exponent field, and a product operation of the decoded value of the significand field.

[0168] Specifically, for example, two ECI numbers x and y are decoded as (s x , e x , m x ) d and y , e y , m y ) d, the product of x and y is stored and represented using an integer, and the calculation formula is expressed as:

[0169] Where S x and S y Represents the sign bits of x and y respectively, Indicates x and y Perform XOR operation, m x and m y Represents the effective number field of x and y respectively, e x and e y Represents the exponent fields of x and y respectively. It can be seen that the multiplication calculation between ECI numbers can be completed without fully decoding into integer format, which saves resources for the operation of quantized neural networks.

[0170] Nonlinear operations in neural networks usually require comparison operations. For example, the activation function ReLU and the MaxPooling operation both require comparison. Although the ECI number is first decoded into an integer and then the comparison operation is performed, since the decoded integer bit width will be nearly doubled, the comparison operation in the integer domain will take up more resources (area and power consumption), and the overhead of the decoding operation cannot be ignored. The comparison operation in the ECI domain of the present invention can be directly calculated using the existing integer comparison operation rules without decoding. Because for the same bit string, the ECI number and the integer it represents have a strict order-preserving relationship. Since the bit width of the ECI number is usually shorter than the number of bits in the integer domain, the comparison in the ECI domain can be implemented with a more efficient circuit.

[0171] The ECI dot product operation can be decomposed into the sign bit XOR, exponent accumulation, effective digit multiplication, complement operation, shift operation and reduction accumulation operation. The main hardware cost of implementing the dot product operation is determined by the effective digit multiplication operation and accumulation operation. Since the complexity (or cost) of the multiplication operation is proportional to the square of the bit width of the operand, that is, O(n 2 ) complexity, so the lower the bit width, the lower the cost of the multiplier.

[0172] The ECI number provided by the present invention has the following characteristics:

[0173] 1. When the hyperparameter E is fixed, all values ​​that ECI(N, E) can represent are unique except 0. The value zero has two legal representations, namely ±0.

[0174] 2. The maximum value that ECI (N, E) can represent is 2 N+E-2 (For the unsigned ECI number system, the maximum value is 2 N+E ); When E=N-1, the maximum value that ECI can represent is 22N-3 .

[0175] 3. By adjusting the hyperparameter E, the interval between adjacent ECI values ​​will be enlarged or reduced accordingly. The adjustment process is as follows: Figure 4 As shown, this is a set of ECI values ​​for different E values ​​when the bit width is fixed.

[0176] 4. When E=0 or 1, ECI(N,1), ECI(N,0) are equivalent to the traditional N-bit integer system. Therefore, the integer system with the same bit width can be regarded as a subset or special case of the ECI number system.

[0177] In a fourth aspect, an embodiment of the present invention provides a quantization method for a deep neural network model, for quantizing data in the deep neural network into the entropy coded integer code described in any one of the above items, wherein the deep neural network model is a model having L network layers, such as Figure 5 As shown, the following steps are included:

[0178] Obtaining a bit width N of the entropy coded integer code;

[0179] Determining optimal encoding hyperparameters for quantization of at least some of the network layers;

[0180] The data to be quantized of each network layer is obtained, and quantized according to the corresponding optimal coding format.

[0181] In some embodiments, the determining of the optimal coding format for each quantization of at least part of the network layers comprises the following steps:

[0182] S1. quantizing the network layer i in the deep neural network model according to the hyperparameters (N, E) of the entropy coded integer coding to obtain an i-layer separate quantization model, where E represents the maximum number of bits of the exponent field of the entropy coded integer coding;

[0183] S2, using the i-layer separate quantization model to perform the prediction task to obtain a prediction result;

[0184] S3, repeat steps S1-S2 until the value range of E is traversed, and the value range of E is 0, 1, 2, ..., N;

[0185] S4, determining the optimal value of E of the i-th layer according to the prediction results of each of the i-th layer individual quantization models;

[0186] S5. Repeat steps S3-S4 until all or part of the value range of i is traversed, and the value range of i is 1, 2, ..., L.

[0187] Preferably, the fifth aspect of the present invention provides a quantization method for a neural network model, which is used to quantize the parameters to be quantized of the neural network model into target quantization codes to obtain a quantization model, such as Figure 6 As shown, including:

[0188] Obtaining a coding value range and a value distribution of the target quantization code;

[0189] Calculating a quantization precision according to a preset range of the parameter to be quantized and the encoding value range, and determining a parameter quantization anchor point corresponding to a value of each target quantization code according to the quantization precision;

[0190] Determine the parameter quantization interval of any target quantization encoding by using the parameter quantization anchor point;

[0191] The parameter to be quantized is quantized according to the corresponding relationship between the parameter quantization interval and the target quantization code.

[0192] In some embodiments, the target quantization code comprises an entropy coded integer code, the entropy coded integer code comprising at least an exponent field and a significand field;

[0193] The exponent field uses unary to represent the exponent size;

[0194] The significant digit field uses binary to represent the significant digit size;

[0195] The hyper parameters of the entropy coded integer coding include the bit width and the maximum number of bits of the exponent field;

[0196] The corresponding relationship between the entropy coded integer code and the real value is expressed as:

[0197]

[0198]

[0199] A=m×2 e

[0200] Among them, A represents the true value, e represents the decoded value of the exponent field, m represents the decoded value of the significand field, and ε represents the unary exponent field. Represents the binary significant digit field, Represents a binary number consisting of 1 and the valid digit field in a bit string.

[0201] In some embodiments, the entropy coded integer code further includes a sign bit, the sign bit is represented by a bit 0 or 1, bit 0 represents that the code is a positive number, and bit 1 represents that the code is a negative number;

[0202] The corresponding relationship between the entropy coded integer code with the sign bit and the real value is expressed as:

[0203] s=δ

[0204]

[0205]

[0206] A=(-1) δ ×m×2 e

[0207] Among them, A represents the real value, s represents the decoded value of the sign bit, e represents the decoded value of the exponent field, m represents the decoded value of the significand field, δ represents the sign bit, ε represents the exponent field, Indicates the valid number field, Represents a binary number consisting of 1 and the valid digit field in a bit string.

[0208] In some embodiments, obtaining the encoding value range of the target quantization encoding includes:

[0209] Obtaining hyperparameters of the target quantization encoding;

[0210] The maximum value and the minimum value that can be represented by the entropy coded integer code are calculated according to the hyper parameters of the entropy coded integer code, and the calculation formula is expressed as:

[0211] For said entropy coded integer code with said sign bit,

[0212] max{ECI(N,E)}=2 N+E-2 , min{ECI(N,E)}=-2 N+E-2 ;or

[0213] For unsigned entropy coded integer encoding,

[0214] max{ECI(N,E)}=2 N+E , min{ECI(N,E)}=0,

[0215] Wherein ECI(N,E) represents the entropy coded integer encoding with hyperparameters (N,E), N represents the bit width, E represents the maximum number of bits of the exponent field, max{ECI(N,E)} represents the maximum value that ECI(N,E) can represent, and min{ECI(N,E)} represents the minimum value that ECI(N,E) can represent.

[0216] In some embodiments, calculating the quantization accuracy according to the preset range of the parameter to be quantized and the encoding value range, and determining the parameter quantization anchor point corresponding to the value of each target quantization code according to the quantization accuracy includes:

[0217] The preset range is [-a, a];

[0218] The calculation formula of the quantization accuracy is expressed as:

[0219] ulp=max{|max{ECI(N,E)}|,|min{ECI(N,E)}|} / a, where ulp represents the quantization accuracy, |max{ECI(N,E)}| represents the absolute value of the maximum value that ECI(N,E) can represent, and |min{ECI(N,E)}| represents the absolute value of the minimum value that ECI(N,E) can represent;

[0220] The corresponding parameter quantization anchor point is obtained by multiplying the value of each target quantization code by the quantization precision.

[0221] In some embodiments, a parameter quantization anchor point corresponding to the target quantization encoding is defined as a first parameter quantization anchor point, a parameter quantization anchor point adjacent to the right of the first parameter quantization anchor point is defined as a second parameter quantization anchor point, and when the first parameter quantization anchor point has no right adjacent parameter quantization anchor point, the second parameter quantization anchor point is defined to coincide with the first parameter quantization anchor point, and a parameter quantization anchor point adjacent to the left of the first parameter quantization anchor point is defined as a third parameter quantization anchor point, and when the first parameter quantization anchor point has no left adjacent parameter quantization anchor point, the third parameter quantization anchor point is defined to coincide with the first parameter quantization anchor point;

[0222] Determining a parameter quantization interval of any target quantization encoding by using the parameter quantization anchor point includes:

[0223] Calculating a right endpoint of the parameter quantization interval according to the first parameter quantization anchor point and the second parameter quantization anchor point, so that the right endpoint is a midpoint between the first parameter quantization anchor point and the second parameter quantization anchor point;

[0224] The left endpoint of the parameter quantization interval is calculated according to the first parameter quantization anchor point and the third parameter quantization anchor point, so that the left endpoint is the midpoint of the first parameter quantization anchor point and the third parameter quantization anchor point.

[0225] In some embodiments, quantizing the parameter to be quantized according to the correspondence between the parameter quantization interval and the target quantization code includes:

[0226] The parameter to be quantized that falls within the parameter quantization interval is mapped to the corresponding target quantization code.

[0227] In some embodiments, a comprehensive experiment is performed using a TSMC 28nm process, 1GHz frequency, 1.0V, 25C process library to obtain the following Figure 7-10Performance comparison results are shown.

[0228] Figure 7 and Figure 8 The energy consumption / area data of multipliers and adders with different bit widths and formats are given. It can be seen that the FP8 multiplier has a high area / energy consumption benefit due to the shortened bit width. However, in order to maintain sufficient accuracy, the FP8 multiplication results usually need to be accumulated using the FP32 format. It can be seen that the accumulation operation overhead of FP32 far exceeds the overhead of FP8 and ECI8 multipliers.

[0229] Fig. 9 and Fig.10 The overall power consumption and area performance comparison of the ECI accelerator and the FP8 accelerator are given respectively. As can be seen from the figure, although FP8 has better energy consumption / area than ECI logic in multiplier logic, the accumulation operation of FP32 requires 6.2× overhead, which offsets the performance advantage brought by the FP8 multiplier; on the other hand, since FP32 requires more pipeline stages and more logic, the register overhead also exceeds the register overhead of ECI. Therefore, whether it is area or energy consumption, or combinational circuit or register overhead, the overall performance of the ECI accelerator is better than that of the FP8 accelerator. The area efficiency of the ECI accelerator is 1.72 times that of the FP8 accelerator, and the energy efficiency is 1.36 times that of the FP8 accelerator.

[0230] The professionals should further realize that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented by electronic hardware, computer software, or a combination of the two. In order to clearly illustrate the interchangeability of hardware and software, the composition and steps of each example have been generally described in the above description according to the function. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professionals and technicians can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of the present invention.

[0231] The steps of the method or algorithm described in conjunction with the embodiments disclosed herein may be implemented using hardware, a software module executed by a processor, or a combination of the two. The software module may be placed in a random access memory (RAM), a memory, a read-only memory (ROM), an electrically programmable ROM, an electrically erasable programmable ROM, a register, a hard disk, a removable disk, a CD-ROM, or any other form of storage medium known in the art.

[0232] The specific implementation methods described above further illustrate the objectives, technical solutions and beneficial effects of the present invention in detail. It should be understood that the above description is only a specific implementation method of the present invention and is not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the scope of protection of the present invention.

Claims

1. A method for encoding an entropy coded integer, characterized in that: The code for generating the entropy coded integer, wherein the entropy coded integer code is used by the operator to perform an arithmetic operation, comprises: An exponent field is represented by a first numerical sequence, the first numerical sequence is unary, and a maximum number of bits of the exponent field is E; The significant number field is represented by a second numerical sequence, wherein the second numerical sequence is binary; Concatenate the first numerical sequence and the second numerical sequence into the N-bit code; The corresponding relationship between the encoding and the true value is expressed as: A=m×2 e Among them, A represents the real value, ε represents the exponential field, Indicates the valid number field, Represents a binary number consisting of 1 and the valid digit field in a bit string.

2. The method for encoding an entropy coded integer according to claim 1, characterized in that: The sign bit is represented by a 1-bit 0 or 1, where bit 0 represents that the code is a positive number, and bit 1 represents that the code is a negative number.

3. The method for encoding an entropy coded integer according to claim 2, characterized in that: From left to right in the code are the sign bit, the exponent field and the valid number field; The corresponding relationship between the code and the real value is expressed as: s=δ A=(-1) δ ×m×2 e Among them, A represents the real value, δ represents the sign bit, and ε represents the exponent field. Indicates the valid number field, Represents a binary number consisting of 1 and the valid digit field in a bit string.

4. The method for encoding an entropy coded integer according to claim 1, characterized in that: The exponent field includes a number of consecutive bit 1s, the number of bit 1s in the number of consecutive bit 1s represents the value of the exponent field, and the value of the exponent field is less than or equal to E.

5. The method for encoding an entropy coded integer according to claim 1, characterized in that: The step of connecting the first numerical sequence and the second numerical sequence into the N-bit code includes: If the value of the exponent field is not equal to E, the first numerical sequence and the second numerical sequence are connected in the following manner: the first numerical sequence, the separation symbol and the second numerical sequence are concatenated according to a bit string, and the separation symbol is bit 0; If the value of the exponent field is equal to E, the first numerical sequence and the second numerical sequence are connected in the following manner: the first numerical sequence and the second numerical sequence are concatenated according to a bit string.

6. A method for decoding an entropy coded integer, characterized in that: for decoding a code of an entropy coded integer into a corresponding decoded value, wherein the code of the entropy coded integer is used by an operator to perform an arithmetic operation; Selecting hyperparameters (N, E) of the encoding, where N represents the bit width of the encoding and E represents the maximum number of bits of the exponent field; Extracting an exponent field and a valid number field in the code according to the hyperparameter; Decoding and calculating the exponent field to obtain a decoded value of the exponent field, and decoding and calculating the significand field to obtain a decoded value of the significand field; The true value of the code is calculated according to the value in the valid number field and the value in the exponent field, and the calculation formula is expressed as: A=m×2 e Where A represents the true value, e represents the decoded value of the exponent field, m represents the decoded value of the significand field, and ε represents the unary exponent field. Represents the binary significant digit field, Represents a binary number consisting of 1 and the valid digit field in a bit string.

7. The method for decoding an entropy coded integer according to claim 6, characterized in that: Before extracting the exponent field and the valid number field, the method further includes: The sign bit at the first position of the code is extracted, and the bit value of the sign bit is used as the decoded value of the sign bit.

8. The method for decoding an entropy coded integer according to claim 7, characterized in that: The step of extracting an exponent field and a valid number field in the encoding according to the hyperparameter comprises: Extracting consecutive bits 1 that are no more than E bits after the sign bit as an exponent field, and the number of bits 1 is the value ε of the exponent field; If the value ε of the exponent field is not equal to E, extract the separator located after the exponent field and the valid number field located after the separator, wherein the separator is a bit 0 and the valid number field is a binary number; If the value of the exponent field is equal to E, a significant number field located after the exponent field is extracted, and the significant number field is a binary number.

9. An entropy coded integer code for use in an operator performing an arithmetic operation, characterized in that: At least include an exponent field and a significand field; The exponent field uses unary to represent the exponent value; The effective number field uses binary to represent the effective number field value; The encoding also has hyperparameters (N, E), where N represents the bit width of the encoding and E represents the maximum number of bits of the exponent field; The decoding process of the encoding includes: Extract the exponent field ε and the valid number field in the encoding according to the hyperparameter Decoding and calculating the exponent field to obtain a decoded value of the exponent field, and decoding and calculating the significand field to obtain a decoded value of the significand field; According to the value ε in the significand field and the value of the exponent field Calculate the true value of the code, the calculation formula is expressed as: A=m×2 e Where A represents the true value, e represents the decoded value of the exponent field, m represents the decoded value of the significand field, and ε represents the unary exponent field. Represents the binary significant digit field, Represents a binary number consisting of 1 and the valid digit field in a bit string.

10. The entropy coded integer code according to claim 9, characterized in that: The code also includes a sign bit, which is represented by a bit 0 or 1, where bit 0 indicates that the code is a positive number, and bit 1 indicates that the code is a negative number.

11. The entropy coded integer code according to claim 10, characterized in that: From left to right in the code are the sign bit, the exponent field and the valid number field; The corresponding relationship between the code and the real value is expressed as: s=δ A=(-1) δ ×m×2 e Among them, A represents the real value, s represents the decoded value of the sign bit, e represents the decoded value of the exponent field, m represents the decoded value of the significand field, δ represents the sign bit, ε represents the exponent field, Indicates the valid number field, Represents a binary number consisting of 1 and the valid digit field in a bit string.

12. The entropy coded integer code according to claim 9, characterized in that: The exponent field includes a number of consecutive bit 1s, the number of bit 1s in the number of consecutive bit 1s represents the value of the exponent field, and the value of the exponent field is less than or equal to E.

13. The entropy coded integer code according to claim 9, characterized in that: If the decoded value of the exponent field is not equal to E, a separator is inserted between the exponent field and the valid number field, and the separator is a bit 0; If the value of the exponent field is equal to E, the significand field is directly concatenated after the exponent field.

14. The entropy coded integer code according to claim 11, characterized in that: The multiplication between the two codes includes an exclusive OR operation of the sign bit, an accumulation operation of the decoded value of the exponent field, and a product operation of the decoded value of the significand field.

15. A quantization method for a deep neural network model, characterized in that: The method is used to quantize the data in the deep neural network into the entropy coded integer code according to any one of claims 9 to 14, wherein the entropy coded integer code is used for the operator to perform arithmetic operations, and the deep neural network model is a model with L network layers, comprising the following steps: Obtaining a bit width N of the entropy coded integer code; Determining optimal encoding hyperparameters for quantization of at least some of the network layers; The data to be quantized of each network layer is obtained, and quantized according to the corresponding optimal coding format.

16. The method for quantizing a deep neural network model according to claim 15, characterized in that: The determining of the optimal coding format for each quantization of at least part of the network layers comprises the following steps: S1. quantize the network layer i in the deep neural network model according to the selected hyperparameters (N, E) of the entropy coded integer coding to obtain an i-layer separate quantization model, where E represents the maximum number of bits of the exponent field of the entropy coded integer coding; S2, using the i-layer separate quantization model to perform the prediction task to obtain a prediction result; S3, repeat steps S1-S2 until the value range of E is traversed, and the value range of E is 0, 1, 2, ..., N; S4, comparing the prediction results of each of the i-layer individual quantization models to determine the optimal value of E of the i-layer; S5. Repeat steps S3-S4 until all or part of the value range of i is traversed, and the value range of i is 1, 2, ..., L.

Images