Floating-point number quantization method and apparatus, floating-point number dequantization method and apparatus, and device and storage medium

Abstract

The present disclosure belongs to the technical field of artificial intelligence. Provided are a floating-point number quantization method and apparatus, a floating-point number dequantization method and apparatus, and a device and a storage medium. The quantization method comprises: on the basis of a compression rule, determining quantization information corresponding to an exponent of each floating-point number among floating-point numbers to be quantized, wherein the quantization information corresponding to the exponent of each floating-point number comprises a first quantization value and / or a second quantization vale, and the compression rule comprises the rule whereby an exponent corresponding to quantization information which indicates a low compression level is greater than an exponent corresponding to quantization information which indicates a high compression level; and for each floating-point number among the floating-point numbers to be quantized, and on the basis of the first quantization value corresponding to the floating-point number, performing quantization processing on the mantissa of the floating-point number, and / or on the basis of the second quantization value corresponding to the floating-point number, performing quantization processing on the exponent of the floating-point number. By using the present disclosure, a required average error can be satisfied during quantization.

Description

Method, device, equipment and storage medium for floating-point quantization and dequantization

[0001] The present application claims priority from the Chinese patent application No. 202410685502.6, filed on May 29, 2024, and entitled "Method, device, equipment and storage medium for floating-point quantization and dequantization", the whole content of which is incorporated herein by reference. TECHNICAL FIELD

[0002] The present disclosure relates to the field of artificial intelligence (AI) technology, and particularly relates to a method, device, equipment and storage medium for floating-point quantization and dequantization. BACKGROUND

[0003] In the AI technology, in the large model training scenario, the training efficiency is limited by the network bandwidth, and in the large model inference scenario, the inference efficiency is limited by the memory bandwidth and the memory capacity. It can be seen that in the training and inference scenarios, the transmission and storage of data are the biggest factors affecting the efficiency, which leads to the idle of computing power and causes the bottleneck of overall performance. Therefore, it is necessary to reduce the data amount of the large model to improve the overall performance.

[0004] At present, there are many floating-point numbers in the data of the large model. The main way to reduce the data amount of the large model is to convert the floating-point numbers with high precision into fixed-point numbers, and then use the integer (int) data format to represent. For example, the half-precision floating-point (FP16) is quantized to int8. In this way, the data amount can be reduced, and a lower-precision multiplier is used for calculation. However, the int8 performs worse than the original FP16 in dynamic range and mantissa precision, resulting in a larger precision error. SUMMARY

[0005] The present disclosure provides a method, device, equipment and storage medium for floating-point quantization and dequantization, which can compress floating-point numbers and reduce precision error.

[0006] In a first aspect, the present disclosure provides a method for floating-point quantization, the method comprising: determining, based on a compression rule, quantization information corresponding to an exponent of each floating-point number in a plurality of floating-point numbers to be quantized, the quantization information corresponding to the exponent of each floating-point number comprising a first quantization value and / or a second quantization value, the compression rule comprising a rule indicating that an exponent to which quantization information of a lower compression degree belongs is greater than an exponent to which quantization information of a higher compression degree belongs; and performing, for each floating-point number in the plurality of floating-point numbers to be quantized, quantization processing on a mantissa of the floating-point number based on the first quantization value corresponding to the floating-point number and / or quantization processing on the exponent of the floating-point number based on the second quantization value corresponding to the floating-point number.

[0007] In the scheme shown in the present disclosure, when quantizing floating-point numbers, the exponent of a floating-point number is quantized based on the size of the exponent and the importance of each bit of the exponent, the larger the exponent, the more important the exponent, the lower the compression degree, and / or the mantissa of a floating-point number is quantized based on the size of the exponent and the importance of each bit of the mantissa, the larger the exponent, the more important the mantissa, the lower the compression degree. Based on this quantization rule, the more important the floating-point number, the lower the compression degree, the smaller the error, and the less important the floating-point number, the higher the compression degree, the larger the error, and an average error meeting a requirement can be obtained.

[0008] In an optional manner, the compression rule further comprises a specified compression rate corresponding to the plurality of floating-point numbers to be quantized. In this way, the compression rule further comprises a specified compression rate, and when quantizing, the compression rate of the floating-point numbers can be adjusted according to the size of the memory bandwidth and the computing power, thereby improving the computing efficiency.

[0009] In an optional manner, the determining, based on the compression rule, quantization information corresponding to an exponent of each floating-point number in a plurality of floating-point numbers to be quantized comprises: determining, based on the compression rule, a plurality of first exponent ranges and quantization information of each first exponent range corresponding to the plurality of floating-point numbers to be quantized; and determining, for each floating-point number in the plurality of floating-point numbers to be quantized, a first quantization value corresponding to the exponent of the floating-point number based on the quantization information of a first exponent range to which the exponent of the floating-point number belongs.

[0010] In the scheme shown in the present disclosure, when determining the first quantization value, the compression rule is used to divide the exponents of the floating-point numbers to be quantized to obtain a plurality of first exponent ranges and quantization information of each first exponent range, and the first quantization value corresponding to each floating-point number is determined based on the quantization information of the first exponent range to which the exponent of each floating-point number belongs, without the need to set a first quantization value for each exponent, thereby improving the efficiency of determining the first quantization value.

[0011] In one optional approach, among the plurality of first exponent ranges, the quantization information of the first exponent range with the smallest endpoint value indicates that the floating-point number is quantized to 0. For each first exponent range other than the first exponent range with the smallest endpoint value, the quantization information of the first exponent range includes the correspondence between the first quantization value of each exponent in the first exponent range and the global quantization value and exponent. Determining the first quantization value corresponding to the exponent of the floating-point number based on the quantization information of the first exponent range to which the exponent of the floating-point number belongs includes: if the exponent of the floating-point number belongs to the first exponent range with the smallest endpoint value, then based on the quantization information of the first exponent range to which the exponent of the floating-point number belongs, determining the first quantization value corresponding to the floating-point number as a quantization value indicating that the floating-point number is quantized to 0; if the exponent of the floating-point number does not belong to the first exponent range with the smallest endpoint value, then based on the quantization information of the first exponent range to which the exponent of the floating-point number belongs and the exponent of the floating-point number, determining the first quantization value corresponding to the floating-point number.

[0012] In the scheme disclosed herein, for floating-point numbers with small exponents, the entire floating-point number is quantized to 0, while for floating-point numbers with large exponents, the mantissa is quantized based on the first quantization value and is not quantized to 0. In this way, for floating-point numbers with small exponents, the absolute value of the floating-point number is also relatively small, which has a smaller impact on the calculation result. Even if the compression degree is high, the error is relatively small. For floating-point numbers with large exponents, the absolute value of the floating-point number is also relatively large, which has a larger impact on the calculation result. The compression degree is low, which can ensure the accuracy of the error.

[0013] In one optional approach, determining the first quantized value corresponding to the floating-point number based on the quantization information of the first exponent range to which the exponent of the floating-point number belongs and the exponent of the floating-point number includes: based on the formula δ1=δ2*2 (E-E1) The first quantization value corresponding to the floating-point number is determined, where δ1 is the first quantization value corresponding to the floating-point number, δ2 is the global quantization value, E is the minimum exponent in the first exponent range with the largest exponent, and E1 is the exponent of the floating-point number.

[0014] In the scheme disclosed herein, within the first exponent range, the larger the exponent, the smaller the first quantization value, and the smaller the error after tail quantization, resulting in a relatively small error caused by quantization.

[0015] In one optional approach, among the plurality of first exponent ranges, the quantization information of the first exponent range with the smallest endpoint value indicates that the floating-point number is quantized to 0, and the quantization information of each first exponent range other than the first exponent range with the smallest endpoint value includes a first quantization value; determining the first quantization value corresponding to the exponent of the floating-point number based on the quantization information of the first exponent range to which the exponent of the floating-point number belongs includes: if the exponent of the floating-point number belongs to the first exponent range with the smallest endpoint value, then based on the quantization information of the first exponent range to which the exponent of the floating-point number belongs, determining the first quantization value corresponding to the floating-point number as a quantization value indicating that the floating-point number is quantized to 0; if the exponent of the floating-point number does not belong to the first exponent range with the smallest endpoint value, then determining the first quantization value of the first exponent range to which the exponent of the floating-point number belongs as the first quantization value corresponding to the floating-point number.

[0016] In the scheme disclosed herein, for floating-point numbers with small exponents, the entire floating-point number is quantized to 0, while for floating-point numbers with large exponents, the mantissa is quantized based on the first quantization value and is not quantized to 0. In this way, for floating-point numbers with small exponents, the absolute value of the floating-point number is also relatively small, which has a smaller impact on the calculation result. Even if the compression degree is high, the error is relatively small. For floating-point numbers with large exponents, the absolute value of the floating-point number is also relatively large, which has a larger impact on the calculation result. The compression degree is low, which can ensure the accuracy of the error.

[0017] In one optional approach, quantizing the mantissa of the floating-point number based on the first quantization value corresponding to the floating-point number includes: if the first quantization value corresponding to the floating-point number indicates that the floating-point number is quantized to 0, then the floating-point number is quantized to 0; if the first quantization value of the floating-point number is not the quantization value indicating that the floating-point number is quantized to 0, then a first division result obtained by dividing the mantissa of the floating-point number by the first quantization value corresponding to the floating-point number is determined; based on the first division result, the mantissa of the quantized floating-point number is determined. In this way, when the floating-point number is not entirely quantized to 0, division can be used to calculate and quantize the mantissa.

[0018] In one optional approach, determining the quantization information corresponding to the exponent of each floating-point number in the floating-point number to be quantized based on the compression rules includes: determining multiple second exponent ranges corresponding to the floating-point number to be quantized and the quantization information of each second exponent range based on the compression rules; and for each floating-point number in the floating-point number to be quantized, determining the second quantization value corresponding to the exponent of the floating-point number based on the quantization information of the second exponent range to which the exponent of the floating-point number belongs.

[0019] In the scheme disclosed herein, when determining the second quantization value, a compression rule is first used to divide the exponent of the floating-point number to be quantized, resulting in multiple second exponent ranges and quantization information for each second exponent range. Based on the quantization information of the second exponent range to which the exponent of each floating-point number belongs, the second quantization value corresponding to each floating-point number is determined, without needing to set a second quantization value separately for each exponent, thus improving the efficiency of determining the second quantization value.

[0020] In one alternative approach, the quantization information for each second exponent range includes a second quantization value; determining the second quantization value corresponding to the exponent of the floating-point number based on the quantization information of the second exponent range to which the exponent of the floating-point number belongs includes: determining the second quantization value of the second exponent range to which the exponent of the floating-point number belongs as the second quantization value corresponding to the floating-point number.

[0021] In one optional approach, quantizing the exponent of the floating-point number based on the second quantized value corresponding to the floating-point number includes: determining a second division result obtained by dividing the exponent of the floating-point number by the second quantized value corresponding to the floating-point number; and determining the quantized exponent of the floating-point number based on the second division result. This allows the quantized exponent to be calculated using division.

[0022] In one alternative approach, the method further includes: performing entropy encoding on the quantized floating-point number to obtain compressed data of the floating-point number to be quantized. Thus, by performing entropy encoding after obtaining the quantized floating-point number, the compression ratio can be further improved.

[0023] Secondly, this disclosure provides a method for dequantizing floating-point numbers, the method comprising:

[0024] Obtain the quantization information corresponding to the exponent of each floating-point number in the floating-point number to be dequantized. The quantization information corresponding to the exponent of each floating-point number includes a first quantization value and / or a second quantization value. The quantization information corresponding to the exponent of each floating-point number is determined based on the compression rule and the exponent of the floating-point number before quantization. The compression rule includes a rule that indicates the exponent of quantization information with a low degree of compression is greater than the exponent of quantization information with a high degree of compression.

[0025] For each floating-point number to be dequantized, the mantissa of the floating-point number is dequantized based on the first quantization value corresponding to the floating-point number, and / or the exponent of the floating-point number is dequantized based on the second quantization value corresponding to the floating-point number.

[0026] In one alternative approach, the compression rule further includes a specified compression ratio corresponding to the floating-point number before quantization.

[0027] In one alternative approach, the exponent of the floating-point number before quantization corresponds to multiple first exponent ranges, which are determined based on the compression rule and the exponent of the floating-point number before quantization. For each floating-point number before quantization, the first quantization value corresponding to the exponent of the floating-point number is determined based on the quantization information of the first exponent range to which the exponent of the floating-point number belongs.

[0028] In one alternative approach, among the plurality of first exponent ranges, the quantization information of the first exponent range with the smallest endpoint value indicates that the floating-point number is quantized to 0. For each first exponent range other than the first exponent range with the smallest endpoint value, the quantization information of the first exponent range includes the correspondence between the first quantized value of each exponent in the first exponent range and the global quantized value and exponent.

[0029] The step of obtaining the quantization information corresponding to the exponent of each floating-point number in the floating-point number to be inverted includes:

[0030] For each floating-point number to be dequantized, if the floating-point number is 0, then the first quantization value corresponding to the floating-point number is determined to be the quantization value indicating that the floating-point number is quantized to 0;

[0031] If the floating-point number is not 0, then the exponent after the dequantization of the floating-point number is determined. Based on the quantization information of the first exponent range to which the dequantized exponent of the floating-point number belongs and the exponent of the floating-point number, the first quantization value corresponding to the floating-point number is determined.

[0032] In one optional approach, determining the first quantized value corresponding to the floating-point number based on the quantization information of the first exponent range to which the dequantized exponent of the floating-point number belongs and the exponent of the floating-point number includes:

[0033] Based on the formula δ1=δ2*2 (E-E1) The first quantization value corresponding to the floating-point number is determined, where δ1 is the first quantization value corresponding to the floating-point number, δ2 is the global quantization value, E is the minimum exponent in the first exponent range with the largest exponent, and E1 is the exponent after the floating-point number is dequantized.

[0034] In one alternative approach, among the plurality of first exponential ranges, the quantization information of the first exponential range with the smallest endpoint value indicates that the floating-point number is quantized to 0, and the quantization information of each first exponential range other than the first exponential range with the smallest endpoint value includes a first quantized value.

[0035] The step of obtaining the quantization information corresponding to the exponent of each floating-point number in the floating-point number to be inverted includes:

[0036] For each floating-point number to be dequantized, if the floating-point number is 0, then the first quantization value corresponding to the floating-point number is determined to be the quantization value indicating that the floating-point number is quantized to 0;

[0037] If the floating-point number is not 0, then the exponent after the dequantization of the floating-point number is determined, and the first quantization value of the first exponent range to which the dequantized exponent of the floating-point number belongs is determined, which is the first quantization value corresponding to the floating-point number.

[0038] In one alternative approach, the floating-point number to be dequantized before quantization corresponds to multiple second exponent ranges, which are determined based on the compression rule and the exponent of the floating-point number before quantization. For each floating-point number, the second quantization value corresponding to the exponent of the floating-point number is determined based on the quantization information of the second exponent range to which the exponent of the floating-point number belongs.

[0039] The step of obtaining the quantization information corresponding to the exponent of each floating-point number in the floating-point number to be inverted includes:

[0040] For each floating-point number to be dequantized, a second quantized value is determined based on the quantization information of the second exponent range corresponding to the exponent of the floating-point number.

[0041] In an alternative approach, the method further includes:

[0042] Get the compressed floating-point number;

[0043] The compressed floating-point number is subjected to entropy decoding to obtain the floating-point number to be dequantized.

[0044] In one optional approach, the step of dequantizing the mantissa of the floating-point number based on the first quantization value corresponding to the floating-point number includes: if the first quantization value corresponding to the floating-point number does not indicate that the floating-point number is quantized to 0, then determining the first multiplication result of multiplying the mantissa of the floating-point number by the first quantization value; and using the second multiplication result as the mantissa of the dequantized floating-point number.

[0045] In one optional approach, the step of dequantizing the mantissa of the floating-point number based on the second quantization value corresponding to the floating-point number includes: determining the second multiplication result of multiplying the exponent of the floating-point number by the second quantization value; and using the second multiplication result as the exponent of the dequantized floating-point number.

[0046] Thirdly, this disclosure provides an apparatus for floating-point quantization, the apparatus comprising one or more modules for implementing the floating-point quantization method provided in the first aspect or any alternative method of the first aspect.

[0047] In one alternative approach, the modules in the device are implemented in software, and the modules in the device are program modules. In another alternative approach, the modules in the device are implemented in hardware or firmware.

[0048] Fourthly, this disclosure provides an apparatus for dequantizing floating-point numbers, the apparatus comprising one or more modules for implementing the method for dequantizing floating-point numbers provided in the second aspect or any alternative method of the second aspect.

[0049] In one alternative approach, the modules in the device are implemented in software, and the modules in the device are program modules. In another alternative approach, the modules in the device are implemented in hardware or firmware.

[0050] Fifthly, this disclosure provides a computing device including a processor and a memory; the processor of the computing device is configured to execute instructions stored in the memory of the computing device to cause the computing device to perform the floating-point quantization method provided in the first aspect or any alternative method of the first aspect.

[0051] In a sixth aspect, this disclosure provides a computing device including a processor and a memory; the processor of the computing device is configured to execute instructions stored in the memory of the computing device to cause the computing device to perform the floating-point dequantization method provided in the second aspect or any alternative method of the second aspect.

[0052] In a seventh aspect, this disclosure provides a computer program product containing instructions that, when executed by a computing device, cause the computing device to perform the floating-point quantization method provided in the first aspect or any alternative method of the first aspect.

[0053] Eighthly, this disclosure provides a computer program product containing instructions that, when executed by a computing device, cause the computing device to perform a floating-point dequantization method as provided in either the second or first aspect above.

[0054] In a ninth aspect, this disclosure provides a computer-readable storage medium including computer program instructions that, when executed by a computing device, perform a floating-point quantization method as provided in the first aspect or any alternative method of the first aspect.

[0055] In a tenth aspect, this disclosure provides a computer-readable storage medium including computer program instructions that, when executed by a computing device, perform a floating-point dequantization method as provided in the second aspect or any alternative method of the second aspect.

[0056] In an eleventh aspect, this disclosure provides a chip that, when running on a computing device, causes the computing device to perform the floating-point quantization method provided in the first aspect or any alternative method of the first aspect.

[0057] In a twelfth aspect, this disclosure provides a chip that, when running on a computing device, causes the computing device to perform a floating-point dequantization method as provided in the second aspect or any alternative method of the second aspect above.

[0058] In a thirteenth aspect, this disclosure provides a quantization system comprising the apparatus as described in the third aspect and the apparatus as described in the fourth aspect.

[0059] In a fourteenth aspect, this disclosure provides a quantization apparatus, comprising a floating-point number format parser, an exponent quantizer, and a mantissa quantizer; the floating-point number format parser is used to split a floating-point number to be quantized into an exponent, a mantissa, and a sign; the exponent quantizer is used to obtain a first quantization value corresponding to the exponent of the floating-point number to be quantized, and based on the obtained first quantization value, to quantize the exponent of the floating-point number to be quantized to obtain an exponent-quantized floating-point number; the mantissa quantizer is used to obtain a second quantization value corresponding to the exponent of the floating-point number to be quantized, and based on the obtained second quantization value, to quantize the mantissa of the floating-point number to be quantized to obtain a mantissa-quantized floating-point number.

[0060] In a fifteenth aspect, this disclosure provides a dequantization device, comprising an exponent dequantizer, a mantissa dequantizer, and a floating-point format combiner; the exponent dequantizer is used to acquire a first quantization value corresponding to the exponent of the floating-point number to be dequantized, and based on the acquired first quantization value, to perform dequantization processing on the exponent of the floating-point number to be dequantized, to obtain a floating-point number after exponent dequantization; the mantissa dequantizer is used to acquire a second quantization value corresponding to the floating-point number to be dequantized, and based on the acquired second quantization value, to perform dequantization processing on the mantissa of the floating-point number to be dequantized, to obtain a floating-point number after mantissa dequantization; the floating-point format combiner is used to combine the floating-point number after exponent dequantization and the floating-point number after mantissa dequantization, to obtain a dequantized floating-point number. Attached Figure Description

[0061] Figure 1 is a schematic diagram of a floating-point number provided in an exemplary embodiment of this disclosure;

[0062] Figure 2 is a schematic diagram of the dynamic range of a floating-point number provided in an exemplary embodiment of this disclosure;

[0063] Figure 3 is a schematic diagram of the quantization comparison of floating-point numbers provided in an exemplary embodiment of this disclosure;

[0064] Figure 4 is a schematic diagram comparing the dynamic range of floating-point numbers and integers provided in an exemplary embodiment of this disclosure;

[0065] Figure 5 is a schematic diagram of the structure of a computing device provided in an exemplary embodiment of the present disclosure;

[0066] Figure 6 is a schematic flowchart of a floating-point quantization method provided in an exemplary embodiment of this disclosure;

[0067] Figure 7 is a schematic diagram of a quantization process provided in an exemplary embodiment of this disclosure;

[0068] Figure 8 is a schematic diagram of the last digit in physical space provided by an exemplary embodiment of this disclosure;

[0069] Figure 9 is a schematic diagram of the quantization mantissa provided in an exemplary embodiment of this disclosure;

[0070] Figure 10 is a schematic diagram of a quantization index provided in an exemplary embodiment of this disclosure;

[0071] Figure 11 is a schematic flowchart of a floating-point dequantization method provided in an exemplary embodiment of this disclosure;

[0072] Figure 12 is a schematic diagram of the inverse quantization process provided in an exemplary embodiment of this disclosure;

[0073] Figure 13 is a schematic diagram of the structure of a floating-point quantization apparatus provided in an exemplary embodiment of the present disclosure;

[0074] Figure 14 is a schematic diagram of the structure of a floating-point dequantization device provided in an exemplary embodiment of the present disclosure. Detailed Implementation

[0075] To make the objectives, technical solutions, and advantages of this disclosure clearer, the embodiments of this disclosure will be described in further detail below with reference to the accompanying drawings.

[0076] The following explains some terms and concepts involved in the embodiments of this disclosure.

[0077] 1. A floating-point number includes a sign bit, an exponent (also called an exponent code), and a mantissa. It can also be understood as including a sign bit, an exponent bit, and a mantissa bit. A floating-point number can be viewed as a three-dimensional vector or a two-dimensional vector. In a three-dimensional vector, the sign, exponent, and mantissa are represented; in a two-dimensional vector, the exponent and mantissa are represented, ignoring the sign bit. Alternatively, they can be represented as a combined number and a combined number, where the sign bit is combined with the exponent. Floating-point numbers include half-precision floating-point (FP16), single-precision floating-point (FP32), double-precision floating-point (FP64), and so on. See Figure 1; FP16 is a floating-point number stored using 16 bits, including 1 bit for the sign, 5 bits for the exponent, and 10 bits for the mantissa. Half-precision floating-point numbers can represent numbers with an absolute value range of approximately [6.10 x 10^-5, 6.55 x 10^4]. FP32 is a floating-point number stored using 32 bits, including 1 bit for the sign, 8 bits for the exponent, and 23 bits for the mantissa. Single-precision floating-point numbers can represent numbers with an absolute value range of approximately [1.18 x 10^-38, 3.40 x 10^38]. FP64 is a floating-point number stored using 64 bits, including 1 bit for the sign, 11 bits for the exponent, and 52 bits for the mantissa. Double-precision floating-point numbers can represent 15 or 16 significant decimal digits, with an absolute value range of approximately [2.23 x 10^-308, 1.80 x 10^308]. FP128 is a floating-point number stored using 128 bits, including 1 bit for the sign, 15 bits for the exponent, and 112 bits for the mantissa. Quad-precision floating-point numbers can represent 33 to 36 significant digits in decimal, and the absolute value range of the numbers they can represent is approximately [3.40 x 10^-4932, 1.18 x 10^4932].

[0078] 2. Compression refers to the process of reducing data size through specific encoding mechanisms.

[0079] 3. Floating-point quantization refers to the process of representing high-precision floating-point numbers as low-precision floating-point numbers.

[0080] 4. Entropy coding involves creating and assigning a unique prefix code to each input symbol, then replacing each fixed-length symbol with a corresponding variable-length, prefix-free output codeword, thereby compressing the data. The length of each codeword is approximately proportional to the negative logarithm of the probability. Therefore, the shortest codeword is used for the most common symbols. Huffman coding and arithmetic coding are two of the most common entropy coding techniques.

[0081] 5. The dynamic range of a floating-point number refers to the range of values ​​a floating-point number can take. The dynamic range differs depending on the precision of the floating-point number. For example, as shown in Figure 2, the dynamic range of a double-precision floating-point number is -2. 1023 to -2 -1022 ,2 -1022 Up to 2 1023 The dynamic range of a single-precision floating-point number is -2. 127 to -2 -126 ,2 -126 Up to 2 127 .

[0082] In AI technology, data transmission and storage are the most influential factors on efficiency in training and inference scenarios of large models (such as large language models, LLMs). Therefore, considering that the weight parameters in large models are floating-point numbers, the weight parameters in large models can be compressed to increase the effective amount of weight parameter reading, thereby improving training and inference efficiency. Moreover, improving data transmission efficiency can also improve the utilization rate of multiply-accumulate (MAC) in training or inference scenarios.

[0083] Currently, floating-point quantization compression techniques convert high-precision floating-point numbers to fixed-point numbers and then represent them using integer data formats. This reduces the data volume and allows for calculations using lower-precision multipliers. However, using fixed-point numbers results in significantly worse performance in terms of dynamic range and mantissa precision compared to the original floating-point numbers. For example, converting a half-precision floating-point number to int8 results in a larger dynamic range and mantissa precision compared to integer int8 (see Figures 3 and 4). After conversion, both the dynamic range and mantissa precision errors increase. Furthermore, further quantization compression to int4 limits the compression space, and the precision error becomes increasingly unacceptable. Therefore, a floating-point compression method that can be tailored to the required compression ratio is the best solution to address the performance limitations of large model computations.

[0084] Based on this, this disclosure provides a method for floating-point quantization. In this method, the exponent is compressed according to the importance of each bit of the exponent, and the mantissa is quantized according to the importance of each bit of the mantissa, thus obtaining a satisfactory average error. In this way, the floating-point compression ratio can be adjusted according to the memory bandwidth and computing power, thereby improving computational efficiency.

[0085] In this embodiment of the disclosure, the subject executing the floating-point quantization method can be a floating-point quantization device. This device can be a hardware device, such as a terminal or server, or a mobile terminal such as a tablet computer, or a desktop computer or a laptop computer. The device can also be a software device that runs on the hardware device.

[0086] Figure 5 provides a schematic diagram of the computing device. As shown in Figure 5, the computing device 500 includes a bus 502, a processor 504, a memory 506, and a communication interface 508. The processor 504, the memory 506, and the communication interface 508 communicate with each other via the bus 502. The computing device 500 can be a server or a terminal device. It should be understood that this disclosure does not limit the number of processors and memories in the computing device 500.

[0087] Bus 502 can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. Buses can be categorized as address buses, data buses, control buses, etc. For ease of illustration, only one line is used in Figure 5, but this does not imply that there is only one bus or one type of bus. Bus 504 can include pathways for transmitting information between various components of computing device 500 (e.g., memory 506, processor 504, communication interface 508).

[0088] Processor 504 may include any one or more processors such as a central processing unit (CPU), an AI processor, a graphics processing unit (GPU), a microprocessor (MP), or a digital signal processor (DSP).

[0089] Memory 506 may include volatile memory, such as random access memory (RAM). Memory 506 may also include non-volatile memory, such as read-only memory (ROM), flash memory, hard disk drive (HDD), or solid state drive (SSD).

[0090] The memory 506 stores executable program code, which the processor 504 executes to implement floating-point quantization or dequantization methods, respectively. That is, the memory 506 stores instructions for performing floating-point quantization or dequantization methods.

[0091] The communication interface 508 uses transceiver modules, such as, but not limited to, network interface cards and transceivers, to enable communication between the computing device 500 and other devices or communication networks.

[0092] The following describes the method flow for floating-point quantization. See steps 601 to 602 in Figure 6. Figure 6 uses a computing device as the execution subject for illustration.

[0093] Step 601: Based on the compression rule, determine the quantization information corresponding to the exponent of each floating-point number in the floating-point number to be quantized. The quantization information corresponding to the exponent of each floating-point number includes a first quantization value and / or a second quantization value. The compression rule includes a rule that indicates the exponent of quantization information with a low degree of compression is greater than the exponent of quantization information with a high degree of compression.

[0094] In this embodiment, the floating-point number to be quantized includes multiple floating-point numbers. These floating-point numbers can be all or part of the weight parameters of a large model; for example, they can be the weight parameters of a model layer within a large model. When determining the quantization information, the quantization information corresponding to the exponent of each floating-point number includes a first quantization value and / or a second quantization value. The first quantization value is used to quantize the mantissa, and the second quantization value is used to quantize the exponent. Quantization of the floating-point number to be quantized is based on the importance of the exponent bits and the mantissa bits. The larger the exponent, the lower the compression degree. That is, if the compression degree indicated by the quantization information corresponding to the first exponent is lower than the compression degree indicated by the quantization information corresponding to the second exponent, then the first exponent is greater than the second exponent.

[0095] It should be noted that if each index corresponds to a specific quantitative information, then the degree of compression will be different for different indices. If the indices correspond to quantitative information according to different regional ranges, then the degree of compression will be different for indices in different regional ranges.

[0096] Optionally, the compression rule also includes a specified compression ratio for the floating-point number to be quantized. This specified compression ratio is a ratio set for the floating-point number to be quantized, equal to the ratio of the data size of the compressed floating-point number to the data size of the floating-point number to be quantized. In other words, it represents the required compression ratio after quantization. The computing device uses the specified compression ratio to set quantization information for each exponent of the floating-point number to be quantized, ensuring that the floating-point number achieves this compression ratio after quantization. This specified compression ratio can be user-inputted or calculated by the computing device based on current computing power and memory.

[0097] Step 602: For each floating-point number to be quantized, the mantissa of the floating-point number is quantized based on the first quantization value corresponding to the floating-point number, and / or the exponent of the floating-point number is quantized based on the second quantization value corresponding to the floating-point number.

[0098] In this embodiment, during quantization, for each floating-point number, if only a first quantization value is determined when determining the quantization information, the first quantization value is used to quantize the mantissa of the floating-point number. If only a second quantization value is determined when determining the quantization information, the second quantization value is used to quantize the exponent of the floating-point number. If both a first quantization value and a second quantization value are determined when determining the quantization information, the first quantization value is used to quantize the mantissa of the floating-point number, and the second quantization value is used to quantize the exponent of the floating-point number. Thus, in this embodiment, a specified compression ratio can be achieved by quantizing only the exponent, only the mantissa, or both the exponent and mantissa.

[0099] Using the process shown in Figure 6, compression rules are used to determine the quantization information during quantization, so that the degree of compression after quantization can meet the requirements. Moreover, the larger the exponent, the lower the degree of compression, retaining more important information and achieving the required average error.

[0100] In one alternative approach, to improve compression performance, after obtaining the quantized floating-point number, entropy encoding is performed on the quantized floating-point number to further enhance the compression ratio. This disclosure does not limit the entropy encoding technique; for example, Huffman coding or arithmetic coding can be used.

[0101] It should be noted that if, after obtaining the quantized floating-point number, entropy encoding is performed on the quantized floating-point number, then the compression rate improvement brought by entropy encoding should be considered when calculating the quantized information.

[0102] In an alternative approach, Figure 7 also provides a schematic diagram of the quantization process framework, namely a structural diagram of a quantization device. The processor of the computing device includes a floating-point format parser, an exponent quantizer, a mantissa quantizer, and an entropy encoder. The floating-point format parser parses each floating-point number into an exponent, a mantissa, and a sign, sends the exponent to the exponent quantizer, the mantissa to the mantissa quantizer, and the sign to the entropy encoder. The mantissa quantizer uses the first quantization value corresponding to each floating-point number to quantize the mantissa of the floating-point number, and sends the quantized mantissa to the entropy encoder. The exponent quantizer uses the second quantization value corresponding to each floating-point number to quantize the exponent of the floating-point number, and sends the quantized exponent to the entropy encoder. The entropy encoder performs entropy encoding processing on the received data and outputs a compressed floating-point number.

[0103] Optionally, the tail quantizer uses the quantization information of the first exponent range to which the exponent of each floating-point number belongs to determine the first quantization value corresponding to each floating-point number, and the exponent quantizer uses the quantization information of the second exponent range to which the exponent of each floating-point number belongs to determine the second quantization value corresponding to each floating-point number. The detailed process will be explained later.

[0104] The processes of quantizing the mantissa and quantizing the index are described below.

[0105] 1. The process of quantifying the last digit.

[0106] First, let's describe the principle of mantissa quantization. In floating-point numbers, the importance of the mantissa is directly related to the size of the exponent. The larger the exponent, the more important the mantissa; conversely, the smaller the exponent, the less important the mantissa. This is because a floating-point number 'a' is represented using an exponent 'e' and a mantissa 'm': a = (-1)^s * m * b^e, where 's' is the sign and 'b' is the base. The larger 'e' is, the larger the absolute value of 'a', and the more important 'm' becomes. Conversely, the smaller 'e' is, the smaller the absolute value of 'a', and the less important 'm' becomes. Based on this principle, the compression level of floating-point numbers with large exponents is set smaller, while the compression level of floating-point numbers with small exponents is set larger. For example, as shown in Figure 8, for the floating-point number FP16, the larger the exponent, the closer the floating-point number is to the integer part, and the larger the floating-point number; conversely, the smaller the exponent, the closer the floating-point number is to the fractional part, and the smaller the floating-point number.

[0107] In this embodiment of the disclosure, two feasible methods for quantizing the mantissa are provided, as described below.

[0108] In one method, the computing device divides the exponent of the floating-point number to be quantized into multiple first exponent ranges. For a portion of the first exponent ranges, the first quantization values ​​of the exponents within each first exponent range are different. The number of multiple first exponent ranges is set according to actual needs, and this embodiment does not limit the number.

[0109] In one alternative approach, the computing device divides the exponent of the floating-point number to be quantized into multiple first exponent ranges and determines the quantization information for each first exponent range.

[0110] Specifically, the computing device determines the exponent distribution information of the floating-point number to be quantized, which records the frequency of each exponent of the floating-point number. Then, according to the exponent distribution information and compression rules, the exponent of the floating-point number to be quantized is divided into multiple first exponent ranges, and quantization information is set for each first exponent range. The multiple first exponent ranges include two or more first exponent ranges. For each first exponent range, the computing device uses the quantization information of that first exponent range to quantize the mantissa of the floating-point numbers belonging to that first exponent range, obtaining quantized floating-point numbers whose exponents belong to that first exponent range. The ratio of the data volume of the quantized floating-point numbers to the data volume of the floating-point numbers to be quantized is calculated to obtain the compression ratio obtained using the current quantization information. It is then determined whether the difference between this compression ratio and a specified compression ratio is less than a target threshold. The target threshold is relatively small and its size can be set according to actual needs. If it is less than the target threshold, the current quantization information is determined as the final quantization information. If it is not less than the target threshold, the quantization information of each first exponent range is updated, and the compression ratio is determined again until the difference between the compression ratio and the specified compression ratio is less than the target threshold, thus obtaining the final quantization information. Here, only the quantization information is adjusted during the adjustment process; however, the division method of multiple first exponent ranges can also be adjusted.

[0111] Alternatively, a computing device stores a first deterministic model containing quantization information. The input to this first deterministic model is exponential distribution information and a specified compression ratio. The output is multiple first exponential ranges and quantization information for each of these ranges. This first deterministic model is trained using multiple samples, each sample including exponential distribution information, a specified compression ratio, multiple exponential ranges, and quantization information corresponding to each exponential range. The exponential distribution information of the floating-point number to be quantized and the specified compression ratio are input to this first deterministic model, which outputs multiple first exponential ranges and quantization information for each first exponential range.

[0112] For each floating-point number to be quantized, the computing device determines a first exponent range to which the exponent of the floating-point number belongs, and determines a first quantization value corresponding to the floating-point number based on the quantization information of the first exponent range. The computing device uses the first quantization value to quantize the mantissa of the floating-point number.

[0113] Optionally, the multiple first exponent ranges include three first exponent ranges. In the first exponent range with the largest exponent, the exponent is equal to the largest exponent of the floating-point number to be quantized. In adjacent first exponent ranges, the exponent is greater than or equal to the largest exponent minus a target value, but less than the largest exponent. In the first exponent range with the smallest exponent, the exponent is less than the largest exponent minus the target value. The target value can be set according to actual needs. For example, if the largest exponent of the floating-point number to be quantized is 15, the target value could be 10, etc.

[0114] For example, when the floating-point number to be quantized is FP16, among the three first exponent ranges, the first exponent range with the largest exponent has an exponent of 15, the adjacent first exponent ranges have an exponent greater than or equal to 5 and less than 15, and the first exponent range with the smallest exponent has an exponent less than 5.

[0115] Optionally, considering that the floating-point numbers corresponding to the first exponent range with the smallest exponent have little impact on the calculation, the floating-point numbers belonging to the exponents in the first exponent range with the smallest exponent can be quantized to 0. However, considering that the floating-point numbers corresponding to the first exponent range with the largest exponent have a significant impact on the calculation, the floating-point numbers corresponding to the first exponent range with the largest exponent can be left unquantized.

[0116] In one alternative approach, for a first exponent range with the smallest exponent, the quantization information for that first exponent range indicates that the floating-point number is quantized to 0. For each first exponent range other than the first exponent range with the smallest exponent, the quantization information includes the correspondence between the first quantized value of each exponent in that first exponent range and the global quantized value and exponent, where the global quantized value is a preset empirical value. For example, the global quantized value is 1. For each floating-point number to be quantized, the computing device determines the first exponent range to which the exponent of the floating-point number belongs. If the first exponent range is the smallest exponent range, the floating-point number is quantized to 0. If the first exponent range is not the smallest exponent range, the quantization information of the first exponent range is used to calculate the first quantized value corresponding to the exponent of the floating-point number. The computing device divides the mantissa of the floating-point number by the first quantized value to obtain the first division result. The first division result is rounded to obtain the mantissa of the quantized floating-point number. Alternatively, the first division result is rounded to obtain the mantissa of the quantized floating-point number. Or, the first division result is rounded down or up to obtain the mantissa of the quantized floating-point number.

[0117] This explanation uses division as an example, but multiplication can also be used. This disclosure does not limit the specific implementation.

[0118] Optionally, for each first index range other than the first index range with the smallest index, the first quantized value corresponding to the target index in that first index range is represented as δ1 = δ2 * 2.(E-E1) The target index is any index within the first index range, δ1 is the first quantized value corresponding to the target index, δ2 is the global quantized value, E is the smallest index within the first index range with the largest index, and E1 is the target index. This formula shows that the larger the index, the smaller the first quantized value, and the lower the degree of mantissa compression.

[0119] As can be seen from the description of Method 1, in the first exponent range except for the first exponent range with the smallest mantissa, the first quantization value is related to the size of the exponent. In this way, each floating-point number can be reasonably quantized based on the size of the exponent.

[0120] Method 2: The computing device divides the exponent of the floating-point number to be quantized into multiple first exponent ranges. For each first exponent range, the first quantization value of each exponent in the first exponent range is the same.

[0121] In one alternative approach, the computing device divides the exponent of the floating-point number to be quantized into multiple first exponent ranges and determines quantization information for each first exponent range, the quantization information for each first exponent range including a first quantization value.

[0122] Specifically, the computing device determines the exponent distribution information of the floating-point number to be quantized, which records the frequency of each exponent of the floating-point number. Then, according to the exponent distribution information, the exponent of the floating-point number to be quantized is divided into multiple first exponent ranges, and quantization information is set for each first exponent range. These multiple first exponent ranges may include two or more first exponent ranges. For each first exponent range, the computing device uses the quantization information of that first exponent range to quantize the mantissa of the floating-point number belonging to that first exponent range, obtaining the quantized floating-point number within that first exponent range. The ratio of the data volume of the quantized floating-point number to the data volume of the floating-point number to be quantized is calculated to obtain the compression ratio obtained using the current quantization information. It is then determined whether the difference between this compression ratio and a specified compression ratio is less than a target threshold. The target threshold is relatively small, and its size is set according to actual needs. If it is less than the target threshold, the current quantization information is determined as the final quantization information. If it is not less than the target threshold, the quantization information of each first exponent range is updated, and the compression ratio is continued to be determined until the difference between the compression ratio and the specified compression ratio is less than the target threshold, thus obtaining the final quantization information. During the adjustment process here, only quantitative information is adjusted, but the division method of multiple first index ranges can also be adjusted.

[0123] Alternatively, a second deterministic model storing quantization information in a computing device. The second deterministic model takes exponential distribution information and a specified compression ratio as input, and outputs multiple first exponential ranges and quantization information for each of the first exponential ranges. This second deterministic model is trained using multiple samples, each sample including exponential distribution information, a specified compression ratio, multiple exponential ranges, and quantization information corresponding to each exponential range. The exponential distribution information of the floating-point number to be quantized and the specified compression ratio are input into this second deterministic model, which outputs multiple first exponential ranges and quantization information for each first exponential range.

[0124] For each floating-point number to be quantized, the computing device determines the first exponent range to which the exponent of the floating-point number belongs, and determines the first quantization value of the first exponent range as the first quantization value corresponding to the floating-point number. The computing device uses the first quantization value to quantize the mantissa of the floating-point number.

[0125] In one alternative approach, for the first exponent range with the smallest exponent, the first quantization value in the quantization information of this first exponent range indicates that the floating-point number is quantized to 0. For each first exponent range other than the first exponent range with the smallest exponent, the first quantization value included in the quantization information is a specific numerical value. For example, referring to Figure 9, the multiple first exponent ranges include three first exponent ranges, which are separated by a first threshold and a second threshold, where the first threshold is less than the second threshold. The first exponent range with the smallest exponent indicates that the floating-point number is quantized to 0, and the first quantization values ​​of the other two first exponent ranges are Δ1 and Δ2, where Δ1 is greater than Δ2.

[0126] When the first exponent range includes the first quantized value, the tail quantization method is as follows:

[0127] For each floating-point number to be quantized, the computing device determines the first exponent range to which the exponent of the floating-point number belongs. If the first exponent range is the smallest exponent range, the floating-point number is quantized to 0. If the first exponent range is not the smallest exponent range, the first quantization value of the first exponent range is determined as the first quantization value corresponding to the floating-point number. The computing device divides the mantissa of the floating-point number by the first quantization value to obtain the first division result. The first division result is then rounded to obtain the mantissa of the quantized floating-point number, or the first division result is rounded to obtain the mantissa of the quantized floating-point number, or the first division result is rounded down or up to obtain the mantissa of the quantized floating-point number.

[0128] This explanation uses division as an example, but multiplication can also be used. This disclosure does not limit the specific implementation.

[0129] 2. The process of quantifying indices.

[0130] First, let's describe the principle of quantization exponents. In floating-point numbers, the exponent is considered a logarithmic encoding method. The precision under different exponents is not uniform. You can understand it as increasing the exponent by one bit not increasing the distance by an equal amount. In floating-point numbers, the larger the exponent, the more space the floating-point number occupies in physical memory. Therefore, you can set the compression degree of the exponent for floating-point numbers with larger exponents to be smaller, and the compression degree of the exponent for floating-point numbers with smaller exponents to be larger.

[0131] In one alternative approach, the computing device divides the exponent of the floating-point number to be quantized into multiple second exponent ranges and determines the quantization information for each second exponent range. The number of multiple second exponent ranges is set according to actual needs, and this embodiment of the present disclosure does not limit it. The second exponent range may be the same as or different from the first exponent range.

[0132] Specifically, the computing device determines the exponent distribution information of the floating-point number to be quantized, which records the frequency of each exponent of the floating-point number. Then, according to the exponent distribution information, the exponent of the floating-point number to be quantized is divided into multiple second exponent ranges, and quantization information is set for each second exponent range. These multiple second exponent ranges may include two or more second exponent ranges. For each second exponent range, the computing device uses the quantization information of that second exponent range to quantize the exponents within that range, obtaining the quantized floating-point numbers. The ratio of the data volume of the quantized floating-point numbers to the data volume of the floating-point number to be quantized is calculated to obtain the compression ratio obtained using the current quantization information. It is then determined whether the difference between this compression ratio and a specified compression ratio is less than a target threshold. If the target threshold is relatively small, and the difference is less than the target threshold, the current quantization information is determined as the final quantization information. If the difference is not less than the target threshold, the quantization information for each second exponent range is updated, and the compression ratio is determined again until the difference between the compression ratio and the specified compression ratio is less than the target threshold, thus obtaining the final quantization information. During the adjustment process here, only quantitative information is adjusted, but multiple second index ranges can also be adjusted.

[0133] Alternatively, the computing device stores a third determining model for the second quantized values. This third determining model takes exponential distribution information and a specified compression ratio as input, and outputs multiple second exponential ranges and their corresponding second quantized values. This third determining model is trained using multiple samples, each sample including exponential distribution information, a specified compression ratio, multiple exponential ranges, and quantization information corresponding to each exponential range. The exponential distribution information of the floating-point number to be quantized and the specified compression ratio are input into this third determining model, which outputs multiple second exponential ranges and their corresponding quantization information.

[0134] For each floating-point number to be quantized, the computing device determines a second exponent range to which the exponent of that floating-point number belongs. Based on the quantization information of this second exponent range, the computing device determines a second quantization value corresponding to that floating-point number. The computing device then uses this second quantization value to quantize the exponent of the floating-point number.

[0135] In one alternative approach, for each second index range, the quantization information includes a second quantization value. For example, referring to Figure 10, the multiple second index ranges include three second index ranges separated by a third threshold and a fourth threshold, and the second quantization values ​​of the three second index ranges are Δ3, Δ4, and Δ5, which decrease sequentially.

[0136] When the quantized information includes a second quantized value, the method of exponential quantization is as follows:

[0137] In multiple second exponent ranges, the exponent of the first second exponent range is greater than the exponent of the second second exponent range, the second quantization value of the first second exponent range is less than the second quantization value of the second second exponent range, and the second quantization values ​​of all multiple second exponent ranges are greater than or equal to 1. For each floating-point number to be quantized, the computing device determines the second exponent range to which the exponent of the floating-point number belongs, and determines the second quantization value of the second exponent range as the second quantization value corresponding to the floating-point number. The computing device divides the exponent of the floating-point number by the second quantization value to obtain the second division result, and rounds the second division result to obtain the quantized exponent of the floating-point number, or rounds the second division result down or up to obtain the quantized exponent of the floating-point number.

[0138] This explanation uses division as an example, but multiplication can also be used. This disclosure does not limit the specific implementation.

[0139] Optionally, the exponent of the second exponent range with the smallest exponent is not important. The second quantization value of the second exponent range with the smallest exponent is infinity, which means that the exponent of the floating-point number is 0 after quantization, indicating that the exponent of the floating-point number has become smaller after quantization and has been compressed. The exponent of the second exponent range with the largest exponent is the most important. The second quantization value of the second exponent range with the largest exponent is 1, which means that the exponent of the floating-point number has not changed after quantization. The second quantization value of the remaining second exponent ranges is greater than 1, which means that the exponent of the floating-point number has become smaller after quantization and has been compressed.

[0140] 3. The process of quantifying the index and the last digit.

[0141] In the preceding text, floating-point quantization separately quantizes the exponent and mantissa of a floating-point number. Alternatively, in another approach, both the exponent and mantissa can be quantized simultaneously. For each exponent, the corresponding quantization information includes a first quantization value and a second quantization value. Therefore, when determining the quantization information, both the first and second quantization values ​​can be determined simultaneously, enabling the use of these values ​​to achieve the specified compression ratio after quantization of the floating-point number. The principle by which the computing device determines the first and second quantization values ​​is the same as described earlier and will not be repeated here.

[0142] For each floating-point number to be quantized, the computing device determines the first exponent range to which the exponent belongs, and uses the quantization information of the first exponent range to quantize the mantissa of the floating-point number. The computing device also determines the second exponent range to which the exponent belongs, and uses the quantization information of the second exponent range to quantize the exponent of the floating-point number. Finally, the quantized floating-point number is obtained. The quantization process is described above and will not be repeated here.

[0143] As described above, the floating-point number to be quantized is divided into an exponent range to determine the quantization information. In another optional approach, the quantization information is directly determined for each exponent of the floating-point number to be quantized.

[0144] The following describes the process flow of floating-point dequantization. Refer to steps 1101 to 1102 in Figure 11. The execution entity for the floating-point dequantization method can be a device performing the dequantization. This device can be a hardware device, such as a terminal or server, or a mobile terminal like a tablet, or a desktop or laptop computer. Alternatively, it can be a software device running on the hardware device. The structure of the computing device is shown in Figure 5. Figure 11 uses a computing device as the execution entity as an example. The computing devices performing quantization and dequantization can be the same or different devices.

[0145] Step 1101: The computing device obtains the quantization information corresponding to the exponent of each floating-point number in the floating-point number to be dequantized. The quantization information corresponding to the exponent of each floating-point number includes a first quantization value and / or a second quantization value. The quantization information corresponding to the exponent of each floating-point number is determined based on the compression rules and the exponent of the floating-point number before quantization. The compression rules include rules that indicate that the exponent of quantization information with a low degree of compression is greater than the exponent of quantization information with a high degree of compression.

[0146] In this embodiment, after quantizing the floating-point number to be quantized, the computing device performing quantization sends the quantization information to the computing device performing dequantization. In this way, the computing device performing dequantization can obtain the quantization information corresponding to the exponent of each floating-point number.

[0147] Optionally, when the quantization information corresponding to the index is the quantization information of the index range to which it belongs, the endpoint value of the index range can be sent when sending the quantization information corresponding to the index.

[0148] Step 1102: For each floating-point number to be dequantized, dequantize the mantissa of the floating-point number based on the first quantization value corresponding to the floating-point number, and / or dequantize the exponent of the floating-point number based on the second quantization value corresponding to the floating-point number.

[0149] In this embodiment, during dequantization, for each floating-point number, if it corresponds only to a first quantization value, the first quantization value is used to dequantize the mantissa of the floating-point number; if it corresponds only to a second quantization value, the second quantization value is used to dequantize the exponent of the floating-point number; if it corresponds to both a first quantization value and a second quantization value, the first quantization value is used to dequantize the mantissa of the floating-point number, and the second quantization value is used to dequantize the exponent of the floating-point number. Based on this dequantization process, a dequantized floating-point number is obtained, which can then be used for subsequent calculations, such as training or inference of large models.

[0150] In one alternative approach, during quantization, to improve compression, after obtaining the quantized floating-point number, entropy encoding is performed on it to further enhance the compression ratio. The quantization processing device then sends the entropy encoding method to the dequantization processing device; alternatively, the dequantization processing device is pre-configured with an entropy decoding method that corresponds to the entropy encoding method used by the quantization processing device.

[0151] The computing device that performs dequantization obtains the compressed floating-point number, performs entropy decoding on the compressed floating-point number, and obtains the floating-point number to be dequantized.

[0152] Figure 12 also provides a schematic diagram of the dequantization process, i.e., a structural diagram of a dequantization device. The processor of the computing device includes an entropy decoder, an exponent dequantizer, a mantissa dequantizer, and a floating-point format combiner. The entropy decoder obtains the compressed floating-point number, performs entropy decoding on the compressed floating-point number to obtain the exponent, mantissa, and sign, sends the sign to the floating-point format combiner, sends the exponent to the exponent dequantizer, and sends the mantissa to the mantissa dequantizer. The exponent dequantizer uses a second quantization value to dequantize the exponent, obtaining the dequantized exponent, and sends the dequantized exponent to the floating-point format combiner. The mantissa dequantizer uses a first quantization value to dequantize the mantissa, obtaining the dequantized mantissa, and sends the dequantized mantissa to the floating-point format combiner. The floating-point format combiner combines the sign, mantissa, and exponent to obtain the dequantized floating-point number.

[0153] Optionally, the mantissa dequantizer uses the quantization information of the first exponent range to which the exponent of each floating-point number belongs to determine the first quantization value corresponding to each floating-point number, and the exponent dequantizer uses the quantization information of the second exponent range to which the exponent of each floating-point number belongs to determine the second quantization value corresponding to each floating-point number. The detailed process will be explained later.

[0154] The following describes the process of dequantizing the exponent and the mantissa, based on the quantization process mentioned above.

[0155] 1. The process of dequantizing the mantissa.

[0156] In the case of quantization processing in mode one, for each floating-point number to be dequantized, if the floating-point number is 0, then the dequantized floating-point number is determined to be 0. If the floating-point number is not 0, then the first exponent range to which the dequantized exponent belongs is determined. Based on the quantization information corresponding to the first exponent range, the first quantization value corresponding to the floating-point number is calculated. Using the first quantization value corresponding to the floating-point number, the mantissa of the floating-point number is dequantized to obtain the dequantized floating-point number.

[0157] Optionally, for each first index range other than the first index range with the smallest index, the first quantized value corresponding to the target index in that first index range is represented as δ1 = δ2 * 2. (E-E1) The target index is any index within the first index range, δ1 is the first quantized value of the target index, δ2 is the global quantized value, E is the smallest index within the first index range with the largest index, and E1 is the target index. This formula shows that the larger the index, the smaller the first quantized value, and the lower the degree of mantissa compression.

[0158] When defining quantization information in this way, during dequantization of floating-point numbers, for each floating-point number, if its exponent is quantized, the dequantized exponent is determined. Using the dequantized exponent, the first exponent range to which the dequantized exponent belongs is determined among multiple first exponent ranges. If the floating-point number's exponent is not quantized, its exponent is used, and the first exponent range to which the floating-point number's exponent belongs is determined among multiple first exponent ranges. Based on the quantization information of the first exponent range, the floating-point number is then quantized.

[0159] In the case of quantization processing method two, for each floating-point number to be dequantized, if the floating-point number is 0, then the dequantized floating-point number is determined to be 0. If the floating-point number is not 0, then the first exponent range to which the dequantized exponent belongs is determined, and the first quantization value in the quantization information of the first exponent range is determined as the first quantization value corresponding to the floating-point number. Using the first quantization value, the mantissa of the floating-point number is dequantized to obtain the dequantized floating-point number.

[0160] Optionally, for each floating-point number, after determining the first quantization value corresponding to the floating-point number, the first quantization value of the floating-point number is used to dequantize the mantissa of the floating-point number to obtain the dequantized mantissa.

[0161] Alternatively, in both cases, the process of dequantizing the mantissa of the floating-point number using the first quantization value is as follows:

[0162] For each floating-point number, the mantissa of the floating-point number is multiplied by the first quantization value to obtain the multiplication result, which is called the first multiplication result. The first multiplication result is used as the mantissa of the dequantized floating-point number.

[0163] 2. The process of inverse quantification.

[0164] For each floating-point number to be dequantized, determine the second exponent range corresponding to the exponent of that floating-point number. For example, the quantization information of each second exponent range includes a second quantized value. For each second exponent range, divide the endpoint value of the second exponent range by the second quantized value of the second exponent range, and then perform a rounding operation to obtain the two endpoint values ​​corresponding to the second exponent range, forming a third exponent range. In this way, the third exponent range corresponding to each second exponent range can be obtained. The rounding operation here is the same as the rounding operation when quantizing the exponent. Then, for each floating-point number to be dequantized, determine the third exponent range to which the exponent of the floating-point number belongs, and determine the second exponent range corresponding to the third exponent range as the second exponent range corresponding to the exponent of the floating-point number. Based on the quantization information of the second exponent range, determine the second quantized value corresponding to the floating-point number. Using the second quantized value corresponding to the floating-point number, perform dequantization on the exponent of the floating-point number to obtain the dequantized floating-point number. For the dequantization mantissa mentioned above, this method can also be used to determine the first exponent range corresponding to each floating-point number to be dequantized.

[0165] Optionally, for each floating-point number, the second quantization value of the second exponent range to which the exponent of the floating-point number belongs is determined as the second quantization value corresponding to the floating-point number.

[0166] Optionally, for each floating-point number, after determining the second quantization value corresponding to the floating-point number, the exponent of the floating-point number is dequantized using the second quantization value to obtain the dequantized exponent.

[0167] Optionally, the process of dequantizing the exponent of the floating-point number using the second quantization value is as follows:

[0168] For each floating-point number, the exponent of the floating-point number is multiplied by the second quantization value to obtain the multiplication result, which is called the second multiplication result. The second multiplication result is used as the exponent of the dequantized floating-point number.

[0169] 3. The process of reverse quantification index and tail number.

[0170] When quantizing floating-point numbers, both the exponent and mantissa are quantized simultaneously. Similarly, when dequantizing the exponent and mantissa, both the exponent and mantissa are also dequantized. The process of dequantizing the exponent and mantissa is described in the previous text and will not be repeated here.

[0171] This disclosure also provides another dequantization method. In this method, instead of using quantization information, the exponent is dequantized based on the length of the exponent bits, and the mantissa is dequantized based on the length of the mantissa bits. For example, if the length of the exponent bits in a half-precision floating-point number is 5, but it is now 4, then 1 bit needs to be padded during dequantization, which can be done by padding with 0 or 1.

[0172] The quantization and dequantization methods in this disclosure can also be applied to chips, including but not limited to CPUs, AI processors, or GPUs. These chips all have arithmetic logic units (ALUs), which include floating-point multipliers and accumulators. Using the quantization methods described above can further improve computational performance and increase end-to-end computational throughput in floating-point computation scenarios where memory bandwidth is limited. For example, in some scenarios, the utilization rate of multipliers and accumulators in chips is relatively low, but due to insufficient memory bandwidth, it is impossible to improve computational performance. Adding floating-point compression and decompression modules to these chips can increase the throughput of floating-point reading and storage by tens of times under the same bandwidth, and the accuracy error in AI computation is within a tolerable range. In this way, the end-to-end computational performance can be improved by several to tens of times.

[0173] The apparatus provided in the embodiments of this disclosure is described below.

[0174] Figure 13 is a structural diagram of the floating-point quantization apparatus provided in an embodiment of this disclosure. This apparatus can be implemented as part or all of the device through software, hardware, or a combination of both. The apparatus provided in this embodiment can implement the process described in Figure 6 of this embodiment. The apparatus includes: a determination module 1310 and a quantization module 1320, wherein:

[0175] The determination module 1310 is used to determine the quantization information corresponding to the exponent of each floating-point number in the floating-point number to be quantized based on the compression rules. The quantization information corresponding to the exponent of each floating-point number includes a first quantization value and / or a second quantization value. The compression rules include rules that indicate that the exponent of the quantization information with a low degree of compression is greater than the exponent of the quantization information with a high degree of compression. Specifically, it can be used to implement the determination function of step 601 and execute the implicit steps included in step 601.

[0176] The quantization module 1320 is used to, for each floating-point number to be quantized, quantize the mantissa of the floating-point number based on a first quantization value corresponding to the floating-point number, and / or quantize the exponent of the floating-point number based on a second quantization value corresponding to the floating-point number. Specifically, it can be used to implement the quantization function of step 602 and execute the implicit steps included in step 602. Here, the quantization module 1320 may include the exponent quantizer and mantissa quantizer mentioned above.

[0177] In one alternative approach, the compression rule further includes a specified compression ratio corresponding to the floating-point number to be quantized.

[0178] In an alternative embodiment, the determining module 1310:

[0179] Based on the compression rules, multiple first exponent ranges corresponding to the floating-point number to be quantized and the quantization information of each first exponent range are determined.

[0180] For each floating-point number to be quantized, a first quantized value corresponding to the exponent of the floating-point number is determined based on the quantization information of the first exponent range to which the exponent of the floating-point number belongs.

[0181] In one alternative approach, among the plurality of first exponent ranges, the quantization information of the first exponent range with the smallest endpoint value indicates that the floating-point number is quantized to 0. For each first exponent range other than the first exponent range with the smallest endpoint value, the quantization information of the first exponent range includes the correspondence between the first quantized value of each exponent in the first exponent range and the global quantized value and exponent.

[0182] The determining module 1310 is used for:

[0183] If the exponent of the floating-point number belongs to the first exponent range with the smallest endpoint value, then based on the quantization information of the first exponent range to which the exponent of the floating-point number belongs, the first quantization value corresponding to the floating-point number is determined to be the quantization value indicating that the floating-point number is quantized to 0.

[0184] If the exponent of the floating-point number does not belong to the first exponent range with the smallest endpoint value, then the first quantized value corresponding to the floating-point number is determined based on the quantization information of the first exponent range to which the exponent of the floating-point number belongs and the exponent of the floating-point number.

[0185] In an alternative embodiment, the determining module 1310 is configured to:

[0186] Based on the formula δ1=δ2*2 (E-E1) The first quantization value corresponding to the floating-point number is determined, where δ1 is the first quantization value corresponding to the floating-point number, δ2 is the global quantization value, E is the minimum exponent in the first exponent range with the largest exponent, and E1 is the exponent of the floating-point number.

[0187] In one alternative approach, among the plurality of first exponential ranges, the quantization information of the first exponential range with the smallest endpoint value indicates that the floating-point number is quantized to 0, and the quantization information of each first exponential range other than the first exponential range with the smallest endpoint value includes a first quantized value.

[0188] The determining module 1310 is used for:

[0189] If the exponent of the floating-point number belongs to the first exponent range with the smallest endpoint value, then based on the quantization information of the first exponent range to which the exponent of the floating-point number belongs, the first quantization value corresponding to the floating-point number is determined to be the quantization value indicating that the floating-point number is quantized to 0.

[0190] If the exponent of the floating-point number does not belong to the first exponent range with the smallest endpoint value, then the first quantization value of the first exponent range to which the exponent of the floating-point number belongs is determined as the first quantization value corresponding to the floating-point number.

[0191] In an alternative embodiment, the quantization module 1320 is configured to:

[0192] The step of quantizing the mantissa of the floating-point number based on the first quantization value corresponding to the floating-point number includes:

[0193] If the first quantization value corresponding to the floating-point number is a quantization value that indicates the floating-point number to be quantized to 0, then the floating-point number is quantized to 0;

[0194] If the first quantization value of the floating-point number is not a quantization value that indicates the floating-point number to be quantized to 0, then determine the first division result obtained by dividing the mantissa of the floating-point number by the first quantization value corresponding to the floating-point number.

[0195] Based on the result of the first division, the mantissa of the quantized floating-point number is determined.

[0196] In an alternative embodiment, the determining module 1310 is configured to:

[0197] Based on the compression rules, multiple second exponent ranges corresponding to the floating-point number to be quantized and the quantization information of each second exponent range are determined;

[0198] For each floating-point number to be quantized, a second quantized value corresponding to the exponent of the floating-point number is determined based on the quantization information of the second exponent range to which the exponent of the floating-point number belongs.

[0199] In one alternative approach, the quantization information for each second exponential range includes a second quantization value;

[0200] The determining module 1310 is used for:

[0201] The second quantization value of the second exponent range to which the exponent of the floating-point number belongs is determined as the second quantization value corresponding to the floating-point number.

[0202] In an alternative embodiment, the quantization module 1320 is configured to:

[0203] Determine the second division result obtained by dividing the exponent of the floating-point number by the second quantized value corresponding to the floating-point number;

[0204] Based on the result of the second division, the quantized exponent of the floating point is determined.

[0205] In an alternative embodiment, the quantization module 1320 is also used for:

[0206] The quantized floating-point number is entropy encoded to obtain the compressed data of the floating-point number to be quantized.

[0207] For a detailed description of the floating-point quantization process performed by the device shown in Figure 13, please refer to the descriptions in the preceding embodiments; it will not be repeated here. The floating-point quantization device shown in Figure 13 is the computing device 500 in Figure 5.

[0208] Figure 14 is a structural diagram of the floating-point dequantization device provided in an embodiment of this disclosure. This device can be implemented as part or all of the device through software, hardware, or a combination of both. The device provided in this embodiment can implement the process described in Figure 10 of this embodiment. The device includes: an acquisition module 1410 and a dequantization module 1420, wherein:

[0209] The acquisition module 1410 is used to acquire the quantization information corresponding to the exponent of each floating-point number in the floating-point number to be dequantized. The quantization information corresponding to the exponent of each floating-point number includes a first quantization value and / or a second quantization value. The quantization information corresponding to the exponent of each floating-point number is determined based on the compression rule and the exponent of the floating-point number before quantization. The compression rule includes a rule that indicates that the exponent of the quantization information with a low degree of compression is greater than the exponent of the quantization information with a high degree of compression. Specifically, it can be used to implement the determination function of step 1101 and execute the implicit steps included in step 1101.

[0210] The dequantization module 1420 is used to, for each floating-point number to be dequantized, perform dequantization processing on the mantissa of the floating-point number based on the first quantization value corresponding to the floating-point number, and / or perform dequantization processing on the exponent of the floating-point number based on the second quantization value corresponding to the floating-point number. Specifically, it can be used to implement the dequantization function of step 1102 and execute the implicit steps included in step 1102. Here, the dequantization module 1420 may include the exponent dequantizer and mantissa dequantizer mentioned above.

[0211] In one alternative approach, the compression rule further includes a specified compression ratio corresponding to the floating-point number before quantization.

[0212] In one alternative approach, the exponent of the floating-point number before quantization corresponds to multiple first exponent ranges, which are determined based on the compression rule and the exponent of the floating-point number before quantization. For each floating-point number before quantization, the first quantization value corresponding to the exponent of the floating-point number is determined based on the quantization information of the first exponent range to which the exponent of the floating-point number belongs.

[0213] In one alternative approach, among the plurality of first exponent ranges, the quantization information of the first exponent range with the smallest endpoint value indicates that the floating-point number is quantized to 0. For each first exponent range other than the first exponent range with the smallest endpoint value, the quantization information of the first exponent range includes the correspondence between the first quantized value of each exponent in the first exponent range and the global quantized value and exponent.

[0214] The acquisition module 1410 is used for:

[0215] For each floating-point number to be dequantized, if the floating-point number is 0, then the first quantization value corresponding to the floating-point number is determined to be the quantization value indicating that the floating-point number is quantized to 0;

[0216] If the floating-point number is not 0, then the exponent after the dequantization of the floating-point number is determined. Based on the quantization information of the first exponent range to which the dequantized exponent of the floating-point number belongs and the exponent of the floating-point number, the first quantization value corresponding to the floating-point number is determined.

[0217] In an alternative embodiment, the acquisition module 1410 is configured to:

[0218] Based on the formula δ1=δ2*2 (E-E1) The first quantization value corresponding to the floating-point number is determined, where δ1 is the first quantization value corresponding to the floating-point number, δ2 is the global quantization value, E is the minimum exponent in the first exponent range with the largest exponent, and E1 is the exponent after the floating-point number is dequantized.

[0219] In one alternative approach, among the plurality of first exponential ranges, the quantization information of the first exponential range with the smallest endpoint value indicates that the floating-point number is quantized to 0, and the quantization information of each first exponential range other than the first exponential range with the smallest endpoint value includes a first quantized value.

[0220] The acquisition module 1410 is used for:

[0221] For each floating-point number to be dequantized, if the floating-point number is 0, then the first quantization value corresponding to the floating-point number is determined to be the quantization value indicating that the floating-point number is quantized to 0;

[0222] If the floating-point number is not 0, then the exponent after the dequantization of the floating-point number is determined, and the first quantization value of the first exponent range to which the dequantized exponent of the floating-point number belongs is determined, which is the first quantization value corresponding to the floating-point number.

[0223] In one alternative approach, the floating-point number to be dequantized before quantization corresponds to multiple second exponent ranges, which are determined based on the compression rule and the exponent of the floating-point number before quantization. For each floating-point number, the second quantization value corresponding to the exponent of the floating-point number is determined based on the quantization information of the second exponent range to which the exponent of the floating-point number belongs.

[0224] The acquisition module 1410 is used for:

[0225] For each floating-point number to be dequantized, a second quantized value is determined based on the quantization information of the second exponent range corresponding to the exponent of the floating-point number.

[0226] In one alternative approach, the dequantization module 1420 is configured to: if the first quantization value corresponding to the floating-point number is not an indication that the floating-point number is quantized to 0, determine the first multiplication result of multiplying the mantissa of the floating-point number by the first quantization value; and use the second multiplication result as the mantissa of the dequantized floating-point number.

[0227] In one alternative approach, the dequantization module 1420 is configured to: determine a second multiplication result of the exponent of the floating-point number multiplied by a second quantization value; and use the second multiplication result as the exponent of the dequantized floating-point number.

[0228] In an alternative embodiment, the inverse quantization module 1420 is also used for:

[0229] Get the compressed floating-point number;

[0230] The compressed floating-point number is subjected to entropy decoding to obtain the floating-point number to be dequantized.

[0231] The detailed process of floating-point dequantization shown in Figure 14 is described in the preceding embodiments and will not be repeated here. The floating-point dequantization device shown in Figure 14 is the computing device 500 in Figure 5.

[0232] In some embodiments, a computer program product is provided, the computer program product including computer instructions stored in a computer-readable storage medium. A processor of a computing device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, causing the computing device to perform the process shown in FIG6.

[0233] In some embodiments, a computer program product is provided, the computer program product including computer instructions stored in a computer-readable storage medium. A processor of a computing device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, causing the computing device to perform the process shown in FIG11.

[0234] Those skilled in the art will recognize that the method steps and units described in the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of both. To clearly illustrate the interchangeability of hardware and software, the steps and components of each embodiment have been generally described in terms of functionality in the foregoing description. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this disclosure.

[0235] In the embodiments provided in this disclosure, it should be understood that the disclosed system architecture, apparatus, and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative. For instance, the division of modules is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple modules or components may be combined or integrated into another system, or some features may be ignored or not executed. In addition, the coupling or direct coupling or communication connection shown or discussed may be an indirect coupling or communication connection through some interfaces, devices, or modules, or it may be an electrical, mechanical, or other form of connection.

[0236] The modules described as separate components may or may not be physically separate. The components shown as modules may or may not be physical modules; that is, they may be located in one place or distributed across multiple network modules. Some or all of the modules can be selected to achieve the purpose of the embodiments of this disclosure, depending on actual needs.

[0237] Furthermore, the modules in the various embodiments of this disclosure can be integrated into one processing module, or each module can exist physically separately, or two or more modules can be integrated into one module. The integrated modules described above can be implemented in hardware or in software.

[0238] If the integrated module is implemented as a software functional module and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this disclosure, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods in the various embodiments of this disclosure. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, ROM, RAM, magnetic disks, or optical disks.

[0239] In this disclosure, the terms "first" and "second," etc., are used to distinguish identical or similar items that have substantially the same function and purpose. It should be understood that there is no logical or temporal dependency between "first" and "second," nor does it limit the quantity or order of execution. It should also be understood that although the following description uses the terms "first" and "second," etc., to describe various elements, these elements should not be limited by the terms. These terms are merely used to distinguish one element from another. For example, without departing from the scope of various examples, a first index can be referred to as a second index, and similarly, a second index can be referred to as a first index. Both the first and second indices can be indices, and in some cases, they can be separate and distinct indices.

[0240] In this disclosure, the term "at least one" means one or more, and the term "multiple" means two or more.

[0241] The above description is merely an exemplary embodiment of this disclosure, but the scope of protection of this disclosure is not limited thereto. Any person skilled in the art can easily conceive of various equivalent modifications or substitutions within the technical scope disclosed in this disclosure, and such modifications or substitutions should all be covered within the scope of protection of this disclosure. Therefore, the scope of protection of this disclosure should be determined by the scope of the claims.

Claims

1. A method for floating-point quantization, characterized in that, The method includes: Based on compression rules, the quantization information corresponding to the exponent of each floating-point number in the floating-point number to be quantized is determined. The quantization information corresponding to the exponent of each floating-point number includes a first quantization value and / or a second quantization value. The compression rules include rules that indicate the exponent of quantization information with a low degree of compression is greater than the exponent of quantization information with a high degree of compression. For each floating-point number to be quantized, the mantissa of the floating-point number is quantized based on the first quantization value corresponding to the floating-point number, and / or the exponent of the floating-point number is quantized based on the second quantization value corresponding to the floating-point number.

2. The method according to claim 1, characterized in that, The compression rule also includes a specified compression ratio corresponding to the floating-point number to be quantized.

3. The method according to claim 1 or 2, characterized in that, The process of determining the quantization information corresponding to the exponent of each floating-point number in the floating-point number to be quantized based on compression rules includes: Based on the compression rules, multiple first exponent ranges corresponding to the floating-point number to be quantized and the quantization information of each first exponent range are determined. For each floating-point number to be quantized, a first quantized value corresponding to the exponent of the floating-point number is determined based on the quantization information of the first exponent range to which the exponent of the floating-point number belongs.

4. The method according to claim 3, characterized in that, Among the plurality of first exponent ranges, the quantization information of the first exponent range with the smallest endpoint value indicates that the floating-point number is quantized to 0. For each first exponent range other than the first exponent range with the smallest endpoint value, the quantization information of the first exponent range includes the correspondence between the first quantized value of each exponent in the first exponent range and the global quantized value and exponent. The determination of the first quantized value corresponding to the exponent of the floating-point number based on the quantization information of the first exponent range to which the exponent of the floating-point number belongs includes: If the exponent of the floating-point number belongs to the first exponent range with the smallest endpoint value, then based on the quantization information of the first exponent range to which the exponent of the floating-point number belongs, the first quantization value corresponding to the floating-point number is determined to be the quantization value indicating that the floating-point number is quantized to 0. If the exponent of the floating-point number does not belong to the first exponent range with the smallest endpoint value, then the first quantized value corresponding to the floating-point number is determined based on the quantization information of the first exponent range to which the exponent of the floating-point number belongs and the exponent of the floating-point number.

5. The method according to claim 4, characterized in that, The determination of the first quantized value corresponding to the floating-point number based on the quantization information of the first exponent range to which the exponent of the floating-point number belongs and the exponent of the floating-point number includes: Based on the formula δ1=δ2*2 (E-E1) The first quantization value corresponding to the floating-point number is determined, where δ1 is the first quantization value corresponding to the floating-point number, δ2 is the global quantization value, E is the minimum exponent in the first exponent range with the largest exponent, and E1 is the exponent of the floating-point number.

6. The method according to claim 3, characterized in that, Among the plurality of first exponential ranges, the quantization information of the first exponential range with the smallest endpoint value indicates that the floating-point number is quantized to 0, and the quantization information of each first exponential range other than the first exponential range with the smallest endpoint value includes a first quantized value. The determination of the first quantized value corresponding to the exponent of the floating-point number based on the quantization information of the first exponent range to which the exponent of the floating-point number belongs includes: If the exponent of the floating-point number belongs to the first exponent range with the smallest endpoint value, then based on the quantization information of the first exponent range to which the exponent of the floating-point number belongs, the first quantization value corresponding to the floating-point number is determined to be the quantization value indicating that the floating-point number is quantized to 0. If the exponent of the floating-point number does not belong to the first exponent range with the smallest endpoint value, then the first quantization value of the first exponent range to which the exponent of the floating-point number belongs is determined as the first quantization value corresponding to the floating-point number.

7. The method according to any one of claims 1 to 6, characterized in that, The step of quantizing the mantissa of the floating-point number based on the first quantization value corresponding to the floating-point number includes: If the first quantization value corresponding to the floating-point number is a quantization value that indicates the floating-point number to be quantized to 0, then the floating-point number is quantized to 0; If the first quantization value of the floating-point number is not a quantization value that indicates the floating-point number to be quantized to 0, then determine the first division result obtained by dividing the mantissa of the floating-point number by the first quantization value corresponding to the floating-point number. Based on the result of the first division, the mantissa of the quantized floating-point number is determined.

8. The method according to any one of claims 1 to 7, characterized in that, The process of determining the quantization information corresponding to the exponent of each floating-point number in the floating-point number to be quantized based on compression rules includes: Based on the compression rules, multiple second exponent ranges corresponding to the floating-point number to be quantized and the quantization information of each second exponent range are determined; For each floating-point number to be quantized, a second quantized value corresponding to the exponent of the floating-point number is determined based on the quantization information of the second exponent range to which the exponent of the floating-point number belongs.

9. The method according to claim 8, characterized in that, The quantification information for each second index range includes the second quantification value; The determination of the second quantized value corresponding to the exponent of the floating-point number based on the quantization information of the second exponent range to which the exponent of the floating-point number belongs includes: The second quantization value of the second exponent range to which the exponent of the floating-point number belongs is determined as the second quantization value corresponding to the floating-point number.

10. The method according to any one of claims 1 to 9, characterized in that, The step of quantizing the exponent of the floating-point number based on the second quantization value corresponding to the floating-point number includes: Determine the second division result obtained by dividing the exponent of the floating-point number by the second quantized value corresponding to the floating-point number; Based on the result of the second division, the quantized exponent of the floating point is determined.

11. The method according to any one of claims 1 to 10, characterized in that, The method further includes: The quantized floating-point number is entropy encoded to obtain the compressed data of the floating-point number to be quantized.

12. A method for dequantizing floating-point numbers, characterized in that, The method includes: Obtain the quantization information corresponding to the exponent of each floating-point number in the floating-point number to be dequantized. The quantization information corresponding to the exponent of each floating-point number includes a first quantization value and / or a second quantization value. The quantization information corresponding to the exponent of each floating-point number is determined based on the compression rule and the exponent of the floating-point number before quantization. The compression rule includes a rule that indicates the exponent of quantization information with a low degree of compression is greater than the exponent of quantization information with a high degree of compression. For each floating-point number to be dequantized, the mantissa of the floating-point number is dequantized based on the first quantization value corresponding to the floating-point number, and / or the exponent of the floating-point number is dequantized based on the second quantization value corresponding to the floating-point number.

13. The method according to claim 12, characterized in that, The compression rule also includes a specified compression ratio corresponding to the floating-point number before quantization.

14. The method according to claim 12 or 13, characterized in that, The exponent of the floating-point number before quantization corresponds to multiple first exponent ranges. These multiple first exponent ranges are determined based on the compression rules and the exponent of the floating-point number before quantization. For each floating-point number before quantization, the first quantization value corresponding to the exponent of the floating-point number is determined based on the quantization information of the first exponent range to which the exponent of the floating-point number belongs.

15. The method according to claim 14, characterized in that, Among the plurality of first exponent ranges, the quantization information of the first exponent range with the smallest endpoint value indicates that the floating-point number is quantized to 0. For each first exponent range other than the first exponent range with the smallest endpoint value, the quantization information of the first exponent range includes the correspondence between the first quantized value of each exponent in the first exponent range and the global quantized value and exponent. The step of obtaining the quantization information corresponding to the exponent of each floating-point number in the floating-point number to be inverted includes: For each floating-point number to be dequantized, if the floating-point number is 0, then the first quantization value corresponding to the floating-point number is determined to be the quantization value indicating that the floating-point number is quantized to 0; If the floating-point number is not 0, then the exponent after the dequantization of the floating-point number is determined. Based on the quantization information of the first exponent range to which the dequantized exponent of the floating-point number belongs and the exponent of the floating-point number, the first quantization value corresponding to the floating-point number is determined.

16. The method according to claim 15, characterized in that, The step of determining the first quantized value corresponding to the floating-point number based on the quantization information of the first exponent range to which the exponent belongs after dequantization of the floating-point number and the exponent of the floating-point number includes: Based on the formula δ1=δ2*2 (E-E1) The first quantization value corresponding to the floating-point number is determined, where δ1 is the first quantization value corresponding to the floating-point number, δ2 is the global quantization value, E is the minimum exponent in the first exponent range with the largest exponent, and E1 is the exponent after the floating-point number is dequantized.

17. The method according to claim 14, characterized in that, Among the plurality of first exponential ranges, the quantization information of the first exponential range with the smallest endpoint value indicates that the floating-point number is quantized to 0, and the quantization information of each first exponential range other than the first exponential range with the smallest endpoint value includes a first quantized value. The step of obtaining the quantization information corresponding to the exponent of each floating-point number in the floating-point number to be inverted includes: For each floating-point number to be dequantized, if the floating-point number is 0, then the first quantization value corresponding to the floating-point number is determined to be the quantization value indicating that the floating-point number is quantized to 0; If the floating-point number is not 0, then the exponent after the dequantization of the floating-point number is determined, and the first quantization value of the first exponent range to which the dequantized exponent of the floating-point number belongs is determined, which is the first quantization value corresponding to the floating-point number.

18. The method according to claim 12, characterized in that, The floating-point number to be dequantized before quantization corresponds to multiple second exponent ranges. The multiple second exponent ranges are determined based on the compression rule and the exponent of the floating-point number before quantization. For each floating-point number, the second quantization value corresponding to the exponent of the floating-point number is determined based on the quantization information of the second exponent range to which the exponent of the floating-point number belongs. The step of obtaining the quantization information corresponding to the exponent of each floating-point number in the floating-point number to be inverted includes: For each floating-point number to be dequantized, a second quantized value is determined based on the quantization information of the second exponent range corresponding to the exponent of the floating-point number.

19. The method according to any one of claims 12 to 18, characterized in that, The method further includes: Get the compressed floating-point number; The compressed floating-point number is subjected to entropy decoding to obtain the floating-point number to be dequantized.

20. A device for floating-point quantization, characterized in that, The device includes: The determination module is used to determine the quantization information corresponding to the exponent of each floating-point number in the floating-point number to be quantized based on the compression rules. The quantization information corresponding to the exponent of each floating-point number includes a first quantization value and / or a second quantization value. The compression rules include rules that indicate the exponent of quantization information with a low degree of compression is greater than the exponent of quantization information with a high degree of compression. The quantization module is used to quantize the mantissa of each floating-point number in the floating-point numbers to be quantized based on a first quantization value corresponding to the floating-point number, and / or to quantize the exponent of the floating-point number based on a second quantization value corresponding to the floating-point number.

21. A device for dequantizing floating-point numbers, characterized in that, The device includes: The acquisition module is used to acquire the quantization information corresponding to the exponent of each floating-point number in the floating-point number to be dequantized. The quantization information corresponding to the exponent of each floating-point number includes a first quantization value and / or a second quantization value. The quantization information corresponding to the exponent of each floating-point number is determined based on the compression rule and the exponent of the floating-point number before quantization. The compression rule includes a rule that indicates the exponent of quantization information with a low degree of compression is greater than the exponent of quantization information with a high degree of compression. The dequantization module is used to, for each floating-point number to be dequantized, perform dequantization processing on the mantissa of the floating-point number based on the first quantization value corresponding to the floating-point number, and / or, perform dequantization processing on the exponent of the floating-point number based on the second quantization value corresponding to the floating-point number.

22. A computing device, characterized in that, The computing device includes a processor and memory; The processor of the computing device is configured to execute instructions stored in the memory of the computing device to cause the computing device to perform the method as described in any one of claims 1 to 10.

23. A computing device, characterized in that, The computing device includes a processor and memory; The processor of the computing device is configured to execute instructions stored in the memory of the computing device to cause the computing device to perform the method as described in any one of claims 11 to 19.

24. A computer-readable storage medium, characterized in that, It includes computer program instructions, which, when executed by a computing device, cause the computing device to perform the method as described in any one of claims 1 to 10.

25. A computer-readable storage medium, characterized in that, It includes computer program instructions, which, when executed by a computing device, cause the computing device to perform the method as described in any one of claims 11 to 19.

26. A chip, characterized in that, When the chip is running on a computing device, the computing device performs the method as described in any one of claims 1 to 10.

27. A chip, characterized in that, When the chip is running on a computing device, the computing device performs the method as described in any one of claims 11 to 19.

28. A quantization device, characterized in that, The quantization device includes a floating-point format parser, an exponential quantizer, and a tail quantizer; The floating-point format parser is used to split the floating-point number to be quantized into an exponent, a mantissa, and a sign. The exponential quantizer is used to obtain a first quantization value corresponding to the exponent of the floating-point number to be quantized, and to quantize the exponent of the floating-point number to be quantized based on the obtained first quantization value to obtain the exponentially quantized floating-point number. The mantissa quantizer is used to obtain a second quantization value corresponding to the exponent of the floating-point number to be quantized, and based on the obtained second quantization value, to quantize the mantissa of the floating-point number to be quantized, thereby obtaining the mantissa quantized floating-point number.

29. An inverse quantization device, characterized in that, The quantization device includes an exponential dequantizer, a mantissa dequantizer, and a floating-point format combiner; The exponent dequantizer is used to obtain the first quantization value corresponding to the exponent of the floating-point number to be dequantized, and based on the obtained first quantization value, to perform dequantization processing on the exponent of the floating-point number to be dequantized to obtain the floating-point number after exponent dequantization. The mantissa dequantizer is used to obtain the second quantization value corresponding to the floating-point number to be dequantized, and based on the obtained second quantization value, to perform dequantization processing on the mantissa of the floating-point number to be dequantized, so as to obtain the floating-point number after mantissa dequantization. The floating-point format combiner is used to combine the floating-point number after dequantization of the exponent and the floating-point number after dequantization of the mantissa to obtain the dequantized floating-point number.

Images