Floating-point number compression method, arithmetic device and calculator readable storage medium

Abstract

The invention discloses a floating-point number compression method, which comprises the following steps by using an arithmetic unit: obtaining a plurality of floating-point numbers; generating a common multiplying power factor for the floating-point numbers; compressing each floating-point number into a plurality of fixed-point number mantissas; and outputting a compression result, wherein the compression result comprises the common multiplying power factors and the fixed-point number mantissas.

Description

A floating point number compression method, computing device and computer-readable storage medium Technical Field

[0001] The present invention relates to an application of floating point operation, in particular to a floating point operation method and a related arithmetic unit. Background Art

[0002] With the increasing popularity of machine learning, which generates a massive amount of floating-point computation, researchers in the field are actively researching how to compress floating-point data to speed up instruction cycles and reduce power consumption. Conventional floating-point technology stores and computes multiple floating-point numbers individually, storing the sign, exponent, and mantissa for each floating-point number. This not only consumes storage space due to the large amount of data stored, but also increases transmission time and computational power consumption.

[0003] Microsoft has proposed a floating-point compression method known as MSFP (Microsoft Floating Point). This method compresses the exponents of multiple floating-point numbers into a single exponent to simplify overall calculations. However, the large compression error significantly reduces computational accuracy. Machine learning (such as neural-like algorithms) requires a certain level of accuracy, so this method is not ideal for practical applications.

[0004] In summary, there is a real need for a novel floating-point operation method and hardware architecture to improve the problems of the existing technology.

[0005] Summary of the Invention

[0006] In response to the above requirements, one of the objectives of the present invention is to provide an efficient floating-point number compression (also known as encoding) and calculation method to improve the defects of floating-point number operations in the prior art without significantly increasing costs, thereby speeding up the instruction cycle and reducing power consumption.

[0007] An embodiment of the present invention provides a floating-point number compression method, comprising using an arithmetic unit to perform the following steps: A) obtaining b floating-point numbers f1-fb, where b is a positive integer greater than 1; B) generating k common rate factors r1-rk for the b floating-point numbers, where k is a positive integer greater than 1, wherein the k common rate factors r1-rk include at least one floating-point number having a mantissa; C) compressing each floating-point number fi in the b floating-point numbers into k fixed-point mantissas mi_1-mi_k to generate a total of b×k fixed-point mantissas mi_j, where i is a positive integer not greater than b and j is a positive integer not greater than k; and D) outputting a compression result, wherein the compression result includes the k common rate factors r1-rk and the b×k fixed-point mantissas mi_j, representing b compressed floating-point numbers cf1-cfb, and the value of each compressed floating-point number cfi is:

[0008] Optionally, according to one embodiment of the present invention, before the computing device executes step D), the computing device further executes the following steps: generating a quasi-compression result, the quasi-compression result including the k common rate factors r1~rk and the b×k fixed-point mantissas mi_j; calculating a compression error for the quasi-compression result; setting a threshold; and adjusting the k common rate factors r1~rk and the b×k fixed-point mantissas mi_j according to the compression error and the threshold.

[0009] Optionally, according to an embodiment of the present invention, the step of calculating the compression error for the quasi-compression result includes: calculating the compression error Ei for each floating-point number fi in the b floating-point numbers according to the following equation: Calculate the sum of squares SE of b errors E1 to Eb according to the following equation: and comparing the square sum with a threshold value. If the square sum is not greater than the threshold value, taking the quasi-compression result as the compression result.

[0010] Optionally, according to an embodiment of the present invention, if the compression error is greater than the threshold, steps B) and C) are re-executed.

[0011] Optionally, according to one embodiment of the present invention, the step of adjusting the k common multiplication factors r1~rk and the b×k fixed-point number mantissas mi_j is: iteratively processing the k common multiplication factors r1~rk and the b×k fixed-point number mantissas mi_j using one of a heuristic algorithm, a random algorithm, or an exhaustive method.

[0012] Optionally, according to an embodiment of the present invention, the step of setting the threshold includes: jointly proposing common multiplication factors r1'-rk' for the b floating-point numbers; compressing each floating-point number fi in the b floating-point numbers into k fixed-point mantissas mi_1'-mi_k' to generate b×k fixed-point mantissas mi_j'; and calculating a compression error Ei' for each floating-point number fi of the b floating-point numbers according to the following equation: Calculate the sum of squares SE' of b errors E1' to Eb' according to the following equation: And the threshold is set to the compression error SE'.

[0013] Optionally, according to an embodiment of the present invention, the mantissas mi_j of the b×k fixed-point numbers are all signed numbers.

[0014] Optionally, according to an embodiment of the present invention, at least one of the b×k fixed-point number mantissas mi_j is a signed number, and the numerical range expressed by the signed number is asymmetric relative to 0.

[0015] Optionally, according to an embodiment of the present invention, the signed number is a 2's complement number.

[0016] Optionally, according to an embodiment of the present invention, the floating-point number compression method further includes: storing the b×k fixed-point number mantissas mi_j and the k common multiplication factors in a memory of a network server for remote download and calculation.

[0017] Optionally, according to an embodiment of the present invention, the floating-point number compression method further includes: storing the b×k fixed-point number mantissas mi_j and all of the common rate factors r1-rk in a memory, but excluding some of the b×k fixed-point number mantissas mi_j and some of the common rate factors r1-rk from the calculation. Optionally, according to an embodiment of the present invention, k is equal to 2, and the common rate factors r1-rk are all floating-point numbers no larger than 16 bits.

[0018] Optionally, according to an embodiment of the present invention, step D) includes: calculating a quasi-compression result, the quasi-compression result including the k common rate factors r1~rk and the b×k fixed-point number mantissas mi_j; calculating a compression error for the quasi-compression result; setting a threshold; and adjusting the quasi-compression result according to the compression error and the threshold to serve as the compression result.

[0019] An embodiment of the present invention provides a computing device, comprising a first register, a second register, and an arithmetic unit, wherein the arithmetic unit comprises at least one multiplier and at least one adder, and the arithmetic unit is coupled to the first register and the second register, wherein: the first register stores b excitation values ​​a1 to ab, where b is a positive integer greater than 1; the second register stores b compressed floating-point numbers cf1 to cfb; the b compressed floating-point numbers include k common multiplication factors r1 to rk, where k is a positive integer greater than 1; each compressed floating-point number cfi in the b compressed floating-point numbers includes k fixed-point mantissas mi_1 to mi_k, for a total of b×k fixed-point mantissas mi_j, where i is a positive integer not greater than b, j is a positive integer not greater than k, and the value of each compressed floating-point number cfi is The arithmetic unit calculates a dot product multiplication result of the b excitation values ​​(a1, a2, ..., ab) and the b compressed floating-point numbers (cf1, cf2, ..., cfb).

[0020] Optionally, according to an embodiment of the present invention, the computing device performs the following steps: A) obtaining b floating-point numbers f1-fb, where b is a positive integer greater than 1; B) generating k common rate factors r1-rk for the b floating-point numbers, where k is a positive integer greater than 1, wherein the k common rate factors r1-rk include at least one floating-point number having a mantissa; C) compressing each floating-point number fi in the b floating-point numbers into k fixed-point mantissas mi_1-mi_k to generate b×k fixed-point mantissas mi_j, where i is a positive integer not greater than b, and j is a positive integer not greater than k; and D) outputting a compression result, the compression result including the k common rate factors r1-rk and the b×k fixed-point mantissas mi_j, representing b compressed floating-point numbers cf1-cfb, and the value of each compressed floating-point number cfi is

[0021] Optionally, according to one embodiment of the present invention, before the computing device executes step D), the computing device further performs the following steps: calculating a quasi-compression result, the quasi-compression result including the k common rate factors r1~rk and the b×k fixed-point number mantissas mi_j; calculating a compression error for the quasi-compression result; setting a threshold; and adjusting the k common rate factors r1~rk and the b×k fixed-point number mantissas mi_j according to the compression error and the threshold.

[0022] Optionally, according to an embodiment of the present invention, the step of calculating the compression error for the quasi-compression result includes: calculating the compression error Ei for each floating-point number fi in the b floating-point numbers according to the following equation: Calculate the sum of squares SE of b errors E1 to Eb according to the following equation: and comparing the square sum with a threshold; wherein if the square sum is not greater than the threshold, taking the quasi-compression result as the compression result.

[0023] Optionally, according to an embodiment of the present invention, if the compression error is greater than the threshold, steps B) and C) are re-executed.

[0024] Optionally, according to one embodiment of the present invention, the step of adjusting the k common multiplication factors r1~rk and the b×k fixed-point number mantissas mi_j is: iteratively processing the compression results of the k common multiplication factors r1~rk and the b×k fixed-point number mantissas mi_j using one of a heuristic algorithm, a random algorithm, or an exhaustive method.

[0025] Optionally, according to an embodiment of the present invention, the step of setting the threshold includes: jointly proposing common multiplication factors r1'-rk' for the b floating-point numbers, compressing each floating-point number fi in the b floating-point numbers into k fixed-point mantissas mi_1'-mi_k' to generate b×k fixed-point mantissas mi_j'; and calculating a compression error Ei' for each floating-point number fi of the b floating-point numbers: Calculate the sum of squares SE' of b errors E1' to Eb' according to the following equation: And the threshold is set to the compression error SE'.

[0026] Optionally, according to an embodiment of the present invention, the b stimulus values ​​a1-ab are integers, fixed-point numbers, or mantissas of MSFP block floating-point numbers.

[0027] Optionally, according to one embodiment of the present invention, the computing device further includes: all of the b×k fixed-point mantissas mi_j and all of the common multiplication factors r1~rk are stored in the second register, but some of the b×k fixed-point mantissas mi_j and some of the common multiplication factors r1~rk do not participate in the calculation.

[0028] An embodiment of the present invention provides a computer-readable storage medium storing computer-readable instructions executable by a computer. When the computer-readable instructions are executed by the computer, a program is triggered for the computer to output b compressed floating-point numbers, where b is a positive integer greater than 1. The program includes the following steps: A) generating k common rate factors r1 to rk, where k is a positive integer of 1 or greater than 1, wherein the k common rate factors r1 to rk include at least one floating-point number with a mantissa, a rate factor exponent, and a rate factor mantissa; B) generating k fixed-point mantissas mi_1 to mi_k to generate b×k fixed-point mantissas mi_j, where i is a positive integer not greater than b, and j is a positive integer not greater than k; and C) outputting the k common rate factors r1 to rk and the b×k fixed-point mantissas mi_j, representing b compressed floating-point numbers cf1 to cfb, where the value of each compressed floating-point number cfi is

[0029] In summary, the present invention's block floating-point number compression can save storage space, reduce power consumption, and accelerate computing speed while meeting the accuracy requirements of the application. Furthermore, thanks to the adjustability of the first and second modes, the electronic products used can flexibly trade off between high-performance and low-power modes, thus embracing a wider range of product applications. Furthermore, compared to Microsoft MSFP and other existing technologies, the present invention's floating-point number compression method can provide optimized computing performance and accuracy, thus saving power and accelerating computing speed while meeting the accuracy requirements of the application.

[0030] The above description is only an overview of the technical solution of the present invention. In order to more clearly understand the technical means of the present invention, which can be implemented in accordance with the contents of the specification, and to make the above and other purposes, features and advantages of the present invention more obvious and easy to understand, the following specifically cites preferred embodiments and describes them in detail with reference to the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0031] FIG1 is a schematic diagram of a floating-point number in the prior art.

[0032] FIG. 2 is a schematic diagram illustrating an arithmetic unit applied to a computing device according to an embodiment of the present invention.

[0033] FIG3 is a schematic diagram of compression processing performed by MSFP in the prior art.

[0034] FIG4 is a schematic diagram of compression processing performed by an arithmetic unit according to an embodiment of the present invention.

[0035] FIG5 is a schematic diagram of compression processing performed by an arithmetic unit according to another embodiment of the present invention.

[0036] FIG6 is a schematic diagram of the present invention using an arithmetic unit and a buffer to perform floating-point multiplication operations of weight values ​​and stimulus values.

[0037] FIG7 is a flowchart of a floating point number compression method according to an embodiment of the present invention.

[0038] FIG8 illustrates the differences between the method of the present invention and the method of MSFP. DETAILED DESCRIPTION

[0039] The present invention is particularly described with reference to the following examples, which are intended to be illustrative only. As various modifications and variations are apparent to those skilled in the art without departing from the spirit and scope of the present disclosure, the scope of protection of the present disclosure shall be determined by the appended claims. Throughout the specification and claims, unless the context clearly dictates otherwise, the meanings of "a," "an," and "the" include references to "one or at least one" element or component. Furthermore, as used herein, singular articles include references to plural elements or components unless the context clearly dictates otherwise. Furthermore, as used in this description and throughout the claims that follow, the meaning of "in which" includes "in which" and "on which" unless the context clearly dictates otherwise. Terms used throughout the specification and claims generally have their ordinary meanings as used in the art, in the context of this disclosure, and in the particular context, unless otherwise noted. Certain terms used to describe the present invention will be discussed below or elsewhere in this specification to provide practitioners with additional guidance regarding the present invention. The use of examples anywhere throughout this specification, including examples of any term discussed herein, is intended to be illustrative only and does not limit the scope or meaning of the invention or any exemplified term. Likewise, the present invention is not limited to the various embodiments set forth in this specification.

[0040] As used herein, the terms "substantially," "around," "about," or "approximately" shall generally mean within 20%, preferably within 10%, of a given value or range. Furthermore, the quantities provided herein may be approximate, thus meaning that unless otherwise stated, they may be expressed using the terms "approximately," "about," or "approximately." When a quantity, concentration, or other numerical value or parameter is given a range, a preferred range, or lists upper and lower ideal values, all ranges consisting of any upper and lower limit pairs or ideal values ​​are specifically disclosed, regardless of whether equivalent ranges are disclosed separately. For example, if a range is disclosed as a length from X cm to Y cm, it is considered to disclose a length of H cm, and H can be any real number between X and Y.

[0041] Furthermore, the term "electrically coupled" or "electrically connected" is intended to encompass any direct and indirect electrical connection means. For example, if a first device is described as being electrically coupled to a second device, this means that the first device may be directly connected to the second device or indirectly connected to the second device through other devices or connection means. Furthermore, when describing the transmission or provision of an electrical signal, those skilled in the art will understand that the transmission of an electrical signal may be accompanied by attenuation or other non-ideal changes. However, unless otherwise specified, the source and receiving end of the transmitted or provided electrical signal should be considered to be substantially the same signal. For example, if an electrical signal S is transmitted (or provided) from terminal A of an electronic circuit to terminal B of the electronic circuit, a voltage drop may occur across the source and drain of a transistor switch and / or possible stray capacitance. However, unless the design purpose is to intentionally exploit the attenuation or other non-ideal changes caused by the transmission (or provision) to achieve a specific technical effect, the electrical signal S should be considered to be substantially the same signal at terminals A and B of the electronic circuit.

[0042] It should be understood that the terms "comprising," "including," "having," "containing," "involving," and the like as used herein are open-ended, meaning to include but not be limited to. Furthermore, any embodiment or claim of the present invention need not achieve all of the objects, advantages, or features disclosed herein. Furthermore, the abstract and title are provided solely to assist in searching for patent documents and are not intended to limit the scope of the present invention.

[0043] Neural algorithms involve a large number of floating-point multiplication operations on weights and activations, so it is crucial to properly compress floating-point numbers while still meeting accuracy requirements.

[0044] Please refer to Figure 1, which is a schematic diagram of a conventional floating-point arithmetic method. As shown in Figure 1, the weight value is an array (or vector) containing 16 words, which can be represented by the floating-point number on the right. Each floating-point number is divided into three different fields of the register: sign, exponent, and mantissa. During the decoding operation, it is decoded as: (-1) Sign ×(1.Mantissa)×2 Exponent

[0045] Among them, Sign represents the sign of this floating-point number, Exponent represents the exponent of this floating-point number, and the mantissa is also called the significand. When stored in the register, the leftmost bit of the register will be allocated as the sign bit to store the sign, and the remaining multiple bits (for example, 15 to 18 bits) will be allocated as the exponent bit and the mantissa bit to store the exponent and mantissa respectively. The practice of the prior art is to treat each word independently as a floating-point number for calculation and storage. Therefore, the register must store 16 to 19 bits for each word, which is not only time-consuming in calculation, but also involves more hardware circuits, resulting in reduced product performance, increased cost and power consumption. Please note that the number of bits of the architecture mentioned in the full text and figures is only for the purpose of ease of understanding, and is not used to limit the scope of the present invention. In practice, the present invention can increase or decrease the number of bits mentioned according to design requirements.

[0046] Please refer to Figure 2, which is a schematic diagram illustrating an arithmetic unit 110 implemented in a computing device 100 according to one embodiment of the present invention. As shown in Figure 2, computing device 100 includes arithmetic unit 110, a first register 111, a second register 112, a third register 113, and a memory 114. Arithmetic unit 110 is coupled to first register 111, second register 112, and third register 113, and memory 114 is coupled to first register 111, second register 112, and third register 113. It should be noted that memory 114 is merely a general term for the storage units within computing device 100. That is, memory 114 can be a standalone storage unit or a general term for all possible storage units within computing device 100. For example, first register 111, second register 112, and third register 113 can each be coupled to a different memory. Furthermore, the memory described herein is merely one example of various available storage media. Those skilled in the art will appreciate that other types of storage media can be used in place of the memory. The computing device 100 can be any device with computing capabilities, such as a central processing unit (CPU), a graphics processing unit (GPU), an artificial intelligence accelerator (AI accelerator), a field programmable gate array (FPGA), a desktop calculator, a notebook calculator, a smartphone, a tablet calculator, a smart wearable device, etc. The present invention can ignore the mantissa of floating-point numbers stored in the first register 111 and the second register 112 and not store them in the memory 114, thereby saving memory space. Furthermore, the memory 114 can store computer-readable instructions executable by the computing device 100. When executed by the computing device 100, the computing device 100 (including the computing unit 110, the first register 111, the second register 112, and the third register 113) will execute a method for compressing floating-point numbers. The memory 114 can also store a complex array of batch normalization coefficients (Batch Normalization Coefficients). Batch norm coefficients are coefficients used to adjust the mean and standard deviation of numerical values ​​in artificial intelligence operations. Typically, a piece of feature map numerical data corresponds to a specific set of batch norm coefficients.

[0047] Please refer to Figure 3, which is a schematic diagram of the MSFP compression process. As shown in Figure 3, unlike treating each word independently as a floating-point number for calculation and storage, MSFP compresses 16 floating-point numbers as a "block". The common exponent portion of each of the 16 floating-point numbers is extracted (denoted as an 8-bit common exponent in the figure). After extraction, only the sign portion and the mantissa portion of these floating-point numbers remain. Please refer to Figure 4, which is a schematic diagram of the arithmetic unit 110 compressing floating-point numbers according to one embodiment of the present invention. Figure 4 compresses each floating-point number into two 2-bit 2'complement fixed-point mantissas m1 and m2. Then, two 7-bit floating-point numbers, namely scales r1 and r2, or scaling factors, are compressed for each block. Next, for each floating-point number, perform integer operations on m1, m2, r1, and r2, so that "m1 × r1 + m2 × r2" has the minimum mean square error with the floating-point number. Note that the fixed-point mantissas m1 and m2 can be signed integers (integers with a positive or negative sign) or unsigned integers (integers without a positive or negative sign).

[0048] Furthermore, the present invention does not limit the number of m and r. For example, the arithmetic unit 110 performs the following steps: obtaining b floating-point numbers f1-fb, where b is a positive integer greater than 1; jointly proposing common rate factors r1-rk for the b floating-point numbers, where k is a positive integer greater than 1; compressing each floating-point number fi among the b floating-point numbers into k fixed-point mantissas mi_1-mi_k to generate b×k fixed-point mantissas mi_j; compressing each floating-point number fi among the b floating-point numbers into k fixed-point mantissas mi_1-mi_k to generate b×k fixed-point mantissas mi_j, where i is a positive integer not greater than b and j is a positive integer not greater than k; and outputting a compression result, the compression result including the k common rate factors r1-rk and the b×k fixed-point mantissas mi_j. Referring to Figure 5 , a schematic diagram illustrating compression processing performed by the arithmetic unit 110 according to another embodiment of the present invention is shown. As shown in Figure 5 , the memory 114 of the computing device 100 can store two sets of complex array batch norm coefficients, corresponding to two floating-point number compression processing modes. The first mode is the complete calculation shown in Figure 4 , while the second mode intentionally ignores the (m²×r²) term to reduce computational complexity. The arithmetic unit 110 can determine whether to use the first or second mode based on the current state of the computing device 100 (e.g., whether it is overheating or overloaded), or based on the accuracy requirements of the current application. For example, if the current temperature of the computing device 100 is too high and needs to be cooled, the second mode can be selected to enable the arithmetic unit 110 to operate in a low-power, low-temperature state. Furthermore, if the computing device 100 is a mobile device and is low on battery power, the second mode can be selected to extend the device's standby time. Furthermore, if the arithmetic unit 110 needs to perform precision calculations, the first mode can be selected to further improve computational accuracy.

[0049] Please refer to Figure 6, which illustrates a schematic diagram of the present invention utilizing registers and an arithmetic unit to perform floating-point dot product multiplication of weight values ​​(Weight) and activation values ​​(Activation). The first, second, and third registers correspond to the first, second, and third registers 111, 112, and 113 in Figure 2, respectively, and the multiplier and adder correspond to the arithmetic unit 110 in Figure 2. As shown in Figure 6, the second register stores the common multiplication factors r1 and r2, as well as the two's complement fixed-point mantissas m1_1, m1_2, etc. corresponding to each floating-point number, each of which is 2 bits. The first register stores the activation values ​​a1, ..., a14, a15, and a16. In the architecture of Figure 6 , a1 is multiplied by m1_1 and m1_2, respectively, and a2 is multiplied by m2_1 and m2_2, respectively. Similarly, a16 is multiplied by m16_1 and m16_2, respectively. These multiplication results are added together by adders 601 and 602, and then processed by multipliers 611 and 612, respectively, and adder 603. Adder 603 outputs the dot product result. Compared to the prior art, the present invention simplifies the hardware architecture, thereby saving power and time for data storage and data transmission.

[0050] Furthermore, to ensure that the required accuracy is maintained after floating-point number compression, the present invention can check for compression errors before generating a compressed result. For example, a quasi-compressed result is generated, which includes k common multiplication factors r1-rk and b×k fixed-point mantissas mi_j. A compression error is then calculated for the quasi-compressed result, and a threshold is set. Finally, the quasi-compressed result is adjusted based on the compression error and the threshold to serve as the compressed result.

[0051] Specifically, the compression error Ei may be calculated for each floating-point number fi among the b floating-point numbers according to the following equation:

[0052] Next, calculate the sum of squares SE of the b errors E1 to Eb according to the following equation:

[0053] Furthermore, the sum of squares is compared with a threshold. If the sum of squares is not greater than the threshold, indicating a small compression error, a quasi-compression result is output as the compression result. If the sum of squares is greater than the threshold, a quasi-compression result is regenerated, for example, by iteratively processing the compression result. Iterative processing includes heuristic algorithms, randomized algorithms, or brute-force algorithms. Heuristic algorithms include evolutionary algorithms and simulated annealing algorithms. For example, if an evolutionary algorithm is used, one bit of the common multiplication factors r1 and r2 can be changed (mutated). If a simulated annealing algorithm is used, for example, the common multiplication factors r1 and r2 can be increased or decreased by a small value d, respectively, to generate four types of common multiplication factors after iteration: r1+d, r2+d, r1+d, r2-d, r1-d, r2+d, or r1-d, r2-d. If a randomized algorithm is used, for example, a random number function can be used to generate the common multiplication factors r1' and r2'. If an exhaustive method is used, for example, if r1 and r2 are both 7 bits, then there are a total of 2 to the power of 14 combinations of r1 and r2, and all of them are iterated once. The above algorithms are only examples and are not intended to limit the scope of the present invention. For example, although evolutionary algorithms and simulated annealing algorithms are almost the most common and common heuristic algorithms, there are others such as bee colony algorithms, ant colony algorithms, whale optimization algorithms, etc. For another example, in addition to mutation operations, evolutionary algorithms also have selection operations and crossover operations, which are not described in detail for the sake of brevity. Those skilled in the art should understand that other types of algorithms can be used for substitution.

[0054] The present invention does not limit the method for generating the threshold. In addition to the absolute value threshold, one method is a relative threshold, which can be summarized as the following steps: generating common multiplication factors r1'-rk' for b floating-point numbers; compressing each floating-point number fi in the b floating-point numbers into k fixed-point mantissas mi_1'-mi_k' to generate b×k fixed-point mantissas mi_j'; and calculating the compression error Ei' for each floating-point number fi in the b floating-point numbers according to the following equation:

[0055] Next, calculate the square sum SE' of the b errors E1' to Eb' according to the following equation:

[0056] And, the threshold is set to the compression error SE'. Those skilled in the art will appreciate that the threshold generation method can be combined with the aforementioned heuristic algorithm (evolutionary algorithm, simulated annealing algorithm, etc.), random algorithm, exhaustive method, etc.

[0057] Optionally, according to one embodiment of the present invention, the step of jointly proposing some common multiplication factors r1~rk for b floating-point numbers includes: jointly proposing signs for the b floating-point numbers so that the b×k fixed-point number mantissa mi_j is unsigned; or when jointly proposing some common multiplication factors r1~rk for the b floating-point numbers, no signs may be proposed so that the b×k fixed-point number mantissa mi_j is signed.

[0058] Optionally, according to an embodiment of the present invention, the mantissas mi_j of the b×k fixed-point numbers may be 2's complement numbers, or may not be 2's complement numbers.

[0059] Optionally, according to one embodiment of the present invention, the floating-point number compression method further includes: storing part of the b×k fixed-point number mantissas mi_j and part of the common multiplication factors r1~rk in a buffer for subsequent calculations, that is, part of the fixed-point number mantissas and / or common multiplication factors will be discarded, which can further speed up the device calculation and reduce the device power consumption.

[0060] Optionally, according to one embodiment of the present invention, the floating-point number compression method further includes: storing all b×k fixed-point number mantissas mi_j and all of the common multiplication factors r1~rk in a buffer, but some of the b×k fixed-point number mantissas mi_j and some of the common multiplication factors r1~rk do not participate in the calculation, that is, not all stored common multiplication factors will participate in the calculation, which can further speed up the device calculation and reduce the device power consumption.

[0061] Please refer to FIG7 , which is a flow chart of a floating-point number compression method according to one embodiment of the present invention. Please note that the steps do not necessarily need to be executed in the order shown in FIG7 , provided that substantially the same results are achieved. The floating-point number operation method shown in FIG7 can be employed by the computing device 100 or arithmetic unit 110 shown in FIG2 , and can be simply summarized as follows:

[0062] Step S702: Obtain b floating point numbers f1-fb;

[0063] Step S704: Propose common multiplication factors r1-rk for the b floating-point numbers;

[0064] Step S706: compress each floating-point number fi in the b floating-point numbers into k fixed-point mantissas mi_1 to mi_k to generate b×k fixed-point mantissas mi_j;

[0065] Step S708: Output a compression result, the compression result including the k common rate factors r1-rk and the b×k fixed-point number mantissas mi_j.

[0066] Since those skilled in the art can easily understand the details of each step in FIG. 7 after reading the above paragraphs, further description will be omitted here for the sake of brevity.

[0067] In summary, the present invention proposes a novel floating-point number compression method with optimized computational efficiency and the advantages of uniform quantization. The present invention approximates each full-precision weight vector (i.e., an uncompressed floating-point number) using the sum of two subword vectors with two magnification ratios. More specifically, each subword is a low-bit (e.g., 2-bit), signed (2's complement) integer, and each magnification ratio is a low-bit floating-point (LBFP) number (e.g., 7-bit). The following details the reasons why the present invention outperforms Microsoft's MSFP algorithm in performance.

[0068] One embodiment of the present invention adopts two multipliers (i.e., r1 and r2), and each floating-point number is compressed into two fixed-point mantissas (i.e., m1 and m2), wherein the computational cost of the multiplier is amortized over 16 weights, and each multiplier is a low-bit floating-point number LBFP, involving only low-bit operations.

[0069] Referring to FIG. 8 , FIG. 8 illustrates the differences between the method of the present invention and the MSFP algorithm, comparing the results of the weight vector compression method of the present invention and the MSFP compression method. The figure clearly illustrates that the present invention requires only fewer quantization levels but achieves smaller quantization error than MSFP, which uses more quantization levels. The advantages of the present invention over MSFP are further listed below.

[0070] 1. No Waste of Quantization Levels: The floating-point number compression method of the present invention uses 2's complement without wasting quantization levels. In contrast, MSFP uses sign and magnitude, consuming an extra quantization level (positive 0 and negative 0 are both 0, so one is wasted. For example, 2 bits can only represent three values: -1, 0, and 1, rather than four values ​​of 2 to the power of 2). When the number of bits is low, the impact of wasting a quantization level is significant.

[0071] 2. Adapting to skewed distribution: The floating-point number compression method of the present invention utilizes the property of 2's complement being asymmetric with respect to 0 (for example, the range of 2's complement of 2 bits is -2, -1, 0, 1) and the multiplier to adapt to the asymmetric weight distribution of the weight vector. In contrast, MSFP uses a sign and magnitude whose range is symmetric with respect to 0 (for example, the range of 2-bit sign and magnitude is -1, 0, 1, symmetric with respect to 0). Therefore, the quantization levels of MSFP are fixed to be symmetric, resulting in the need to consume additional quantization levels to adapt to the asymmetric weight distribution. As shown in FIG8 , while MSFP requires 15 quantization levels (4 bits), the present invention only uses 8 quantization levels (3 bits).

[0072] 3. Adaptability to Non-Uniform Distribution: The floating-point compression method of the present invention can provide non-uniform quantization levels by combining two magnifications (r1, r2). In contrast, MSFP only provides uniform quantization levels. In other words, the floating-point compression method of the present invention is more flexible in compressing non-uniformly distributed weights.

[0073] 4. More Flexible Quantization Step Size: The quantization step size of the floating-point compression method of the present invention is defined by two multipliers (r1, r2), which are low-bitwidth floating-point values. In contrast, the quantization step size of MSFP can only be a power-of-two value, such as 0.5, 0.25, and 0.125.

[0074] The table below shows experimental data comparing the present invention and MSFP for a neural network image classification operation. Both methods compress data using blocks of 16 floating-point numbers. In comparison, the present invention requires fewer bits per 16 floating-point number, achieving higher classification accuracy.

[0075] The number of bits of the fixed-point number mantissa m1 and m2 of the present invention may be preferably shown in the following table, but is not limited thereto.

[0076] The number of bits of the common ratios r1 and r2 of the present invention may be shown in the following table in a preferred embodiment, but is not limited thereto.

[0077] In summary, the present invention's block floating-point number compression can save power and accelerate instruction cycles while meeting the accuracy requirements of applications. Furthermore, thanks to the adjustability of the first and second modes, compatible electronic products can flexibly trade off between high-performance and low-power modes, thus embracing a wider range of product applications. Furthermore, compared to Microsoft MSFP and other existing technologies, the present invention's floating-point number compression method can provide optimized computing performance and accuracy, thus saving power and accelerating computing speed while meeting the accuracy requirements of applications.

[0078] The above description is merely a preferred embodiment of the present invention and does not constitute any form of limitation to the present invention. Although the present invention has been disclosed as a preferred embodiment, it is not intended to limit the present invention. Any technician familiar with the present profession can make slight changes or modifications to equivalent embodiments of equivalent changes using the methods and technical contents disclosed above without departing from the scope of the technical solution of the present invention. However, any simple modifications, equivalent changes and modifications made to the above embodiments based on the technical essence of the present invention without departing from the content of the technical solution of the present invention are still within the scope of the technical solution of the present invention.

Claims

1. A floating point number compression method, characterized in that: The method comprises using an arithmetic unit to perform the following steps: A) Obtain b floating-point numbers f1 to fb, where b is a positive integer greater than 1; B) generating k common rate factors r1-rk for the b floating-point numbers, where k is a positive integer of 1 or greater, and wherein the k common rate factors r1-rk include at least one floating-point number having a mantissa; C) compressing each floating-point number fi in the b floating-point numbers into k fixed-point mantissas mi_1 through mi_k to generate a total of b×k fixed-point mantissas mi_j, where i is a positive integer not greater than b, and j is a positive integer not greater than k; as well as D) Output a compression result, which includes the k common multiplication factors r1~rk and the b×k fixed-point number mantissas mi_j, representing b compressed floating-point numbers cf1~cfb, and the value of each compressed floating-point number cfi is 2. The floating point number compression method according to claim 1, wherein Before the computing device performs step D), the computing device further performs the following steps: generating a quasi-compression result, the quasi-compression result including the k common rate factors r1-rk and the b×k fixed-point number mantissas mi_j; and calculating a compression error for the quasi-compression result; Set a threshold; as well as The k common magnification factors r1-rk and the b×k fixed-point number mantissas mi_j are adjusted according to the compression error and the threshold.

3. The floating point number compression method according to claim 2, wherein: The step of calculating the compression error for the quasi-compression result includes: The compression error Ei is calculated for each floating-point number fi in the b floating-point numbers according to the following equation: Calculate the sum of squares SE of b errors E1 to Eb according to the following equation: and comparing the sum of squares with a threshold; If the square sum is not greater than the threshold, the quasi-compression result is used as the compression result.

4. The floating point number compression method according to claim 2, wherein: If the compression error is greater than the threshold, steps B) and C) are executed again.

5. The floating point number compression method according to claim 4, wherein: The steps of adjusting the k common multiplication factors r1-rk and the b×k fixed-point number mantissas mi_j are as follows: An iterative process using one of a heuristic algorithm, a random algorithm, or an exhaustive method is performed on the k common multiplication factors r1-rk and the b×k fixed-point number mantissas mi_j.

6. The floating point number compression method according to claim 2, wherein: The steps to set the threshold include: Propose common multiplication factors r1'~rk' for the b floating-point numbers; For each floating-point number fi in the b floating-point numbers, compress it into k fixed-point mantissas mi_1' to mi_k' to generate b×k fixed-point mantissas mi_j'; For each of the b floating-point numbers fi, the compression error Ei' is calculated according to the following equation: Calculate the sum of squares SE' of b errors E1' to Eb' according to the following equation: as well as This threshold is defined as the compression error SE'.

7. The floating point number compression method according to claim 1, wherein: The mantissas mi_j of the b×k fixed-point numbers are all signed numbers.

8. The floating point number compression method according to claim 1, wherein: At least one of the b×k fixed-point mantissas mi_j is a signed number, and the numerical range expressed by the signed number is asymmetric relative to 0.

9. The floating point number compression method according to claim 8, wherein: The number is 2's complement.

10. The floating point number compression method according to claim 1, wherein: The floating point number compression method further comprises: The b×k fixed-point number mantissas mi_j and the k common multiplication factors are stored in a memory of a network server for remote download and calculation.

11. The floating point number compression method according to claim 1, wherein: The floating point number compression method further comprises: The b×k fixed-point mantissas mi_j and all of the common multiplication factors r1-rk are stored in a memory, but some of the b×k fixed-point mantissas mi_j and some of the common multiplication factors r1-rk do not participate in the calculation.

12. The floating point number compression method according to claim 1, wherein: k is equal to 2, and the common multiplication factors r1-rk are all floating point numbers no larger than 16 bits.

13. A computing device, characterized in that: The invention comprises a first register, a second register and an arithmetic unit, wherein the arithmetic unit comprises at least one multiplier and at least one adder, and the arithmetic unit is coupled to the first register and the second register, wherein: The first buffer stores b excitation values ​​a1-ab, where b is a positive integer greater than 1; The second buffer stores b compressed floating point numbers cf1-cfb; The b compressed floating-point numbers include k common rate factors r1-rk, where k is a positive integer of 1 or greater; Each compressed floating-point number cfi in the b compressed floating-point numbers contains k fixed-point mantissas mi_1 to mi_k, totaling b×k fixed-point mantissas mi_j, where i is a positive integer not greater than b, j is a positive integer not greater than k, and the value of each compressed floating-point number cfi is as well as The arithmetic unit calculates a dot product multiplication result of the b excitation values ​​(a1, a2, ..., ab) and the b compressed floating-point numbers (cf1, cf2, ..., cfb).

14. The computing device according to claim 13, wherein: The computing device performs the following steps: A) Obtain b floating-point numbers f1 to fb, where b is a positive integer greater than 1; B) generating k common rate factors r1-rk for the b floating-point numbers, where k is a positive integer greater than 1, and wherein the k common rate factors r1-rk include at least one floating-point number having a mantissa; C) compressing each floating-point number fi in the b floating-point numbers into k fixed-point mantissas mi_1 through mi_k to generate b×k fixed-point mantissas mi_j, where i is a positive integer not greater than b, and j is a positive integer not greater than k; as well as D) Output a compression result, which includes the k common multiplication factors r1~rk and the b×k fixed-point number mantissas mi_j, representing b compressed floating-point numbers cf1~cfb, and the value of each compressed floating-point number cfi is 15. The computing device according to claim 14, wherein: Before the computing device performs step D), the computing device further performs the following steps: Calculating a quasi-compression result, the quasi-compression result including the k common rate factors r1-rk and the b×k fixed-point number mantissas mi_j; Calculating a compression error for the quasi-compression result; Set a threshold; as well as The k common magnification factors r1-rk and the b×k fixed-point number mantissas mi_j are adjusted according to the compression error and the threshold.

16. The computing device according to claim 15, wherein: The step of calculating the compression error for the quasi-compression result includes: The compression error Ei is calculated for each of the b floating-point numbers fi according to the following equation: Calculate the sum of squares SE of b errors E1 to Eb according to the following equation: and comparing the sum of squares with a threshold; If the square sum is not greater than the threshold, the quasi-compression result is used as the compression result.

17. The computing device according to claim 15, wherein: If the compression error is greater than the threshold, steps B) and C) are executed again.

18. The computing device according to claim 17, wherein: The steps of adjusting the k common multiplication factors r1-rk and the b×k fixed-point number mantissas mi_j are as follows: An iterative process using one of a heuristic algorithm, a random algorithm, or an exhaustive method is performed on the compression results of the k common multiplication factors r1-rk and the b×k fixed-point number mantissas mi_j.

19. The computing device according to claim 14, wherein: The steps to set the threshold include: Propose a common multiplication factor r1'~rk' for the b floating point numbers For each floating-point number fi in the b floating-point numbers, compress it into k fixed-point mantissas mi_1' to mi_k' to generate b×k fixed-point mantissas mi_j'; Calculate the compression error Ei' for each floating point number fi of the b floating point numbers according to the following equation: Calculate the sum of squares SE' of b errors E1' to Eb' according to the following equation: as well as This threshold is defined as the compression error SE'.

20. The computing device according to claim 13, wherein: The b excitation values ​​a1-ab are integers, fixed-point numbers, or mantissas of MSFP block floating-point numbers.

21. The computing device according to claim 13, wherein: The computing device further comprises: All of the b×k fixed-point mantissas mi_j and all of the common rate factors r1 ˜rk are stored in the second register, but some of the b×k fixed-point mantissas mi_j and some of the common rate factors r1 ˜rk do not participate in the calculation.

22. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer-readable instructions that can be executed by the computer. When the computer-readable instructions are executed by the computer, the computer is triggered to output b compressed floating-point numbers, where b is a positive integer greater than 1. The method includes: A) generating k common multiplication factors r1-rk, where k is a positive integer of 1 or greater, wherein the k common multiplication factors r1-rk include at least one floating point number having a multiplication factor exponent and a multiplication factor mantissa; B) generating k fixed-point mantissas mi_1 to mi_k to generate b×k fixed-point mantissas mi_j, where i is a positive integer not greater than b, and j is a positive integer not greater than k; and C) Output the k common multiplication factors r1~rk and the b×k fixed-point number mantissas mi_j, representing b compressed floating-point numbers cf1~cfb, and the value of each compressed floating-point number cfi is

Images