Floating point index operation method, tensor processor, equipment and storage medium
By converting floating-point exponentiation into base-2 exponentiation and splitting the fractional terms, combined with Taylor expansion, the problem of low efficiency of floating-point exponentiation in tensor processors is solved, and efficient parallel computing is achieved.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- SOPHGO TECH LTD
- Filing Date
- 2025-12-31
- Publication Date
- 2026-05-08
AI Technical Summary
Existing tensor processors are inefficient in floating-point exponentiation, mainly because Taylor expansion calculations require multiple iterations, resulting in long processing times.
The floating-point exponentiation with base e is converted to exponentiation with base 2, and the decimal terms are split into fixed-point terms and floating-point terms. The exponentiation results are obtained from a preset table and then calculated using Taylor expansion to reduce the number of iterations.
While maintaining accuracy, it improves the efficiency of floating-point exponentiation operations and supports parallel operations of large-scale floating-point numbers, significantly reducing computation time.
Smart Images
Figure CN121996202A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of data processing technology, and in particular to a floating-point exponentiation method, a tensor processor, a device, and a storage medium. Background Technology
[0002] Tensor Processing Units (TPUs) are processors specifically designed for machine learning. Trained for deep machine learning, they offer higher efficiency and can run neural network models more quickly. The execution of these models involves calculations of functions like Softmax, which require extensive floating-point exponentiation (operations with a base of e and a floating-point exponent) EXP(x). Currently, TPUs typically perform floating-point exponentiation by converting the expression to a base-2 exponentiation expression, then decomposing it into integer and fractional terms. The integer terms are raised to powers using bit shifting, while the fractional terms are raised using Taylor series expansion. However, achieving the target precision with Taylor series expansion requires multiple polynomial iterations, which is time-consuming, and floating-point exponentiation is inefficient. Therefore, how to improve the efficiency of floating-point exponentiation while ensuring the accuracy of floating-point exponentiation is an urgent problem to be solved. Summary of the Invention
[0003] This invention provides a floating-point exponentiation method, a tensor processor, a device, and a storage medium, aiming to improve the efficiency of floating-point exponentiation while ensuring the accuracy of floating-point exponentiation.
[0004] In a first aspect, embodiments of the present invention provide a floating-point exponentiation method applied to a tensor processor, the tensor processor including a control unit and multiple computing units, the method comprising: The calculation unit is controlled to convert a first formula with the real number e as the base and the target floating-point number as the exponent into a second formula with the base 2 and the reciprocal of the target floating-point number multiplied by ln2 as the exponent, and to obtain the integer term of the exponent of the second formula. The calculation unit is controlled to obtain the first decimal term of the exponent of the second formula and to split the first decimal term into a fixed-point term and a first floating-point term, and to obtain the first exponent calculation result corresponding to the fixed-point term from a preset table; The calculation unit is controlled to calculate the second exponentiation result of the integer term and multiply the first exponentiation result and the second exponentiation result to obtain the initial natural exponentiation result of the target floating-point number; The calculation unit is controlled to perform Taylor expansion calculation on the first floating-point number term to obtain the third exponent operation result, and the initial natural exponent operation result of the target floating-point number and the third exponent operation result are multiplied to obtain the target natural exponent operation result of the target floating-point number.
[0005] The floating-point exponentiation method provided in the first aspect converts a first formula with a real number e as the base and the target floating-point number as the exponent into a second formula with a base of 2 and the reciprocal of the target floating-point number multiplied by ln2 as the exponent, and obtains the integer term of the exponent of the second formula. Then, the first decimal term of the second formula is split into a fixed-point term and a first floating-point term, and the first exponentiation result corresponding to the fixed-point term is obtained from a preset table. The second exponentiation result of the integer term is calculated, and Taylor expansion is performed on the first floating-point term to obtain the third exponentiation result. Finally, the first exponentiation result, the second exponentiation result, and the third exponentiation result are multiplied together to obtain the target natural exponentiation result of the target floating-point number. This reduces the number of polynomial iteration calculations in Taylor expansion, thereby improving the efficiency of floating-point exponentiation while ensuring the accuracy of floating-point exponentiation.
[0006] Secondly, embodiments of the present invention also provide a floating-point exponentiation method applied to a tensor processor, the tensor processor including a control unit and multiple computing units, the method comprising: The calculation unit is controlled to execute a first instruction based on the input target floating-point number to convert a first formula with the real number e as the base and the target floating-point number as the exponent into a second formula with the base 2 and the reciprocal of the target floating-point number multiplied by ln2 as the exponent, and to obtain the integer term of the exponent of the second formula. The calculation unit used for each of the multiple operation instructions included in the execution of the first instruction is different. The calculation unit is controlled to execute a second instruction based on the exponent and the integer term of the second formula to obtain the first decimal term of the exponent of the second formula, split the first decimal term into a fixed-point term and a first floating-point term, obtain the first exponent operation result corresponding to the fixed-point term from a preset table, calculate the second exponent operation result of the integer term, multiply the first exponent operation result and the second exponent operation result to obtain the initial natural exponent operation result of the target floating-point number, wherein the calculation unit used for each of the multiple operation instructions included in the execution of the second instruction is different; The control unit executes a third instruction based on the first floating-point number term and the initial natural exponent calculation result of the target floating-point number to perform Taylor expansion calculation on the first floating-point number term to obtain a third exponent calculation result. The initial natural exponent calculation result of the target floating-point number and the third exponent calculation result are multiplied to obtain the target natural exponent calculation result of the target floating-point number. The calculation unit used for each of the multiple calculation instructions included in the execution of the third instruction is different.
[0007] The second aspect provides a floating-point exponentiation method that, on the one hand, transforms a first formula with the real number e as the base and the target floating-point number as the exponent into a second formula with the base 2 and the exponent of the target floating-point number multiplied by the reciprocal of ln2, and obtains the integer term of the exponent of the second formula. Then, it splits the first decimal term of the second formula into a fixed-point term and a first floating-point term, and obtains the first exponentiation result corresponding to the fixed-point term from a preset table. It calculates the second exponentiation result of the integer term and performs Taylor expansion on the first floating-point term to obtain the third exponentiation result. Finally, it multiplies the first, second, and third exponentiation results to obtain the target natural exponentiation result of the target floating-point number. This reduces the number of polynomial iteration calculations in Taylor expansion, thereby improving the efficiency of floating-point exponentiation while ensuring the accuracy of floating-point exponentiation. On the other hand, each of the multiple arithmetic instructions included in the same instruction uses a different computing unit. In this way, when performing floating-point exponentiation on large-scale floating-point numbers, parallel floating-point exponentiation can be achieved for different floating-point numbers, further improving the efficiency of floating-point exponentiation.
[0008] Thirdly, embodiments of the present invention also provide a tensor processor, the tensor processor including a control unit and a plurality of computing units, the control unit being connected to the plurality of computing units, the control unit being used to implement the floating-point exponentiation method as described in the first or second aspect.
[0009] Fourthly, embodiments of the present invention also provide a computer device, including one or more tensor processors, one or more memories, a computer program stored in the memory and executable by the tensor processor, and a data bus for implementing connection communication between the tensor processor and the memory, wherein when the computer program is executed by the tensor processor, it implements the floating-point exponentiation method described in the first or second aspect.
[0010] Fifthly, embodiments of the present invention also provide a storage medium for computer-readable storage, the storage medium storing one or more programs, the one or more programs being executable by one or more tensor processors to implement the floating-point exponentiation method as described in the first or second aspect. Attached Figure Description
[0011] To more clearly illustrate the technical solutions of the embodiments of this application, the drawings used in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.
[0012] Figure 1 This is a flowchart illustrating the steps of a floating-point exponentiation method provided in an embodiment of the present invention. Figure 2 for Figure 1 A flowchart illustrating the sub-steps of the floating-point exponentiation method in [the document / technology]. Figure 3 This is a schematic block diagram of a tensor processor provided in an embodiment of the present invention; Figure 4 This is a schematic block diagram of the structure of a computer device provided in an embodiment of the present invention. Detailed Implementation
[0013] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.
[0014] The flowchart shown in the attached diagram is for illustrative purposes only and does not necessarily include all content and operations / steps, nor does it necessarily have to be performed in the described order. For example, some operations / steps can be broken down, combined, or partially merged, so the actual execution order may change depending on the actual situation.
[0015] It should be understood that the terminology used in this specification is for the purpose of describing particular embodiments only and is not intended to limit the invention. As used in this specification and the appended claims, the singular forms “a,” “an,” and “the” are intended to include the plural forms unless the context clearly indicates otherwise.
[0016] Tensor Processing Units (TPUs) are processors specifically designed for machine learning. Trained for deep machine learning, they offer higher efficiency and can run neural network models more quickly. The execution of these models involves calculations of functions like Softmax, which require extensive floating-point exponentiation (operations with a base of e and a floating-point exponent) EXP(x). Currently, TPUs typically perform floating-point exponentiation by converting the expression to a base-2 exponentiation expression, then decomposing it into integer and fractional terms. The integer terms are raised to powers using bit shifting, while the fractional terms are raised using Taylor series expansion. However, achieving the target precision with Taylor series expansion requires multiple polynomial iterations, which is time-consuming, and floating-point exponentiation is inefficient.
[0017] To address the aforementioned problems, embodiments of the present invention provide a floating-point exponentiation method, a tensor processor, a computer device, and a storage medium. This floating-point exponentiation method converts a first expression with a base of real number e and an exponent of the target floating-point number into a second expression with a base of 2 and an exponent of the reciprocal of the target floating-point number multiplied by ln2. It then obtains the integer term of the exponent of the second expression. The first decimal term of the second expression is then split into a fixed-point term and a first floating-point term. The first exponentiation result corresponding to the fixed-point term is obtained from a preset table. The second exponentiation result of the integer term is calculated, and a Taylor expansion is performed on the first floating-point term to obtain a third exponentiation result. Finally, the first, second, and third exponentiation results are multiplied to obtain the target natural exponentiation result of the target floating-point number. This reduces the number of polynomial iterations in the Taylor expansion, thereby improving the efficiency of floating-point exponentiation while maintaining its accuracy.
[0018] It should be noted that the floating-point exponentiation method provided in this embodiment of the invention can be applied to computer devices, which may include servers and electronic devices. The server can be a standalone server or a cloud server providing basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, content delivery networks (CDNs), and big data and artificial intelligence platforms. The electronic device may include smartphones, tablets, laptops, desktop computers, personal digital assistants, and wearable devices.
[0019] The following detailed description of some embodiments of the present invention is provided in conjunction with the accompanying drawings. Unless otherwise specified, the following embodiments and features can be combined with each other.
[0020] Please see Figure 1 , Figure 1 This is a flowchart illustrating the steps of a floating-point exponentiation method provided in an embodiment of the present invention.
[0021] like Figure 1 As shown, the floating-point exponentiation method includes steps S101 to S104.
[0022] Step S101: The control calculation unit converts the first formula with the real number e as the base and the target floating-point number as the exponent into the second formula with the base 2 and the reciprocal of the target floating-point number multiplied by ln2 as the exponent, and obtains the integer term of the exponent of the second formula.
[0023] In this embodiment, the target floating-point number may include at least one of the following: FP32 format floating-point number (float32, single-precision 32-bit floating-point number), FP16 format floating-point number (float16, half-precision 16-bit floating-point number), and BF16 format floating-point number (Bfloat16, half-precision 16-bit floating-point number, different from FP16 format). The floating-point number is stored in binary code. Taking the FP32 format floating-point number in IEEE 754 as an example, the floating-point number is stored in 32-bit binary code. The first bit is the sign bit S, the second to ninth bits are the exponent E, and the remaining 23 bits are the mantissa. The normalized mantissa actually has 24 bits, with a leading 1 implicitly preceding the most significant bit. Therefore, the value of the normalized FP32 format floating-point number can be represented by the following formula:
[0024] Where S is the sign bit, E is the exponent, M is the mantissa, the subscript (10) indicates that it is to be converted to decimal, and 127 indicates the bias required for the exponent E to be stored as a biased binary number. The structure of the binary code of floating-point numbers in other formats is consistent with the structure of the binary code of floating-point numbers in FP32 format, but the number of bits of the exponent and mantissa are different, which will not be elaborated here.
[0025] Understandably, through the formula The first formula with the real number e as the base and the target floating-point number as the exponent can be... Convert to a second formula with base 2 and exponent of the target floating-point number multiplied by the reciprocal of ln2. Then, for the second equation index Round down to get Integer terms Where x is the target floating-point number.
[0026] Step S102: The control calculation unit obtains the first decimal term of the exponent of the second formula and splits the first decimal term into a fixed-point term and a first floating-point term, and obtains the first exponent calculation result corresponding to the fixed-point term from a preset table.
[0027] In this embodiment, the second formula can be used. index Subtract the integer term of the exponent in the second expression. The second equation is obtained. index First decimal term = - The preset table stores the pre-calculated exponential results (base 2) corresponding to different numerical values.
[0028] In some embodiments, obtaining the first exponentiation result corresponding to the fixed-point number term from a preset table includes: converting the fixed-point number term into a second floating-point number term, and obtaining the first exponentiation result corresponding to the second floating-point number term from the preset table. This embodiment can quickly obtain the exponentiation result by looking up the table, reducing the number of Taylor expansion calculations.
[0029] In some embodiments, splitting the first decimal term into a fixed-point term and a first floating-point term may include: converting the first decimal term into a fixed-point format decimal term, and then splitting the fixed-point format decimal term into a fixed-point term and a first floating-point term. For example, padding the first decimal term with implicit leading 1s yields a second decimal term; shifting the second decimal term right by -E bits yields a fixed-point format decimal term; and then splitting the fixed-point format decimal term into a fixed-point term and a first floating-point term using a mask. This embodiment can quickly and accurately split the term into fixed-point and first floating-point terms.
[0030] In some embodiments, converting the first decimal term to a fixed-point number format decimal term and splitting the fixed-point number format decimal term into a fixed-point number term and a first floating-point number term may include: obtaining the exponent value represented by the exponent of the target floating-point number; in response to the exponent value represented by the exponent of the target floating-point number being greater than or equal to a preset exponent value and less than zero, adding an implicit leading 1 to the first decimal term to obtain a second decimal term; right-shifting the second decimal term by a first number of bits to obtain a third decimal term, taking the mantissa corresponding to the high n bits of the third decimal term as a fixed-point number term, the first number being equal to the negative of the exponent value minus 1; left-shifting the mantissa of the third decimal term (excluding the fixed-point number term) by a second number of bits to obtain a remaining mantissa term; and determining the first floating-point number term based on the exponent of the target floating-point number and the remaining mantissa term. This embodiment ensures that the remaining mantissa term does not discard its last bits, thereby improving the exponent calculation results of subsequent decimal terms.
[0031] It should be noted that the preset exponent value can be set based on actual conditions, and this embodiment of the invention does not impose specific limitations on it. For example, the preset exponent value is -4. Therefore, the result of the exponent calculation for the decimal term can be expressed as... Where rem is the decimal term, It is a fixed-point number term (a binary integer fixed-point number term consisting of the high 4 bits of the decimal term rem). It is the remaining term in fixed-point format, which can be converted into the first floating-point term (a floating-point number composed of the remaining bits of the fractional term rem) after normalization.
[0032] In some embodiments, after obtaining the exponent value represented by the exponent of the target floating-point number, the method further includes: in response to the exponent value represented by the exponent of the target floating-point number being less than a preset exponent value, determining the first decimal term as the first floating-point term, and setting the fixed-point term to zero. For example, if the preset exponent value is 4, and the exponent value represented by the exponent of the target floating-point number is less than -4, it indicates that if the first decimal term is converted to a decimal term in fixed-point format, the high 4 bits of the decimal term in fixed-point format will all be 0. Therefore, it is not necessary to convert the first decimal term to a decimal term in fixed-point format, and the fixed-point term is then used. Take 0, first floating-point term You can directly take the value of the first decimal term, rem.
[0033] Step S103: The control calculation unit calculates the second exponent operation result of the integer term and multiplies the first exponent operation result and the second exponent operation result to obtain the initial natural exponent operation result of the target floating-point number.
[0034] In this embodiment, the second exponent operation result of the integer term can be calculated by shifting the integer term into the exponent. Multiplying the first exponent operation result and the second exponent operation result yields the low-precision natural exponent operation result exp_main of the target floating-point number.
[0035] Step S104: The control calculation unit performs Taylor expansion calculation on the first floating-point number term to obtain the third exponent operation result, and multiplies the initial natural exponent operation result of the target floating-point number with the third exponent operation result to obtain the target natural exponent operation result of the target floating-point number.
[0036] In this embodiment, the first floating-point number can be... The Taylor expansion at 0 yields the result of the third exponentiation of the first floating-point term. The result of the third exponent operation, denoted as frac, is multiplied by the result of the initial natural exponent operation, exp_main, to obtain the target natural exponent operation result of the target floating-point number, denoted as exp. Compared with existing methods, the floating-point exponent operation method provided in this embodiment only requires a small number of table lookup operations and bit operations, which can reduce a lot of polynomial iteration calculations in Taylor expansion, thereby improving the efficiency of floating-point exponent operation.
[0037] The following calculates the error between the floating-point exponentiation method provided in this embodiment and existing methods. The error is calculated as follows:
[0038] For example, taking 10,000 random data points within the FP32 format exponent calculation range as input, calculating exp based on the floating-point exponent calculation method provided in this embodiment and existing methods respectively, and then performing error calculation according to the above error calculation method, the FP32 calculation result table shown in Table 1 can be obtained.
[0039] Table 1
[0040] As shown in Table 1 above, under the same target accuracy requirements, the floating-point exponentiation method provided by this invention requires far fewer Taylor expansion iterations (4 times) to calculate the exp of a floating-point number in FP32 format than the existing method (9 times). Therefore, the floating-point exponentiation method provided by this invention is more efficient in calculating exp.
[0041] For example, taking 10,000 random data points within the FP16 format exponent calculation range as input, calculating exp based on the floating-point exponent calculation method provided in this embodiment and existing methods respectively, and then performing error calculation according to the above error calculation method, the FP16 calculation result table shown in Table 2 can be obtained.
[0042] Table 2
[0043] As can be seen from Table 2 above, under the same target accuracy requirements, the floating-point exponentiation method provided by the present invention requires far fewer Taylor expansion iterations (2 times) to calculate the exp of a floating-point number in FP16 format than the existing method (7 times). Therefore, the floating-point exponentiation method provided by the present invention is more efficient in calculating exp.
[0044] For example, taking 10,000 random data points within the BF16 format exponent calculation range as input, calculating exp based on the floating-point exponent calculation method provided in this embodiment and existing methods respectively, and then performing error calculation according to the above error calculation method, we can obtain the BF16 calculation result table shown in Table 3.
[0045] Table 3
[0046] As shown in Table 3 above, under the same target accuracy requirements, the floating-point exponentiation method provided by this invention requires far fewer Taylor expansion iterations (2 times) to calculate the exp of a floating-point number in BF16 format than the existing method (7 times). Therefore, the floating-point exponentiation method provided by this invention is more efficient in calculating exp.
[0047] Please see Figure 2 , Figure 2 This is a flowchart illustrating another floating-point exponentiation method provided in an embodiment of the present invention.
[0048] like Figure 2 As shown, the floating-point exponentiation method includes steps S201 to S203.
[0049] Step S201: The control calculation unit executes a first instruction based on the input target floating-point number to convert the first formula with the real number e as the base and the target floating-point number as the exponent into a second formula with the base 2 and the reciprocal of the target floating-point number multiplied by ln2 as the exponent, and to obtain the integer term of the exponent of the second formula.
[0050] In this embodiment, the computing unit used for each of the multiple arithmetic instructions included in the first instruction is different. For example, as shown in Table 4, the first instruction is CMD instruction 1, with input operands p and max_p, and output operands res0: x_iln2 (the exponent in the second formula) and res1: fn (the integer term of the exponent in the second formula). It includes 4 arithmetic instructions and 1 output instruction. The computing units used to execute these 4 arithmetic instructions are different. In CMD instruction 1, the floating-point adder FADD is used to execute the arithmetic instruction p0 = p - max_p, the arithmetic logic unit ALU is used to execute the arithmetic instruction p1 = min(max(p0, RANG_START), RANG_END), the floating-point multiplier FMUL is used to execute the arithmetic instruction x_iln2 = p1 * inv_ln2, and the type converter F2F is used to execute the arithmetic instruction fn = f2f(x_iln2), round_floor.
[0051] Table 4
[0052] For example, controlling the calculation unit to execute the first instruction based on the input target floating-point number may include: controlling the floating-point adder FADD to execute the operation instruction p0 = p - max_p according to the softmax calculation rule, so that the target floating-point number p is subtracted by an offset max_p; controlling the arithmetic logic unit ALU to execute the operation instruction p1 = min(max(p0, RANG_START), RANG_END) to control the boundary value; and then controlling the floating-point multiplier FMUL to execute the operation instruction x_iln2 = p1 * inv_ln2 to obtain the second formula. index It is stored in res0; it controls F2F to execute the operation instruction fn = f2f(x_iln2), round_floor, so as to obtain the result by rounding down. Integer terms And store it in res1.
[0053] Step S202: The control calculation unit executes a second instruction based on the exponent and integer terms of the second formula to obtain the first decimal term of the exponent of the second formula, splits the first decimal term into a fixed-point term and a first floating-point term, obtains the first exponent operation result corresponding to the fixed-point term from a preset table, calculates the second exponent operation result of the integer term, and multiplies the first exponent operation result and the second exponent operation result to obtain the initial natural exponent operation result of the target floating-point number.
[0054] In this embodiment, the computing unit used for each of the multiple arithmetic instructions included in the execution of the second instruction is different. For example, as shown in Table 4, the second instruction is CMD instruction 2, with input operands x_iln2 and fn, and output operands res0: y (the first floating-point term) and res1: exp_main (the result of the initial natural exponent operation). It includes 5 operation instructions and 1 output instruction. The computing units used to execute these 5 operation instructions are different. In CMD instruction 2, the floating-point adder FADD is used to execute the operation instruction p1_rem=x_iln2 – fn, the type converter F2F is used to execute the operation instructions f2f(p1_rem)→rem_h, rem_l and rem_l<<=clz(rem_l), y={sign,exp,rem_l}, the lookup table SG_Line is used to execute the operation instruction fh = scan_table(rem_h), the shifter F2I is used to execute the operation instruction n=f2i(fn), res_int =(n+127)<<23, and the floating-point multiplier FMUL is used to execute the operation instruction exp_main= fh * as_fp(rem_int).
[0055] For example, the control calculation unit executes the second instruction based on the exponent and integer terms of the second formula, including: controlling the floating-point adder FADD to execute p1_rem=x_iln2 – fn to extract the first decimal term p1_rem of x_iln2; controlling the type converter F2F to execute the operation instruction f2f(p1_rem)→rem_h, rem_l to split the first decimal term into a fixed-point term rem_h and a residual term rem_l, executing the operation instruction rem_l<<=clz(rem_l), y={sign,exp,rem_l} to normalize the residual term rem_l back to a floating-point type, denoted as the first floating-point term y, and stored in res0; controlling the lookup table SG_Line to execute the operation instruction fh = scan_table(rem_h) to obtain the exponent operation result of the fixed-point term rem_h by looking up the table, denoted as fh; and controlling the shifter F2I to execute the operation instruction n=f2i(fn). res_int = (n+127)<<23, resulting in integer terms. The result of the exponentiation operation is denoted as res_int. Note that the exponent bias needs to be added. The floating-point multiplier FMUL is controlled to execute the operation instruction exp_main= fh * as_fp(rem_int)fh to obtain the initial natural exponentiation operation result, denoted as exp_main, which is stored in res1.
[0056] Step S203: The control calculation unit executes a third instruction based on the first floating-point number term and the initial natural exponent calculation result of the target floating-point number to perform Taylor expansion calculation on the first floating-point number term to obtain the third exponent calculation result. The initial natural exponent calculation result of the target floating-point number is multiplied by the third exponent calculation result to obtain the target natural exponent calculation result of the target floating-point number.
[0057] In this embodiment, the computational unit used for each of the multiple arithmetic instructions included in the third instruction is different. For example, as shown in Table 4, the third instruction is CMD instruction 3, with input operands y and exp_main, and output operand res0:exp (the result of the natural exponent calculation). It includes two arithmetic instructions and one output instruction. The computational units used to execute these two arithmetic instructions are different. In CMD instruction 3, the floating-point adder FADD and the floating-point multiplier FMUL are used to execute the arithmetic instruction frac = taylor_4x(y) (Taylor expansion calculation), and the floating-point multiplier FMUL is used to execute the arithmetic instruction exp = exp_main * frac.
[0058] For example, the control calculation unit may execute a third instruction based on the initial natural exponent calculation result of the first floating-point number and the target floating-point number, which may include: controlling the floating-point adder FADD and the floating-point multiplier FMUL to execute the operation instruction frac = taylor_4x(y) to perform Taylor expansion calculation on the first floating-point number to obtain the third exponent calculation result frac; controlling the floating-point multiplier FMUL to execute the operation instruction exp = exp_main * frac to obtain the target natural exponent calculation result exp of the target floating-point number p, which is stored in res0.
[0059] The floating-point exponentiation method provided in the above embodiments, on the one hand, converts the first formula with the real number e as the base and the target floating-point number as the exponent into the second formula with the base 2 and the exponent of the target floating-point number multiplied by the reciprocal of ln2 as the exponent, and obtains the integer term of the exponent of the second formula. Then, the first decimal term of the second formula is split into a fixed-point term and a first floating-point term, and the first exponent operation result corresponding to the fixed-point term is obtained from a preset table. The second exponent operation result of the integer term is calculated, and Taylor expansion is performed on the first floating-point term to obtain the third exponent operation result. Finally, the first exponent operation result, the second exponent operation result, and the third exponent operation result are multiplied to obtain the target natural exponent operation result of the target floating-point number. This reduces the number of polynomial iteration calculations in Taylor expansion, thereby improving the efficiency of floating-point exponentiation while ensuring the accuracy of floating-point exponentiation. On the other hand, each operation instruction in the multiple operation instructions included in the same instruction uses a different computing unit. In this way, when performing floating-point exponentiation on large-scale floating-point numbers, parallel floating-point exponentiation can be achieved for different floating-point numbers, further improving the efficiency of floating-point exponentiation.
[0060] In Table 4, EU represents the computational unit used. Since different computational units are used in each computational process within each instruction, parallel computation of different data can be achieved during large-scale data computation. Therefore, the computation time of the floating-point exponentiation method provided in this embodiment is masked by the computation of the decimal and integer terms of the EXP inherent in existing floating-point exponentiation methods, thus avoiding additional time consumption. Therefore, the performance gain of the floating-point exponentiation method provided in this embodiment is equal to the reduction in Taylor expansion computation in the EXP calculation.
[0061] By combining the fusion instructions (including the first instruction, the second instruction, and the third instruction) provided in this embodiment of the invention, the fixed-point splitting and table lookup process are integrated with the left shift of integer terms and the subtraction calculation of decimal terms, thereby achieving fast EXP calculation and effectively improving the computational efficiency of EXP. Taking the calculation of operators of mainstream large models (such as the attention_QKV process of the LLama2 model) as an example, this calculation process involves many softmax operations and requires a large number of FP16 format EXP calculations. The following table analyzes the computation count of the EXP calculation process in SoftMax in this embodiment of the invention. The matrix input size of the EXP to be calculated is [1,32,1,256], and the EXP computation counts without and with the fusion instructions are shown in Table 5 below.
[0062] Table 5
[0063] As shown in Table 5 above, the matrix input size for calculating EXP is [1,32,1,256]. The number of calculations for EXP calculation without using the fusion instruction is 204800, while the number of calculations for EXP calculation with the fusion instruction is 147456, resulting in a reduction rate of (204800-147456) / 204800=28%.
[0064] The computational efficiency of the entire Attention_QKV is shown in Table 6 below, which uses the fusion instruction to perform EXP calculation.
[0065] Table 6
[0066] As shown in Table 6 above, the computation time of the entire Attention_QKV is 331.4us when EXP is calculated without using the fusion instruction, and 320.6us when EXP is calculated with the fusion instruction. The computation efficiency of the entire Attention_QKV is improved by (331.4-320.6) / 331.4=3.26% when EXP is calculated with the fusion instruction compared to when EXP is calculated without the fusion instruction.
[0067] Please see Figure 3 , Figure 3 This is a schematic block diagram of a tensor processor provided in an embodiment of the present invention.
[0068] like Figure 3 As shown, the tensor processor 110 includes a control unit 111 and multiple computing units 112, with the control unit 11 connected to the multiple computing units 112.
[0069] In some embodiments, the control unit 111 is configured to perform the following steps: The calculation unit is controlled to convert a first formula with the real number e as the base and the target floating-point number as the exponent into a second formula with the base 2 and the reciprocal of the target floating-point number multiplied by ln2 as the exponent, and to obtain the integer term of the exponent of the second formula. The calculation unit is controlled to obtain the first decimal term of the exponent of the second formula and to split the first decimal term into a fixed-point term and a first floating-point term, and to obtain the first exponent calculation result corresponding to the fixed-point term from a preset table; The calculation unit is controlled to calculate the second exponentiation result of the integer term and multiply the first exponentiation result and the second exponentiation result to obtain the initial natural exponentiation result of the target floating-point number; The calculation unit is controlled to perform Taylor expansion calculation on the first floating-point number term to obtain the third exponent operation result, and the initial natural exponent operation result of the target floating-point number and the third exponent operation result are multiplied to obtain the target natural exponent operation result of the target floating-point number.
[0070] In some embodiments, the control unit 111 is configured to perform the following steps: The calculation unit is controlled to execute a first instruction based on the input target floating-point number to convert a first formula with the real number e as the base and the target floating-point number as the exponent into a second formula with the base 2 and the reciprocal of the target floating-point number multiplied by ln2 as the exponent, and to obtain the integer term of the exponent of the second formula. The calculation unit used for each of the multiple operation instructions included in the execution of the first instruction is different. The calculation unit is controlled to execute a second instruction based on the exponent and the integer term of the second formula to obtain the first decimal term of the exponent of the second formula, split the first decimal term into a fixed-point term and a first floating-point term, obtain the first exponent operation result corresponding to the fixed-point term from a preset table, calculate the second exponent operation result of the integer term, multiply the first exponent operation result and the second exponent operation result to obtain the initial natural exponent operation result of the target floating-point number, wherein the calculation unit used for each of the multiple operation instructions included in the execution of the second instruction is different; The control unit executes a third instruction based on the first floating-point number term and the initial natural exponent calculation result of the target floating-point number to perform Taylor expansion calculation on the first floating-point number term to obtain a third exponent calculation result. The initial natural exponent calculation result of the target floating-point number and the third exponent calculation result are multiplied to obtain the target natural exponent calculation result of the target floating-point number. The calculation unit used for each of the multiple calculation instructions included in the execution of the third instruction is different.
[0071] In some embodiments, when the control unit 111 splits the first decimal term into a fixed-point term and a first floating-point term, it is configured to: The first decimal term is converted into a fixed-point number format decimal term, and the fixed-point number format decimal term is split into a fixed-point number term and a first floating-point number term.
[0072] In some embodiments, when the control unit 111 converts the first decimal item into a fixed-point number format decimal item and splits the fixed-point number format decimal item into a fixed-point number item and a first floating-point number item, it is configured to: Obtain the exponent value of the target floating-point number in its exponent representation; In response to the fact that the exponent value of the exponent representation of the target floating-point number is greater than or equal to a preset exponent value and less than zero, an implicit leading 1 is added to the first decimal term to obtain the second decimal term; Shift the second fractional term to the right by a first number of bits to obtain the third fractional term. Take the mantissa corresponding to the high n bits of the third fractional term as the fixed-point number term. The first number is equal to the opposite of the exponent value minus 1. Shift the mantissa items (excluding the fixed-point items) in the third fractional item to the left by a second number of bits to obtain the remaining mantissa items. Determine the first floating-point item based on the exponent of the target floating-point number and the remaining mantissa items.
[0073] In some embodiments, after acquiring the exponent value of the exponent representation of the target floating-point number, the control unit 111 is further configured to: In response to the fact that the exponent value of the exponent representation of the target floating-point number is less than a preset exponent value, the first decimal term is determined as the first floating-point term, and the fixed-point term is zero.
[0074] It should be noted that those skilled in the art will understand that, for the sake of convenience and brevity, the specific working process of the tensor processor described above can be referred to the corresponding process in the aforementioned floating-point exponentiation method embodiment, and will not be repeated here.
[0075] Please see Figure 4 , Figure 4 This is a schematic block diagram of the structure of a computer device provided in an embodiment of the present invention.
[0076] like Figure 4 As shown, the computer device 100 includes one or more tensor processors 110 and one or more memories 120, which are connected via a bus 130, such as an I2C (Inter-integrated Circuit) bus.
[0077] In some embodiments, the computer device 100 further includes a main processor, which provides computing and control capabilities to support the operation of the entire computer device. The main processor may be a Central Processing Unit (CPU), or it may be other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor or any conventional processor.
[0078] Specifically, the memory 120 can be a Flash chip, a read-only memory (ROM) disk, an optical disk, a USB flash drive, or a portable hard drive, etc.
[0079] Those skilled in the art will understand that Figure 4 The structures shown are merely block diagrams of the item structures related to the embodiments of the present invention, and do not constitute a limitation on the computer devices on which the embodiments of the present invention are applied. Specific computer devices may include more or fewer components than those shown in the figures, or combine certain components, or have different component arrangements.
[0080] The tensor processor 110 is used to run a computer program stored in a memory, and implements any of the floating-point exponentiation methods provided in the embodiments of the present invention when executing the computer program.
[0081] In some embodiments, the tensor processor 110 is configured to run a computer program stored in memory, and when executing the computer program, perform the following steps: The calculation unit is controlled to convert a first formula with the real number e as the base and the target floating-point number as the exponent into a second formula with the base 2 and the reciprocal of the target floating-point number multiplied by ln2 as the exponent, and to obtain the integer term of the exponent of the second formula. The calculation unit is controlled to obtain the first decimal term of the exponent of the second formula and to split the first decimal term into a fixed-point term and a first floating-point term, and to obtain the first exponent calculation result corresponding to the fixed-point term from a preset table; The calculation unit is controlled to calculate the second exponentiation result of the integer term and multiply the first exponentiation result and the second exponentiation result to obtain the initial natural exponentiation result of the target floating-point number; The calculation unit is controlled to perform Taylor expansion calculation on the first floating-point number term to obtain the third exponent operation result, and the initial natural exponent operation result of the target floating-point number and the third exponent operation result are multiplied to obtain the target natural exponent operation result of the target floating-point number.
[0082] In some embodiments, the tensor processor 110 is configured to run a computer program stored in memory, and when executing the computer program, perform the following steps: The calculation unit is controlled to execute a first instruction based on the input target floating-point number to convert a first formula with the real number e as the base and the target floating-point number as the exponent into a second formula with the base 2 and the reciprocal of the target floating-point number multiplied by ln2 as the exponent, and to obtain the integer term of the exponent of the second formula. The calculation unit used for each of the multiple operation instructions included in the execution of the first instruction is different. The calculation unit is controlled to execute a second instruction based on the exponent and the integer term of the second formula to obtain the first decimal term of the exponent of the second formula, split the first decimal term into a fixed-point term and a first floating-point term, obtain the first exponent operation result corresponding to the fixed-point term from a preset table, calculate the second exponent operation result of the integer term, multiply the first exponent operation result and the second exponent operation result to obtain the initial natural exponent operation result of the target floating-point number, wherein the calculation unit used for each of the multiple operation instructions included in the execution of the second instruction is different; The control unit executes a third instruction based on the first floating-point number term and the initial natural exponent calculation result of the target floating-point number to perform Taylor expansion calculation on the first floating-point number term to obtain a third exponent calculation result. The initial natural exponent calculation result of the target floating-point number and the third exponent calculation result are multiplied to obtain the target natural exponent calculation result of the target floating-point number. The calculation unit used for each of the multiple calculation instructions included in the execution of the third instruction is different.
[0083] In some embodiments, when the tensor processor 110 implements the splitting of the first decimal term into a fixed-point term and a first floating-point term, it is configured to: The first decimal term is converted into a fixed-point number format decimal term, and the fixed-point number format decimal term is split into a fixed-point number term and a first floating-point number term.
[0084] In some embodiments, when the tensor processor 110 converts the first decimal term into a fixed-point number format decimal term and splits the fixed-point number format decimal term into a fixed-point number term and a first floating-point number term, it is configured to: Obtain the exponent value of the target floating-point number in its exponent representation; In response to the fact that the exponent value of the exponent representation of the target floating-point number is greater than or equal to a preset exponent value and less than zero, an implicit leading 1 is added to the first decimal term to obtain the second decimal term; Shift the second fractional term to the right by a first number of bits to obtain the third fractional term. Take the mantissa corresponding to the high n bits of the third fractional term as the fixed-point number term. The first number is equal to the opposite of the exponent value minus 1. Shift the mantissa items (excluding the fixed-point items) in the third fractional item to the left by a second number of bits to obtain the remaining mantissa items. Determine the first floating-point item based on the exponent of the target floating-point number and the remaining mantissa items.
[0085] In some embodiments, after acquiring the exponent value of the exponent representation of the target floating-point number, the tensor processor 110 is further configured to: In response to the fact that the exponent value of the exponent representation of the target floating-point number is less than a preset exponent value, the first decimal term is determined as the first floating-point term, and the fixed-point term is zero.
[0086] It should be noted that those skilled in the art will understand that, for the sake of convenience and brevity, the specific working process of the computer device described above can be referred to the corresponding process in the aforementioned floating-point exponentiation method embodiment, and will not be repeated here.
[0087] This invention also provides a storage medium for computer-readable storage, wherein the storage medium stores one or more programs, which can be executed by one or more tensor processors to implement any of the floating-point exponentiation methods provided in the specification of this invention.
[0088] The storage medium can be an internal storage unit of the computer device described in the foregoing embodiments, such as the hard disk or memory of the computer device. The storage medium can also be an external storage device of the computer device, such as a plug-in hard disk, Smart Media Card (SMC), Secure Digital (SD) card, or Flash Card.
[0089] Those skilled in the art will understand that all or some of the steps, systems, or apparatuses disclosed above, and their functional modules / units, can be implemented as software, firmware, hardware, or suitable combinations thereof. In hardware embodiments, the division between functional modules / units mentioned in the above description does not necessarily correspond to the division of physical components; for example, a physical component may have multiple functions, or a function or step may be performed collaboratively by several physical components. Some or all physical components may be implemented as software executed by a processor, such as a central processing unit, digital signal processor, or microprocessor, or as hardware, or as an integrated circuit, such as an application-specific integrated circuit. Such software may be distributed on a computer-readable medium, which may include computer storage media (or non-transitory media) and communication media (or transient media). As is known to those skilled in the art, the term computer storage media includes volatile and non-volatile, removable and non-removable media implemented in any method or technology for storing information (such as computer-readable instructions, data structures, program modules, or other data). Computer storage media include, but are not limited to, RAM, ROM, EEPROM, flash memory or other memory technologies, CD-ROM, digital versatile disc (DVD) or other optical disc storage, magnetic cartridges, magnetic tape, disk storage or other magnetic storage devices, or any other medium that can be used to store desired information and can be accessed by a computer. Furthermore, it is well known to those skilled in the art that communication media typically contain computer-readable instructions, data structures, program modules, or other data in modulated data signals such as carrier waves or other transmission mechanisms, and may include any information delivery medium.
[0090] It should be understood that the term "and / or" as used in this specification and the appended claims refers to any combination and all possible combinations of one or more of the associated listed items, and includes such combinations. It should be noted that, herein, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or system that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or system. Without further limitation, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or system that includes that element.
[0091] The sequence numbers of the above embodiments of the present invention are merely for descriptive purposes and do not represent the superiority or inferiority of the embodiments. The above descriptions are only specific embodiments of the present invention, but the scope of protection of the present invention is not limited thereto. Any person skilled in the art can easily conceive of various equivalent modifications or substitutions within the technical scope disclosed in the present invention, and these modifications or substitutions should all be covered within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the scope of the claims.
Claims
1. A floating-point exponentiation method, characterized in that, Applied to a tensor processor, the tensor processor including a control unit and multiple computing units, the method includes: The calculation unit is controlled to convert a first formula with the real number e as the base and the target floating-point number as the exponent into a second formula with the base 2 and the reciprocal of the target floating-point number multiplied by ln2 as the exponent, and to obtain the integer term of the exponent of the second formula. The calculation unit is controlled to obtain the first decimal term of the exponent of the second formula and to split the first decimal term into a fixed-point term and a first floating-point term, and to obtain the first exponent calculation result corresponding to the fixed-point term from a preset table; The calculation unit is controlled to calculate the second exponentiation result of the integer term and multiply the first exponentiation result and the second exponentiation result to obtain the initial natural exponentiation result of the target floating-point number; The calculation unit is controlled to perform Taylor expansion calculation on the first floating-point number term to obtain the third exponent operation result, and the initial natural exponent operation result of the target floating-point number and the third exponent operation result are multiplied to obtain the target natural exponent operation result of the target floating-point number.
2. The floating-point exponentiation method according to claim 1, characterized in that, The step of splitting the first decimal term into a fixed-point term and a first floating-point term includes: The first decimal term is converted into a fixed-point number format decimal term, and the fixed-point number format decimal term is split into a fixed-point number term and a first floating-point number term.
3. The floating-point exponentiation method according to claim 2, characterized in that, The step of converting the first decimal term to a fixed-point number format decimal term, and splitting the fixed-point number format decimal term into a fixed-point number term and a first floating-point number term, includes: Obtain the exponent value of the target floating-point number in its exponent representation; In response to the fact that the exponent value of the exponent representation of the target floating-point number is greater than or equal to a preset exponent value and less than zero, an implicit leading 1 is added to the first decimal term to obtain the second decimal term; Shift the second fractional term to the right by a first number of bits to obtain the third fractional term. Take the mantissa corresponding to the high n bits of the third fractional term as the fixed-point number term. The first number is equal to the opposite of the exponent value minus 1. Shift the mantissa items (excluding the fixed-point items) in the third fractional item to the left by a second number of bits to obtain the remaining mantissa items. Determine the first floating-point item based on the exponent of the target floating-point number and the remaining mantissa items.
4. The floating-point exponentiation method according to claim 3, characterized in that, After obtaining the exponent value of the exponent representation of the target floating-point number, the method further includes: In response to the fact that the exponent value of the exponent representation of the target floating-point number is less than a preset exponent value, the first decimal term is determined as the first floating-point term, and the fixed-point term is zero.
5. A floating-point exponentiation method, characterized in that, Applied to a tensor processor, the tensor processor including a control unit and multiple computing units, the method includes: The calculation unit is controlled to execute a first instruction based on the input target floating-point number to convert a first formula with the real number e as the base and the target floating-point number as the exponent into a second formula with the base 2 and the reciprocal of the target floating-point number multiplied by ln2 as the exponent, and to obtain the integer term of the exponent of the second formula. The calculation unit used for each of the multiple operation instructions included in the execution of the first instruction is different. The calculation unit is controlled to execute a second instruction based on the exponent and the integer term of the second formula to obtain the first decimal term of the exponent of the second formula, split the first decimal term into a fixed-point term and a first floating-point term, obtain the first exponent operation result corresponding to the fixed-point term from a preset table, calculate the second exponent operation result of the integer term, multiply the first exponent operation result and the second exponent operation result to obtain the initial natural exponent operation result of the target floating-point number, wherein the calculation unit used for each of the multiple operation instructions included in the execution of the second instruction is different; The control unit executes a third instruction based on the first floating-point number term and the initial natural exponent calculation result of the target floating-point number to perform Taylor expansion calculation on the first floating-point number term to obtain a third exponent calculation result. The initial natural exponent calculation result of the target floating-point number and the third exponent calculation result are multiplied to obtain the target natural exponent calculation result of the target floating-point number. The calculation unit used for each of the multiple calculation instructions included in the execution of the third instruction is different.
6. The floating-point exponentiation method according to claim 5, characterized in that, The step of splitting the first decimal term into a fixed-point term and a first floating-point term includes: The first decimal term is converted into a fixed-point number format decimal term, and the fixed-point number format decimal term is split into a fixed-point number term and a first floating-point number term.
7. The floating-point exponentiation method according to claim 6, characterized in that, The step of converting the first decimal term to a fixed-point number format decimal term, and splitting the fixed-point number format decimal term into a fixed-point number term and a first floating-point number term, includes: Obtain the exponent value of the target floating-point number in its exponent representation; In response to the fact that the exponent value of the exponent representation of the target floating-point number is greater than or equal to a preset exponent value and less than zero, an implicit leading 1 is added to the first decimal term to obtain the second decimal term; Shift the second fractional term to the right by a first number of bits to obtain the third fractional term. Take the mantissa corresponding to the high n bits of the third fractional term as the fixed-point number term. The first number is equal to the opposite of the exponent value minus 1. Shift the mantissa items (excluding the fixed-point items) in the third fractional item to the left by a second number of bits to obtain the remaining mantissa items. Determine the first floating-point item based on the exponent of the target floating-point number and the remaining mantissa items.
8. A tensor processor, characterized in that, The tensor processor includes a control unit and a plurality of computing units, the control unit being connected to the plurality of computing units, and the control unit being used to implement the floating-point exponentiation method as described in any one of claims 1 to 8.
9. A computer device, characterized in that, It includes one or more tensor processors, one or more memories, a computer program stored in the memory and executable by the tensor processor, and a data bus for implementing communication between the tensor processor and the memory, wherein the computer program, when executed by the tensor processor, implements the floating-point exponentiation method as described in any one of claims 1 to 7.
10. A storage medium for computer-readable storage, characterized in that, The storage medium stores one or more programs, which can be executed by one or more tensor processors to implement the floating-point exponentiation method according to any one of claims 1 to 7.