Processing unit with mixed precision arithmetic

Abstract

A graphics processing unit (GPU) [100] implements an operation [105] having an associated operation code to perform a mixed-precision mathematical operation. The GPU includes an arithmetic logic unit (ALU) [104] having different execution paths [106, 107], where each execution path performs a different mixed-precision operation. By implementing the mixed-precision operation at the ALU in response to an operation code that specifies a description of the operation, the GPU efficiently improves the precision of the specified mathematical operation while reducing execution overhead.

Description

Background Art

[0001] A processor employs one or more processing units that are specifically designed and configured to perform specified operations on behalf of the processor. For example, some processors employ graphics processing units (GPUs) and other parallel processing units that typically implement multiple processing elements (also known as processor cores or compute units) that execute multiple instances of a single program on multiple data sets simultaneously to perform graphics, vector, and other computational processing operations. The processor's central processing unit (CPU) provides commands to the GPU, and the GPU's command processor (CP) decodes the commands into one or more operations. The GPU's execution units, such as one or more arithmetic logic units (ALUs), perform operations to perform graphics and vector processing operations. BRIEF DESCRIPTION OF THE DRAWINGS

[0002] By referencing the accompanying drawings, the present disclosure may be better understood, and its numerous features and advantages become apparent to those skilled in the art. The same reference numerals used in different drawings indicate similar or identical items.

[0003] Figure 1 is a block diagram of a portion of a processing unit employing opcodes for mixed-precision arithmetic, according to some embodiments.

[0004] Figure 2 According to some embodiments Figure 1 Block diagram of the mixed-precision floating-point execution path of the processing unit.

[0005] Figure 3 According to some embodiments Figure 1 Block diagram of another mixed-precision floating-point execution path of a processing unit.

[0006] Figure 4 According to some embodiments Figure 1 Block diagram of the mixed-precision integer execution path of the processing unit. DETAILED DESCRIPTION

[0007] Figures 1 to 4 Techniques are shown in which a parallel processing unit, in this example a graphics processing unit (GPU), implements operations with associated opcodes to perform mixed-precision mathematical operations. The GPU includes an arithmetic logic unit (ALU) with different execution paths, each of which performs a different mixed-precision operation. By implementing mixed-precision operations at the ALU in response to an opcode that specifies the operation, the GPU effectively increases the precision of the specified mathematical operation while reducing execution overhead.

[0008] For example, in the process of executing an instruction, the GPU performs a mathematical operation specified by an opcode associated with the instruction. The opcode indicates the precision of the mathematical operation at least in part by specifying the size of the operands used for the mathematical operation. For example, some opcodes specify 16-bit floating-point operations to be performed with 16-bit operands, while other opcodes specify 32-bit operations to be performed with 32-bit operands. Conventionally, all operands used in the operation have the same size and therefore have the same precision. However, for some operations, such as some vector processing mathematical operations using operands of the same size, the overall precision of the operation will be lost. For example, the result of a dot product operation in which all operands are limited to 16 bits has relatively low precision for some applications. Using the techniques described herein, the GPU performs mathematical operations with mixed-precision operands in response to the corresponding opcodes, thereby efficiently supporting increased mathematical precision.

[0009] Figure 1 A GPU 100 supporting mixed-precision operations according to some embodiments is shown. For purposes of this description, it is assumed that the GPU 100 is part of a processor that executes an instruction set (e.g., a computer program) to perform tasks on behalf of an electronic device. Thus, in various embodiments, the GPU 100 is part of an electronic device such as a desktop computer, a laptop computer, a server, a tablet computer, a smartphone, a game console, etc. Furthermore, it is assumed that the processor including the GPU 100 includes a central processing unit (CPU) that executes the instruction set.

[0010] The GPU 100 is designed and manufactured to perform designated operations on behalf of the CPU. In particular, the GPU 100 performs graphics and vector processing operations on behalf of the CPU. For example, in some embodiments, in the process of executing instructions, the CPU generates commands associated with graphics and vector processing operations. The CPU provides the commands to the GPU 100, which employs a command processor (not shown) to decode the commands into an instruction set for execution at the GPU 100.

[0011] To facilitate the execution of instructions, GPU 100 includes a decode stage 102 and an ALU 104. In some embodiments, decode stage 102 is part of an instruction pipeline (not shown) that includes additional stages to support instruction execution, including a fetch stage that fetches instructions from an instruction buffer, additional decode stages, execution units other than ALU 104, and a retire stage that retires instructions from execution. Decode stage 102 includes circuitry for decoding instructions received from the fetch stage (e.g., instruction 101) into one or more operations (e.g., operation 105) and dispatching these operations to one of the execution units based on the type of operation. In some embodiments, each operation is identified by a corresponding opcode, and the decode stage identifies the execution unit based on the opcode and provides information indicating the opcode to the execution unit. The execution unit uses the opcode or information based on the opcode to determine the type of operation to be performed and performs the indicated operation.

[0012] For example, some operations and associated opcodes indicate arithmetic operations. In response to recognizing that a received instruction indicates an arithmetic operation, decode stage 102 determines the opcode for the operation and provides the opcode, along with other information such as the operands to be used for the arithmetic operation, to ALU 104. ALU 104 performs the operation indicated by the opcode using the indicated operands stored at register file 110. In some embodiments, the operation provided by ALU 104 indicates the precision of the operands and the operation to be performed. For example, in some embodiments, decode stage 102 provides an operation (and corresponding opcode) for a 16-bit multiplication operation using 16-bit operands, and provides another operation (and corresponding opcode) for a 32-bit multiplication operation using 32-bit operands.

[0013] Additionally, the decode stage 102 generates operations with corresponding opcodes for mixed-precision mathematical operations that employ operands of different sizes. For example, in some embodiments, the decode stage generates a multiply-accumulate (MACC) operation based on a corresponding instruction that multiplies operands of one size (e.g., 16 bits) and adds the result to operands of a different size (e.g., 32 bits). In some embodiments, these operations include: 1) a mixed-precision dot product operation (designated DOT4_F32_F16) that multiplies two sets of four 16-bit floating-point operands and adds the multiplication results to each other and to a 32-bit floating-point operand; 2) a mixed-precision dot product operation (designated DOT2_F32_F16) that multiplies two sets of two 16-bit floating-point operands and adds the multiplication results to each other and to a 32-bit floating-point operand; and 3) a mixed-precision dot product operation (designated DOT_I32_I16) that multiplies two sets of four 16-bit integer operands and adds the multiplication results to each other and to a 32-bit integer operand.

[0014] ALU 104 includes a different execution path to perform each mixed-precision operation. In some embodiments, the different execution paths share electronic components or modules, such as registers, adders, multipliers, etc. In other embodiments, some or all of the different execution paths are independent and do not share arithmetic circuits or modules. In the illustrated embodiment, ALU 104 includes a path 106 for performing a DOT4_F32_F16 operation, a path 107 for performing a DOT2_F32_F16 operation, and a path 108 for performing a DOT4_I32_I16 operation. In response to receiving an opcode or other indicator for a mixed-precision operation, ALU 104 performs the operation using the corresponding execution path and stores the result in a register of register file 110. In some embodiments, each mixed-precision operation is specified by a single opcode. In other words, ALU 104 does not require multiple opcodes or operations to perform mixed-precision operations, thereby reducing processing overhead while supporting increased precision.

[0015] Figure 2 The DOT4_F32_F16 execution path 106 is illustrated according to some embodiments. As described above, the DOT4_F32_F16 operation multiplies a set of four 16-bit operands (named A0, A1, A2, and A3 for descriptive purposes) by corresponding operands in another set of four 16-bit operands (named B0, B1, B2, and B3 for descriptive purposes) and adds the result to a third 32-bit operand (named C). Therefore, the DOT4_F32_F16 operation is represented by the following formula:

[0016] D.f32=A.f16[0]*B.f16[0]+A.f16[1]*B.f16[1]+A.f16[2]*B.f16[2]+A.f16[3]*B.f16[3]+C.f32

[0017] In addition, the functionality of the DOT_F32_F16 operation is represented by the following pseudo code:

[0018]

[0019]

[0020] To implement this operation, execution path 106 includes: a set of 16-bit registers (e.g., register 212) for storing operands A0, A1, A2, and A3 and operands B0, B1, B2, and B3; and a 32-bit register for storing operand C. Each of these operands is represented as a floating-point number including a mantissa and an exponent. Execution path 106 also includes a set of adders (e.g., adder 216) and multipliers (e.g., multiplier 218), wherein each adder adds the exponents of a corresponding pair of A and B operands, and each multiplier multiplies the mantissas of a corresponding pair of A and B operands. Thus, for example, adder 216 adds the exponents of operands A0 and B0, while multiplier 218 multiplies the mantissas of operands A0 and B0.

[0021] Execution path 106 also includes an exponent comparison module 220 and a mantissa sorting and alignment module 222. Exponent comparison module 220 receives summed exponents from the adders and compares these sums to determine any mismatches in the exponents and determine a temporary exponent value for the subsequent normalized result D.f32, as described below. Exponent comparison module 220 provides control signaling indicating the identified mismatch to mantissa sorting and alignment module 222. Mantissa sorting and alignment module 222 receives mantissa products from the multipliers and, based on information provided by the exponent comparison module, shifts the mantissa products so that each shifted mantissa product is represented by the same exponent value. Mantissa sorting and alignment module 222 thereby aligns the mantissa products for addition.

[0022] To add the aligned mantissa products, the execution path 106 includes a fused adder 224. In some embodiments, to increase precision, the fused adder 224 adds values ​​having a larger bit size than the A, B, and C operands. For example, in some embodiments, the A and B operands are 16-bit values, the C operand is a 32-bit value, and the mantissa ordering and alignment module 222 generates mantissa values ​​that are 82 bits wide. In these embodiments, the fused adder 224 is capable of adding 82-bit (or larger) values, thereby preventing loss of precision during mantissa addition.

[0023] Fused adder 224 adds the mantissa values to generate a temporary value for the mantissa of D.f32, and provides the temporary mantissa value to normalization module 226, which normalizes the temporary mantissa value. For example, in some embodiments, normalization module 226 shifts the temporary mantissa value to remove any leading zeros in the mantissa. In some embodiments, normalization module 226 adjusts the temporary mantissa to cause the integer portion of the temporary mantissa to be a specified value (e.g., 1). Based on the adjustments made to the mantissa, normalization module adjusts the temporary exponent value provided by exponent comparison module 220 to preserve the overall value of the temporary mantissa. In addition, normalization module sets the sticky bit for the mantissa based on tail bits 229 received from mantissa ordering and alignment module 222.

[0024] Normalization module 226 provides the adjusted mantissa value and exponent value for D.f32 to rounding module 228. Rounding module 228 rounds the mantissa value based on a specified rounding rule, such as rounding D.f32 to the nearest even value, thereby generating a final value for D.f32. Rounding module 228 provides the final D.f32 value to register file 110 for storage at the register indicated by the received operation.

[0025] Figure 3 Figure illustrates DOT2_F32_F16 execution path 107, according to some embodiments. As described above, the DOT2_F32_F16 operation multiplies a set of two 16-bit operands (named A0 and Al for purposes of description) with corresponding operands in another set of two 16-bit operands (named B0 and Bl for purposes of description), and adds the results to a third 32-bit operand (named C). Thus, the DOT2_F32_F16 operation is represented by the following equation:

[0026] D.f32 = A.f16[0] * B.f16[0] + A.f16[l] * B.f16[l] + C.f32

[0027] To implement this operation, execution path 107 includes a set of 32-bit registers (registers 320, 321, and 323) to store operands A0, A1, B0, B1, and operand C. In some embodiments, the operands are stored in different ones of registers 320, 321, and 323, depending on the specific instruction or operation being executed. For example, for one instance of a DOT2_F32_F16 operation, register 320 stores the A operand, register 321 stores the B operand, and register 323 stores the C operand. For another instance of a DOT2_F32_F16 operation, register 320 stores the C operand, register 321 stores the B operand, and register 323 stores the A operand. Furthermore, for different instances of the DOT2_F32_F16 operation, different portions of the 32-bit registers store different ones of the 16-bit operands. For example, for some instances, the A0 operand is stored in the upper 16 bits of one of the registers 320, 321, and 323, while for other instances, the A0 operand is stored in the lower 16 bits of one of the registers 320, 321, and 323. Execution path 106 also includes an operand selection module for selecting an operand from registers 320, 321, and 323 based on control information provided by a received operation.

[0028] Each of the operands is represented as a floating point number including a mantissa and an exponent. The execution path 107 includes a set of prenormalization modules (e.g., prenormalization module 324) to prenormalize the 16-bit operands A0, A1, B0, and B1 by, for example, converting the 16-bit operands to 32-bit operands (e.g., by converting the exponent values ​​of the 16-bit operands). In addition, the execution path 107 includes a denormalization refresh module 326 that refreshes the value of C to zero when the C operand is a denormalized value.

[0029] To multiply the A and B operands, execution path 107 includes a set of AND gates (e.g., AND gate 328), a set of adders (e.g., adder 330), and a set of multipliers (e.g., multiplier 332). Each AND gate performs a logical AND operation on the sign bits of a corresponding pair of prenormalized A and B operands to generate a sign bit for the corresponding multiplication operation. Thus, for example, in one embodiment, AND gate 328 performs an AND operation on the sign bits of prenormalized operands A0 and B0 to generate a sign bit for the A0*B0 operation. Each adder adds the exponents of a corresponding pair of prenormalized A and B operands, and each multiplier multiplies the mantissas of a corresponding pair of prenormalized A and B operands. Thus, for example, adder 216 adds the exponents of prenormalized operands A0 and B0, and multiplier 218 multiplies the mantissas of prenormalized operands A0 and B0.

[0030] To add the products generated by the AND gates, multipliers, and adders, execution path 107 includes a fused adder 334. In some embodiments, to increase precision, the fused adder 224 adds values ​​having a larger bit size than the A, B, and C operands. For example, in some embodiments, the A and B operands are 16-bit values, the C operand is a 32-bit value, the adder generates a 7-bit exponent, and the multiplier generates a 22-bit mantissa product. In these embodiments, the fused adder 334 is capable of adding 52-bit values, thereby preventing loss of precision during mantissa addition.

[0031] The fused adder 224 adds the mantissa product values ​​to generate a temporary mantissa value for D.f32 and provides the temporary mantissa value and the exponent value to the normalization module 336, which normalizes the temporary D.f32 value. For example, in some embodiments, the normalization module 336 shifts the temporary mantissa value to remove any leading zeros in the mantissa. In some embodiments, the normalization module 336 adjusts the temporary mantissa to force the integer portion of the temporary mantissa to a specified value (e.g., 1). Based on the adjustments made to the mantissa, the normalization module adjusts the temporary exponent value to preserve the overall value of D.f32.

[0032] Normalization module 336 provides the adjusted mantissa value and exponent value of D.f32 to rounding module 338. Rounding module 338 rounds the mantissa value based on a specified rounding rule, such as rounding D.f32 to the nearest even value, thereby generating a final value of D.f32. Rounding module 228 provides the final D.f32 value to register file 110 for storage.

[0033] Figure 4 The DOT2_I32_I16 execution path 108 is shown according to some embodiments. As described above, the DOT2_I32_I16 operation multiplies one set of two 16-bit integer operands (named A0 and A1 for descriptive purposes) with corresponding integer operands in another set of two 16-bit integer operands (named B0 and B1 for descriptive purposes) and adds the result to a third 32-bit integer operand (named C). Therefore, the DOT2_I32_I16 operation is represented by the following formula:

[0034] D.i32=A.i16[0]*B.i16[0]+A.i16[1]*B.i16[1]+C.i32

[0035] To implement this operation, execution path 108 includes a set of 32-bit registers (registers 440, 441, and 443) to store operands A0, A1, B0, B1, and operand C. In some embodiments, the operands are stored in different ones of registers 440, 441, and 443 depending on the particular instruction or operation being executed, as described above with respect to Figure 3 Similar to execution path 107 described above. Execution path 108 also includes an operand selection module to select the operands from registers 440, 441, and 443 based on control information provided by the received operation.

[0036] To multiply the A and B operands, execution path 108 includes multipliers 444 and 446. Each of multipliers 444 and 446 multiplies a respective pair of 16-bit operands to generate a 32-bit product. Execution path 108 also includes a 32-bit adder that adds the products generated by multipliers 444 and 446 to each other and to the C operand to generate a temporary value of D.i32. Execution path 108 includes a saturation module 450 that receives the temporary value of D.i32 as well as a clamp value (designated CLMP). Saturation module 450 compares the temporary D.i32 value to the CLMP value. In response to the D.i32 value exceeding the CLMP value, saturation module 450 sets the final D.i32 value to the CLMP value, otherwise the final D.i32 value is set to the temporary D.i32 value. Saturation module 450 provides the final D.i32 value to register file 110 for storage.

[0037] As described herein, in some embodiments, a method includes decoding a first instruction at a processing unit to identify a first multi-precision operation; and performing the first multi-precision operation at an arithmetic logic unit (ALU) by performing a first mathematical operation using operands of different precisions. In one aspect, the first mathematical operation includes a floating point multiply-accumulate operation. In another aspect, the floating point multiply-accumulate operation multiplies two sets of N operands of a first precision and adds an operand of a second precision that is different than the first precision. In one aspect, N is at least two. In another aspect, N is at least four.

[0038] In one aspect, the first mathematical operation includes an integer multiply-accumulate operation. In another aspect, the method includes decoding a second instruction at the processing unit to identify a second multi-precision operation that is different than the first multi-precision operation; and performing the second multi-precision operation at the ALU by performing a second mathematical operation using operands of different precisions, the second mathematical operation being different than the first mathematical operation. In another aspect, performing the first multi-precision operation includes performing the first multi-precision operation at a first execution path of the ALU; and performing the second multi-precision operation includes performing the second multi-precision operation at a second execution path of the ALU, the second execution path being different than the first execution path.

[0039] In some embodiments, a processing unit includes: a decode stage for decoding a first instruction to identify a first multi-precision operation; and an arithmetic logic unit (ALU) for performing the first multi-precision operation by performing the first mathematical operation using operands of different precisions. In one aspect, the first mathematical operation comprises a floating-point multiply-accumulate operation. In another aspect, the floating-point multiply-accumulate operation multiplies two sets of N operands of a first precision and adds operands of a second precision different from the first precision. In one aspect, N is at least two. In another aspect, N is at least four.

[0040] In one aspect, the first mathematical operation comprises an integer multiply-accumulate operation. In another aspect, the decode stage is to decode the second instruction to identify a second multi-precision operation different from the first multi-precision operation; and the ALU is to perform the second multi-precision operation at the ALU by performing the second mathematical operation using operands of different precision, the second mathematical operation different from the first mathematical operation. In another aspect, the ALU comprises: a first execution path for performing the first multi-precision operation; and a second execution path for performing the second multi-precision operation, the second execution path different from the first execution path.

[0041] In some embodiments, the processing unit includes an arithmetic logic unit (ALU) comprising a first execution path for performing a first multi-precision operation by performing a first mathematical operation using operands of different precision; and a second execution path for performing a second multi-precision operation by performing a second mathematical operation using operands of different precision, the second mathematical operation being different from the first execution path. In one aspect, the first mathematical operation includes a first floating-point multiply-accumulate operation, and the second mathematical operation includes a second floating-point multiply-accumulate operation. In another aspect, the first floating-point multiply-accumulate operation multiplies two sets of N operands of first precision and adds operands of second precision different from the first precision, and wherein the second floating-point multiply-accumulate operation multiplies two sets of M operands of first precision and adds operands of second precision. In one aspect, N is at least two and M is at least four.

[0042] In some embodiments, certain aspects of the technology described above may be implemented by one or more processors of a processing system that executes software. The software includes one or more executable instruction sets stored or otherwise tangibly embodied on a non-transitory computer-readable storage medium. The software includes instructions and certain data that manipulate one or more processors to perform one or more aspects of the technology described above when executed by one or more processors. Non-transitory computer-readable storage media include, for example, magnetic or optical disk storage devices, solid-state storage devices (such as flash memory, cache, random access memory (RAM) or one or more other non-volatile memory devices), etc. The executable instructions stored on the non-transitory computer-readable storage medium may be presented in source code, assembly language code, object code, or other instruction formats that are interpreted or otherwise executed by one or more processors.

[0043] It should be noted that not all activities or elements described above in the general description are required, and a portion of a specific activity or device may not be required, and one or more other activities may be performed, or elements other than those described may be included. In addition, the order in which the activities are listed is not necessarily the order in which the activities are performed. Moreover, the concepts have been described with reference to specific embodiments. However, those skilled in the art will appreciate that various modifications and changes may be made without departing from the scope of the present disclosure as set forth in the appended claims. Accordingly, this specification and drawings are to be regarded as illustrative rather than restrictive, and all such modifications are intended to be included within the scope of the present disclosure.

[0044] The benefits, other advantages, and solutions to problems have been described above with reference to specific embodiments. However, the benefits, advantages, solutions to problems, and any features that may cause any benefit, advantage, or solution to the problem to appear or become more prominent should not be construed as key, required, or essential features of any or all of the claims. Furthermore, the specific embodiments disclosed above are illustrative only, as the disclosed subject matter may be modified and practiced in different but equivalent manners that will be apparent to those skilled in the art having the benefit of the teachings herein. No limitation is intended to the details of construction or design shown herein, except as described in the appended claims. Therefore, it is apparent that the specific embodiments disclosed above may be changed or modified, and all such variations are considered to be within the scope of the disclosed subject matter. Accordingly, the protection sought herein is as set forth in the appended claims.

Claims

1. A method comprising: decoding a first instruction [101] at a processing unit [100] to identify a first multiple-precision operation [105] indicated by a single opcode; selecting a first execution path from a plurality of execution paths available at an arithmetic logic unit (ALU) based on the single opcode [106], wherein each execution path in the plurality of execution paths performs a different multiple-precision operation; as well as The first multiple-precision operation is performed via a first execution path of the ALU by performing a first mathematical operation using operands of different precision. 2 . The method of claim 1 , wherein the first mathematical operation comprises a floating-point multiply-accumulate operation.

3. The method of claim 2, wherein the floating-point multiply-accumulate operation multiplies two sets of N operands of a first precision and adds operands of a second precision different from the first precision. The method of claim 3 , wherein N is at least two.

5. The method of claim 4, wherein N is at least four. The method of claim 1 , wherein the first mathematical operation comprises an integer multiply-accumulate operation.

7. The method of claim 1 , further comprising: decoding a second instruction at the processing unit to identify a second multiple-precision operation different from the first multiple-precision operation; as well as The second multiple-precision operation is performed at the ALU by performing a second mathematical operation using operands of different precision, the second mathematical operation being different from the first mathematical operation.

8. The method of claim 7, wherein: Performing the second multiple-precision operation includes performing the second multiple-precision operation at a second execution path [107] of the ALU, the second execution path being different from the first execution path.

9. A processing unit [100] comprising: a decode stage [102] for decoding a first instruction [101] to identify a first multiple precision operation [105] indicated by a single opcode; as well as an arithmetic logic unit (ALU) [104] for selecting a first execution path from a plurality of execution paths available at the arithmetic logic unit based on the single opcode, and performing the first multi-precision operation by performing a first mathematical operation using operands of different precisions, wherein each of the plurality of execution paths performs a different multi-precision operation.

10. The processing unit of claim 9, wherein the first mathematical operation comprises a floating-point multiply-accumulate operation.

11. The processing unit of claim 10, wherein the floating-point multiply-accumulate operation multiplies two sets of N operands of a first precision and adds operands of a second precision different from the first precision.

12. The processing unit of claim 11, wherein N is at least two.

13. The processing unit of claim 12, wherein N is at least four.

14. The processing unit of claim 9, wherein the first mathematical operation comprises an integer multiply-accumulate operation.

15. The processing unit of claim 9, wherein: the decode stage is to decode a second instruction to identify a second multiple-precision operation different from the first multiple-precision operation; and The ALU performs the second multiple-precision operation at the ALU by performing a second mathematical operation using operands of different precision, the second mathematical operation being different from the first mathematical operation.

Images