Apparatus and method for multiple register block by block matrix multiplication

Abstract

Apparatuses and methods for multi-register block-wise matrix multiplication are disclosed. An example processor executes an instruction having an operation object field to indicate that a multi-register cross product matrix multiplication is to be performed with a first plurality of submatrices and a second plurality of submatrices to generate a third plurality of submatrices. An execution circuit executes the instruction to generate each of the third plurality of submatrices in a corresponding vector register of a third plurality of vector registers by multiplying a submatrix in a corresponding vector register of the first plurality of vector registers with a submatrix in a corresponding vector register of the second plurality of vector registers.

Description

Technical Field

[0001] This invention generally relates to the field of computer processors. More specifically, this invention relates to apparatus and methods for multi-register block-by-block matrix multiplication. Background Technology

[0002] A matrix is ​​typically represented as a two-dimensional arrangement of numbers in rows and columns. For example, a 4×4 matrix may have 16 numbers arranged in 4 rows and 4 columns. Matrices are used in a variety of applications, including graphics processing and high-performance computing. Matrix processing instructions have been developed to perform common machine learning tasks such as matrix multiplication and convolution. For example, a matrix multiplication instruction can specify two source matrix operands (matrix A and matrix B), which are multiplied to produce a result matrix. Some matrix multiplication instructions add the result of the matrix multiplication to the values ​​in an accumulation matrix (matrix C) to produce a result matrix (matrix C / D).

[0003] Existing matrix processing instructions double the number of computations by using vector length orthogonality. As the demands of machine learning data processing continue to increase, new instructions and microarchitectures will be needed to keep up with these needs. Attached Figure Description

[0004] A better understanding of the invention can be obtained from the following detailed description, in conjunction with the accompanying drawings, in which:

[0005] Figure 1 This is a block diagram of an embodiment of a processor that operates to execute instructions (e.g., vector compaction matrix multiplication and accumulation instructions).

[0006] Figure 2 This is a block diagram of an embodiment of a processor that uses a first matrix and a second matrix, both having 32-bit data elements, to perform vector compaction matrix multiplication and accumulation instructions.

[0007] Figure 3 This is a block diagram of an embodiment of a processor that uses a first matrix and a second matrix, both having 16-bit data elements, to perform vector compaction matrix multiplication and accumulation instructions.

[0008] Figure 4 This is a block diagram of an embodiment of a processor that uses a first matrix and a second matrix, both having 8-bit data elements, to perform vector compaction matrix multiplication and accumulation instructions.

[0009] Figure 5 This is a block diagram of a more detailed example embodiment of a processor that operates to perform vector-compacted matrix multiplication and accumulation instructions.

[0010] Figure 6This is a flowchart of an embodiment of a method for executing vector-compacted matrix multiplication and accumulation instructions.

[0011] Figure 7 This is a flowchart of an embodiment of a method for performing vector-compacted matrix multiplication and accumulation instructions using simulation or binary translation.

[0012] Figure 8 This is a block diagram of an example embodiment of a suitable set of vector registers.

[0013] Figure 9 The diagram illustrates a computing system.

[0014] Figure 10 The diagram shows a block diagram of an example processor and / or System on a Chip (SoC) that may have one or more cores and an integrated memory controller.

[0015] Figure 11(A) is a block diagram illustrating both the example ordered pipeline and the example register renaming, out-of-order issue / execution pipeline, based on the example.

[0016] Figure 11(B) is a block diagram illustrating both the example ordered architecture core to be included in the processor and the example register renaming, out-of-order issue / execution architecture core, according to the example.

[0017] Figure 12 The diagram illustrates an example of one or more execution unit circuits.

[0018] Figure 13 It is a block diagram of a register architecture based on some examples.

[0019] Figure 14 An example of a command format is shown in the diagram.

[0020] Figure 15 This diagram illustrates an example of an addressing information field.

[0021] Figure 16 The diagram shows an example of the first prefix.

[0022] Figures 17(A)-17(D) How to use the diagram Figure 16 Examples of the R, X, and B fields in the first prefix.

[0023] Figures 18(A)-18(B) The illustration shows an example of the second prefix.

[0024] Figure 19 The diagram shows an example of the third prefix.

[0025] Figure 20This is a block diagram illustrating, based on an example, the use of a software instruction converter to translate binary instructions from a source instruction set architecture into binary instructions from a target instruction set architecture.

[0026] Figures 21A-21B The diagram illustrates an example implementation of multiplying two source matrices to generate a result matrix.

[0027] Figure 22 The diagram illustrates an example implementation of generating a result matrix through partial cross-multiplication of two source matrices.

[0028] Figure 23 The diagram illustrates the architecture on which matrix multiplication instructions can be executed.

[0029] Figures 24A-24C The diagram illustrates various configurations of tensor processing circuitry within a processor or core.

[0030] Figure 25 The diagram illustrates methods based on some implementation approaches.

[0031] Figures 26A-26B The diagram illustrates different combinations of vector registers assigned to perform matrix multiplication between two source matrices to generate a result matrix.

[0032] Figure 27 The diagram shows an accumulation matrix array based on some implementation methods.

[0033] Figure 28 This diagram illustrates an example implementation of a core or processor architecture.

[0034] Figure 29 The diagram illustrates a set of example instructions for performing matrix operations.

[0035] Figure 30 The diagram illustrates a method for performing multi-register block-by-block matrix multiplication. Detailed Implementation

[0037] This document discloses embodiments of instructions, embodiments of processors for executing instructions, embodiments of methods performed by a processor when executing instructions, embodiments of systems comprising one or more processors for executing instructions, and embodiments of programs or machine-readable media that store or otherwise provide instructions. Numerous specific details (e.g., specific instruction operations, data formats, processor configurations, microarchitectural details, sequences of operations, etc.) are set forth in the following description. However, embodiments may be practiced without these specific details. In other instances, well-known circuits, structures, and techniques have not been shown in detail to avoid obscuring the understanding of this specification.

[0038] Figure 1This is a block diagram of an embodiment of a processor 100 that operates to execute instructions 101 (e.g., vector-compacted matrix multiplication and accumulation instructions). In some embodiments, the processor may be a general-purpose processor (e.g., a general-purpose microprocessor or central processing unit (CPU) of the type used in desktop computers, laptop computers, servers, smartphones, and other computer systems). Alternatively, the processor may be a special-purpose processor. Examples of suitable special-purpose processors include, but are not limited to, machine learning processors, artificial intelligence processors, coprocessors, graphics processors, network processors, communication processors, cryptographic processors, and digital signal processors (DSPs). The processor may have any of a variety of complex instruction set computing (CISC) architectures, reduced instruction set computing (RISC) architectures, very long instruction word (VLIW) architectures, hybrid architectures, other types of architectures, or combinations of different architectures (e.g., different cores may have different architectures). In some embodiments, the processor may include at least one integrated circuit or semiconductor die (e.g., disposed on at least one integrated circuit or semiconductor die). In some embodiments, the processor may include at least some kind of hardware (e.g., transistors, capacitors, circuits, non-volatile memory for storing circuit-level instructions / control signals).

[0039] Processor 100 may be coupled to receive instruction 101. For example, the processor may have an interface for coupling with an interconnect to receive instructions from memory via that interconnect. Instructions may represent macro instructions, machine code instructions, or other instructions from the processor's instruction set. Instructions may have various formats or encodings, such as, for example, hereinafter (e.g., for...) Figures 14-19 The format or encoding described further. The instruction has one or more fields for the opcode, which specifies at least partially or completely the operation to be performed (e.g., matrix multiplication and accumulation).

[0040] In some embodiments, instructions may explicitly specify (e.g., by one or more fields or sets of bits) or otherwise indicate (e.g., implicitly) a first (e.g., source) matrix 105, a second (e.g., source) matrix 107, and a third (e.g., source) matrix 109. For example, as shown in the illustrated embodiment, instructions may specify or otherwise indicate a first (e.g., source) vector register 106 storing the first matrix, a second (e.g., source) vector register 108 storing the second matrix, and a third (e.g., source / destination) vector register 110 storing the third matrix. Vector registers are sometimes also referred to in the art as compact data registers and single instruction multiple data (SIMD) registers. Alternatively, one or more of the first, second, and third matrices may optionally be stored in memory or other storage locations. For example, as indicated by the dashed line representing the second vector register, in another embodiment, the second matrix may optionally be stored in a memory location. Instructions may have source and / or destination operation object specification fields to specify registers, memory locations, or other storage locations for the first, second, and third matrices.

[0041] In the illustrated embodiment, the first vector register has a 128-bit channel or portion 111 for storing a first matrix 105, the second vector register has a 128-bit channel or portion 112 for storing a second matrix 107, and the third vector register has a 128-bit channel or portion 113 for storing a third matrix 109. The first matrix may have 2 rows (e.g., M=2) multiplied by K columns of data elements, each of which has a first number of bits, where K equals 64 bits divided by the first number of bits. The second matrix may have K rows multiplied by 2 columns (e.g., N=2) of data elements, each of which has a first number of bits. K may represent a common dimension shared by the two matrices to be multiplied, commonly referred to as the K-dimensionality or inner product dimension. The third matrix may have 2 rows multiplied by 2 columns of data elements, each of which has a second number of bits, where the second number of bits is greater than the first number of bits. In some embodiments, each of the first, second, and third vector registers may be a 128-bit register with only the corresponding 128-bit channel. In other embodiments, each of the first, second, and third vector registers may be a 256-bit register with two 128-bit channels. In still other embodiments, each of the first, second, and third vector registers may be a 512-bit register with four 128-bit channels. In still other embodiments, each of the first, second, and third vector registers may have another number of 128-bit channels (e.g., a 384-bit register may have three 128-bit channels, a 640-bit register may have five 128-bit channels, a 768-bit register may have six 128-bit channels, a 1024-bit register may have eight 128-bit channels, a 2048-bit register may have sixteen 128-bit channels, and so on). In some embodiments, each of two or more 128-bit channels of a vector register may be used to store a corresponding matrix with the same dimensions and data element sizes as those mentioned for the first 128-bit channel of the same vector register. Specific examples of suitable 128-bit, 256-bit, and 512-bit vector registers include, but are not limited to, the 128-bit xmm, 256-bit ymm, and 512-bit zmm registers mentioned elsewhere in this document.

[0042] In various embodiments, the first number of bits for the data elements of the first and second matrices can be 2 bits, 4 bits, 8 bits, 16 bits, or 32 bits. For a 2-bit data element, K can be 32. For a 4-bit data element, K can be 16. For an 8-bit data element, K can be 8. For a 16-bit data element, K can be 4. For a 32-bit data element, K can be 2. Examples of suitable types for the 2-bit and 4-bit data elements of the first and second matrices include, but are not limited to, 2-bit and 4-bit signed and unsigned integers. Examples of suitable types for the 8-bit data elements of the first and second matrices include, but are not limited to, 8-bit signed integers (S8), 8-bit unsigned integers (U8), and 8-bit floating-point data elements (FP8). Examples of suitable types for the 8-bit floating-point data elements of the first and second matrices include, but are not limited to, bfloat8 (BF8) with 5 exponent bits and 2 explicit mantissa / significant bits, and hfloat8 (HF8) with 4 exponent bits and 3 explicit mantissa / significant bits. In various embodiments, both the first matrix and the second matrix may have S8 data elements, both the first matrix and the second matrix may have U8 data elements, the first matrix may have S8 data elements and the second matrix may have U8 data elements, or the first matrix may have U8 data elements and the second matrix may have S8 data elements. In various other embodiments, both the first matrix and the second matrix may have BF8 data elements, both the first matrix and the second matrix may have HF8 data elements, the first matrix may have BF8 data elements and the second matrix may have HF8 data elements, or the first matrix may have HF8 data elements and the second matrix may have BF8 data elements. Examples of suitable types of 16-bit data elements for the first matrix and the second matrix include, but are not limited to, 16-bit signed integers (S16), 16-bit unsigned integers (U16), half-precision floating-point data elements (float16, i.e., F16), and bfloat16 (BF16) floating-point data elements. In various embodiments, both the first matrix and the second matrix may have F16 data elements, or both the first matrix and the second matrix may have BF16 data elements. Examples of suitable types of 32-bit data elements for the first and second matrices include, but are not limited to, 32-bit signed integers (S32), 32-bit unsigned integers (U32), single-precision floating-point data elements (float32, or F32), and tensor floating-point 32-bit floating-point data elements (TF32) with 8 exponent bits and 10 explicit mantissa / significant bits. In some embodiments, both the first and second matrices may have TF32 data elements.

[0043] Vector registers are conventionally used to store vectors, but not typically for storing matrices. In the art, vectors are sometimes also referred to as compressed data and SIMD data. A vector represents a one-dimensional array or data structure. In contrast, in some embodiments, at least one vector register may be used to store a matrix. A matrix may represent data elements (e.g., numbers or values) arranged in rows and columns in a two-dimensional layout. However, a matrix may be stored in a single vector register (e.g., in a single vector and / or one-dimensional layout) rather than in a slice, a two-dimensional collection of registers, or other two-dimensional storage structures. Such matrices may also be referred to herein as vector-compressed matrices because the matrix is ​​compressed into a vector or one-dimensional layout.

[0044] Vector registers 106, 108, and 110 may represent architecture-visible or architectural registers that are visible to software and / or programmers, and / or registers indicated by instructions in the processor's instruction set to identify the objects of operation. These architectural registers contrast with other non-architectural registers in the microarchitecture, such as temporary registers, reorder buffers, retirement registers, etc. These vector registers may be implemented in different microarchitectures and in different ways, and they are not limited to any particular design. Examples of suitable types of vector registers include, but are not limited to, dedicated physical registers, dynamically allocated physical registers using register renaming, and combinations thereof.

[0045] Refer again Figure 1The processor includes decoder circuitry 102 (e.g., an instruction decoder). The decoder circuitry may be coupled to receive instructions. The decoder circuitry may operate to decode instructions into one or more lower-level control signals, operations, or decoded instructions 104 (e.g., one or more microinstructions, microoperations, microcode entry points, etc.). In some embodiments, the decoder circuitry may include: at least one input structure (e.g., a port, interconnect, or interface) coupled to receive instructions; instruction recognition and decoding logic coupled to at least one input structure to recognize instructions and decode them into one or more lower-level control signals, operations, or decoded instructions; and at least one output structure (e.g., a port, interconnect, or interface) coupled to the instruction recognition and decoding logic to output the one or more lower-level control signals, operations, or decoded instructions. Various instruction decoding mechanisms may be used to implement the decoder circuitry and / or its instruction recognition and decoding logic, including but not limited to microcode read-only memory (ROM), lookup tables, hardware implementations, programmable logic arrays (PLAs), other mechanisms suitable for implementing the instruction decoder circuitry, and combinations thereof. In some embodiments, the decoder circuitry may include at least some type of hardware (e.g., transistors, integrated circuits, on-die read-only memory, or other non-volatile memory, and any combination thereof, storing microcode or other hardware-level instructions). In some embodiments, the decoder circuitry may be included on a die, integrated circuit, or semiconductor substrate.

[0046] Execution circuitry 103 (e.g., an execution unit) is coupled to decoder circuitry 102 (e.g., to receive one or more lower-level control signals, operations, or decoded instructions 104). The execution circuitry is also coupled to receive a first matrix 105, a second matrix 107, and a third matrix 109 (e.g., coupled to a first vector register 106, a second vector register 108, and a third vector register 110). In some embodiments, the execution circuitry may be located on a die or integrated circuit together with the decoder circuitry. The execution circuitry is operable to perform operations corresponding to and / or specified by and / or controlled by instruction 101. For example, one or more lower-level control signals, operations, or decoded instructions may be executed by the execution circuitry to control the execution circuitry to perform an operation corresponding to the instruction (e.g., an operation at least partially specified by the instruction's opcode).

[0047] In some embodiments, the operation may include generating a result matrix 115 having 2 rows by 2 columns (e.g., M=2, N=2) result data elements, each having a second number of bits. In some embodiments, the second number of bits is 32 bits. In some embodiments, the result matrix may represent the sum of a third matrix 109 (e.g., an accumulation matrix) and a product matrix (e.g., having 2 rows by 2 columns), which is generated from matrix multiplication using and / or involving and / or based on the first matrix 105 and the second matrix 107. As an example, a product matrix having M rows by N columns may be generated from matrix multiplication involving a first matrix having M rows by K columns and a second matrix having K rows by N columns, where M and N are each 2. In some embodiments, the product matrix may be generated from matrix multiplication using the first matrix and the second matrix, wherein, prior to the matrix multiplication, the data elements of the first matrix and the second matrix are converted from a first number of bits to a larger number of bits (e.g., from 8 bits or 16 bits to 32 bits). In some embodiments, the product matrix may be generated from matrix multiplication using a first matrix and a second matrix, wherein floating-point rounding may be optionally performed as needed during matrix multiplication and / or accumulation. In various embodiments, rounding may be performed as needed after each multiplication used to generate each result element, rounding may be performed as needed after each accumulation operation used to generate each result element, rounding may be performed as needed after each multiplication used to generate each result element and also as needed after each accumulation operation used to generate each result element, and rounding may be performed once as needed after all operations used to generate the result element. In some embodiments, the operation may include storing the result matrix in a 128-bit channel of a third vector register initially used to store a third matrix. That is, the third vector register may be a source / destination vector register initially used as a source of the third vector and subsequently implicitly reused as a source / destination vector register for the destination where the result matrix is ​​to be stored.

[0048] In some embodiments, in order to generate and store the result matrix, the execution circuit may perform the following operations on each column n of the two columns of the second matrix and on each row m of the two rows of the first matrix: (1) converting K data elements from row m of the first matrix into K corresponding converted data elements each having more bits than a first number of bits, and converting K data elements from column n of the second matrix into K corresponding converted data elements each having more bits than a first number of bits; (2) generating K products, including multiplying the K converted data elements corresponding to row m with the K converted data elements corresponding to column n, wherein generating K products may optionally include performing rounding as needed; (3) generating a result data element having a second number of bits, including accumulating the K products with the data elements of the corresponding row m and the corresponding column n of the two rows of the third matrix, wherein generating the result data element may optionally include performing rounding as needed; and (4) storing the result data element in a 128-bit channel of a third vector register at a position corresponding to row m and column n of the third matrix. As mentioned immediately above with respect to item (3), the accumulation of the K products with data elements from the third matrix can be performed in various different ways, including in various different orders. In some embodiments, in order to generate and store the resulting matrix, the execution circuitry may perform any of the operations of the example embodiments of the instructions further shown and described below.

[0049] In some embodiments, the first vector register 106 may optionally have a second 128-bit channel to store a fourth matrix having 2 rows by K columns of data elements, each of which has a first number of bits; the second vector register 108 may optionally have a second 128-bit channel to store a fifth matrix having K rows by 2 columns of data elements, each of which has a first number of bits; and the third vector register 110 may optionally have a second 128-bit channel to store a sixth matrix having 2 rows by 2 columns of data elements, each of which has a second number of bits. In such embodiments, to perform an operation corresponding to an instruction, the execution circuitry may further generate a second result matrix having 2 rows by 2 columns (e.g., M=2, N=2) of result data elements, each of which has a second number of bits. In some embodiments, the second result matrix may represent the sum of the sixth matrix (e.g., a second accumulation matrix) and a product matrix (e.g., having 2 rows by 2 columns) generated from matrix multiplication using the fourth and fifth matrices. The second result matrix may be stored in the second 128-bit channel of the third vector register. In other embodiments, an additional 128-bit channel or portion may be present. Individual 2×2×K matrix multiplication and accumulation operations can be performed on each of these 128-bit channels or portions of the vector register.

[0050] In some embodiments, the processor may have registers (e.g., MXCSR or other floating-point control registers) for controlling floating-point operations. In some embodiments, the registers may have one or more fields (e.g., rounding mode fields) for specifying one of a plurality of supported rounding modes to be used in the floating-point operation. In some embodiments, in order to perform an operation corresponding to an instruction, when at least some data elements of a matrix are floating-point data elements, the execution circuitry may perform rounding according to only a single rounding mode (e.g., nearest-even rounding mode), regardless of and / or regardless of one or more fields in the register (e.g., rounding mode fields), and / or without checking those one or more fields. In some such embodiments, rounding may be performed according to a single rounding mode in a mathematically precise manner, rather than by simplification that results in mathematically inaccurate results. In some embodiments, the registers may have one or more fields (e.g., denormals-are-zero (DAZ) bits or fields) for specifying whether denormalized values ​​in the input (e.g., source matrix) to the floating-point instruction / operation are to be set to zero, zeroed, or treated as zero. In some such embodiments, in order to perform an operation corresponding to an instruction, when at least some data elements of the matrix are floating-point data elements, the execution circuit may set or make the non-regular number in the input to the floating-point operation zero, or treat the non-regular number in the input to the floating-point operation as zero, regardless of and / or regardless of whether one or more fields in the register (e.g., DAZ bits or fields) specify whether the non-regular number in the input to the floating-point operation should be forced to zero, and / or need not check whether the one or more fields specify whether the non-regular number in the input to the floating-point operation should be forced to zero. In some embodiments, the register may have one or more fields (e.g., Flush-To-Zero (FTZ) bits or fields) for specifying whether the non-regular result of the floating-point operation should be set to zero, forced to zero, or zeroed. In some such embodiments, in order to perform an operation corresponding to an instruction, when at least some data elements of the matrix are floating-point data elements, the execution circuitry may set the non-normal result of the floating-point operation to zero, force the non-normal result to zero, or set the non-normal result to zero, regardless of whether one or more fields in the register (e.g., FTZ bits or fields) specify whether the non-normal result of the floating-point operation should be set to zero, forced to zero, or set to zero, and / or without checking whether the one or more fields specify whether the non-normal result of the floating-point operation should be set to zero, forced to zero, or set to zero. In some embodiments, when at least some data elements of the matrix are floating-point data elements, the execution circuitry may perform the operation corresponding to the instruction without accessing registers (e.g., MXCSR or other floating-point control registers) (e.g., without reading from or writing to registers).

[0051] The execution circuitry and / or processor may include specific or particular logic (e.g., transistors, integrated circuits, or other hardware potentially combined with firmware (e.g., instructions stored in non-volatile memory) and / or software) for operating to execute instructions and / or in response to and / or due to instructions (e.g., in response to one or more lower-level control signals, operations, or decoded instructions that have been decoded from the instructions). In some embodiments, the execution circuitry may include multiplication circuitry (e.g., matrix multiplication circuitry), addition circuitry (e.g., matrix addition circuitry), floating-point rounding circuitry, and circuitry for performing other operations described herein. In some embodiments, the execution circuitry may include: one or more input structures (e.g., ports, interconnects, or interfaces) coupled to receive first, second, and third matrices; circuitry or logic coupled to the one or more input structures for receiving and processing these matrices to generate a result matrix; and one or more output structures (e.g., ports, interconnects, or interfaces) coupled to the circuitry or logic for outputting the result matrix (e.g., storing the result matrix in a vector register).

[0052] Figure 2 This is a block diagram of an embodiment of processor 200, which executes vector compacted matrix multiplication and accumulation instructions 201 to perform vector compacted matrix multiplication and accumulation operations using a first matrix 205 and a second matrix 207, each having 32-bit data elements. The processor includes decoder circuitry 202 for decoding the vector compacted matrix multiplication and accumulation instructions. The decoder circuitry is compatible with... Figure 1 The described decoder circuit 102 is similar or identical. Instructions may specify or otherwise indicate a first vector register 206 having a 128-bit channel for storing a first matrix 205, a second vector register 208 having a 128-bit channel for storing a second matrix 207, and a third vector register 210 having a 128-bit channel for storing a third matrix 209.

[0053] For these 32-bit data elements, the first and second matrices share a common dimension K=2 (e.g., K=2=64 / 32). The first matrix has four 32-bit data elements arranged in 2 rows by 2 columns, labeled as: A 0,0 In bits [31:0]; A 1,0 In bits [63:32]; A 0,1 In bits [95:64]; and A 1,1 In bits [127:96], where element A is ordered by A k,mThis is used for indexing. The second matrix has four 32-bit data elements arranged in 2 rows by 2 columns, labeled as: B 0,0 In bits [31:0]; B 0,1 In bits [63:32]; B 1,0 In bits [95:64]; and B 1,1 In bits [127:96], where element B is ordered by B n,k The third matrix has four 32-bit data elements arranged in 2 rows by 2 columns, and these data elements are labeled with C in bits [31:0]. 0,0 C in bit [63:32] 1,0 C in bit [95:64] 0,1 And C in bits [127:96] 1,1 , where C elements are arranged according to C n,m This arrangement of 32-bit data elements within a matrix can tend to be more efficient than some other arrangements (e.g., more efficient in managing smaller matrices as part of a general algorithm that multiplies and accumulates larger matrices), but other arrangements may be used optionally where desired.

[0054] The execution circuit 203 is coupled to the decoder circuit 202. The execution circuit performs vector-compacted matrix multiplication and accumulation operations corresponding to the instructions. These operations may include generating a result matrix 215. The result matrix may have four 32-bit data elements arranged in a 2x2 column configuration, labeled as: D 0,0 In bits [31:0]; D 1,0 In bits [63:32]; D 0,1 In bits [95:64]; and D 1,1 In bits [127:96], where the D element is ordered by D n,m This is used for indexing. The resulting matrix can represent the sum of the product matrix of the third matrix 209 and the matrix generated from the matrix multiplication using the first matrix 205 and the second matrix 207. This can be done as elsewhere in this document (e.g., for...). Figure 1 The execution circuitry may perform the operation as described, for example, by optionally using conversion, optionally by using rounding as needed, etc. The execution circuitry may also store the resulting matrix in a 128-bit channel of the third vector register. For example, the resulting matrix may overwrite the third matrix in the 128-bit channel of the third vector register.

[0055] In some embodiments, the 32-bit data elements of the first and second matrices may each be TF32 data elements. In some embodiments, the 32-bit data elements of the third and fourth matrices may each be single-precision floating-point (F32) data elements. In some embodiments, each vector register may optionally include more than one 128-bit channel (e.g., two, four, eight, or some other number of 128-bit channels), wherein each 128-bit channel stores a corresponding matrix, and the operation may further include performing separate multiplication and accumulation operations on the matrices of these different channels, similar to the multiplication and accumulation operations described herein. In some embodiments, the vector compact matrix multiplication and accumulation instruction 201 may be a VMMTF32PS instruction as further described below, and the execution circuitry may perform any of the operations described below for the VMMTF32PS instruction.

[0056] Figure 3 This is a block diagram of an embodiment of processor 300, which executes vector compacted matrix multiplication and accumulation instructions 301 to perform vector compacted matrix multiplication and accumulation operations using a first matrix 305 and a second matrix 307, each having 16-bit data elements. The processor includes a decoder circuit 302 for decoding the vector compacted matrix multiplication and accumulation instructions. The decoder circuit can be used with... Figure 1 The described decoder circuit 102 is similar or identical. Instructions may specify or otherwise indicate a first vector register 306 having a 128-bit channel for storing a first matrix 305, a second vector register 308 having a 128-bit channel for storing a second matrix 307, and a third vector register 310 having a 128-bit channel for storing a third matrix 309.

[0057] For these 16-bit data elements, the first and second matrices share a common dimension K=4 (e.g., K=4=64 / 16). The first matrix has eight 16-bit data elements arranged in 2 rows by 4 columns, labeled as: A 0,0 In bits [15:0]; A 1,0 In bits [31:16]; A 2,0 In bits [47:32]; A 3,0 In bits [63:48]; A 0,1 In bits [79:64]; A 1,1 In bits [95:80]; A 2,1 In bits [111:96]; and A 3,1 In bits [127:112], where element A is ordered by A k,mTo index. The second matrix has eight 16-bit data elements arranged in 4 rows by 2 columns, labeled as: B 0,0 In bits [15:0]; B 0,1 In bits [31:16]; B 0,2 In bits [47:32]; B 0,3 In bits [63:48]; B 1,0 In bits [79:64]; B 1,1 In bits [95:80]; B 1,2 In bits [111:96]; and B 1,3 In bits [127:112], where element B is ordered by B n,k To index. The third matrix has four 32-bit data elements arranged in 2 rows by 2 columns, labeled as: C 0,0 In bits [31:0]; C 1,0 In bits [63:32]; C 0,1 In bits [95:64]; and C 1,1 In bits [127:96], where C elements are arranged in C... n,m This arrangement of 16-bit and 32-bit data elements within a matrix tends to be more efficient than some other arrangements (e.g., more efficient in managing smaller matrices as part of a larger overall algorithm for multiplying and accumulating larger matrices), but other arrangements may be used optionally where desired.

[0058] The execution circuit 303 is coupled to the decoder circuit 302. The execution circuit performs vector-compacted matrix multiplication and accumulation operations corresponding to the instructions. These operations may include generating a result matrix 315. The result matrix may have four 32-bit data elements arranged in a 2x2 column configuration, labeled as: D 0,0 In bits [31:0]; D 1,0 In bits [63:32]; D 0,1 In bits [95:64]; and D 1,1 In bits [127:96], where the D element is ordered by D n,m This is used for indexing. The resulting matrix can represent the sum of the product matrix of the third matrix 309 and the matrix generated from the matrix multiplication using the first matrix 305 and the second matrix 307. This can be done as elsewhere in this document (e.g., for...). Figure 1The execution circuitry may perform the operation as described, for example, by optionally using conversion, optionally by using rounding as needed, etc. The execution circuitry may also store the resulting matrix in a 128-bit channel of the third vector register. For example, the resulting matrix may overwrite the third matrix in the 128-bit channel of the third vector register.

[0059] In some embodiments, the 16-bit data elements of the first and second matrices may each be bfloat16 (BF16) data elements. In other embodiments, the 16-bit data elements of the first and second matrices may each be half-precision floating-point (float16, i.e., F16) data elements. In some embodiments, the 32-bit data elements of the third and fourth matrices may each be single-precision floating-point (F32) data elements. In some embodiments, each vector register may optionally include more than one 128-bit channel (e.g., two, four, eight, or some other number of 128-bit channels), wherein each 128-bit channel stores a corresponding matrix, and the operation may further include performing separate multiplication and accumulation operations on the matrices of these different channels, similar to the multiplication and accumulation operations described herein. In some embodiments, the vector compact matrix multiplication and accumulation instruction 301 may be the VMMBF16PS instruction described further below, and the execution circuitry may perform any of the operations described below for the VMMBF16PS instruction. In other embodiments, the vector compaction matrix multiplication and accumulation instruction 301 may be a VMMF16PS instruction as further described below, and the execution circuitry may perform any of the operations described below for the VMMF16PS instruction.

[0060] Figure 4 This is a block diagram of an embodiment of processor 400, which executes vector compaction matrix multiplication and accumulation instructions 401 to perform vector compaction matrix multiplication and accumulation operations using a first matrix 405 and a second matrix 407, each having 8-bit data elements. The processor includes decoder circuitry 402 for decoding the vector compaction matrix multiplication and accumulation instructions. The decoder circuitry is compatible with... Figure 1 The described decoder circuit 102 is similar or identical. Instructions may specify or otherwise indicate a first vector register 406 having a 128-bit channel for storing a first matrix 405, a second vector register 408 having a 128-bit channel for storing a second matrix 407, and a third vector register 410 having a 128-bit channel for storing a third matrix 409.

[0061] For these 8-bit data elements, the first and second matrices share a common dimension K=8 (e.g., K=8=64 / 8). The first matrix has sixteen 8-bit data elements arranged in 2 rows by 8 columns, labeled as: A 0,0 In bits [7:0]; A 1,0 In bits [15:8]; A 2,0 In bits [23:16]; A 3,0 In bits [31:24]; A 4,0 In bits [39:32]; A 5,0 In bits [47:40]; A 6,0 In bits [55:48]; and A 7,0 In bits [63:56]; A 0,1 In bits [71:64]; A 1,1 In bits [79:72]; A 2,1 In bits [87:80]; A 3,1 In bits [95:88]; A 4,1 In bits [103:96]; A 5,1 In bits [111:104]; A 6,1 In bits [119:112]; and A 7,1 In bits [127:120], where element A is ordered by A k,m This is used for indexing. The second matrix has sixteen 8-bit data elements arranged in 8 rows and 2 columns, labeled as: B 0,0 In bits [7:0]; B 0,1 In bits [15:8]; B 0,2 In bits [23:16]; B 0,3 In bits [31:24]; B 0,4 In bits [39:32]; B 0,5 In bits [47:40]; B 0,6 In bits [55:48]; B 0,7 In bits [63:56]; B 1,0 In bits [71:64]; B 1,1 In bits [79:72]; B 1,2 In bits [87:80]; B 1,3 In bits [95:88]; B 1,4 In bits [103:96]; B 1,5 In bits [111:104]; B 1,6 In bits [119:112]; and B1,7 In bits [127:120], where element B is ordered by B n,k This is used for indexing. The third matrix has four 32-bit data elements arranged in 2 rows by 2 columns, labeled as: C 0,0 In bits [31:0]; C 1,0 In bits [63:32]; C 0,1 In bits [95:64]; and C 1,1 In bits [127:96], where C elements are arranged in C... n,m This arrangement of 8-bit and 32-bit data elements within a matrix tends to be more efficient than some other arrangements (e.g., more efficient in managing smaller matrices as part of a larger overall algorithm for multiplying and accumulating matrices), but other arrangements may be used optionally where desired.

[0062] The execution circuit 403 is coupled to the decoder circuit 402. The execution circuit performs vector-compacted matrix multiplication and accumulation operations corresponding to the instructions. These operations may include generating a result matrix 415. The result matrix may have four 32-bit data elements arranged in a 2x2 column configuration, labeled as: D 0,0 In bits [31:0]; D 1,0 In bits [63:32]; D 0,1 In bits [95:64]; and D 1,1 In bits [127:96], where the D element is ordered by D n,m This is used for indexing. The resulting matrix can represent the sum of the product matrix of the third matrix 409 and the matrix generated from the matrix multiplication using the first matrix 405 and the second matrix 407. This can be done as elsewhere in this document (e.g., for...). Figure 1 The execution circuitry may perform the operation as described, for example, by optionally using conversion, optionally by using rounding as needed, etc. The execution circuitry may also store the resulting matrix in a 128-bit channel of the third vector register. For example, the resulting matrix may overwrite the third matrix in the 128-bit channel of the third vector register.

[0063] In some embodiments, the 8-bit data elements of the first and second matrices may each be bfloat8 (BF8) data elements. In other embodiments, the 8-bit data elements of the first and second matrices may each be hfloat8 (HF8) data elements. In other embodiments, the 8-bit data elements of the first matrix may each be bfloat8 (BF8) data elements, and the 8-bit data elements of the second matrix may each be hfloat8 (HF8) data elements. In other embodiments, the 8-bit data elements of the first matrix may each be hfloat8 (HF8) data elements, and the 8-bit data elements of the second matrix may each be bfloat8 (BF8) data elements. In any of the embodiments described above in this paragraph, the 32-bit data elements of the third and fourth matrices may each be single-precision floating-point (F32) data elements.

[0064] In other embodiments, the 8-bit data elements of the first and second matrices may each be 8-bit signed integer (S8) data elements. In other embodiments, the 8-bit data elements of the first and second matrices may each be 8-bit unsigned integer (U8) data elements. In other embodiments, the 8-bit data elements of the first matrix may each be 8-bit signed integer (S8) data elements, and the 8-bit data elements of the second matrix may each be 8-bit unsigned integer (U8) data elements. In other embodiments, the 8-bit data elements of the first matrix may each be 8-bit unsigned integer (U8) data elements, and the 8-bit data elements of the second matrix may each be 8-bit signed integer (S8) data elements. In any of the embodiments described above in this paragraph, the 32-bit data elements of the third and fourth matrices may each be 32-bit signed integer (int32) data elements. In some embodiments, each vector register may optionally include more than one 128-bit channel (e.g., two, four, eight, or some other number of 128-bit channels), wherein each 128-bit channel stores a corresponding matrix, and the operation may further include performing separate multiplication and accumulation operations on the matrices of these different channels, similar to the multiplication and accumulation operations described below. In some embodiments, the vector compaction matrix multiplication and accumulation instruction 401 may be any of the VPMMUUBD, VPMMSSBD, VPMMUSBD, VPMMSUBD, VMMBF8PS, VMMHF8PS, VMMBHF8PS, and VMMHBF8PS instructions described further below, and the execution circuitry may perform any of the operations described below for any of these instructions.

[0065] The following describes example embodiments of the instructions. These instructions are labeled VMMTF32PS, VMM[BF16,F16]PS, VPMM[UU,SS,US,SU]BD, and VMM[B,H,BH,HB]F8PS. In some embodiments, VMM[BF16,F16]PS may be two separate instructions (e.g., one instruction for BF16 and another for F16), while in other embodiments, it may be a single instruction having an immediate value, field, or operand for selecting or indicating one of BF16 and F16. Similarly, in some embodiments, VPMM[UU,SS,US,SU]BD may be four separate instructions (e.g., one instruction for each of UU, SS, US, and SU), while in other embodiments, it may be a single instruction having an immediate value, field, or operand for selecting or indicating one of UU, SS, US, and SU. Similarly, in some embodiments, VMM[B,H,BH,HB]F8PS can be four separate instructions (e.g., one instruction for each of B, H, BH, and HB), while in other embodiments it can be a single instruction having an immediate value, field, or operand for selecting or indicating one of B, H, BH, and HB.

[0066] In the pseudocode below, "src1 (source 1)" specifies the first source operand, "src2 (source 2)" specifies the second source operand, "DEST (destination)" specifies the destination operand, "srcdest (source destination)" specifies the source operand implicitly reused as the destination operand, and "TMP (temporary)" specifies the temporary value generated during the operation. Further, in the x86 instruction set architecture (ISA), "xmm", "ymm", and "zmm" specify 128-bit, 256-bit, and 512-bit registers, respectively. These broadly represent 128-bit, 256-bit, and 512-bit registers. "ModRM:reg(r,w)", "ModRM: 11:rrr:bbb", "VEX.vvvv(r)", and "ModRM:reg(r)" specify the operand addressing mode used in the x86 ISA. ModRM: 11:rrr:bbb The first two set bits "11" specify register / register access where the operand is optionally only allowed in a register rather than memory, which is not required for other embodiments. "VL" (e.g., as in "VL=128") specifies the vector length, while "KL" specifies the number of 128-bit segment channels in a vector of length VL. "FOR" (e.g., as in "FOR n := 0 to x") specifies the start of a loop for a certain number of iterations (e.g., x+1 iterations). The symbol ":=" specifies that the value on the right-hand side is assigned to the variable on the left-hand side. The notation "[x:0]" specifies a range of bit positions. For example, "src1[127:0]" specifies bit positions 0 to 127 of src1, and [31:0] in "TMP1.fp32[31:0]" specifies bits 0 to 31 of TMP1.fp32, and so on. The asterisk " The symbol "" indicates multiplication. There are two backslashes " / / " before the comment about the pseudocode. "RET" indicates a return (e.g., storage of the destination operand object into the destination register).

[0067] Furthermore, in the pseudocode below, the following items specify how data elements are accessed or addressed: (1) .128b specifies that the vector element is interpreted as 128 bits wide; (2) .fp32 specifies that the vector element is interpreted as F32 (float32) values; (3) .tf32 specifies that the vector element is interpreted as TF32 (tensor floating-point 32) values; (4) .bf16 specifies that the vector element is interpreted as BF16 (bfloat16) values; (5) .fp16 specifies that the vector element is interpreted as F16 (float16) values; (6) .bf8 specifies that the vector element is interpreted as BF8 (bfloat8) values; (7) .hf8 specifies that the vector element is interpreted as HF8 (hfloat8) values; (8) .i32 specifies that the vector element is interpreted as I32 (signed int32) values; (9) .s8 Specifies that the vector elements are interpreted as S8 (signed int8) values; (10) .u8 Specifies that the vector elements are interpreted as U8 (unsigned int8) values.

[0068] VMMTF32PS instruction: Compact TF32 small matrix multiplication for one or more 128-bit channels: Each 128-bit channel in src1 and src2 is interpreted as a 2x2 (A) matrix or a 2x2 (B) matrix, and the A and B matrices in the corresponding 128-bit channel are multiplied together. The result is a 2×2 intermediate matrix in FP32 format, the elements of which are added to the corresponding elements of the accumulation matrix in srcdest. VL=128 VMMTF32PS xmm1, xmm2, xmm3 VL=256 VMMTF32PS ymm1, ymm2, ymm3 VL=512 VMMTF32PS zmm1, zmm2, zmm3 Operation object 1: ModRM:reg(r,w) Operation object 2: VEX.vvvv(r) Operation object 3: ModRM:reg(r)

[0069] VMMTF32PS dest, src1, src2 VL=(128,256,512) KL = VL / 128 FOR i := 0 to KL-1: TMP[127:0] := tf32fp32_2x2x2_matmul(src1.128b[i], src2.128b[i]) FOR j := 0 to 3: dest.fp32[i 4 + j] := dest.fp32[i 4 + j] + TMP.fp32[j] DEST[MAX_VL-1:VL] := 0

[0070] The pseudocode operation is tf32fp32_2x2x2_matmul(src1[127:0], src2[127:0]). DEST[127:0] = 0 FOR n := 0 to 1: FOR m := 0 to 1: P0[31:0] := TF32FP32Mul(src1.tf32[m 2+0], src2.tf32[n 2+0]) P1[31:0] := TF32FP32Mul(src1.tf32[m 2+1], src2.tf32[n 2+1]) DEST.fp32[m 2+n] := P0.fp32[31:0] + P1.fp32[31:0]; RET DEST

[0071] TF32FP32Mul represents a scalar multiplication of two TF32 elements that produce an FP32 product. In some embodiments, the source data elements may optionally be converted to larger (e.g., more precise) data elements (e.g., 32-bit single-precision floating-point data elements). The operation can also perform floating-point rounding as needed, handle nonnormal numbers as needed (e.g., whether to perform FTZ, whether to perform DAZ), and propagate NaN and infinity as needed.

[0072] VMM[BF16 / F16]PS instruction: Compacts one or more 128-bit channels of BF16 and / or FP16 small matrix multiplication: Each 128-bit channel in src1 and src2 is interpreted as a 2×4 (A) matrix or a 4×2 (B) matrix, and the A and B matrices in the corresponding 128-bit channel are matrices multiplied by each other. The result is an intermediate 2×2 matrix in FP32, whose elements are added to the corresponding elements of the accumulation matrix in srcdest. VL=128 VMM[BF16 / F16]PS xmm1, xmm2, xmm3 VL=256 VMM[BF16 / F16]PS ymm1, ymm2, ymm3 VL=512 VMM[BF16 / F16]PS zmm1, zmm2, zmm3 Operation object 1: ModRM:reg(r,w) Operand 2: c(r) Operation object 3: ModRM:reg(r)

[0073] VMMBF16PS dest, src1, src2 VL=(128,256,512) KL = VL / 128 FOR i := 0 to KL-1: TMP[127:0] := bf16fp32_2x2x4_matmul(src1.128b[i], src2.128b[i]) FOR j := 0 to 3: dest.fp32[i 4 + j] := dest.fp32[i 4 + j] + TMP.fp32[j] DEST[MAX_VL-1:VL] := 0

[0074] The pseudocode operation is bf16fp32_2x2x4_matmul(src1[127:0], src2[127:0]). DEST[127:0] = 0 FOR n := 0 to 1: FOR m := 0 to 1: P0[31:0] := BF16FP32Mul(src1.bf16[m 4+0], src2.bf16[n 4+0]) P1[31:0] := BF16FP32Mul(src1.bf16[m 4+1], src2.bf16[n 4+1]) P2[31:0] := BF16FP32Mul(src1.bf16[m 4+2], src2.bf16[n 4+2]) P3[31:0] := BF16FP32Mul(src1.bf16[m 4+3], src2.bf16[n 4+3]) TMP0[31:0] = P0.fp32[31:0] + P2.fp32[31:0]; TMP1[31:0] = P1.fp32[31:0] + P3.fp32[31:0]; DEST.fp32[m 2+n] := TMP0.fp32[31:0] + TMP1.fp32[31:0]; RET DEST

[0075] BF16FP32Mul represents a scalar multiplication of two BF16 elements that produce an FP32 product. In some embodiments, the source data elements may optionally be converted to larger (e.g., more precise) data elements (e.g., 32-bit single-precision floating-point data elements). The operation can also perform floating-point rounding as needed, handle nonnormal numbers as needed (e.g., whether to perform FTZ, whether to perform DAZ), and propagate NaN and infinity as needed.

[0076] VMMF16PS dest, src1, src2 VL=(128,256,512) KL = VL / 128 FOR i := 0 to KL-1: TMP[127:0] := f16fp32_2x2x4_matmul(src1.128b[i], src2.128b[i]) FOR j := 0 to 3: dest.fp32[i 4 + j] := dest.fp32[i 4 + j] + TMP.fp32[j] DEST[MAX_VL-1:VL] := 0

[0077] The pseudocode operation is f16fp32_2x2x4_matmul(src1[127:0], src2[127:0]). DEST[127:0] = 0 FOR n := 0 to 1: FOR m := 0 to 1: P0[31:0] := F16FP32Mul(src1.f16[m 4+0], src2.f16[n 4+0]) P1[31:0] := F16FP32Mul(src1.f16[m 4+1], src2.f16[n 4+1]) P2[31:0] := F16FP32Mul(src1.f16[m 4+2], src2.f16[n 4+2]) P3[31:0] := F16FP32Mul(src1.f16[m 4+3], src2.f16[n 4+3]) TMP0[31:0] = P0.fp32[31:0] + P2.fp32[31:0]; TMP1[31:0] = P1.fp32[31:0] + P3.fp32[31:0]; DEST.fp32[m 2+n] := TMP0.fp32[31:0] + TMP1.fp32[31:0]; RET DEST

[0078] F16FP32Mul represents a scalar multiplication of two FP16 elements that produce an FP32 product. In some embodiments, the source data elements may optionally be converted to larger (e.g., more precise) data elements (e.g., 32-bit single-precision floating-point data elements). The operation can also perform floating-point rounding as needed, handle nonnormal numbers as needed (e.g., whether to perform FTZ, whether to perform DAZ), and propagate NaN and infinity as needed.

[0079] VPMM[UU / SS / US / SU]BD instruction: Compact unsigned-unsigned, signed-signed, unsigned-signed, or signed-unsigned 8-bit integer small matrix multiplication of one or more 128-bit channels: Each 128-bit channel in src1 and src2 is interpreted as a 2×8 (A) matrix or an 8×2 (B) matrix, and the A and B matrices in the corresponding 128-bit channels are matrices multiplied by each other. A and B can be signed or unsigned int8. The result is an intermediate 2×2 matrix in int32, whose elements are added to the corresponding elements of the accumulation matrix in srcdest. VL=128 VPMM[UU / SS / US / SU]BD xmm1, xmm2, xmm3 VL=256 VPMM[UU / SS / US / SU]BD ymm1, ymm2, ymm3 VL=512 VPMM[UU / SS / US / SU]BD zmm1, zmm2, zmm3 Operation object 1: ModRM:reg(r,w) Operation object 2: VEX.vvvv(r) Operation object 3: ModRM:reg(r)

[0080] VPMMUUBD dest, src1, src2 VL=(128,256,512) KL = VL / 128 FOR i := 0 to KL-1: TMP[127:0] := uu8i32_2x2x8_matmul(src1.128b[i], src2.128b[i]) FOR j := 0 to 3: dest.i32[i 4 + j] := dest.i32[i 4 + j] + TMP.i32[j] DEST[MAX_VL-1:VL] := 0

[0081] The pseudocode operation is uu8i32_2x2x8_matmul(src1[127:0], src2[127:0]). DEST[127:0] = 0 FOR n := 0 to 1: FOR m := 0 to 1: P0[31:0] := UU8I32Mul(src1.u8[m 8+0], src2.u8[n 8+0]) P1[31:0] := UU8I32Mul(src1.u8[m 8+1], src2.u8[n 8+1]) P2[31:0] := UU8I32Mul(src1.u8[m 8+2], src2.u8[n 8+2]) P3[31:0] := UU8I32Mul(src1.u8[m 8+3], src2.u8[n 8+3]) P4[31:0] := UU8I32Mul(src1.u8[m 8+4], src2.u8[n 8+4]) P5[31:0] := UU8I32Mul(src1.u8[m 8+5], src2.u8[n 8+5]) P6[31:0] := UU8I32Mul(src1.u8[m 8+6], src2.u8[n 8+6]) P7[31:0] := UU8I32Mul(src1.u8[m 8+7], src2.u8[n 8+7]) / / Because it is unsaturated addition, the order is not required (e.g., FP8 order). DEST.i32[m 2+n] := int32_reduce_nonsat( P0, P1, P2, P3, P4, P5, P6,P7 )

[0082] UU8I32Mul represents a scalar multiplication of two U8 elements that produce an I32 product. In some embodiments, the source data elements may optionally be converted into larger (e.g., more precise) data elements (e.g., 32-bit integer data elements).

[0083] VPMMSSBD dest, src1, src2 VL=(128,256,512) KL = VL / 128 FOR i := 0 to KL-1: TMP[127:0] := ss8i32_2x2x8_matmul(src1.128b[i], src2.128b[i]) FOR j := 0 to 3: dest.i32[i 4 + j] := dest.i32[i 4 + j] + TMP.i32[j] DEST[MAX_VL-1:VL] := 0

[0084] The pseudocode operation is ss8i32_2x2x8_matmul(src1[127:0], src2[127:0]). DEST[127:0] = 0 FOR n := 0 to 1: FOR m := 0 to 1: P0[31:0] := SS8I32Mul(src1.s8[m 8+0], src2.s8[n 8+0]) P1[31:0] := SS8I32Mul(src1.s8[m 8+1], src2.s8[n 8+1]) P2[31:0] := SS8I32Mul(src1.s8[m [8+2], src2.s8[n 8+2]) P3[31:0] := SS8I32Mul(src1.s8[m [8+3], src2.s8[n 8+3]) P4[31:0] := SS8I32Mul(src1.s8[m [8+4], src2.s8[n 8+4]) P5[31:0] := SS8I32Mul(src1.s8[m [8+5], src2.s8[n 8+5]) P6[31:0] := SS8I32Mul(src1.s8[m [8+6], src2.s8[n 8+6]) P7[31:0] := SS8I32Mul(src1.s8[m [8+7], src2.s8[n 8+7]) / / Because it is unsaturated addition, the order is not required (e.g., FP8 order). DEST.i32[m 2+n] := int32_reduce_nonsat( P0, P1, P2, P3, P4, P5, P6,P7 )

[0085] SS8I32Mul represents a scalar multiplication of two S8 elements that produce an I32 product. In some embodiments, the source data elements may optionally be converted into larger (e.g., more precise) data elements (e.g., 32-bit integer data elements).

[0086] VPMMUSBD dest, src1, src2 VL=(128,256,512) KL = VL / 128 FOR i := 0 to KL-1: TMP[127:0] := us8i32_2x2x8_matmul(src1.128b[i], src2.128b[i]) FOR j := 0 to 3: dest.i32[i 4 + j] := dest.i32[i 4 + j] + TMP.i32[j] DEST[MAX_VL-1:VL] := 0

[0087] The pseudocode operation is us8i32_2x2x8_matmul(src1[127:0], src2[127:0]). DEST[127:0] = 0 FOR n := 0 to 1: FOR m := 0 to 1: P0[31:0] := US8I32Mul(src1.u8[m 8+0], src2.s8[n 8+0]) P1[31:0] := US8I32Mul(src1.u8[m 8+1], src2.s8[n 8+1]) P2[31:0] := US8I32Mul(src1.u8[m [8+2], src2.s8[n 8+2]) P3[31:0] := US8I32Mul(src1.u8[m [8+3], src2.s8[n 8+3]) P4[31:0] := US8I32Mul(src1.u8[m [8+4], src2.s8[n 8+4]) P5[31:0] := US8I32Mul(src1.u8[m [8+5], src2.s8[n 8+5]) P6[31:0] := US8I32Mul(src1.u8[m [8+6], src2.s8[n 8+6]) P7[31:0] := US8I32Mul(src1.u8[m [8+7], src2.s8[n 8+7]) / / Because it is unsaturated addition, the order is not required (e.g., FP8 order). DEST.i32[m 2+n] := int32_reduce_nonsat( P0, P1, P2, P3, P4, P5, P6,P7 )

[0088] US8I32Mul represents a scalar multiplication of the U8 and S8 elements that generate an I32 product. In some embodiments, the source data elements may optionally be converted into larger (e.g., more precise) data elements (e.g., 32-bit integer data elements).

[0089] VPMMSUBD dest, src1, src2 VL=(128,256,512) KL = VL / 128 FOR i := 0 to KL-1: TMP[127:0] := su8i32_2x2x8_matmul(src1.128b[i], src2.128b[i]) FOR j := 0 to 3: dest.i32[i 4 + j] := dest.i32[i 4 + j] + TMP.i32[j] DEST[MAX_VL-1:VL] := 0

[0090] The pseudocode operation is su8i32_2x2x8_matmul(src1[127:0], src2[127:0]). DEST[127:0] = 0 FOR n := 0 to 1: FOR m := 0 to 1: P0[31:0] := SU8I32Mul(src1.s8[m 8+0], src2.u8[n 8+0]) P1[31:0] := SU8I32Mul(src1.s8[m 8+1], src2.u8[n 8+1]) P2[31:0] := SU8I32Mul(src1.s8[m 8+2], src2.u8[n 8+2]) P3[31:0] := SU8I32Mul(src1.s8[m 8+3], src2.u8[n 8+3]) P4[31:0] := SU8I32Mul(src1.s8[m 8+4], src2.u8[n 8+4]) P5[31:0] := SU8I32Mul(src1.s8[m 8+5], src2.u8[n 8+5]) P6[31:0] := SU8I32Mul(src1.s8[m 8+6], src2.u8[n 8+6]) P7[31:0] := SU8I32Mul(src1.s8[m 8+7], src2.u8[n 8+7]) / / Because it is unsaturated addition, the order is not required (e.g., FP8 order). DEST.i32[m 2+n] := int32_reduce_nonsat( P0, P1, P2, P3, P4, P5, P6,P7 )

[0091] SU8I32Mul represents a scalar multiplication of the S8 and U8 elements that generate an I32 product. In some embodiments, the source data elements may optionally be converted into larger (e.g., more precise) data elements (e.g., 32-bit integer data elements).

[0092] The VMM[B / H / BH / HB]F8PS instruction: Compact 8-bit floating-point small matrix multiplication of one or more 128-bit channels: Each 128-bit channel in src1 and src2 is interpreted as a 2×8 (A) matrix or an 8×2 (B) matrix, and the A and B matrices in the corresponding 128-bit channels are matrices multiplied by each other. The A and B matrices can have BF8 (also known as E5M2, with 5 exponent bits and 2 explicit mantissa bits) elements or HF8 (also known as E4M3, with 4 exponent bits and 3 explicit significant bits) elements independently of each other. The result is an intermediate 2×2 matrix in FP32, the elements of which are added to the corresponding elements of the accumulation matrix in srcdest. VL=128 VMM[B / H / BH / HB]F8PS xmm1, xmm2, xmm3 VL=256 VMM[B / H / BH / HB]F8PS ymm1, ymm2, ymm3 VL=512 VMM[B / H / BH / HB]F8PS zmm1, zmm2, zmm3 Operation object 1: ModRM:reg(r,w) Operation object 2: VEX.vvvv(r) Operation object 3: ModRM:reg(r)

[0093] VMMBF8PS dest, src1, src2 VL=(128,256,512) KL = VL / 128 FOR i := 0 to KL-1: TMP[127:0] := bf8fp32_2x2x8_matmul(src1.128b[i], src2.128b[i]) FOR j := 0 to 3: dest.fp32[i 4 + j] := dest.fp32[i 4 + j] + TMP.fp32[j] DEST[MAX_VL-1:VL] := 0

[0094] The pseudocode operation is bf8fp32_2x2x8_matmul(src1[127:0], src2[127:0]). DEST[127:0] = 0 FOR n := 0 to 1: FOR m := 0 to 1: P0e.fp32[31:0] := BF8FP32Mul(src1.bf8[m 8+0], src2.bf8[n 8+0]) P0o.fp32[31:0] := BF8FP32Mul(src1.bf8[m 8+1], src2.bf8[n 8+1]) P1e.fp32[31:0] := BF8FP32Mul(src1.bf8[m 8+2], src2.bf8[n 8+2]) P1o.fp32[31:0] := BF8FP32Mul(src1.bf8[m 8+3], src2.bf8[n 8+3]) P2e.fp32[31:0] := BF8FP32Mul(src1.bf8[m 8+4], src2.bf8[n 8+4]) P2o.fp32[31:0] := BF8FP32Mul(src1.bf8[m 8+5], src2.bf8[n 8+5]) P3e.fp32[31:0] := BF8FP32Mul(src1.bf8[m 8+6], src2.bf8[n 8+6]) P3o.fp32[31:0] := BF8FP32Mul(src1.bf8[m 8+7], src2.bf8[n 8+7]) / / Column horizontal reduction TMP0e.fp32[31:0] := P0e.fp32[31:0] + P0o.fp32[31:0]; TMP0o.fp32[31:0] := P1e.fp32[31:0] + P1o.fp32[31:0]; TMP1e.fp32[31:0] := P2e.fp32[31:0] + P2o.fp32[31:0]; TMP1o.fp32[31:0] := P3e.fp32[31:0] + P3o.fp32[31:0]; / / Vertical Reduction TMP0.fp32[31:0] := TMP0e.fp32[31:0] + TMP1e.fp32[31:0]; TMP1.fp32[31:0] := TMP0o.fp32[31:0] + TMP1o.fp32[31:0]; / / Horizontal reduction DEST.fp32[m 2+n] := TMP0.fp32[31:0] + TMP1.fp32[31:0]; RET DEST

[0095] BF8FP32Mul represents a scalar multiplication of two BF8 elements that produces an FP32 product. In some embodiments, the source data elements may optionally be converted to larger (e.g., more precise) data elements (e.g., 32-bit single-precision floating-point data elements). The operation can also perform floating-point rounding as needed, handle nonnormal numbers as needed (e.g., whether to perform FTZ, whether to perform DAZ), and propagate NaN and infinity as needed.

[0096] VMMHF8PS dest, src1, src2 VL=(128,256,512) KL = VL / 128 FOR i := 0 to KL-1: TMP[127:0] := hf8fp32_2x2x8_matmul(src1.128b[i], src2.128b[i]) FOR j := 0 to 3: dest.fp32[i 4 + j] := dest.fp32[i 4 + j] + TMP.fp32[j] DEST[MAX_VL-1:VL] := 0

[0097] The pseudocode operation is hf8fp32_2x2x8_matmul(src1[127:0], src2[127:0]). DEST[127:0] = 0 FOR n := 0 to 1: FOR m := 0 to 1: P0e.fp32[31:0] := HF8FP32Mul(src1.hf8[m 8+0], src2.hf8[n 8+0]) P0o.fp32[31:0] := HF8FP32Mul(src1.hf8[m 8+1], src2.hf8[n 8+1]) P1e.fp32[31:0] := HF8FP32Mul(src1.hf8[m 8+2], src2.hf8[n 8+2]) P1o.fp32[31:0] := HF8FP32Mul(src1.hf8[m 8+3], src2.hf8[n 8+3]) P2e.fp32[31:0] := HF8FP32Mul(src1.hf8[m 8+4], src2.hf8[n 8+4]) P2o.fp32[31:0] := HF8FP32Mul(src1.hf8[m 8+5], src2.hf8[n 8+5]) P3e.fp32[31:0] := HF8FP32Mul(src1.hf8[m 8+6], src2.hf8[n 8+6]) P3o.fp32[31:0] := HF8FP32Mul(src1.hf8[m 8+7], src2.hf8[n 8+7]) / / Column horizontal reduction TMP0e.fp32[31:0] := P0e.fp32[31:0] + P0o.fp32[31:0]; TMP0o.fp32[31:0] := P1e.fp32[31:0] + P1o.fp32[31:0]; TMP1e.fp32[31:0] := P2e.fp32[31:0] + P2o.fp32[31:0]; TMP1o.fp32[31:0] := P3e.fp32[31:0] + P3o.fp32[31:0]; / / Vertical Reduction TMP0.fp32[31:0] := TMP0e.fp32[31:0] + TMP1e.fp32[31:0]; TMP1.fp32[31:0] := TMP0o.fp32[31:0] + TMP1o.fp32[31:0]; / / Horizontal reduction DEST.fp32[m 2+n] := TMP0.fp32[31:0] + TMP1.fp32[31:0]; RET DEST

[0098] HF8FP32Mul represents a scalar multiplication of two HF8 elements that produces an FP32 product. In some embodiments, the source data elements may optionally be converted to larger (e.g., more precise) data elements (e.g., 32-bit single-precision floating-point data elements). The operation can also perform floating-point rounding as needed, handle nonnormal numbers as needed (e.g., whether to perform FTZ, whether to perform DAZ), and propagate NaN and infinity as needed.

[0099] VMMBHF8PS dest, src1, src2 VL=(128,256,512) KL = VL / 128 FOR i := 0 to KL-1: TMP[127:0] := bhf8fp32_2x2x8_matmul(src1.128b[i], src2.128b[i]) FOR j := 0 to 3: dest.fp32[i 4 + j] := dest.fp32[i 4 + j] + TMP.fp32[j] DEST[MAX_VL-1:VL] := 0

[0100] The pseudocode operation is bhf8fp32_2x2x8_matmul(src1[127:0], src2[127:0]). DEST[127:0] = 0 FOR n := 0 to 1: FOR m := 0 to 1: P0e.fp32[31:0] := BHF8FP32Mul(src1.bf8[m 8 + 0], src2.hf8[n 8 + 0]) P0o.fp32[31:0] := BHF8FP32Mul(src1.bf8[m 8 + 1], src2.hf8[n 8 + 1]) P1e.fp32[31:0] := BHF8FP32Mul(src1.bf8[m 8 + 2], src2.hf8[n 8 + 2]) P1o.fp32[31:0] := BHF8FP32Mul(src1.bf8[m 8 + 3], src2.hf8[n 8 + 3]) P2e.fp32[31:0] := BHF8FP32Mul(src1.bf8[m 8 + 4], src2.hf8[n 8 + 4]) P2o.fp32[31:0] := BHF8FP32Mul(src1.bf8[m 8 + 5], src2.hf8[n 8 + 5]) P3e.fp32[31:0] := BHF8FP32Mul(src1.bf8[m 8 + 6], src2.hf8[n 8 + 6]) P3o.fp32[31:0] := BHF8FP32Mul(src1.bf8[m 8 + 7], src2.hf8[n 8 + 7]) / / Column horizontal reduction TMP0e.fp32[31:0] := P0e.fp32[31:0] + P0o.fp32[31:0]; TMP0o.fp32[31:0] := P1e.fp32[31:0] + P1o.fp32[31:0]; TMP1e.fp32[31:0] := P2e.fp32[31:0] + P2o.fp32[31:0]; TMP1o.fp32[31:0] := P3e.fp32[31:0] + P3o.fp32[31:0]; / / Vertical Reduction TMP0.fp32[31:0] := TMP0e.fp32[31:0] + TMP1e.fp32[31:0]; TMP1.fp32[31:0] := TMP0o.fp32[31:0] + TMP1o.fp32[31:0]; / / Horizontal reduction DEST.fp32[m 2+n] := TMP0.fp32[31:0] + TMP1.fp32[31:0]; RET DEST

[0101] BHF8FP32Mul represents a scalar multiplication of BF8 and HF8 elements that produces an FP32 product. In some embodiments, the source data elements may optionally be converted to larger (e.g., more precise) data elements (e.g., 32-bit single-precision floating-point data elements). The operation can also perform floating-point rounding as needed, handle nonnormal numbers as needed (e.g., whether to perform FTZ, whether to perform DAZ), and propagate NaN and infinity as needed.

[0102] VMMHBF8PS dest, src1, src2 VL=(128,256,512) KL = VL / 128 FOR i := 0 to KL-1: TMP[127:0] := hbf8fp32_2x2x8_matmul(src1.128b[i], src2.128b[i]) FOR j := 0 to 3: dest.fp32[i 4 + j] := dest.fp32[i 4 + j] + TMP.fp32[j] DEST[MAX_VL-1:VL] := 0

[0103] The pseudocode operation is hbf8fp32_2x2x8_matmul(src1[127:0], src2[127:0]). DEST[127:0] = 0 FOR n := 0 to 1: FOR m := 0 to 1: P0e.fp32[31:0] := HBF8FP32Mul(src1.hf8[m 8+0], src2.bf8[n 8+0]) P0o.fp32[31:0] := HBF8FP32Mul(src1.hf8[m 8+1], src2.bf8[n 8+1]) P1e.fp32[31:0] := HBF8FP32Mul(src1.hf8[m 8+2], src2.bf8[n 8+2]) P1o.fp32[31:0] := HBF8FP32Mul(src1.hf8[m​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​ TMP1e.fp32[31:0] := P2e.fp32[31:0] + P2o.fp32[31:0]; TMP1o.fp32[31:0] := P3e.fp32[31:0] + P3o.fp32[31:0]; / / Vertical Reduction TMP0.fp32[31:0] := TMP0e.fp32[31:0] + TMP1e.fp32[31:0]; TMP1.fp32[31:0] := TMP0o.fp32[31:0] + TMP1o.fp32[31:0]; / / Horizontal reduction DEST.fp32[m 2+n] := TMP0.fp32[31:0] + TMP1.fp32[31:0]; RET DEST

[0104] HBF8FP32Mul represents a scalar multiplication of HF8 and BF8 elements that produces an FP32 product. In some embodiments, the source data elements may optionally be converted to larger (e.g., more precise) data elements (e.g., 32-bit single-precision floating-point data elements). The operation can also perform floating-point rounding as needed, handle nonnormal numbers as needed (e.g., whether to perform FTZ, whether to perform DAZ), and propagate NaN and infinity as needed.

[0105] It is important to understand that, for each of the specific instructions described above, the specific order in which the products of matrix multiplications are accumulated and their accumulation with the accumulated values ​​from the source / destination matrices is not required. Each of the specific instructions described above indicates a possible order for such addition / accumulation, but this specific order is merely an example and is not mandatory. For each instruction, multiple (if not many) other orders are also possible. Floating-point addition is not strictly correlated. As a result, slightly different results may be obtained when floating-point addition is performed in different orders, which is why a particular example order is specified in the architectural description of these instructions. However, many variations of the instructions in which the order is changed in various different ways are conceived. As an example, for the bf8fp32_2x2x78_matmul operation, the illustrated embodiment performs column horizontal reduction, vertical reduction, and then horizontal reduction, but this is not required. In other embodiments, many other addition orders may optionally be used (e.g., P0e.fp32[31:0] may be added to P1e.fp32[31:0] to form a temporary value, P0e.fp32[31:0] may be added to P2e.fp32[31:0] to form a temporary value, P0e.fp32[31:0] may be added to P20.fp32[31:0] to form a temporary value, and these temporary values ​​may be added in different orders, etc.).

[0106] In some embodiments, accumulation can be performed "late" because the accumulation of the product with the accumulated value and / or the register used to hold the accumulated value is performed only after all multiplications have been performed. In some embodiments, all products are accumulated with the accumulated value after all products have been generated. Another possible approach is to sequentially access the accumulated value and / or the register used to hold the accumulated value two or more times, including once after one part (e.g., half) of the multiplication has been performed, and again after another part (e.g., half) of the multiplication has been performed. A possible advantage of such "late" accumulation is that it may help address latency characteristics and / or improve the quality of numerical results. Due to late accumulation, only the final result may be exposed to the scheduler of an out-of-order (OoO) machine. Moreover, resetting the running accumulator (e.g., as might be done for late accumulation) can tend to improve the overall result quality because rounding errors (which may be encountered in floating-point operations) can tend to be reduced.

[0107] In some embodiments, one or more of the instructions disclosed above, or each instruction, may have one or more optional attributes as described below to achieve certain advantages (e.g., improved performance, simplified implementation, making the instructions better suited for certain types of workloads, etc.). One area where such simplifications can be made is when dealing with floating-point numbers. These simplifications apply only to instructions that operate on floating-point data elements, not to instructions that operate on integers.

[0108] In some embodiments, the processor may have an MXCSR, a floating-point control and / or status register, or other registers for controlling floating-point operations. In some embodiments, the register may have one or more fields (e.g., a rounding mode field) for specifying one of a plurality of supported rounding modes to be used for floating-point operations. In some embodiments, one or more instructions, or each instruction, of the instructions disclosed above may control rounding to be performed only according to a single rounding mode (e.g., in some cases, a nearest-even rounding mode), regardless of and / or regardless of one or more fields in the register (e.g., the rounding mode field), and / or without checking those one or more fields. In some such embodiments, rounding may be performed according to a single rounding mode in a mathematically precise manner, rather than through simplification that results in mathematically inaccurate results.

[0109] In some embodiments, the MXCSR, the floating-point control and / or status register, or other registers for controlling floating-point operations may have one or more fields (e.g., dump clear to zero (FTZ) bits or fields) for specifying whether a non-normal result of a floating-point operation is to be set to zero, forced to zero, or zeroed. In some such embodiments, one or more instructions, or each instruction, of the instructions disclosed above may control the processor or cause the processor to set, force, or zero a non-normal result of a floating-point operation, regardless of whether one or more fields in the register (e.g., FTZ bits or fields) specify that the non-normal result of a floating-point operation is to be set to zero, forced to zero, or zeroed, and / or without checking whether the one or more fields specify that the non-normal result of a floating-point operation is to be set to zero, forced to zero, or zeroed.

[0110] In some embodiments, the MXCSR, floating-point control and / or status register, or other registers for controlling floating-point operations may have one or more fields (e.g., nonregular to zero (DAZ) bits or fields) for specifying whether a nonregular value in the input to a floating-point instruction / operation (e.g., a source matrix) is to be set to zero, zeroed, or treated as zero. In some such embodiments, one or more instructions, or each instruction, disclosed above may control or cause the processor to set a nonregular number in the input to a floating-point operation to zero, zero the nonregular number, or treat a nonregular number in the input to a floating-point operation as zero, regardless of and / or regardless of whether one or more fields in the register (e.g., DAZ bits or fields) specify that a nonregular number in the input to a floating-point operation is to be forced to zero, and / or without checking whether the one or more fields specify that a nonregular number in the input to a floating-point operation is to be forced to zero. Alternatively, in other embodiments, instructions operating on FP16 and FP8 source floating-point data elements (e.g., VMMF16PS, VMMBF8PS, VMMHF8PS, VMMBHF8PS, and VMMHBF8PS instructions) may optionally conform to DAZ bits or fields in registers (e.g., unequal values ​​in the input of a floating-point operation are not treated as zero regardless of whether one or more fields specify that unequal values ​​in the input of a floating-point operation should be treated as zero). One possible reason for disabling DAZ for FP16 and / or FP8 data element formats is that FP16 and / or FP8 data element formats have fewer exponent bits (e.g., a smaller numerical range), making it more appropriate to process unequal values ​​in the input / source rather than setting, zeroing, or treating unequal values ​​in the input / source as zero.

[0111] In some embodiments, the MXCSR, the floating-point control and / or status register, or other registers for controlling floating-point operations may have one or more fields for controlling whether floating-point instructions / operations report floating-point exceptions. In some such embodiments, one or more instructions, or each instruction, of the instructions disclosed above may control or cause the processor not to report floating-point exceptions, regardless of and / or regardless of whether one or more fields in the register specify that a floating-point exception should be reported, and / or without needing to check whether the one or more fields specify that a floating-point exception should be reported. In some embodiments, the instructions may optionally cause the processor to propagate a Not-a-Number (NaN) or an Infinity (Inf).

[0112] In some such embodiments, one or more of the instructions disclosed above, or each instruction, may control or cause the processor to perform the operation corresponding to the instruction without accessing the MXCSR, the floating-point control and / or status registers, or other registers used to control floating-point operations (e.g., without reading from or writing to these registers).

[0113] While such floating-point features may not be suitable for all workloads, some workloads can benefit from the enhanced performance and / or simpler implementations that can result from them. Strict adherence to the floating-point standard and support for all variations of floating-point features typically come at a cost. Some workloads (e.g., artificial intelligence) may not require all of these floating-point features to be supported by the instructions and can benefit more from higher performance and / or simpler implementations.

[0114] In some embodiments, one or more of the instructions disclosed above, or each instruction, may optionally allow only the source and destination operands to be in registers rather than memory, but this is not required. This may also be the case in some embodiments, even if the instructions are implemented using an ISA that is not a load-store ISA, but rather includes other data processing instructions capable of specifying and operating on operands in memory. As an example, in an x86 ISA, instructions may use the ModRM:11:rrr:bbb operand addressing mode. One possible reason for this is to improve the performance of instruction execution (e.g., no latency for memory operand access), which may be more beneficial for certain workloads than requiring access to operands in memory. Another possible reason is to reduce the risk of side-channel attacks. However, in other embodiments, it may optionally be allowed to retrieve the source matrix from memory.

[0115] In some embodiments, one or more, or each, of the instructions disclosed above may optionally be allowed to operate only in 64-bit mode (e.g., not in 32-bit mode), but this is not required. Supporting only 64-bit mode may potentially help improve performance (e.g., potentially allow access to a greater number of registers, potentially simplify implementation (e.g., eliminate one or more checks), etc.). However, in other embodiments, the instructions may also optionally be supported in 32-bit mode.

[0116] In some embodiments, any one or more of the features described above may optionally be implicit to instructions (e.g., implicit to their opcodes), such that instructions can only have these features. In other embodiments, any one or more of the features described above may optionally be configured or configurable to instructions (e.g., the opcode of an instruction), such that one or more features can be configured or configurable to instructions (e.g., enabled or disabled). In some embodiments, the processor may have one or more bits, flags, or configurable controls (e.g., in floating-point status and / or control registers, in model-specific registers (MSRs), etc.) to store such configuration information. Each control may have a first value specifying whether a feature or an alternative feature is to be used. For example, a control may indicate whether the FTZ field is to be followed or overridden by instructions, etc.

[0117] The detailed instructions described above represent specific examples of suitable instructions. However, many modifications to these instructions are possible. For example, the instructions above refer to the xmm, ymm, and zmm registers, which represent registers in an x86 ISA. In other embodiments, in a different non-x86 ISA, these xmm, ymm, and zmm registers may optionally be replaced by other 128-bit, 256-bit, or 512-bit registers. For example, the xmm, ymm, and zmm registers may optionally be replaced by the Scalable Vector Extension (SVE) or SVE2 Scalable Vector Registers Z0-Z31 used in the ARM architecture. Thus, other embodiments of the instructions above can widely use 128-bit, 256-bit, or 512-bit registers or Scalable Vector Registers instead of the xmm, ymm, and zmm registers shown in the examples above. In other embodiments, the instructions may also support different vector register sizes (e.g., 640-bit, 768-bit, 1024-bit, etc.) including different numbers of 128-bit channels. As another example, masking / predicates have optionally not been used in these instructions, but in other embodiments, masking / predicates may optionally be used. As another example, some of the instructions above may use one or more of “ModRM:reg(r,w)”, “ModRM: 11:rrr:bbb”, “VEX.vvvv(r)”, or “ModRM:reg(r)”, which represent operand addressing modes specific to the instruction encoding format used in the x86 ISA. In other embodiments, these operand addressing modes may broadly represent operand addressing modes that may be optionally replaced by other operand addressing modes used by the instruction encoding and / or used in other non-x86 ISAs. Further, the first two set bits “11” of ModRM:11:rrr:bbb specify register / register access where the operand is optionally only allowed in a register rather than memory, but this is not necessary for other embodiments where the source operand may originate from memory. As yet another example, some of the instructions above reference the MXCSR, which is a floating-point status and / or control register in the x86 ISA. In other embodiments, the MXCSR may optionally be replaced by another floating-point status and / or control register in a non-x86 ISA. Thus, other embodiments of the instructions above may reference another floating-point status and / or control register.

[0118] Figure 5 This is an operation used to execute instruction 501 (e.g., for...). Figure 1A more detailed block diagram of an example embodiment of the processor 500 described in the instructions 101 is provided. The processor includes a decoder circuit 502 for decoding instructions, an execution circuit 503 for performing operations corresponding to the instructions, and a register and / or memory 522 for storing a matrix for the instructions. Except for aspects relating to additional components of the processor 500, the instructions 501, decoder circuit 502, execution circuit 503, and register / memory 522 may optionally be coupled with components already designed for the processor 500. Figure 1 The components described are the same or similar. To avoid ambiguity in this specification, the main description will be... Figure 5 Different and / or additional components and aspects of the embodiments, without repetition, may optionally be combined with those already addressed. Figure 1 The aspects that are the same or similar to those described.

[0119] Optional storage device 520 may be used to store instruction 501. As an example, the storage device may be a cache (e.g., instruction cache, unified cache) or memory coupled to the processor. Decoder circuitry 502 may receive and decode instructions as previously described. The decoder circuitry may be optionally coupled to the storage device via optional fetch circuitry (not shown) to receive instructions.

[0120] In some examples, optional register renaming, register allocation, and / or scheduling circuitry 521 may provide functionality for one or more of the following: (1) renaming logical operand values ​​to physical operand values ​​(e.g., register alias tables in some examples); (2) assigning status bits and flags to decoded instructions; and (3) scheduling decoded instructions from the instruction pool for execution circuitry to execute (e.g., using reserved stations in some examples). As previously described, registers and / or memory 522 may store source and result matrices of instructions.

[0121] Execution circuitry 503 is coupled to decoder circuitry 502 and register / memory 522 via intermediate register renaming, register allocation, and / or scheduling circuitry 521. As previously described, execution circuitry performs operations corresponding to instructions. A possible example of execution circuitry is execution cluster(s) 1160 shown in Figure 11(B). Optional write-back / retrieval circuitry 523 can architecturally commit the destination register to register / memory 522 and commit the instruction.

[0122] Figure 6This is a flowchart of an embodiment of method 625, which executes vector-compacted matrix multiplication and accumulation instructions. In various embodiments, the method may be executed by a processor, digital logic device, or integrated circuit. As an example, such a method may be executed by processor 100, 200, 300, 400, or 500, processor core 1190 of FIG. 11(B), or the pipeline further described below. The components, features, and specific optional details described herein for processor 100, 200, 300, 400, or 500, processor core 1190, or pipeline 1100 may optionally be applied to the method. Alternatively, method 625 may be executed by similar or different processors, cores, or pipelines. Furthermore, processor 100, 200, 300, 400, or 500 may execute the same, similar, or different methods as method 625.

[0123] At position 626, an instance of a single vector compact matrix multiplication and accumulation instruction is obtained. The single instruction can be any of those instructions described elsewhere in this document (e.g., one of instructions 101, 201, 301, 401, etc.). In some examples, the instruction is retrieved from the instruction cache. The instructions may explicitly specify or otherwise indicate a first (e.g., source) matrix (e.g., indicating a first vector register having a 128-bit channel for storing the first matrix), the first matrix having 2 rows by K columns of data elements, each of which has a first number of bits, where K equals 64 bits divided by the first number of bits; may explicitly specify or otherwise indicate a second (e.g., source) matrix (e.g., indicating a second vector register having a 128-bit channel for storing the second matrix), the second matrix having K rows by 2 columns of data elements, each of which has a first number of bits; and may explicitly specify or otherwise indicate a third (e.g., source) matrix (e.g., indicating a third vector register having a 128-bit channel for storing the third matrix), the third matrix having 2 rows by 2 columns of data elements, each of which has a second number of bits, the second number of bits being greater than the first number of bits. At least one, at least two, or all of these matrices may be stored in the vector register and / or stored in a one-dimensional vector format. These matrices may optionally be associated with those described elsewhere herein (e.g., as previously described for...). Figures 1-4 The matrices described are the same or similar.

[0124] At point 627, the acquired instruction can be decoded. For example, the acquired instruction can be decoded by a decoder circuit (e.g., decoder circuit 102).

[0125] At 628, when the decoded instruction is scheduled, the source operands can be retrieved. For example, when one or more of the source operands are memory operands, the data can be retrieved from the indicated memory location.

[0126] At 629, the decoded instructions are executed by execution circuitry (e.g., one of execution circuits 103, 203, 303, 403, or one or more execution clusters 1160 of FIG. 11(B)). This may include performing operations corresponding to and / or pursuant to the instructions. In some embodiments, the operations may include generating a result matrix having 2 rows by 2 columns (e.g., M=2, N=2) result data elements, each having a second number of bits. In some embodiments, the result matrix may represent the sum of a third matrix (e.g., an accumulation matrix) and a product matrix (e.g., having 2 rows by 2 columns), which is generated from matrix multiplication using and / or involving and / or based on the first and second matrices. In some embodiments, the product matrix may be generated from matrix multiplication using the first and second matrices, wherein prior to the matrix multiplication, the data elements of the first and second matrices are converted from a first number of bits to a larger number of bits (e.g., from 8 bits or 16 bits to 32 bits). In some embodiments, the product matrix may be generated from matrix multiplication using the first and second matrices, wherein floating-point rounding may be optionally performed as needed during matrix multiplication. In some embodiments, the operation may include storing the resulting matrix in a 128-bit channel of a third vector register initially used to store the third matrix. In some embodiments, the operation may include any of those operations described elsewhere herein, including the operations described with respect to the detailed example instructions described herein (e.g., any of VMMTF32PS, VMM[BF16,F16]PS, VPMM[UU,SS,US,SU]BD, and VMM[B,H,BH,HB]F8PS).

[0127] In some examples, at 630, the instruction can be committed or decommissioned. This allows the resulting operand to be stored in a destination register (e.g., a renamed register committed to an architecture register).

[0128] Figure 7This is a flowchart of an embodiment of method 732, which uses simulation or binary translation to execute vector compact matrix multiplication and accumulation instructions. In various embodiments, method 732 may be executed by a processor, digital logic device, or integrated circuit. As an example, processor 100, 200, 300, 400, or 500, processor core 1190 of FIG. 11(B), or pipelines further described below can execute method 732. The components, features, and specific optional details described herein for processor 100, 200, 300, 400, or 500, processor core 1190, or pipeline 1100 may optionally be applied to this method. Alternatively, method 732 may be executed by similar or different processors, cores, or pipelines. Moreover, processor 100, 200, 300, 400, or 500 may execute methods that are the same as, similar to, or different from method 732.

[0129] At 733, an instance of a single vector compact matrix multiplication and accumulation instruction of the first instruction set architecture is obtained. The single instruction can be any of those instructions described elsewhere herein (e.g., any of instructions 101, 201, 301, 401). In some examples, the instruction is fetched from the instruction cache. The instructions may explicitly specify or otherwise indicate a first (e.g., source) matrix (e.g., indicating a first vector register having a 128-bit channel for storing the first matrix), the first matrix having 2 rows by K columns of data elements, each of which has a first number of bits, where K equals 64 bits divided by the first number of bits; may explicitly specify or otherwise indicate a second (e.g., source) matrix (e.g., indicating a second vector register having a 128-bit channel for storing the second matrix), the second matrix having K rows by 2 columns of data elements, each of which has a first number of bits; and may explicitly specify or otherwise indicate a third (e.g., source) matrix (e.g., indicating a third vector register having a 128-bit channel for storing the third matrix), the third matrix having 2 rows by 2 columns of data elements, each of which has a second number of bits, the second number of bits being greater than the first number of bits. At least one, at least two, or all of these matrices may be stored in the vector register and / or stored in a one-dimensional vector format. These matrices may optionally be associated with those described elsewhere herein (e.g., as previously described for...). Figures 1-4 The matrices described are the same or similar.

[0130] At 734, the single vector compact matrix multiplication and accumulation instructions of the first instruction set architecture are translated into one or more instructions of a different second instruction set architecture. In some examples, this translation may be performed by a software translation and / or emulation layer. In some examples, this translation may be performed by, for example, Figure 20The instruction translator 2012 shown in the figure performs the translation. In some examples, the translation may be performed by hardware translation circuitry.

[0131] At 735, one or more translated instructions of the second instruction set architecture are decoded. For example, the translated instructions may be decoded by a decoder circuit (e.g., one of decoder circuits 102, 202, 302, 402). In some examples, the operations of translating 734 and decoding 735 may optionally be combined.

[0132] At 736, when the decoded instruction is scheduled, the source operands can be retrieved. For example, when one or more of the source operands are memory operands, the data can be retrieved from the indicated memory location.

[0133] At 737, the execution circuitry (e.g., execution circuitry 103, 203, 303, 403, or execution cluster 1160 of FIG. 11(B)) executes one or more decoded instructions of the second instruction set architecture to perform one or more operations indicated by and / or corresponding to a single instruction of the first instruction set architecture. In some embodiments, the operation may include generating a result matrix having 2 rows by 2 columns (e.g., M=2, N=2) result data elements, each having a second number of bits. In some embodiments, the result matrix may represent the sum of a third matrix (e.g., an accumulation matrix) and a product matrix (e.g., having 2 rows by 2 columns), which is generated from matrix multiplication using and / or involving and / or based on the first and second matrices. In some embodiments, the product matrix may be generated from matrix multiplication using the first and second matrices, wherein prior to the matrix multiplication, the data elements of the first and second matrices are converted from a first number of bits to a larger number of bits (e.g., from 8 bits or 16 bits to 32 bits). In some embodiments, the product matrix may be generated from matrix multiplication using the first and second matrices, wherein floating-point rounding may be optionally performed as needed during matrix multiplication. In some embodiments, the operation may include storing the resulting matrix in a 128-bit channel of a third vector register initially used to store the third matrix. In some embodiments, the operation may include any of those operations described elsewhere herein, including the operations described with respect to the detailed example instructions described herein (e.g., any of VMMTF32PS, VMM[BF16,F16]PS, VPMM[UU,SS,US,SU]BD, and VMM[B,H,BH,HB]F8PS).

[0134] In some examples, at 738, one or more decoded instructions of the second instruction set architecture are committed or retired. This allows the resulting floating-point operand to be stored in an architecture storage location for and / or corresponding to a destination register (e.g., a renamed register committed to an architecture register).

[0135] Figure 8 This is a block diagram of an example embodiment of a suitable vector register set 840. The vector registers include 32 512-bit vector registers labeled zmm0 to zmm31. In the illustrated embodiment, the lower-order 256 bits of the lower 16 registers (i.e., zmm0-zmm15) are aliased as corresponding 256-bit vector registers labeled ymm0-ymm15 or overlapped on these 256-bit vector registers, but this is not mandatory. Similarly, in the illustrated embodiment, the lower-order 128 bits of registers ymm0-ymm15 are aliased as corresponding 128-bit vector registers labeled xmm0-xmm15, or overlapped on these 128-bit vector registers, but this is also not mandatory. The 512-bit registers zmm0 to zmm31 operate to hold 512-bit vectors, 256-bit vectors, or 128-bit vectors. The 256-bit registers ymm0-ymm15 operate to hold 256-bit vectors or 128-bit vectors. The 128-bit registers xmm0-xmm15 operate to hold a 128-bit vector. In some embodiments, each register in the registers can be used to store compressed floating-point data or compressed integer data. Supported data element sizes and types include any of the 8-bit, 16-bit, and 32-bit integer and floating-point formats disclosed elsewhere herein. It should be understood that this is merely an illustrative example of a suitable set of registers. In alternative embodiments, different numbers of registers may be used, and / or different register sizes may be used, and / or larger registers may be aliased as smaller registers, etc., may or may not be used.

[0136] Example computer architecture.

[0137] The example computer architecture is described in detail below. Other system designs and configurations known in the art for laptops, desktop computers, personal computers (PCs), personal digital assistants, engineering workstations, servers, discrete servers, network devices, network hubs, switches, routers, embedded processors, digital signal processors (DSPs), graphics devices, video game devices, set-top boxes, microcontrollers, cellular phones, portable media players, handheld devices, and various other electronic devices are also suitable. In summary, various systems or electronic devices capable of incorporating the processors and / or other execution logic disclosed herein are generally suitable.

[0138] Figure 9 An example computing system is illustrated. The multiprocessor system 900 is an interface-based system and includes multiple processors or cores, including a first processor 970 and a second processor 980 coupled via an interface 950 (e.g., a point-to-point (PP) interconnect, fabric, and / or bus). In some examples, the first processor 970 and the second processor 980 are homogeneous. In some examples, the first processor 970 and the second processor 980 are heterogeneous. While the example system 900 is shown as having two processors, the system may have three or more processors, or it may be a single-processor system. In some embodiments, the computing system is a system-on-a-chip (SoC).

[0139] Processors 970 and 980 are shown to include integrated memory controller (IMC) circuitry 972 and 982, respectively. Processor 970 also includes interface circuitry 976 and 978; similarly, the second processor 980 includes interface circuitry 986 and 988. Processors 970 and 980 can exchange information via interface 950 using interface circuitry 978 and 988. IMCs 972 and 982 couple processors 970 and 980 to their respective memories, namely memories 932 and 934, which may be part of the main memory locally attached to each processor.

[0140] Processors 970 and 980 can each use interface circuits 976, 994, 986, and 998, respectively, to exchange information with network interface (NWI / F) 990 via interfaces 952 and 954. Network interface 990 (e.g., one or more of interconnects, buses, and / or structures, in some examples a chipset) can optionally exchange information with coprocessor 938 via interface circuit 992. In some examples, coprocessor 938 is a dedicated processor, such as a high-throughput processor, network or communication processor, compression engine, graphics processor, general-purpose graphics processing unit (GPGPU), neural-network processing unit (NPU), embedded processor, etc.

[0141] A shared cache (not shown) may be included in either processor 970, 980, or connected to these processors outside of the two processors via an interface (e.g., PP interconnect), such that if one processor is placed in a low-power mode, the local cache information of either or both processors may also be stored in the shared cache.

[0142] Network interface 990 may be coupled to first interface 916 via interface circuitry 996. In some examples, first interface 916 may be an interface such as a Peripheral Component Interconnect (PCI) interconnect, a PCI Express interconnect, or another I / O interconnect. In some examples, first interface 916 is coupled to power control unit (PCU) 917, which may include circuitry, software, and / or firmware to perform power management operations with respect to processors 970, 980, and / or coprocessor 938. PCU 917 provides control information to a voltage regulator (not shown) to cause the voltage regulator to generate an appropriate regulated voltage. PCU 917 also provides control information to control the generated operating voltage. In various examples, PCU 917 may include various power management logic units (circuitry) to perform hardware-based power management. Such power management can be entirely controlled by the processor (e.g., controlled by various processor hardware and can be triggered by workload and / or power constraints, thermal constraints or other processor constraints), and / or power management can be performed in response to external sources (e.g., platform or power management sources or system software).

[0143] The PCU 917 is illustrated as logic separate from the processor 970 and / or processor 980. In other cases, the PCU 917 may execute on one or more cores of the processor 970 or 980 (not shown). In some cases, the PCU 917 may be implemented as a microcontroller (dedicated or general-purpose) or other control logic configured to execute its own dedicated power management code (sometimes called P-code). In still other examples, the power management operations to be performed by the PCU 917 may be implemented externally to the processor, for example, by a separate power management integrated circuit (PMIC) or another component external to the processor. In still other examples, the power management operations to be performed by the PCU 917 may be implemented within the BIOS or other system software.

[0144] Various I / O devices 914 and a bus bridge 918 can be coupled to a first interface 916, which in turn couples the first interface 916 to a second interface 920. In some examples, one or more additional processors 915 are coupled to the first interface 916, such as coprocessors, high-throughput many-integrated-core (MIC) processors, GPGPUs, accelerators (e.g., graphics accelerators or digital signal processing (DSP) units), field-programmable gate arrays (FPGAs), or any other processor. In some examples, the second interface 920 can be a low-pin-count (LPC) interface. Various devices can be coupled to the second interface 920, including, for example, a keyboard and / or mouse 922, a communication device 927, and storage circuitry 928. Storage circuitry 928 can be one or more non-transitory machine-readable storage media as described below, such as disk drives or other mass storage devices, which in some examples may include instruction / code and data 930 and may implement a storage device `ISAB03`. Additionally, the audio I / O 924 can be coupled to the second interface 920. Note that other architectures besides the point-to-point architecture described above are also possible. For example, a system such as the multiprocessor system 900 can implement a multi-drop interface or other such architectures instead of a point-to-point architecture.

[0145] Example core architecture, processor, and computer architecture.

[0146] Processor cores can be implemented in different ways, for different purposes, and in different processors. For example, these core implementations can include: 1) general-purpose ordered cores for general computing purposes; 2) high-performance general-purpose out-of-order cores for general computing purposes; and 3) dedicated cores primarily for graphics and / or scientific (throughput) computing purposes. Different processor implementations can include: 1) CPUs, comprising one or more general-purpose ordered cores and / or one or more general-purpose out-of-order cores for general computing purposes; and 2) coprocessors, comprising one or more dedicated cores primarily for graphics and / or scientific (throughput) computing purposes. These different processors lead to different computer system architectures, which can include: 1) coprocessors and CPUs on separate chips; 2) coprocessors and CPUs on separate dies within the same package; 3) coprocessors and CPUs on the same die (in this case, such coprocessors are sometimes referred to as dedicated logic, such as integrated graphics and / or scientific (throughput) logic, or dedicated cores); and 4) system-on-a-chip (SoC), which may be included on the same die as the described CPU (sometimes referred to as application cores or application processors), the aforementioned coprocessors, and additional functionality. An example core architecture is described next, followed by a description of an example processor and computer architecture.

[0147] Figure 10 A block diagram of an example processor and / or SoC 1000 is illustrated, which may have one or more cores and an integrated memory controller. The processor 1000 illustrated by solid-line boxes has a single core 1002(A), system proxy unit circuitry 1010, and a set of one or more interface controller unit circuitry 1016, while alternative processors 1000 are illustrated by optional dashed-line boxes having multiple cores 1002(A)-(N), a set of one or more integrated memory control unit circuitry 1014 from the system proxy unit circuitry 1010, dedicated logic 1008, and a set of one or more interface controller unit circuitry 1016. Note that the processor 1000 may be... Figure 9 It is one of the processors 970 or 980 or the coprocessor 938 or 915.

[0148] Therefore, different implementations of processor 1000 may include: 1) a CPU, where dedicated logic 1008 is integrated graphics and / or scientific (throughput) logic (which may include one or more cores, not shown), and cores 1002(A)-(N) are one or more general-purpose cores (e.g., general-purpose ordered cores, general-purpose out-of-order cores, or a combination of both); 2) a coprocessor, where cores 1002(A)-(N) are a large number of dedicated cores primarily for graphics and / or scientific (throughput) purposes; and 3) a coprocessor, where cores 1002(A)-(N) are a large number of general-purpose ordered cores. Thus, processor 1000 can be a general-purpose processor, a coprocessor, or a dedicated processor, such as a network or communication processor, a compression engine, a graphics processor, a GPGPU (General-Purpose Graphics Processing Unit), a high-throughput integrated many-core (MIC) coprocessor (including 30 or more cores), an embedded processor, etc. The processor may be implemented on one or more chips. The processor 1000 may be part of one or more substrates and / or may be implemented on one or more substrates using any of a variety of process technologies, such as complementary metal oxide semiconductor (CMOS), bipolar CMOS (BiCMOS), P-type metal oxide semiconductor (PMOS), or N-type metal oxide semiconductor (NMOS).

[0149] The memory hierarchy includes one or more levels of cache cell circuitry 1004(A)-(N) within cores 1002(A)-(N), a group of one or more shared cache cell circuitry 1006, and external memory (not shown) coupled to the group of integrated memory controller cell circuitry 1014. The group of one or more shared cache cell circuitry 1006 may include one or more intermediate level caches, such as Level 2 (L2), Level 3 (L3), Level 4 (4), or other levels of cache, such as the last level cache (LLC), and / or combinations thereof. While in some examples interface network circuitry 1012 (e.g., a ring interconnect) provides an interface to dedicated logic 1008 (e.g., integrated graphics logic), the group of shared cache cell circuitry 1006, and system agent cell circuitry 1010, alternative examples use any number of known techniques to interface to these units. In some examples, one or more circuits in the shared cache cell circuitry 1006 maintain consistency with cores 1002(A)-(N). In some examples, the interface controller unit circuit 1016 couples these cores 1002 to one or more other devices 1018, such as one or more I / O devices, storage devices, one or more communication devices (e.g., wireless networks, wired networks, etc.).

[0150] In some examples, one or more of cores 1002(A)-(N) have multi-threading capabilities. System agent unit circuitry 1010 includes those components that coordinate and operate cores 1002(A)-(N). System agent unit circuitry 1010 may include, for example, power control unit (PCU) circuitry and / or display unit circuitry (not shown). The PCU may be (or may include) the logic and components required to regulate the power state of cores 1002(A)-(N) and / or dedicated logic 1008 (e.g., integrated graphics logic). Display unit circuitry is used to drive one or more externally connected displays.

[0151] Cores 1002(A)-(N) can be homogeneous in terms of instruction set architecture (ISA). Alternatively, cores 1002(A)-(N) can be heterogeneous in terms of ISA; that is, a subset of cores 1002(A)-(N) may be able to execute an ISA, while other cores may be able to execute only a subset of that ISA or be able to execute another ISA.

[0152] Example core architecture—ordered and out-of-order core block diagrams.

[0153] Figure 11(A) is a block diagram illustrating both the example ordered pipeline and the example register renaming, out-of-order issue / execution pipeline, according to some examples. Figure 11(B) is a block diagram illustrating both the example ordered architecture core and the example register renaming, out-of-order issue / execution architecture core, according to examples, to be included in the processor. Figures 11(A)-11(B) The solid boxes in the diagram illustrate ordered pipelines and ordered kernels, while the optional dashed boxes illustrate register renaming, out-of-order issue / execution pipelines, and kernels. Since ordered aspects are a subset of out-of-order aspects, out-of-order aspects will be described.

[0154] In Figure 11(A), the processor pipeline 1100 includes a fetch phase 1102, an optional length-decode phase 1104, a decode phase 1106, an optional alloc phase 1108, an optional rename phase 1110, a scheduling (also known as dispatch or dispatch) phase 1112, an optional register read / memory read phase 1114, an execution phase 1116, a write-back / memory write phase 1118, an optional exception handling phase 1122, and an optional commit phase 1124. One or more operations can be performed in each of these processor pipeline phases. For example, during the fetch phase 1102, one or more instructions are fetched from instruction memory, and during the decode phase 1106, the fetched instructions can be decoded, an address using a forward register port (e.g., a load store unit (LSU) address) can be generated, and branch forwarding (e.g., an immediate offset or a link register (LR)) can be performed. In one example, the decoding phase 1106 and the register read / memory read phase 1114 can be combined into a single pipeline phase. In another example, during the execution phase 1116, decoded instructions can be executed, LSU address / data pipelined to the Advanced Microcontroller Bus (AMB) interface can be performed, multiplication and addition operations can be performed, arithmetic operations with branch results can be performed, and so on.

[0155] As an example, the core of the example register renaming, out-of-order issue / execution architecture in Figure 11(B) can be implemented by pipeline 1100 in the following ways: 1) Instruction fetch circuit 1138 executes fetch and length decoding stages 1102 and 1104; 2) Decoding circuit 1140 executes decoding stage 1106; 3) Rename / allocator unit circuit 1152 executes allocation stage 1108 and rename stage 1110; 4) (one or more) scheduler circuits 1156 execute scheduling stage 1112; 5) (one or more) physical registers The register stack circuit 1158 and memory cell circuit 1170 execute register read / memory read phase 1114; (one or more) execution cluster 1160 executes execution phase 1116; 6) memory cell circuit 1170 and (one or more) physical register stack circuit 1158 execute write-back / memory write phase 1118; 7) various circuits may be involved in exception handling phase 1122; and 8) retirement unit circuit 1154 and (one or more) physical register stack circuit 1158 execute commit phase 1124.

[0156] Figure 11(B) shows that processor core 1190 includes front-end unit circuitry 1130 coupled to execution engine unit circuitry 1150, and both are coupled to memory unit circuitry 1170. Core 1190 can be a reduced instruction set architecture computing (RISC) core, a complex instruction set architecture computing (CISC) core, a very long instruction word (VLIW) core, or a hybrid or alternative core type. Alternatively, core 1190 can be a dedicated core, such as a network or communication core, a compression engine, a coprocessor core, a general purpose computing graphics processing unit (GPGPU) core, a graphics core, etc.

[0157] Front-end unit circuitry 1130 may include branch prediction circuitry 1132 coupled to instruction cache circuitry 1134, which is coupled to translation lookaside buffer (TLB) 1136, which is coupled to instruction fetch circuitry 1138, which is coupled to decode circuitry 1140. In one example, instruction cache circuitry 1134 is included in memory unit circuitry 1170 instead of front-end circuitry 1130. Decoding circuitry 1140 (or decoder) can decode instructions and generate one or more micro-operations, microcode entry points, microinstructions, other instructions, or other control signals as outputs, which are decoded from, or otherwise reflect or are derived from, the original instruction. Decoding circuitry 1140 may also include address generation unit (AGU, not shown) circuitry. In one example, the AGU uses the forwarded register port to generate the LSU address and can further perform branch forwarding (e.g., immediate offset branch forwarding, LR register branch forwarding, etc.). Various different mechanisms can be used to implement the decoding circuit 1140. Examples of suitable mechanisms include, but are not limited to, lookup tables, hardware implementations, programmable logic arrays (PLAs), microcode read-only memory (ROMs), etc. In one example, core 1190 includes a microcode ROM (not shown) or other medium that stores microcode for certain macro instructions (e.g., in the decoding circuit 1140 or otherwise within the front-end circuit 1130). In one example, the decoding circuit 1140 includes micro-ops or operation caches (not shown) to save / cachive decoded operations, micro-tags, or micro-operations generated during decoding or other stages of the processor pipeline 1100. The decoding circuit 1140 may be coupled to the renaming / allocator unit circuit 1152 in the execution engine circuitry 1150.

[0158] The execution engine circuitry 1150 includes a renaming / allocator unit circuitry 1152, which is coupled to a retirement unit circuitry 1154 and one or more scheduler circuits 1156. The scheduler circuits 1156 represent any number of different schedulers, including reservation stations, central instruction windows, etc. In some examples, the scheduler circuits 1156 may include an arithmetic logic unit (ALU) scheduler / scheduling circuit, an ALU queue, an address generation unit (AGU) scheduler / scheduling circuit, an AGU queue, etc. The scheduler circuits 1156 are coupled to one or more physical register file circuits 1158. Each of the physical register file circuits 1158 represents one or more physical register files, which store one or more different data types, such as scalar integers, scalar floating-point numbers, compressed integers, compressed floating-point numbers, vector integers, vector floating-point numbers, status (e.g., an instruction pointer, i.e., the address of the next instruction to be executed), etc. In one example, one or more physical register file circuits 1158 include vector register cell circuits, write mask register cell circuits, and scalar register cell circuits. These register cells can provide architectural vector registers, vector mask registers, general-purpose registers, etc. One or more physical register file circuits 1158 are coupled to retirement cell circuits 1154 (also called retirement queues) to demonstrate various ways that can be used to implement register renaming and out-of-order execution (e.g., utilizing one or more reorder buffers (ROBs) and one or more retirement register files; utilizing one or more future heaps, one or more history buffers, and one or more retirement register files; utilizing register maps and register pools; etc.). Retirement cell circuits 1154 and one or more physical register file circuits 1158 are coupled to one or more execution clusters 1160. One or more execution clusters 1160 include a set of one or more execution cell circuits 1162 and a set of one or more memory access circuits 1164. One or more execution unit circuits 1162 can perform various arithmetic, logical, floating-point, or other types of operations (e.g., shift, addition, subtraction, multiplication) on various types of data (e.g., scalar integers, scalar floating-points, compressed integers, compressed floating-points, vector integers, vector floating-points). While some examples may include several execution units or execution unit circuits dedicated to a particular function or set of functions, other examples may include only one execution unit circuit or multiple execution units / execution unit circuits that perform all functions.One or more scheduler circuits 1156, one or more physical register file circuits 1158, and one or more execution clusters 1160 are shown as potentially multiple, because some examples create separate pipelines for certain types of data / operations (e.g., scalar integer pipelines, scalar floating-point / compact integer / compact floating-point / vector integer / vector floating-point pipelines, and / or memory access pipelines, each with its own scheduler circuitry, one or more physical register file circuits, and / or execution clusters—and in the case of separate memory access pipelines, in some examples of implementations only the execution cluster of that pipeline has one or more memory access unit circuits 1164). It should also be understood that, in the case of using separate pipelines, one or more of these pipelines may be issued / executed out of order, while the rest are ordered.

[0159] In some examples, the execution engine unit circuit 1150 can perform load memory unit (LSU) address / data pipelined operations to the Advanced Microcontroller Bus (AMB) interface (not shown), as well as address phases and write-back, data phase load, store, and branch.

[0160] A set of memory access circuitry 1164 is coupled to memory cell circuitry 1170, which includes data TLB circuitry 1172, which is coupled to data cache circuitry 1174, which is coupled to Level 2 (L2) cache circuitry 1176. In one example, memory access circuitry 1164 may include load cell circuitry, memory address cell circuitry, and memory data cell circuitry, each of which is coupled to data TLB circuitry 1172 in memory cell circuitry 1170. Instruction cache circuitry 1134 is further coupled to Level 2 (L2) cache circuitry 1176 in memory cell circuitry 1170. In one example, instruction cache 1134 and data cache 1174 are combined into L2 cache circuitry 1176, Level 3 (L3) cache circuitry (not shown), and / or a single instruction and data cache (not shown) in main memory. L2 cache circuitry 1176 is coupled to one or more other levels of cache and ultimately coupled to main memory.

[0161] Core 1190 may support one or more instruction sets (e.g., x86 instruction set architecture (optionally with some extensions added with later versions); MIPS instruction set architecture; ARM instruction set architecture (optionally with optional additional extensions, such as NEON)) that include the instructions(s) described herein. In one example, Core 1190 includes logic supporting compressed data instruction set architecture extensions (e.g., AVX1, AVX2), thereby allowing the use of compressed data to perform operations used by many multimedia applications.

[0162] Example execution unit circuit.

[0163] Figure 12 Examples of execution unit circuits, such as execution unit circuits 1162 in Figure 11(B), are illustrated. As shown, execution unit circuits 1162 may include one or more ALU circuits 1201, optional vector / single instruction multiple data (SIMD) circuits 1203, load / store circuits 1205, branch / jump circuits 1207, and / or floating-point unit (FPU) circuits 1209. ALU circuits 1201 perform integer arithmetic and / or Boolean operations. Vector / SIMD circuits 1203 perform vector / SIMD operations on compressed data (e.g., SIMD / vector registers). Load / store circuits 1205 execute load and store instructions to load data from memory into registers or store data from registers into memory. Load / store circuits 1205 may also generate addresses. Branch / jump circuits 1207 cause a branch or jump to a memory address depending on the instruction. FPU circuit 1209 performs floating-point arithmetic. The width of (one or more) execution unit circuits 1162 varies depending on the example and can range from, for example, 16 bits to 1024 bits. In some examples, two or more smaller execution units are logically combined to form a larger execution unit (e.g., two 128-bit execution units are logically combined to form a 256-bit execution unit).

[0164] Example register architecture.

[0165] Figure 13This is a block diagram of register architecture 1300 based on some examples. As shown, register architecture 1300 includes vector / SIMD register 1310, whose width varies from 128 bits to 1024 bits. In some examples, vector / SIMD register 1310 is physically 512 bits, and depending on the mapping, only some low-order bits are used. For example, in some examples, vector / SIMD register 1310 is a 512-bit ZMM register: the lower 256 bits are used for the YMM register, and the lower 128 bits are used for the XMM register. Therefore, register overriding exists. In some examples, the vector length field is chosen between a maximum length and one or more other shorter lengths, where each such shorter length is half the length of the previous length. Scalar operations are performed on the lowest-order data element positions in the ZMM / YMM / XMM registers; higher-order data element positions are either kept the same as before the instruction or zeroed out, depending on the example.

[0166] In some examples, the register architecture 1300 includes a write mask / predicate register 1315. For example, in some examples, there are eight write mask / predicate registers (sometimes referred to as k0 to k7), each with a size of 16 bits, 32 bits, 64 bits, or 128 bits. The write mask / predicate register 1315 may allow merging (e.g., allowing any set of elements in the destination to be protected from updates during the execution of any operation) and / or zeroing (e.g., a zeroing vector mask allows any set of elements in the destination to be zeroed during the execution of any operation). In some examples, each data element position in a given write mask / predicate register 1315 corresponds to a data element position in the destination. In other examples, the write mask / predicate register 1315 is scalable and consists of a set number of enable bits for a given vector element (e.g., 8 enable bits for each 64-bit vector element).

[0167] The register architecture 1300 includes multiple general-purpose registers 1325. These registers can be 16-bit, 32-bit, 64-bit, etc., and can be used for scalar operations. In some examples, these registers are referred to by the names RAX, RBX, RCX, RDX, RBP, RSI, RDI, RSP, and R8 through R15.

[0168] In some examples, register architecture 1300 includes a scalar floating-point (FP) register file 1345, which is used to perform scalar floating-point operations on 32 / 64 / 80-bit floating-point data using x87 instruction set architecture extensions, or as an MMX register to perform operations on 64-bit compressed integer data, and to store operation objects for some operations performed between the MMX and XMM registers.

[0169] One or more flag registers 1340 (e.g., EFLAGS, RFLAGS, etc.) store status and control information used for arithmetic, comparison, and system operations. For example, one or more flag registers 1340 may store condition code information such as carry, parity, auxiliary carry, zero, sign, and overflow. In some examples, one or more flag registers 1340 are referred to as program status and control registers.

[0170] Segment register 1320 contains segment points used to access memory. In some examples, these registers are referred to by the names CS, DS, SS, ES, FS, and GS.

[0171] Machine-specific register (MSR) 1335 controls and reports processor performance. Most MSR 1335 handles system-related functions and is not accessible to applications. Machine check register 1360 consists of control, status, and error reporting MSRs used for detecting and reporting hardware errors.

[0172] One or more instruction pointer registers 1330 store instruction pointer values. One or more control registers 1355 (e.g., CR0-CR4) determine the operating mode of the processor (e.g., processors 970, 980, 938, 918, and / or 1000) and the characteristics of the currently executing task. Debug register 1350 controls and allows monitoring of debug operations on the processor or core.

[0173] Memory (mem) management register 1365 specifies the location of data structures used in protected-mode memory management. These registers may include the global descriptor table register (GDTR), the interrupt descriptor table register (IDTR), the task register, and the local descriptor table register (LDTR).

[0174] Alternative examples may use wider or narrower registers. Furthermore, alternative examples may use more, fewer, or different register files and registers. Register architecture 1300 may be used, for example, in register file / memory `ISAB08` or in physical register file circuitry 1158.

[0175] Instruction set architecture.

[0176] An instruction set architecture (ISA) may include one or more instruction formats. A given instruction format may define various fields (e.g., number of bits, bit positions) to specify the operation to be performed (e.g., opcode) and the operand(s) to be performed on, and / or other data fields(e.g., mask), etc. Some instruction formats are further decomposed through the definition of instruction templates (or subformats). For example, an instruction template for a given instruction format may be defined as having different subsets of the fields of that instruction format (the included fields are usually in the same order, but at least some have different bit positions because fewer fields are included) and / or be defined as having given fields interpreted in different ways. Thus, each instruction in an ISA is expressed using a given instruction format (and, if defined, as a given instruction template in the instruction templates of that instruction format) and includes fields for specifying the operation and operand. For example, the sample ADD instruction has a specific opcode and instruction format, which includes an opcode field to specify the opcode and an operand field to select the operand (source 1 / destination and source 2); and the appearance of this ADD instruction in the instruction stream will have specific content in the operand field that selects the specific operand. Furthermore, although the following description is made in the context of an x86 ISA, it is within the knowledge of those skilled in the art to apply the teachings of this disclosure to other ISAs.

[0177] Example instruction format.

[0178] Examples of the instructions(s) described herein can be implemented in different formats. Furthermore, example systems, architectures, and pipelines are detailed below. The examples of the instructions(s) can be executed on these systems, architectures, and pipelines, but are not limited to those detailed herein.

[0179] Figure 14An example of an instruction format is illustrated. As shown, an instruction may include multiple components, including but not limited to one or more fields for the following: one or more prefixes 1401, opcode 1403, addressing information 1405 (e.g., register identifier, memory addressing information, etc.), offset value 1407, and / or immediate value 1409. Note that some instructions utilize some or all of the fields of this format, while others may only use the fields of opcode 1403. In some examples, the order shown is the order in which these fields are encoded; however, it should be understood that in other examples, these fields may be encoded in a different order, combined, etc.

[0180] One or more prefix fields 1401 modify instructions when used. In some examples, one or more prefixes are used for repeating string instructions (e.g., 0xF0, 0xF2, 0xF3, etc.), providing section override (e.g., 0x2E, 0x36, 0x3E, 0x26, 0x64, 0x65, 0x2E, 0x3E, etc.), performing bus locking operations, and / or changing the operand (e.g., 0x66) and address size (e.g., 0x67). Some instructions require mandatory prefixes (e.g., 0x66, 0xF2, 0xF3, etc.). Some of these prefixes can be considered "traditional" prefixes. Other prefixes (one or more examples of which are detailed herein) indicate and / or provide further capabilities, such as specifying particular registers, etc. These other prefixes typically follow "traditional" prefixes.

[0181] Opcode field 1403 is used to at least partially define the operation to be performed when the instruction is decoded. In some examples, the main opcode encoded in opcode field 1403 is one, two, or three bytes long. In other examples, the main opcode can be of other lengths. An additional 3-bit opcode field is sometimes encoded in another field.

[0182] Addressing information field 1405 is used to address one or more operands of an instruction, such as a location in memory or one or more registers. Figure 15An example of addressing information field 1405 is illustrated. This illustration shows the optional MOD R / M byte 1502 and the optional Scale, Index, Base (SIB) byte 1504. MOD R / M byte 1502 and SIB byte 1504 are used to encode up to two operands of an instruction, each operand being either a direct register or an effective memory address. Note that both fields are optional; that is, not all instructions include one or more of these fields. MOD R / M byte 1502 includes MOD field 1542, register field 1544, and R / M field 1546.

[0183] The contents of MOD field 1542 distinguish between memory access and non-memory access modes. In some examples, when MOD field 1542 has a binary value of 11 (11b), register direct addressing mode is used; otherwise, register indirect addressing mode is used.

[0184] Register field 1544 can encode either the destination register operand or the source register operand, or it can encode an opcode extension without being used to encode any instruction operand. The contents of register field 1544 directly specify or are generated from an address to specify the location of the source or destination operand (in a register or in memory). In some examples, register field 1544 is supplemented with extra bits from a prefix (e.g., prefix 1401) to allow for larger addressing.

[0185] R / M field 1546 can be used to encode instruction operands that reference memory addresses, or it can be used to encode destination register operands or source register operands. Note that in some examples, R / M field 1546 can be combined with MOD field 1542 to specify the addressing mode.

[0186] SIB byte 1504 includes a scaling field 1552, an index field 1554, and a base field 1556 for address generation. The scaling field 1552 indicates the scaling factor. The index field 1554 specifies the index register to be used. In some examples, the index field 1554 is supplemented with extra bits from the prefix (e.g., prefix 1401) to allow for larger addressing. The base field 1556 specifies the base address register to be used. In some examples, the base field 1556 is supplemented with extra bits from the prefix (e.g., prefix 1401) to allow for larger addressing. In practice, the contents of the scaling field 1552 allow scaling the contents of the index field 1554 for memory address generation (e.g., for addresses using 2...). 缩放 Address generation from index + base address.

[0187] Some addressing schemes use bitwise shift values ​​to generate memory addresses. For example, they can be based on 2 缩放 Index + Base Address + Displacement, Index Memory addresses are generated using scaling plus offset, r / m plus offset, instruction pointer (RIP / EIP) plus offset, register plus offset, etc. The offset can be a value of 1 byte, 2 bytes, 4 bytes, etc. In some examples, the offset field 1407 provides this value. Additionally, in some examples, the use of an offset factor is encoded in the MOD field of the addressing information field 1405, which indicates a compact offset scheme for which the offset value is calculated and stored in the offset field 1407.

[0188] In some examples, the immediate value field 1409 specifies an immediate value for the instruction. Immediate values ​​can be encoded as 1-byte values, 2-byte values, 4-byte values, and so on.

[0189] Figure 16 An example of the first prefix 1401(A) is illustrated. In some examples, the first prefix 1401(A) is an example of the REX prefix. Instructions using this prefix can specify general-purpose registers, 64-bit compact data registers (e.g., single-instruction multiple-data (SIMD) registers or vector registers), and / or control registers and debug registers (e.g., CR8-CR15 and DR8-DR15).

[0190] Instructions using the first prefix 1401(A) can specify up to three registers using 3-bit fields, depending on the format: 1) using reg field 1544 and R / M field 1546 of MOD R / M byte 1502; 2) using MOD R / M byte 1502 with SIB byte 1504, including using reg field 1544 as well as base field 1556 and index field 1554; or 3) using the register field of the opcode.

[0191] In the first prefix 1401(A), bits 7:4 are set to 0100. Bit 3 (W) can be used to determine the operand size, but cannot determine the operand width alone. Therefore, when W = 0, the operand size is determined by the code segment descriptor (CS.D), while when W = 1, the operand size is 64 bits.

[0192] Note that adding another bit allows for 16(2) pairs. 4It can address 8 registers at a time, while the individual MOD R / M reg field 1544 and MOD R / M R / M field 1546 can each address 8 registers.

[0193] In the first prefix 1401(A), bit position 2(R) can be an extension of the reg field 1544 of the MOD R / M, and can be used to modify the reg field 1544 of the MOD R / M when this field encodes a general-purpose register, a 64-bit compressed data register (e.g., an SSE register), or a control or debug register. R is ignored when MOD R / M byte 1502 specifies other registers or defines extended opcodes.

[0194] Bit position 1 (X) can modify SIB byte index field 1554.

[0195] Setting bit position 0 (B) can modify the base address in the R / M field 1546 or the SIB byte base address field 1556 of MOD R / M; or it can modify the opcode register field used to access general-purpose registers (e.g., general-purpose register 1325).

[0196] Figures 17(A)-17(D) The figures illustrate examples of how the R, X, and B fields of the first prefix 1401(A) are used. Figure 17(A) illustrates how the R and B from the first prefix 1401(A) are used to extend the reg field 1544 and R / M field 1546 of the MOD R / M byte 1502 when SIB byte 1504 is not used for memory addressing. Figure 17(B) illustrates how the R and B from the first prefix 1401(A) are used to extend the reg field 1544 and R / M field 1546 of the MOD R / M byte 1502 (register-to-register addressing) when SIB byte 1504 is not used. Figure 17(C) illustrates how the R, X, and B from the first prefix 1401(A) are used to extend the reg field 1544, index field 1554, and base address field 1556 of the MOD R / M byte 1502 when SIB byte 1504 is used for memory addressing. Figure 17(D) illustrates that when the register is encoded in opcode 1403, the B from the first prefix 1401(A) is used to extend the reg field 1544 of the MOD R / M byte 1502.

[0197] Figures 18(A)-18(B)An example of the second prefix 1401(B) is illustrated. In some examples, the second prefix 1401(B) is an example of a VEX prefix. The second prefix 1401(B) encoding allows instructions to have more than two operands and allows SIMD vector registers (e.g., vector / SIMD register 1310) to be longer than 64 bits (e.g., 128 bits and 256 bits). The use of the second prefix 1401(B) provides syntax for three operands (or more). For example, the previous two-operand instructions performed operations such as A = A + B, which overwrote the source operands. The use of the second prefix 1401(B) allows operands to perform non-destructive operations, such as A = B + C.

[0198] In some examples, the second prefix 1401(B) has two forms—two-byte and three-byte. The two-byte second prefix 1401(B) is mainly used for 128-bit, scalar, and some 256-bit instructions; while the three-byte second prefix 1401(B) provides a compact replacement for 3-byte opcode instructions and the first prefix 1401(A).

[0199] Figure 18(A) illustrates an example of the two-byte form of the second prefix 1401(B). In one example, format field 1801 (byte 0 1803) contains the value C5H. In another example, byte 1 1805 includes the value “R” in bit [7]. This value is the complement of the value of “R” in the first prefix 1401(A). Bit [2] is used to specify the length (L) of the vector (where the value of 0 is a scalar or a 128-bit vector, and the value of 1 is a 256-bit vector). Bits [1:0] provide opcode extensions equivalent to some conventional prefixes (e.g., 00 = no prefix, 01 = 66H, 10 = F3H, and 11 = F2H). The bits [6:3] shown as vvvv can be used to: 1) encode the first source register operand, which is specified in reverse (ones complement) form and is valid for instructions with two or more source operands; 2) encode the destination register operand, which is specified in ones complement form for some vector shift; or 3) not encode any operand, in which case the field is reserved and should contain a value such as 1111b.

[0200] Instructions using this prefix can use the R / M field 1546 of MOD R / M to encode instruction operands that reference memory addresses, or to encode destination register operands or source register operands.

[0201] Instructions using this prefix can use MOD R / M's reg field 1544 to encode either the destination register operand or the source register operand, or they can be treated as an opcode extension without being used to encode any instruction operand.

[0202] For instruction syntax supporting four operands, vvvv, R / M field 1546 of MOD R / M, and reg field 1544 of MOD R / M encode three of the four operands. Then bits [7:4] of immediate value field 1409 are used to encode the third source register operand.

[0203] Figure 18(B) illustrates an example of the three-byte form of the second prefix 1401(B). In one example, format field 1811 (byte 0 1813) contains the value C4H. Byte 1 1815 includes “R”, “X”, and “B” in bits [7:5], which are the complements of these values ​​from the first prefix 1401(A). Bits [4:0] of byte 1 1815 (shown as mmmmm) include the content to encode one or more implicit preamble opcode bytes as needed. For example, 00001 means 0FH preamble opcode, 00010 means 0F38H preamble opcode, 00011 means 0F3AH preamble opcode, and so on.

[0204] The use of bits [7] in byte 2 1817 is similar to that of W in the first prefix 1401(A), including helping to determine the size of the operand that can be promoted. Bit [2] is used to specify the length (L) of the vector (where the value of 0 is a scalar or a 128-bit vector, and the value of 1 is a 256-bit vector). Bits [1:0] provide opcode extensions equivalent to some conventional prefixes (e.g., 00 = no prefix, 01 = 66H, 10 = F3H, and 11 = F2H). Bits [6:3] shown as vvvv can be used to: 1) encode the first source register operand, which is specified in reverse (ones complement) form, valid for instructions with two or more source operands; 2) encode the destination register operand, which is specified in ones complement form, for some vector shift; or 3) not encode any operand, in which case the field is reserved and should contain a value, such as 1111b.

[0205] Instructions using this prefix can use the R / M field 1546 of MOD R / M to encode instruction operands that reference memory addresses, or to encode destination register operands or source register operands.

[0206] Instructions using this prefix can use MOD R / M's reg field 1544 to encode either the destination register operand or the source register operand, or they can be treated as an opcode extension without being used to encode any instruction operand.

[0207] For instruction syntax supporting four operands, vvvv, R / M field 1546 of MOD R / M, and reg field 1544 of MOD R / M encode three of the four operands. Then bits [7:4] of immediate value field 1409 are used to encode the third source register operand.

[0208] Figure 19 The illustration shows an example of the third prefix 1401(C). In some examples, the third prefix 1401(C) is an example of the EVEX prefix. The third prefix 1401(C) is a four-byte prefix.

[0209] The third prefix 1401(C) enables the encoding of 32 vector registers (e.g., 128-bit, 256-bit, and 512-bit registers) in 64-bit mode. In some examples, write masks / operation masks are used (see the discussion of registers in the previous diagrams, e.g.) Figure 13 Instructions that use predicates or operations utilize this prefix. Operation mask registers allow conditional processing or selection control. Operation mask instructions—whose source / destination operands are operation mask registers and whose contents are treated as a single value—are encoded using the second prefix 1401(B).

[0210] The third prefix 1401(C) can encode instruction class-specific features (e.g., a compact instruction with "load + operation" semantics can support embedded broadcast functionality, a floating-point instruction with rounding semantics can support static rounding functionality, a floating-point instruction with non-rounding arithmetic semantics can support "suppress all exceptions" functionality, etc.).

[0211] The first byte of the third prefix 1401(C) is the format field 1911, which in one example has a value of 62H. The subsequent bytes are referred to as payload bytes 1915-1919, and together they form the 24-bit value of P[23:0], which provides specific capabilities in the form of one or more fields (detailed herein).

[0212] In some examples, P[1:0] of payload byte 1919 is the same as the two lower mm bits. In some examples, P[3:2] is reserved. Bit P[4] (R') allows access to the high 16 vector register set when combined with P[7] and MOD R / M's reg field 1544. P[6] can also provide access to the high 16 vector registers when SIB type addressing is not required. P[7:5] consists of R, X, and B, which are operand specifier modifier bits for vector registers, general-purpose registers, and memory addressing, and when combined with MOD R / M's register field 1544 and MOD R / M's R / M field 1546, allows access to the next set of 8 registers beyond the lower 8 registers. P[9:8] provides opcode extensions equivalent to some conventional prefixes (e.g., 00 = no prefix, 01 = 66H, 10 = F3H, and 11 = F2H). P

[10] is a fixed value of 1 in some examples. P[14:11], shown as vvvv, can be used to: 1) encode the first source register operand, which is specified in reverse (ones complement) form, for instructions with two or more source operands; 2) encode the destination register operand, which is specified in ones complement form, for some vector shift; or 3) not encode any operand, in which case the field is reserved and should contain a value such as 1111b.

[0213] P

[15] is similar to W in the first prefix 1401(A) and the second prefix 1411(B), and can be used as an opcode extension bit or an operand size boost.

[0214] P[18:16] specifies the index of the register in the operation mask (write mask) register (e.g., write mask / predicate register 1315). In one example, the specific value aaa = 000 has special behavior, implying that no operation mask is used for this particular instruction (this can be achieved in various ways, including using a hard-wired operation mask to all one or hardware that bypasses the masking hardware). When merging, the vector mask allows any set of elements in the destination to be protected from updates during the execution of any operation (specified by the basic and enhanced operations); in another example, the old value of each element in the destination is preserved (if the corresponding mask bit has a value of 0). In contrast, when zeroing, the vector mask allows any set of elements in the destination to be zeroed during the execution of any operation (specified by the basic and enhanced operations); in one example, the elements in the destination are set to 0 when the corresponding mask bit has a value of 0. A subset of this functionality is the ability to control the vector length of the operation being performed (i.e., the span of the modified elements, from the first to the last); however, the modified elements do not necessarily have to be contiguous. Thus, the operation mask field allows for some vector operations, including load, store, arithmetic, logical, and so on. While in the described example, the content of the operation mask field selects the one among several operation mask registers containing the operation mask to be used (thus the content of the operation mask field indirectly identifies the mask to be performed), alternatively or additionally, alternative examples allow the content of the mask write field to directly specify the mask to be performed.

[0215] P

[19] can be combined with P[14:11] to encode a second source vector register in a non-destructive source syntax that allows access to the high 16 vector registers using P

[19] . P

[20] encodes various functions that differ across different classes of instructions and can affect the meaning of the vector length / rounding control specifier field (P[22:21]). P

[23] indicates support for merge-write masking (e.g., when set to 0) or support for zeroing and merge-write masking (e.g., when set to 1).

[0216] The table below details exemplary examples of register encoding in instructions using the third prefix 1401(C). Table 1: 32 Registers Supported in 64-bit Mode Table 2: Encoding Register Specify in 32-bit Mode Table 3: Operation Mask Register Specifier Encoding

[0217] Program code can be applied to input information to perform the functions described herein and generate output information. The output information can be applied to one or more output devices in a known manner. For the purposes of this application, the processing system includes any system having a processor, such as a digital signal processor (DSP), microcontroller, application-specific integrated circuit (ASIC), field-programmable gate array (FPGA), microprocessor, or any combination thereof.

[0218] The program code can be implemented in a procedural or object-oriented high-level programming language to communicate with the processing system. Assembly or machine language can also be used if desired. In fact, the mechanisms described in this article are not limited to any particular programming language. In any case, the language can be a compiled language or an interpreted language.

[0219] Examples of the mechanisms disclosed herein can be implemented in hardware, software, firmware, or a combination of these approaches. The examples can be implemented as computer programs or program code, executing on a programmable system comprising at least one processor, a storage system (including volatile and non-volatile memory and / or storage elements), at least one input device, and at least one output device.

[0220] One or more aspects of at least one example can be implemented by representative instructions stored on a machine-readable medium, representing various logic within a processor, which, when read by a machine, cause the machine to produce logic for performing the technology described herein. These representations, referred to as “intellectual property (IP) cores,” can be stored on a tangible machine-readable medium and provided to various customers or manufacturing facilities for loading into the manufacturing machine that produces the logic or processor.

[0221] These machine-readable storage media may include—but are not limited to—non-transient tangible arrangements of articles made or formed by machines or equipment, including storage media such as: hard disks, any other type of disk (including floppy disks, optical disks, compact disk read-only memory (CD-ROM), compact disk rewritable (CD-RW), and magneto-optical disks), semiconductor devices (e.g., read-only memory (ROM), random access memory (RAM) such as dynamic random access memory (DRAM), static random access memory (SRAM), erasable programmable read-only memory (EPROM), flash memory, electrically erasable programmable read-only memory (EEPROM), phase change memory (PCM)), magnetic cards or optical cards, or any other type of media suitable for storing electronic instructions.

[0222] Therefore, examples also include non-transitory tangible machine-readable media containing instructions or design data that defines the features of the structures, circuits, devices, processors, and / or systems described herein, such as Hardware Description Language (HDL). Such examples may also be referred to as program products.

[0223] Simulation (including binary translation, code transformation, etc.).

[0224] In some cases, instruction translators can be used to translate instructions from a source instruction set architecture to a target instruction set architecture. For example, an instruction translator can translate (e.g., using static binary translation, including dynamic binary translation with dynamic compilation), transform, emulate, or otherwise translate instructions into one or more other instructions to be processed by the kernel. Instruction translators can be implemented in software, hardware, firmware, or a combination thereof. Instruction translators can be on-processor, off-processor, or partially on-processor and partially off-processor.

[0225] Figure 20The block diagram illustrates the use of a software instruction converter according to an example, which is used to convert binary instructions in a source ISA into binary instructions in a target ISA. In the illustrated example, the instruction converter is a software instruction converter, but alternatively, the instruction converter can be implemented using software, firmware, hardware, or various combinations thereof. Figure 20 A program of high-level language 2002 is shown to be compiled using a first ISA compiler 2004 to generate first ISA binary code 2006, which can be natively executed by a processor 2016 having at least one first ISA core. A processor 2016 having at least one first ISA core represents any processor capable of performing substantially the same function as an Intel processor having at least one first ISA core by compatiblely executing or otherwise processing (1) a substantial portion of the first ISA or (2) a version of object code for an application or other software targeted to run on an Intel® processor having at least one first ISA core, so as to achieve substantially the same results as a processor having at least one first ISA core. The first ISA compiler 2004 represents a compiler operable to generate first ISA binary code 2006 (e.g., object code) that can be executed on a processor 2016 having at least one first ISA core, with or without additional linking processing. Similarly, Figure 20 A program in high-level language 2002 is shown to be compiled using an alternative ISA compiler 2008 to generate alternative ISA binary code 2010, which can be natively executed by a processor 2014 without a first ISA core. An instruction converter 2012 is used to convert the first ISA binary code 2006 into code that can be natively executed by a processor 2014 without a first ISA core. This converted code may not necessarily be identical to the alternative ISA binary code 2010; however, the converted code will implement the overall operation and consist of instructions from the alternative ISA. Thus, the instruction converter 2012 represents software, firmware, hardware, or a combination thereof that, through emulation, simulation, or any other process, allows a processor or other electronic device without a first ISA processor or core to execute the first ISA binary code 2006.

[0226] In the various matrix processing implementations described below, source and destination matrix data elements are packed into a 512-bit vector register and interpreted as matrices rather than vectors. According to these implementations, the matrix multiplication property associated with the 512-bit vector length is utilized by a new set of matrix multiplication instructions, including a first instruction for performing block-by-block matrix multiplication and a second instruction for performing vector burst matrix multiplication (sometimes referred to as the "cross product" of matrix blocks). The first instruction is sometimes referred to herein as VPMM-X (Vector-Extension Packed Matrix Multiplication X-product), and the second instruction is sometimes referred to as VBMM (Vector Burst Matrix Multiplication). Compared to existing matrix multiplication instructions (e.g., VNNI instructions), at least some implementations of these matrix multiplication instructions achieve four times the number of matrix operations without increasing register storage requirements.

[0227] In some implementations, 512-bit consecutive matrix data elements are loaded into a 512-bit vector register (e.g., the "zmm" register), and a 4×4×K matrix multiplication is performed. In these implementations, the basic operation of matrix multiplication (which can be used as a building block for larger matrix multiplications) is the cross product matrix multiplication: xmm (128 bits) = ymm (256 bits). ymm (256 bits) 2×2×K.

[0228] Figures 21A-21B The diagram illustrates two example implementations 2100A-2100B of a cross product operation (e.g., based on the execution of a decoded VPMM-X instruction). Both examples include a 2xK source matrix (source matrix A) stored in ymm0 2110A-2110B and a K×2 source matrix (source matrix B) stored in ymm1 2111A-2111B, which are multiplied by multiplication-addition (or multiplication-accumulation) circuitry 2120A-2120B to produce a result matrix in xmm2 2112A-2112B. In these examples, the ymm and xmm registers can be 256-bit and 128-bit portions of a 512-bit zmm register, respectively. As further described below, multiple such cross product operations can be performed using corresponding portions (sometimes referred to as partitions or slices) of source matrix A and source matrix B to produce the result matrix in xmm2 2112A-2112B.

[0229] exist Figure 21AIn ymm0 2110A and ymm1 2111A, the source data elements of the source matrices A and B are 16-bit data elements (e.g., INT16, BF16, FP16), and... Figure 21B In the example, the source data elements of the source matrices A and B in ymm0 2110B and ymm1 2111B are 8-bit data elements (e.g., INT8, FP8). In these examples, the variable K is the length of the inner product dimension of matrices A and B. In example implementation 2100A, K=8 for the 16-bit source matrix values ​​2110A, 2111A, and in implementation 2100B, K=16 for the 8-bit source matrix values ​​2110B, 2111B. The resulting matrix (matrix C) generated by the multiplication-addition / accumulation circuits 2120A-2120B includes 32-bit data elements (e.g., INT32, FP32) stored in the destination registers xmm2 2112A-2112B (e.g., the 128-bit portion of the zmm register). Specifically, the multiplication-addition / accumulation circuits 2120A-2120B multiply the source data elements in each row of matrix A with the corresponding data elements in each column of matrix B to produce multiple corresponding products, and then add the products together to generate the corresponding result data elements of matrix C. c 00 , c 01 c 10 c 11 Matrix multiplication instructions can also accumulate any data element stored in matrix C with its corresponding product set to generate the corresponding result data element, which is then stored back into matrix C 2112A-2112B. Therefore, in these implementations, matrix C is operated on as both the source and destination matrix.

[0230] Specifically, referring to implementation 2100A, the 16-bit data elements in the first row of matrix A a 0,0 、a 0,1 、a 0,2 、 a 0,3 、a 0,4 、a 0,5 、a 0,6 、a 0,7 The corresponding 16-bit data element in the first column of matrix B b 0,0 、b 1,0、b 2,0 、b 3,0 、b 4,0 、 b 5,0 、b 6,0 、b 7,0 The products are multiplied to produce eight corresponding products, which are then added to the corresponding accumulated values ​​via multiple adders in the multiplication-addition / accumulation circuit 2120A. c’ 0,0 To generate corresponding new result data elements c 0,0 (Right now, c 00 = c’ 0,0 + a 0,0 b 0,0 + a 0,1 b 1,0 + … a 0,7 b 7,0 The corresponding data elements of the second row of matrix A and the first column of matrix B, along with the accumulated value, can be obtained by the multiplication-addition / accumulation circuit 2120A. c’ 1,0 Execute an additional set of multiplication-addition operations in parallel to generate c 1,0 Use the first row of matrix A, the second column of matrix B, and the accumulated value. c’ 0,1 Execute an additional set of multiplication-addition operations in parallel to generate c 0,1 ; and using the second row of matrix A, the second column of matrix B, and the accumulated value. c’ 1,1 Execute an additional set of multiplication-addition operations in parallel to generate c 1,1 .

[0231] In the various implementations described herein, the source matrices A and B can be matrix partitions or slices that include portions of the larger A and B matrices. For example, slices of matrix A and B can be defined based on the size of the source matrix registers and / or the parallel processing capabilities of the matrix / tensor execution circuitry (e.g., the number of processing channels and the width of each channel).

[0232] Figure 22 The diagram shows that the source matrix data elements and the result matrix data elements are packed into multiple 512-bit vector registers (sometimes referred to in this paper as...). zmm Example within the registers. Specifically, the 16-bit data elements of the first matrix (matrix A) are packed in the first vector register zmm0 2200, the 16-bit data elements of the second matrix (matrix B) are packed in the second vector register zmm1 2210, and the 32-bit result data elements are packed in the third vector register zmm22220. In this particular implementation, zmm0 2200 is divided into zmm.lo 2201 and zmm.hi 2202, zmm.lo 2201 stores the first 256-bit half of the rows of the source matrix A, and zmm.hi 2202 stores the other 256-bit half of the rows of the source matrix A. Similarly, zmm1 2210 is divided into zmm.lo 2211 and zmm.hi 2212, with zmm.lo 2211 storing a 256-bit half of the column of source matrix B and zmm.hi 2212 storing the other 256-bit half of the column of source matrix B.

[0233] These implementations facilitate the multiplication of matrices A and B as a cross product operation, where each 2×K submatrix of source matrix A 2201-2202 is multiplied by each K×2 submatrix of source matrix B 2211-2212 to generate four corresponding submatrices 2221-2224 of the result matrix C stored in zmm22220. For example, to generate the top-left submatrix 2221 of the result matrix C, the multipliers and adders of the multiplication-accumulation circuit 2250A multiply the data elements in each row of matrix A in zmm.lo 2201 with the corresponding data elements in each column of matrix B in zmm.lo 2211 to generate corresponding product groups, each product group corresponding to a specific combination of rows of matrix A, columns of matrix B, and specific data elements of submatrix 2221 of matrix C. Each product group is added to the corresponding data element in the corresponding submatrix 2221 of matrix C to generate the corresponding result data element (e.g., ...). C 0,0、 C 0,1、 C 1,0、 C 1,1Similarly, to generate submatrix 2224 of the resulting matrix C, the multipliers and adders of the multiplication-accumulation circuit 2250D multiply the data elements of each row of matrix A in zmm.hi 2202 with the corresponding data elements in each column of matrix B in zmm.hi 2212 (e.g., with...). Figure 21A (as shown in the diagram) to generate product groups. Each product group is added to the corresponding data element of matrix C. C’ 2,2 To generate the corresponding result data elements. C 2,2 .

[0234] As shown in the figure, two of the four submatrices of the resulting matrix C are stored in zmm.lo 2221, and two are stored in zmm.hi 2222. Given that each submatrix 2221-2224 of the resulting matrix C is 128 bits (i.e., an xmm partition of zmm), the entire C matrix can now be held in a single zmm register zmm2 2220, effectively doubling the number of multiplication-accumulation operations without requiring new registers.

[0235] Therefore, according to these implementations, each of the source matrices A and B is partitioned and packed as multiple submatrices into a 512-bit vector register (e.g., a zmm register). A cross product is performed using the submatrices of matrix A and matrix B to generate corresponding submatrices 2221-2224 of the resulting matrix C. For example, in Figure 22 In the process, the two submatrices of matrix A are packed into a 512-bit vector register (zmm0 2200), and the two submatrices of matrix B are packed into another 512-bit vector register (zmm1 2210). Then, a cross product is performed to multiply each submatrix of matrix A with each submatrix of matrix B to generate four corresponding submatrices 2221-2224 (zmm2 2220) of the resulting matrix C.

[0236] Some implementations define multiple different instructions to perform matrix multiplication operations with different source and destination data widths and formats. As an example, and not a limitation, source data formats (e.g., those used for source matrices A and B as described herein) may include TF32 (Tensor 32), FP16 (16-bit floating-point), BF16 (16-bit floating-point), FP8 (BF8 / HF8) (8-bit floating-point with 5 or 4 exponent bits), and INT8. In some implementations, different matrix multiplication instructions may have different opcodes to specify the source data format and / or the dimensions of source matrices A and B, and the format and / or dimensions of the result matrix C. In some implementations, when the source matrices A and B are in TF32, FP16, BF16, or FP8 data formats, the FP32 data format is used for the data elements of the result matrix C. When the INT8 data format is used for source matrices A and B, the INT32 data format can be used for the result matrix C.

[0237] Figure 23 This is a block diagram of an embodiment of a processor or core (e.g., core 1190) of a processor 2300 that operates to execute matrix multiplication instructions 2305 stored in storage device and / or memory 2303. Matrix multiplication instructions 2305 may represent macro instructions, machine code instructions, or other instructions in the processor's instruction set. Matrix multiplication instructions 2305 may have various formats or encodings, such as, for example, regarding... Figure 14-19 The encoding may include one or more fields for an opcode that at least partially or completely specifies the operation to be performed (e.g., matrix multiplication and addition / accumulation), and one or more fields for one or more operands, such as operands of registers that can be used to identify the source matrix (e.g., submatrices A and B packed in a zmm register as described herein) and the result matrix (e.g., matrix C stored in a single zmm register).

[0238] For example, as shown in the illustrated embodiment, each matrix multiplication instruction 2305 may specify or otherwise indicate a first (e.g., source) vector register in vector register file 2350 for storing a first source matrix (A), a second (e.g., source) vector register in vector register file 2350 for storing a second source matrix (B), and a third (e.g., source / destination) vector register in vector register file 2350 for storing a third matrix. Alternatively, one or more of the first, second, and third matrices may optionally be stored in memory 2303 or other locations (e.g., one of multiple cache levels).

[0239] Decoder circuitry 2310 (e.g., an instruction decoder) may be coupled to receive each matrix multiplication instruction 2305 fetched from memory 2303 by instruction fetch circuitry (not shown) and decode the matrix multiplication instruction 2305 into one or more lower-level control signals, operations, or decoded instructions (e.g., one or more microinstructions, microoperations, microcode entry points, etc.). The decoder circuitry may be as described above regarding... Figure 5 The decoder circuit 502 and / or Figure 11B It operates as described in the decoding circuit 1140.

[0240] In some examples, register renaming, allocation, and / or scheduling circuitry 2320 may provide functionality for one or more of the following: (1) renaming logical operand values ​​to physical operand values ​​(e.g., register alias tables in some examples); (2) assigning status bits and flags to decoded instructions; and (3) scheduling decoded instructions from the instruction pool for execution circuitry to execute (e.g., using reserved stations in some examples). As previously described, vector register 2350 (and / or memory 2303) may store the source and result matrices of matrix multiplication instructions. For example, the zmm0, zmm1, and zmm2 registers (and xmm and ymm registers) described herein may be allocated from vector register file 2350.

[0241] Execution circuitry 2330 may be coupled to decoder circuitry 2310, register renaming / allocation / scheduler circuitry 2320, and register / memory 2350, and may perform matrix operations corresponding to instructions as described herein. For example, one or more lower-level control signals, operations, or decoded instructions may be executed by the execution circuitry to control it to perform operations corresponding to the instructions (e.g., operations specified at least in part by the instruction's opcode). Write-back / retrieval circuitry 2340 performs conflict checks before retiring the results produced by execution circuitry 2330 (e.g., a result matrix stored in zmm2).

[0242] In the illustrated implementation, execution circuitry 2330 includes integrated tensor processing circuitry 2335 for performing various operations described herein based on corresponding matrix multiplication instructions. In some embodiments, the operation may include performing cross multiplication by multiplying each submatrix 2201-2202 of matrix A with each submatrix 2210 of matrix B to generate four submatrixes 2221-2224 of matrix C2220.

[0243] In some embodiments, the resulting matrix 2220 may be generated from matrix multiplication using a first matrix 2200 and a second matrix 2210, wherein prior to the matrix multiplication, the data elements of the first and second matrices are converted from a first number of bits to a larger number of bits (e.g., from 8 bits or 16 bits to 32 bits). In some embodiments, the resulting matrix may be generated from matrix multiplication using a first matrix and a second matrix, wherein floating-point rounding may be optionally performed as needed during matrix multiplication and / or accumulation. In various embodiments, rounding may be performed as needed after each multiplication used to generate each resulting element, rounding may be performed as needed after each accumulation operation used to generate each resulting element, rounding may be performed as needed after each multiplication used to generate each resulting element and also as needed after each accumulation operation used to generate each resulting element, and rounding may be performed once as needed after all operations used to generate the resulting element.

[0244] In some embodiments, the processor may have registers (e.g., MXCSR or other floating-point control registers) for controlling floating-point operations. In some embodiments, the registers may have one or more fields (e.g., rounding mode fields) for specifying one of a plurality of supported rounding modes to be used in the floating-point operation. In some embodiments, in order to perform an operation corresponding to an instruction, when at least some data elements of a matrix are floating-point data elements, the execution circuitry may perform rounding according to only a single rounding mode (e.g., nearest-even rounding mode), regardless of and / or regardless of one or more fields in the register (e.g., rounding mode fields), and / or without checking those one or more fields. In some such embodiments, rounding may be performed according to a single rounding mode in a mathematically precise manner, rather than by simplification that results in mathematically inaccurate results. In some embodiments, the registers may have one or more fields (e.g., denormalized zero (DAZ) bits or fields) for specifying whether nonnormal values ​​in the input (e.g., source matrix) to the floating-point instruction / operation are to be set to zero, zeroed, or treated as zero. When rounding and handling nonnormal numbers, various matrix multiplication implementations can be found as described above. Figure 5 Operate as described.

[0245] Figures 24A-24C Different configuration options for the tensor processing circuit 2335 relative to the processor or core 2400 are shown. Figure 24AThe illustration shows an example of tensor processing circuitry 2335 coupled to core 2400 and sharing memory 2403 with core 2400. In this example, tensor processing circuitry 2335 may rely on memory management circuitry of core 2400 or may include its own memory management circuitry. In some implementations, tensor processing circuitry 2335 is coupled to an input / output memory management unit (IOMMU) that performs memory management functions on behalf of tensor processing circuitry 2335. Tensor processing circuitry 2335 may be on the same die as core 2400 and may be shared by core 2400 and one or more other cores. Alternatively, tensor processing circuitry 2335 may be on a separate die, but in the same package as the die of core 2400. In other implementations, tensor processing circuitry 2335 may be in a different package than core 2400 (e.g., in a different socket within the same computer system).

[0246] In various configurations, core 2400 can establish work queues 2451 (e.g., in response to the execution of runtime software) to submit work to tensor processing circuitry 2335. Each work queue 2451 may include multiple entries submitted by core 2400, where each entry identifies a tensor core 2450 (or other binary code) to be executed. Tensor processing circuitry 2335 reads each entry from a given work queue 2451 to identify its corresponding core 2450 to be executed accordingly. One or more cores 2450 may include instances of the matrix multiplication instructions described herein.

[0247] Figure 24B The diagram illustrates a configuration where the tensor processing circuitry 2335 is integrated with the core 2400 (e.g., in relation to...). Figure 23 (Described in the same or similar manner).

[0248] Figure 24CThe diagram illustrates an implementation where the memory subsystem includes system memory 2403 accessible by core 2400 and device memory 2404 associated with tensor processing circuitry 2335 (e.g., dedicated memory configured in the same package as tensor processing circuitry 2335, such as graphics memory integrated on the same graphics card as the graphics processor). As described above, core 2400 can be provided with access to device memory 2404 to establish a work queue 2451 identifying core 2450 to be executed. Alternatively or additionally, in addition to device memory 2404, tensor processing circuitry 2335 may also be provided with access to system memory 2403 (or a portion thereof) and may share a portion of the virtual address space with core 2400 (e.g., in a shared virtual memory (SVM) implementation). In this implementation, device memory 2404 stores program code and data most frequently accessed by tensor processing circuitry 2335 (i.e., to improve performance).

[0249] Depending on the implementation method, it is used for execution Figure 22 The matrix multiplication instructions in the code include the Vector Extended Compact Matrix Multiplication X-product (VPMM-X) instruction and the Vector Burst Matrix Multiplication (VBMM) instruction. The following code sequence defines various embodiments of these instructions.

[0250] Example instruction templates for VPMM-X and VBMM instructions are as follows: VL=512 template zmm1, zmm2, zmm3 Operation object 1: ModRM:reg( r,w ) Operand 2: VEX.vvvv(r) Operation object 3: ModRM:reg( r )

[0251] In the code above, VL indicates a 512-bit vector length (e.g., the zmm register), and ModRM:11:rrr:bbb indicates RISC-type instructions operating in 64-bit mode. In some implementations, floating-point exceptions are not tracked (although NaN / Inf propagation is checked). The control register MXCSR is neither consulted nor updated.

[0252] In some implementations, for SIMD FP rounding and dump clear to zero (FTZ) / denormal number zero (DAZ) processing, the floating-point matrix multiplication instruction described in this paper uses nearest even rounding (RNE) and FTZ=1 (for all true). For FP16 and all BF8 / HF8 inputs, DAZ is 0, and for BF16 and TF32 implementations, DAZ is set to 1.

[0253] In operation, these instructions are 512-bit compact matrix multiplication instructions. Each 256-bit channel in src1 and src2 is interpreted as a 2×X matrix (matrix A) or an X×2 matrix (matrix B). The 256-bit submatrices of source matrices A and B are cross-multiplied, as shown in the diagram. Figure 22 The result is a 2×2 matrix 2220 in FP32 / INT32.

[0254] In some embodiments, the C matrix (e.g., 2220) is both a source and a destination. For example, in Figure 22 In this implementation, zmm22220 may include accumulated source data elements (e.g., from previous matrix operations) that are added to the corresponding product resulting from the multiplication of data elements from matrix A (zmm02200) and matrix B (zmm12210). In this implementation, the multiplication-accumulation circuits 2250A-2250B add the corresponding values ​​from zmm2220 to the corresponding product set and store the resulting data elements back to the corresponding positions in zmm2220 (matrix C).

[0255] The following is a sample instruction operation template for VPMM-X and VBMM instructions: Operation templates: dest, src1, src2 VL=(512) KL = VL / 128 TMP[127:0] := PREC_SPECIFIC_matmul( src1.256b[0], src2.256b[0] ) TMP[255:128] := PREC_SPECIFIC_matmul( src1.256b[0], src2.256b[1] ) TMP[383:256] := PREC_SPECIFIC_matmul( src1.256b[1], src2.256b[0] ) TMP[511:384] := PREC_SPECIFIC_matmul( src1.256b[1], src2.256b[1] ) FOR i := 0 to KL-1: FOR j := 0 to 3: dest.fp32 / i32[i 4 + j] := dest.fp32 / i32[i 4 + j] + TMP.fp32 / i32[i 4 + j]

[0256] The code sequence above specifies a separate product between each 256-bit portion of matrix A (src1.256b[0] and src1.256b[1]) and each 256-bit portion of matrix B (src2.256b[0] and src2.256b[1]). These four products are stored in temporary (TMP) locations and are added to the corresponding data element values ​​in the destination 512-bit vector register (e.g., zmm2 2220) before being stored back to their corresponding locations in the destination. Depending on the capabilities of the tensor processing circuit 2335, the accumulation operations shown here and in the subsequent code sequences can be performed in any order or in parallel.

[0257] When interpreting and addressing elements in a vector register, the following definition can be used: .128b -> Interprets vector elements as 128 bits wide .fp32 -> interprets vector elements as float32 values .tf32 -> interprets vector elements as tensor-float32 values. .bf16-> interprets vector elements as bfloat16 values. .fp16 -> interprets vector elements as float16 values .bf8 -> interprets vector elements as bfloat8 values .hf8 -> interprets vector elements as hfloat8 values .i32-> interprets vector elements as signed int32 values. .s8-> interprets vector elements as signed int8 values. .u8 -> interprets vector elements as unsigned int8 values.

[0258] Different matrix multiplication instructions can be defined based on the following source / destination data formats and other configuration options: ·TF32FP32Mul: TF32 multiplier with FP32 output • BF16FP32Mul: Scalar BF16 multiplication with FP32 output o F16FP32Mul: Scalar FP16 multiplication with FP32 output • BF8FP32Mul: Scalar BF8 multiplication with FP32 output o HF8FP32Mul: Scalar HF8 multiplication with FP32 output o BHF8FP32Mul: Scalar BF8 with FP32 output HF8 Multiplication o HBF8FP32Mul: Scalar HF8 with FP32 output BF8 Multiplication • SS8I32Mul: Scalar signed INT8 multiplication with INT32 output o UU8I32Mul: Scalar unsigned INT8 multiplication with INT32 output o SU8I32Mul: A scalar with signed INT8 outputs. Unsigned INT8 multiplication o US8I32Mul: Scalar unsigned INT8 with INT32 output Signed INT8 multiplication

[0259] The example implementation of VPMM-X and VBMM matrix multiplication instructions is defined as follows:

[0260] The operation is performed using `tf32fp32_2x2x4_matmul( src1[256:0], src2[256:0] )`. DEST[127:0] = 0 FOR n := 0 to 1: FOR m := 0 to 1: P0[31:0] := TF32FP32Mul(src1.tf32[m 4+0], src2.tf32[n 4+0]) P1[31:0] := TF32FP32Mul(src1.tf32[m 4+1], src2.tf32[n 4+1]) P2[31:0] := TF32FP32Mul(src1.tf32[m 4+2], src2.tf32[n 4+2]) P3[31:0] := TF32FP32Mul(src1.tf32[m 4+3], src2.tf32[n 4+3]) DEST.fp32[m 2+n] := fp32_reduce( P0, P1, P2, P3 ) RET DEST

[0261] The code above specifies TF32 data elements for matrices A and B, and FP32 data elements for matrix C. To execute... Figure 22 The cross product shown is the multiplication of each submatrix of matrix A with each submatrix of matrix B. For a variant of the above instruction, each TF32 data element of a given submatrix of matrix A is multiplied with the corresponding TF32 data element of a given submatrix of matrix B, and the products are added (fp32_reduce) to generate the corresponding submatrix of matrix C (e.g., Figure 22 One FP32 data element from submatrices 2221-2224. The resulting submatrices are then combined in the destination register (e.g., submatrices 2221-2224 are combined to form...). Figure 22 (matrix C in zmm2 2220).

[0262] The operation is performed using `bf16fp32_2x2x8_matmul( src1[256:0], src2[256:0] )`. DEST[127:0] = 0 FOR n := 0 to 1: FOR m := 0 to 1: P0[31:0] := BF16FP32Mul(src1.bf16[m 8+0], src2.bf16[n 8+0]) P1[31:0] := BF16FP32Mul(src1.bf16[m 8+1], src2.bf16[n 8+1]) P2[31:0] := BF16FP32Mul(src1.bf16[m [8+2], src2.bf16[n 8+2]) P3[31:0] := BF16FP32Mul(src1.bf16[m 8+3], src2.bf16[n 8+3]) P4[31:0] := BF16FP32Mul(src1.bf16[m [8+4], src2.bf16[n 8+4]) P5[31:0] := BF16FP32Mul(src1.bf16[m [8+5], src2.bf16[n 8+5]) P6[31:0] := BF16FP32Mul(src1.bf16[m 8+6], src2.bf16[n 8+6]) P7[31:0] := BF16FP32Mul(src1.bf16[m [8+7], src2.bf16[n 8+7]) DEST.fp32[m 2+n] := fp32_reduce( P0, P1, P2, P3, P4, P5, P6, P7 RET DEST

[0263] The code above specifies the rows of matrix A, the BF16 data elements in the columns of matrix B, and the FP32 data elements in matrix C. To execute... Figure 22 The cross product shown is the product of each submatrix of matrix A and each submatrix of matrix B (e.g., as shown in the diagram). Figure 21A (As shown for a single submatrix). For the above instruction variant, each FP16 data element in each row of the submatrix of matrix A is multiplied by the corresponding FP16 data element in a column of the submatrix of matrix B. The products are added (fp32_reduce) to generate the corresponding submatrix of matrix C (e.g., ...). Figure 22 One FP32 data element (from one of submatrices 2221-2224 in the matrix). The resulting submatrices are combined in the destination register (e.g., submatrices 2221-2224 are combined to form...). Figure 22 (matrix C in zmm2 2220).

[0264] The same set of operations shown above can be performed on the operation f16fp32_2x2x8_matmul( src1[256:0], src2[256:0] ) with the only difference being that FP16 values ​​are used instead of BF16 values ​​for the source matrix data elements.

[0265] The operation is performed using `bf8fp32_2x2x16_matmul( src1[256:0], src2[256:0] )`. DEST[127:0] = 0 FOR n := 0 to 1: FOR m := 0 to 1: P0[31:0] := BF8FP32Mul(src1.bf8[m 16+0], src2.bf8[n 16+0]) P1[31:0] := BF8FP32Mul(src1.bf8[m 16+1], src2.bf8[n 16+1]) P2[31:0] := BF8FP32Mul(src1.bf8[m 16+2], src2.bf8[n 16+2]) P3[31:0] := BF8FP32Mul(src1.bf8[m 16+3], src2.bf8[n 16+3]) P4[31:0] := BF8FP32Mul(src1.bf8[m 16+4], src2.bf8[n 16+4]) P5[31:0] := BF8FP32Mul(src1.bf8[m 16+5], src2.bf8[n 16+5]) P6[31:0] := BF8FP32Mul(src1.bf8[m 16+6], src2.bf8[n 16+6]) P7[31:0] := BF8FP32Mul(src1.bf8[m 16+7], src2.bf8[n 16+7]) P8[31:0] := BF8FP32Mul(src1.bf8[m 16+0], src2.bf8[n 16+0]) P9[31:0] := BF8FP32Mul(src1.bf8[m 16+1], src2.bf8[n 16+1]) PA[31:0] := BF8FP32Mul(src1.bf8[m 16+2], src2.bf8[n 16+2]) PB[31:0] := BF8FP32Mul(src1.bf8[m 16+3], src2.bf8[n 16+3]) PC[31:0] := BF8FP32Mul(src1.bf8[m 16+4], src2.bf8[n 16+4]) PD[31:0] := BF8FP32Mul(src1.bf8[m 16+5], src2.bf8[n 16+5]) PE[31:0] := BF8FP32Mul(src1.bf8[m 16+6], src2.bf8[n 16+6]) PF[31:0] := BF8FP32Mul(src1.bf8[m 16+7], src2.bf8[n 16+7]) DEST.fp32[m 2+n] := fp32_reduce( P0, P1, P2, P3, P4, P5, P6, P7, P8, P9, PA, PB, PC, PD, PE, PF ) RET DEST

[0266] The code above specifies BF8 data elements for matrices A and B, and FP32 data elements for matrix C. To execute... Figure 22 The cross product shown is the product of each submatrix of matrix A and each submatrix of matrix B (e.g., as shown in the diagram). Figure 21B (As shown for a single submatrix). For the above instruction variant, the FP8 data element in each row of a given submatrix of matrix A is multiplied by the corresponding FP8 data element in a column of a given submatrix of matrix B. The products are added (fp32_reduce) to generate the corresponding submatrix of matrix C (e.g., ...). Figure 22One FP32 data element from submatrices 2221-2224. The resulting submatrices are then combined in the destination register (e.g., submatrices 2221-2224 are combined to form...). Figure 22 (matrix C in zmm2 2220).

[0267] The following additional instructions operate as described above for bf8fp32_2x2x16_matmul, with any specified differences:

[0268] The operation is hf8fp32_2x2x16_matmul( src1[256:0], src2[256:0] )

[0269] Similar to bf8fp32_2x2x16_matmul, but uses HF8 for the source data elements of matrices A and B.

[0270] The operation is performed using `bhf8fp32_2x2x16_matmul( src1[256:0], src2[256:0] )`.

[0271] Similar to bf8fp32_2x2x16_matmul, but use BF8 for matrix A and HF8 for matrix B.

[0272] The operation is performed using `hbf8fp32_2x2x16_matmul( src1[256:0], src2[256:0] )`.

[0273] Similar to bf8fp32_2x2x16_matmul, but use HF8 for matrix A and BF8 for matrix B.

[0274] The following instructions are defined for integer source and destination data elements:

[0275] The operation is performed using `ss8i32_2x2x16_matmul( src1[256:0], src2[256:0] )`. DEST[127:0] = 0 FOR n := 0 to 1: FOR m := 0 to 1: P0[31:0] := SS8I32Mul(src1.s8[m 16+0], src2.s8[n 16+0]) P1[31:0] := SS8I32Mul(src1.s8[m 16+1], src2.s8[n 16+1]) P2[31:0] := SS8I32Mul(src1.s8[m 16+2], src2.s8[n 16+2]) P3[31:0] := SS8I32Mul(src1.s8[m 16+3], src2.s8[n 16+3]) P4[31:0] := SS8I32Mul(src1.s8[m 16+4], src2.s8[n 16+4]) P5[31:0] := SS8I32Mul(src1.s8[m 16+5], src2.s8[n 16+5]) P6[31:0] := SS8I32Mul(src1.s8[m 16+6], src2.s8[n 16+6]) P7[31:0] := SS8I32Mul(src1.s8[m 16+7], src2.s8[n 16+7]) P8[31:0] := SS8I32Mul(src1.s8[m 16+0], src2.s8[n 16+0]) P9[31:0] := SS8I32Mul(src1.s8[m 16+1], src2.s8[n 16+1]) PA[31:0] := SS8I32Mul(src1.s8[m 16+2], src2.s8[n 16+2]) PB[31:0] := SS8I32Mul(src1.s8[m 16+3], src2.s8[n 16+3]) PC[31:0] := SS8I32Mul(src1.s8[m 16+4], src2.s8[n 16+4]) PD[31:0] := SS8I32Mul(src1.s8[m 16+5], src2.s8[n 16+5]) PE[31:0] := SS8I32Mul(src1.s8[m 16+6], src2.s8[n 16+6]) PF[31:0] := SS8I32Mul(src1.s8[m 16+7], src2.s8[n 16+7]) DEST.i32[m 2+n] := int32_reduce_nonsat( P0, P1, P2, P3, P4, P5, P6, P7, P8, P9, PA, PB, PC, PD, PE, PF )

[0276] The code above specifies signed INT8 data elements for matrices A and B, and INT32 (i32) data elements for matrix C. To execute... Figure 22 The cross product shown is generated by multiplying each data element in a row of a given submatrix of matrix A with the corresponding data element in each column of a given submatrix of matrix B to produce a corresponding product. For a variant of the above instruction, each signed INT8 data element in each row of a given submatrix of matrix A is multiplied with the corresponding signed INT8 data element in the corresponding column of a given submatrix of matrix B to produce multiple products, which are then added (fp32_reduce) to generate the corresponding submatrix of matrix C (e.g., ...). Figure 22 One INT32 data element from one of submatrices 2221-2224 in the matrix. The resulting submatrices are then combined in the destination register (e.g., submatrices 2221-2224 are combined to form...). Figure 22 (matrix C in zmm2 2220).

[0277] The following additional instructions operate as described above for ss8i32_2x2x16_matmul( src1[256:0], src2[256:0]), but with different combinations of unsigned / signed INT8 source values ​​as specified:

[0278] The operation is performed using `uu8i32_2x2x16_matmul( src1[256:0], src2[256:0] )`.

[0279] Similar to ss8i32_2x2x16_matmul, but uses unsigned INT8 values ​​in both matrix A and matrix B.

[0280] The operation is su8i32_2x2x16_matmul( src1[256:0], src2[256:0] )

[0281] Similar to ss8i32_2x2x16_matmul, but uses signed INT8 for matrix A and unsigned INT8 for matrix B.

[0282] The operation is us8i32_2x2x16_matmul( src1[256:0], src2[256:0] )

[0283] Similar to ss8i32_2x2x16_matmul, but uses unsigned INT8 for matrix A and signed INT8 for matrix B.

[0284] Some implementations of matrix multiplication instructions support microscaling formats, such as different combinations of Microscaling Floating-Point (MXFP) formats, such as MXFP8 (8-bit floating-point), MXFP6 (6-bit floating-point), and / or MXFP4 (4-bit floating-point). These implementations operate according to the embodiments described above, incorporating a 4+4 scaling factor for calculating 16 inner products within a single VPMM-X / VBMM instruction. For example, the scaling factors can be provided via mask registers (since they are not otherwise used in VPMM-X / VBMM instructions), and they are appropriately sized for this purpose.

[0285] Additionally, in some embodiments, matrix A or matrix B is provided in transposed representation. For example, values ​​can be loaded from the register file (e.g., the corresponding zmm register) to the execution circuitry (e.g., each ALU) in a transposed manner (i.e., transposed at runtime). These embodiments significantly reduce software requirements with small incremental costs in the hardware.

[0286] Figure 25 The diagram illustrates a method based on several implementations. This method can be implemented on various processor, core, and system architectures described herein, but is not necessarily limited to any particular architecture(s).

[0287] At position 2501, the first source matrix is ​​loaded into the first vector register, and the second source matrix is ​​loaded into the second vector register. In various implementations described herein, for example, each vector register is a 512-bit vector register.

[0288] At 2502, an instance of a matrix multiplication instruction is decoded, which has fields for indicating the first source matrix and the second source matrix (e.g., operand fields), fields for indicating the third source matrix and the result matrix to be stored in the third vector register, and operand fields for indicating the cross product matrix multiplication operation to be performed using the submatrices of the first source matrix and the second source matrix.

[0289] At 2503, a matrix multiplication instruction is executed to perform a cross product matrix multiplication operation, including: multiplying each submatrix of the first source matrix with each submatrix of the second source matrix, and adding the corresponding submatrix of the third source matrix in the third vector register to generate a corresponding submatrix of the result matrix to be stored in the corresponding location of the third vector register (e.g., in the same location as the submatrix of the third source matrix).

[0290] In some implementations, the cross product matrix multiplication operation multiplies each submatrix of the first source matrix with each submatrix of the second source matrix to generate the corresponding submatrix of the result matrix, without adding the corresponding submatrix of the third source matrix. For example, some embodiments of the matrix multiplication instruction may not use the third source matrix to store the accumulated data elements and / or the third source matrix may not have received the result from the previous operation (and therefore may be set to all zeros).

[0291] In these implementations, multiplication between each submatrix of the first source matrix and each submatrix of the second source matrix is ​​performed by multiplying the data elements in the rows of each submatrix of the first source matrix with the corresponding data elements in the columns of each submatrix of the second source matrix, thereby generating multiple corresponding products. These multiple products are then added to the corresponding data elements of the corresponding submatrix of the third source matrix to generate the corresponding result data elements of the corresponding submatrix of the result matrix.

[0292] against Figures 1 to 20 The components, features, and details described in any of them can be optionally applied to Figures 21A to 25 Any of the implementations herein. The components, features, and details described for any of the processors and core architectures disclosed herein may optionally be applied to any of the methods disclosed herein, in embodiments where the method may optionally be executed by and / or utilized by such a processor / core. Any of the processors described herein may optionally be included in any of the systems disclosed herein (e.g., Figure 9(in any of the systems). Additionally, in some embodiments, any instruction disclosed herein may optionally have any of the features or details of the instruction format shown herein.

[0293] As mentioned, the matrix multiplication instructions described in this paper (e.g., VPMM-X / VBMM instructions) can utilize matrix multiplication properties (such as those used for performing N...) 3 The N of the computational operation 2 Memory operations quadruple the computation of existing instructions. Modern machine learning implementations require additional floating-point operations (FLOPs) per instruction. To address this need, some implementations described in this paper increase per-instruction FLOPs by extending the features of matrix multiplication instructions to a group of vector registers (register blocks) used for input and output, thus providing additional per-instruction FLOPs in a scalable manner to meet the requirements of modern machine learning applications.

[0294] The multi-register matrix multiplication instructions (e.g., VBMM) described in this paper perform fused multiply-accumulate (FMA) operations using two or more source vector registers (e.g., for storing source matrices A and B) and a source / destination vector register (e.g., for matrix C, which is both the source and destination operand). Some implementations perform 4×4×K matrix multiplications with various precisions and data formats, including floating-point, integer, and micro-scaled data formats. In these implementations, to perform matrix multiplication, an additional level of outer integration block is performed at the register level in both the M-dimensional dimension (the number of rows in matrix C) and the N-dimensional dimension (the number of columns in matrix C).

[0295] Figures 26A-26B The illustrations show five examples (2601-2605) of 512-bit vector register allocations corresponding to different values ​​of N and M, where source matrix A is an M×K matrix, matrix B is a K×N matrix, and matrix C is an M×N matrix. For example, for M=4 and N=4 (shown at 2601), three 512-bit vector registers (zmm0, zmm8, and zmm16) are allocated: a first vector register (zmm8) for storing data elements of matrix A, a second vector register (zmm16) for storing data elements of matrix B, and a third vector register (zmm0) for storing data elements of matrix C. In operation, each row of data elements in the first vector register is multiplied by each column of data elements in the second vector register to generate a corresponding set of products. The products in each set are added to the corresponding data elements of matrix C to generate the corresponding result matrix data elements in the third vector register.

[0296] The matrix multiplication operation and register allocation shown in Example 2601 can be used as Figures 26A-26B and Figure 27 The diagram illustrates the building blocks of a larger matrix multiplication operation. Specifically, to generate a submatrix of matrix C stored in a given vector register in a third plurality of vector registers, corresponding submatrix pairs of matrices A and B, stored respectively in the corresponding first and second source vector registers, are multiplied, as shown in Example 2601. For M=8 and N=16 (shown at 2602) and M=16 and N=8 (shown at 2603), fourteen 512-bit vector registers are allocated—eight for accumulating / resulting matrix (matrix C), two for one source matrix (e.g., matrix A or matrix B), and four for the other source matrix (matrix B or matrix A). In these implementations, each destination vector register zmm0-zmm7 stores a portion (called a submatrix or slice) of matrix C generated by multiplying the submatrixes of the two corresponding source vector registers. For example, when M=8 and N=16, the submatrix of matrix C in zmm0 is generated by multiplying the submatrix of matrix A in zmm8 and the submatrix of matrix B in zmm16. Similarly, when M=16 and N=8, the submatrix in zmm6 is generated by multiplying the submatrix A in zmm11 and the submatrix B in zmm16.

[0297] exist Figure 26B In the examples shown at 2604 and 2605, the data elements of the resulting matrices stored in zmm0 (2604) and zmm0-zmm7 (2605) are 16-bit values ​​(e.g., FP16 values ​​with 5 exponent bits, 10 mantissa bits, and 1 sign bit), meaning that twice the number of data elements can be packed into the zmm register. In the example at 2604, the source matrix in zmm8 is multiplied by the source submatrix in zmm16 to generate the left half of the 16-bit data elements in zmm0, and multiplied by the source submatrix in zmm17 to generate the right half of the data elements in zmm0. Similarly, in 2605, zmm8-zmm11 are each multiplied by zmm16-zmm17 to generate the resulting submatrices in zmm0, zmm2, zmm4, and zmm6, respectively, and multiplied by zmm18-zmm19 to generate the resulting submatrices in zmm1, zmm3, zmm5, and zmm7, respectively.

[0298] exist Figures 26A-26BIn any implementation, any accumulated data elements previously stored in each destination vector register (e.g., results from previous matrix operations) may optionally be added to the product of the two corresponding submatrices in the two source vector registers to generate a result submatrix stored in the destination vector register, where the destination vector register is both a source vector register and a destination vector register. In some implementations, a “late accumulation” is performed after each pair of source matrix multiplications to expose only the delay in storing the final sum of the accumulated values ​​and the final result matrix in each source / destination vector register.

[0299] refer to Figure 27 Some embodiments include an accumulator register array 2700, which is configured to store accumulated data elements of a matrix / submatrix (e.g., a submatrix of matrix C) for improved performance. These registers are prefixed with... zaa Identify them to associate them with the data elements of the storage source matrix. zmm The vector registers are distinct. One or more are stored in the accumulator register array 2700. zaa The C matrix in the registers, as well as the data elements of the A matrix in vector register 2701 and the B matrix in vector register 2702, can be encoded as matrix slices / submatrices comprising groups of vector registers. These implementations provide additional reuse of data elements from the A and B matrices (at low bandwidth), thereby utilizing every N... 3 N computational operations 2 The matrix multiplication property of memory operations can be used to increase the number of FLOPs.

[0300] Therefore, each zmm register in register block A is multiplied by each zmm register matrix in register block B, and the result is optionally added to the corresponding zmm or zaa register in register block C. The resulting data element is stored back into the zmm or zaa register in register block C. Register numbers can be rounded down to the nearest multiple of the block size (e.g., if the operand is 4 (zmm...)). n If the register block is +3, then zmm6 will be rounded down to 4 and the register blocks for zmm4, zmm5, zmm6, and zmm7 will be specified. Note the case where M=8, N=16 ( Figure 26A The zigzag register sorting of the matrix C vector register in 2602).

[0301] In these implementations, different opcodes can be used to indicate different instruction variants, such as matrix multiplication instructions with different numbers of vector registers, different data element sizes and formats, and different matrix dimensions. Alternatively or additionally, the same opcode can be used, and different instruction variants can be indicated in one or more instruction fields (e.g., fields indicating one of multiple M×N×K combinations, source / destination vector registers, immediate values, and / or data formats). As used herein, a group of vector registers allocated to store the data elements of a matrix is ​​called a vector register block. For example, the result matrix in 2602 comprises a block of eight zmm registers: zmm0-zmm7.

[0302] In addition to the cross product matrix multiplication instructions described herein, some implementations include support instructions such as those for copying an accumulator register block (e.g., stored in the zaa accumulator register) to a vector register (e.g., zmm0, zmm1, etc.) and copying an accumulator register block (e.g., stored in the zaa accumulator register) from a vector register (e.g., zmm0, zmm1, etc.) and / or for moving a matrix / submatrix between different zmm registers and between the cache / memory subsystem and the zmm register.

[0303] Return to reference Figure 23 One or both of the decoder 2310 and the register renaming / allocation / scheduling circuit 2320 allocate blocks of vector registers and potentially allocate accumulator registers based on the opcode and / or fields of each matrix multiplication instruction 2305. For example, for Figure 26A In the VBMM.RC.MxKxN.E8M23.E8M7.E8M7 zmm0, zmm8, zmm16, 0x0 configuration, three vector registers (zmm0, zmm8, zmm16) are allocated for M=4 and N=4, while fourteen vector registers are allocated for M=8 and N=16 and M=16 and N=8, respectively. As another example, for... Figure 27 In the VBMM.RC16x8x16.E8M23.E8M7.E8M7 zaa0, zmm8, zmm16, the sixteen zaa0 registers in the corresponding accumulator register array 2700 are allocated to store the sixteen corresponding submatrices of matrix C, and four vector registers are allocated for the submatrices of each of matrices A and B.

[0304] In some implementations, instructions that perform operations on matrices with floating-point data elements (e.g., on source matrices with TF32, FP16, BF16, FP8 (BF8 / HF8), or INT8 data elements) generate output matrices with FP32 data elements (or, in some implementations, FP16 or BF16 data elements). Similarly, instructions that perform operations on matrices with integer data elements (e.g., signed / unsigned INT8) generate output matrices with INT32 data elements.

[0305] In some implementations, multi-register matrix multiplication instructions (e.g., VBMM instructions) operate based on the following opcodes and fields: VBMM.Layout.Dimension.CType. AType.BType zmm1[+i], zmm2[+j], zmm3[+k] [, imm] (VBMM.Layout.Dimension.C-type.A-type.B-type zmm1[+i], zmm2[+j], zmm3[+k][, imm]) Here, Layout indicates whether matrix A or B is a row-major or column-major matrix; Dimension indicates the size of the matrix multiplication; CType, AType, and BType describe the data types of the operands of matrices C, A, and B, respectively (e.g., using EiMj format, where i is the number of bits in the exponent and j is the number of bits in the mantissa). In some implementations, zmm operands represent single registers, such as... Figure 26A zmm0, zmm8, and zmm16 at position 2601, or a vector register block comprising multiple vector registers allocated for storing a given source matrix and / or result matrix across multiple vector registers (e.g., Figure 26A Figure 2602 shows zmm8-zmm9 for storing source matrix A, zmm16-zmm19 for storing source matrix B, and zmm0-zmm7 for storing the accumulation / result matrix. This can be represented by the notation zmm when multiple registers are used to store matrix data elements. n+i This is used to indicate. Depending on the size of the register block, i can be 1, 3, 7, or 15.

[0306] In some implementations, the immediate number field imm Used to provide information related to the Micro-Scaling Floating-Point (MXFP) data format. Supported MXFP formats may include, but are not limited to, MXFP8 (8-bit floating-point), MXFP6 (6-bit floating-point), MXFP4 (4-bit floating-point), and / or MXINT8 (8-bit integer).

[0307] Matrix layout components can include two letters, one for the layout of matrix A and one for the layout of matrix B, as indicated in Table 1 below. In this implementation, R indicates row-first layout and C indicates column-first layout. Row / column (RC) layout is supported for all dimensions and types. Depending on the dimension and type, other layouts besides RC may also be supported. Table 4: Data Layout

[0308] The dimension components in the mnemonic consist of M×K×N values ​​used for matrix multiplication. Some implementations of the matrix multiplication instructions support five different M×N combinations: 4×4, 8×16, and 16×8 for 32-bit result types, and 4×8 and 16×16 for 16-bit result types. K will be 8, 16, or 32, depending on the types of matrices A and B. Various M×K×N combinations can be supported, including but not limited to those described herein.

[0309] In some of the examples provided in this article, E can be used. i M j The format indicates the data type of the data elements in matrices A, B, and C, where i It is the number of bits in the exponent, and j This refers to the number of bits in the mantissa. Types A and B can be: floating-point data types, such as E5M10 (FP16), E8M7 (BF16), E5M2 (BF8), E4M3 (HF8), etc.; integer data types, such as E0M7 (signed INT8), E0M8 (unsigned INT8); or micro-scaling (MX) data types, including 8-bit, 6-bit, and 4-bit data formats for smaller memory footprints. MX format data consists of blocks of 32 elements, all sharing an 8-bit scaling factor. All bits in the scaling factor can be considered as the bias exponent (E8M0). The micro-scaling integer data type MXINT8 can be indicated by Q1F6.

[0310] Table 5 illustrates valid combinations of types A and B. At the bottom of the table, if the data type is supported as a regular data type and / or as a data type with microscaling, the data type is marked with non-MXPF and / or MXFP. Table 5: Support for all precision combinations of multi-register VBMM instructions.

[0311] Refer again Figure 27In addition to or in place of vector registers, multi-register matrix multiplication instructions (e.g., VBMM instructions) may refer to accumulator registers. Specifically, a dedicated zaa register array (such as array 2700) stores the accumulated results of iterations of the matrix multiplication instructions described herein. In some implementations, 32 1-kilobyte zaa arrays (zaa0-zaa31) are provided, each holding the equivalent of 16 zaamm registers of storage. In these implementations, matrix multiplication instructions (such as VBMM instructions) can use the zaa accumulator array 2700 to accumulate the results of successive multiplication-accumulation operations. As described herein, each accumulated data element in the accumulator register array 2700 can be added to the corresponding set of products generated by multiplying the data elements of matrix A with the corresponding data elements of matrix B.

[0312] Some implementations also include move / copy instructions for transferring data between the zaa register array and the zmm register block. The architectural state of the zaa registers can be created using the 8KB of storage added by the AMX extension. 16×K×16 VBMM instructions with 32-bit results can use the zaa accumulator array as a source / destination register (e.g., for storing source / destination C matrices).

[0313] As mentioned above, some implementations utilize the MX data format with MXFP scaling factors. These may require more space, as in the case of basic building block VPMM-X instructions. Some implementations of matrix multiplication instructions (e.g., VBMM-X) that operate on MXFP operands store the corresponding block scaling factors in operation mask registers (e.g., K0, K3, etc.). Alternatively or additionally, the block scaling factors can be stored in one or more zmm registers (e.g., zmm0). Implementations using operation mask registers support both zmm and zaa destinations as described herein. In some implementations, for example, each 64-bit operation mask register can hold eight block scaling factors that are applied to the zmm. n The elements in the +1 register block are eight rows / columns. An operation mask register block with two consecutive operation mask registers numbered even / odd can be used for zmm. n +3 register block.

[0314] Figure 28This is a block diagram of an embodiment of a processor or processor core operable to execute an implementation of a multi-register matrix multiplication instruction 2305 for processing matrix data elements 2890 as described herein. Instruction fetch circuitry 2809 fetches the multi-register matrix multiplication instruction 2305 from memory / cache subsystem 2803 (e.g., from system memory, L1 instruction cache, etc.). The multi-register matrix multiplication instruction 2305 may be a macro instruction, machine code instruction, or other instruction from the processor's instruction set. The multi-register matrix multiplication instruction 2305 may have various formats or encodings, such as, for example, regarding... Figure 11B The encoding may include one or more fields for an opcode that at least partially or completely specifies the operation to be performed (e.g., matrix multiplication using a specified number of source and destination vector registers and / or accumulation vector registers), and one or more fields for one or more operation objects (such as operation objects that can be used to identify source / destination registers and memory locations).

[0315] Decoder circuitry 2310 (e.g., instruction decoder) may be coupled to receive each multi-register matrix multiplication instruction 2305 obtained from cache / memory subsystem 2803 and decode it into one or more lower-level control signals, operations, or decoded instructions (e.g., one or more microinstructions, microoperations, microcode entry points, etc.).

[0316] In some examples, the register renaming, allocation, and / or scheduling circuitry 2320 may provide functionality for one or more of the following: (1) renaming logical operand values ​​to physical operand values ​​(e.g., register alias tables in some examples); (2) assigning status bits and flags to decoded instructions; and (3) scheduling decoded instructions from the instruction pool for execution by the execution circuitry (e.g., using reserved stations in some examples).

[0317] A separate block of vector register 2850 may be allocated by register renaming / allocation / scheduling circuitry 2320 and / or decoder circuitry 2310 for storing matrix data elements of source matrix A, source matrix B, and source / destination matrix C, as described herein. Certain instructions may also use a set of registers allocated from accumulator register array 2852 to store accumulated matrix data elements and result matrix data elements (e.g., matrix C).

[0318] The execution circuit 2330 may be coupled to the decoder circuit 2310, the register renaming / allocation / scheduler circuit 2320, the vector register 2850, and the memory / cache subsystem 2803, and includes tensor processing circuitry 2335 to perform the multi-register matrix multiplication operations described herein.

[0319] Figure 29The illustration shows an example set of VBMM instructions with BF16 source matrices and FP32 accumulator / result matrices according to some implementations. Example instructions for moving submatrices between vector register blocks, the accumulator register array (zaa), and memory are also illustrated.

[0320] As mentioned, VBMM directives can be defined as follows: VBMM.Layout.Dimension.CType.AT ype.BType zmm1[+i], zmm2[+j], zmm3[+k][, imm] Here, Layout indicates whether matrix A or B is row-major (R) or column-major (C); Dimension indicates the size of the matrix multiplication (e.g., 16×8×16 for the first two listed instructions); CType, AType, and BType describe the operand types of matrices C, A, and B respectively (e.g., using the EiMj format, where i is the number of bits in the exponent and j is the number of bits in the mantissa). zmm1, zmm2, and zmm3 indicate a single vector register or a block of vector registers (e.g., storing a submatrix of the A / B source matrix). When multiple registers are needed to hold matrix data, the symbol zmm is used. n +i. Depending on the size of the register block, i will be 1, 3, 7, or 15. For example, to implement Figure 26A Register allocation and operands in 2602 zmm1[+7] zmm0-zmm7 can be allocated for storing matrix C, and the objects to be manipulated can be... zmm2[+1] zmm8 and zmm9 can be assigned to store the source matrix A, and the objects to be manipulated can be... zmm3[+3] zmm16-zmm19 can be allocated to store source matrix B.

[0321] In some implementations, the immediate operand (imm) of the matrix multiplication instruction indicates whether the operand data type is an MXFP data type with a block scaling factor, and which operation mask registers (or vector registers) contain the corresponding block scaling factor. Assume... imm [0:0] is set and imm [3:1] Contains value i Then K i The least significant byte in the array is the block scaling factor for row 0 of A, the next byte is the block scaling factor for row 1, and so on until the block scaling factor for row 7 in the most significant byte. If matrix A is maintained with 16 rows, the block scaling factors for rows 8 to 15 are in the operation mask register K. i +1. For operands with 16 rows or columns, the specified index is reduced to the nearest even number and indicates the start of the operand mask register block of two consecutive operand mask registers.

[0322] In some implementations, if AType or BType is not an MXFP type, then even if it is set... imm[0:0] The block scaling factor of the operand is also ignored. Contrary to the normal use of the operand mask register, where EVEX.aaa==0 means that masking should not be applied, K0 It can be used to maintain the block scaling factor.

[0323] In some implementations, the zmm register is allocated to store the block scaling factor, and the Zaa destination register from the Zaa accumulator register array 2852 is used to accumulate / result matrices (e.g., matrix C). Loop kernels operating on the destination of a 4×4 Zaa register array will not have enough operation mask registers to maintain the block scaling factor for the entire A matrix and the 16 columns of matrix B. For these kernels, using the zmm register to maintain the block scaling factor is more efficient.

[0324] The zmm register can store 64 8-bit block scaling factors, 16 of which can be provided via each 128-bit channel. The least significant byte in the channel contains the block scaling factor for row / column 0, the next byte for row / column 1, and so on. In this implementation, the immediate value (imm) encodes the zmm register block holding the block scaling factor and the 128-bit channel used for A. EVEX.aaa indicates which 128-bit channel will be used for matrix B. The zmm register indices in imm[7:3] can be rounded down to the nearest even index. In the zmm register block, one or more even zmm registers hold the block scaling factor for matrix A, and one or more odd zmm registers hold the block scaling factor for matrix B. The 128-bit channel used by the instruction is encoded in imm[2:1] (A) and EVEX.aaa[1:0] (B).

[0325] In some implementations, the VZAAMOVBLOCK instruction is executed to move a matrix or matrix slice between an accumulator register block, a regular vector register / register block, and a memory location. The VZAAZERO instruction is configured to initialize the corresponding matrix value to all zeros.

[0326] Figure 30 The illustration shows a method for performing multi-register cross product matrix multiplication according to some implementations. This method can be implemented on various architectures described herein, but is not limited to any particular processor or system architecture.

[0327] At 3001, the multi-register cross product matrix multiplication instruction is fetched from the cache / memory subsystem (e.g., from the L1 instruction cache in some implementations).

[0328] At 3002, the multi-register cross product matrix multiplication instruction is decoded, which includes fields indicating the following: a first plurality of submatrices of a first source matrix to be stored in the corresponding first plurality of vector registers, a second plurality of submatrices of a second source matrix to be stored in the corresponding second plurality of vector registers, and a third plurality of submatrices of the result / accumulated matrix to be stored in the corresponding third plurality of vector registers.

[0329] At position 3003, the first plurality of submatrices are loaded into the corresponding first plurality of vector registers, and the second plurality of submatrices are loaded into the corresponding second plurality of vector registers.

[0330] At address 3004, a multi-register cross product matrix multiplication instruction is executed. This multiplies the submatrices in the corresponding vector registers of the first plurality of vector registers with the corresponding submatrices in the corresponding vector registers of the second plurality of vector registers, generating each submatrix in the third plurality of vector registers. Optionally, the accumulated submatrices from the corresponding vector registers of the third plurality of vector registers can be added to the result of the multiplication to generate the corresponding result submatrix.

[0331] At position 3005, submit the third or more submatrices of the result / accumulated matrix.

[0332] References to "an example," "example," etc., indicate that the described example may include a particular feature, structure, or characteristic, but not every example may necessarily include that particular feature, structure, or characteristic. Furthermore, such phrases do not necessarily refer to the same example. Moreover, when a particular feature, structure, or characteristic is described in conjunction with an example, it is assumed that the influence of such feature, structure, or characteristic on other examples, whether explicitly described or not, is within the knowledge of a person skilled in the art.

[0333] The processor components disclosed herein may be described as and / or claimed to be operable, capable of operating, able to be configured, adapted, or otherwise perform operations. For example, a decoder may be described as and / or claimed to be used for decoding instructions, an execution unit may be described as and / or claimed to be used for storing results, and so on. As used herein, these expressions refer to the characteristics, properties, or attributes of the components in a power-off state, and do not imply that these components or the devices or apparatuses in which these components are included are currently powered on or operating. For clarity, it should be understood that the processors and devices claimed herein are not required to be powered on or operating.

[0334] The terms “coupled” and / or “connected” and their derivatives may be used in the specification and claims. These terms are not intended to be synonyms with each other. Rather, in various embodiments, “connected” can be used to indicate that two or more elements are in direct physical and / or electrical contact with each other. “Coupled” can mean that two or more elements are in direct physical and / or electrical contact with each other. However, “coupled” can also mean that two or more elements are not in direct contact with each other, but still cooperate or act on each other. For example, an execution unit may be coupled to a register and / or decoding unit via one or more intermediate components. In the drawings, arrows are used to illustrate connection and coupling.

[0335] Some embodiments include an article of manufacture (e.g., a computer program product) that includes a machine-readable medium. The medium may include mechanisms for providing (e.g., storing) information in a machine-readable form. The machine-readable medium may provide or store instructions or sequences of instructions that, if executed by a machine, are operable to cause the machine to perform and / or induce the machine to perform one or more operations, methods, or techniques disclosed herein.

[0336] In some embodiments, a machine-readable medium may include tangible and / or non-transitory machine-readable storage media. For example, a non-transitory machine-readable storage medium may include floppy disks, optical storage media, optical disks, optical data storage devices, CD-ROMs, magnetic disks, magneto-optical disks, read-only memory (ROM), programmable ROM (PROM), erasable-and-programmable ROM (EPROM), electrically erasable-and-programmable ROM (EEPROM), random access memory (RAM), static RAM (SRAM), dynamic RAM (DRAM), flash memory, phase-change memory, phase-change data storage materials, non-volatile memory, non-volatile data storage devices, non-transitory memory, or non-transitory data storage devices, etc. A non-transitory machine-readable storage medium does not consist of transient propagating signals. In some embodiments, the storage medium may include tangible media, which include solid substances or materials such as, for example, semiconductor materials, phase-change materials, magnetic solid materials, solid data storage materials, etc. Alternatively, non-tangible, transient, computer-readable transmission media may be used, such as, for example, electrical, optical, acoustic, or other forms of propagation signals—such as carrier waves, infrared signals, and digital signals.

[0337] Examples of suitable machines include, but are not limited to, general-purpose processors, special-purpose processors, digital logic circuits, integrated circuits, etc. Other examples of suitable machines include computer systems or other electronic devices that contain processors, digital logic circuits, or integrated circuits. Examples of such computer systems or electronic devices include, but are not limited to, desktop computers, laptop computers, notebook computers, tablet computers, netbooks, smartphones, cellular phones, servers, network equipment (e.g., routers and switches), mobile internet devices (MIDs), media players, smart TVs, internet access devices, set-top boxes, and video game controllers.

[0338] Moreover, in the examples described above, unless otherwise specifically stated, delimited language such as the phrase “at least one of A, B or C” or “A, B and / or C” is intended to be understood as referring to A, B, or C, or any combination thereof (i.e., A and B, A and C, B and C, and A, B and C).

[0339] In the foregoing description, specific details have been set forth to provide a thorough understanding of the embodiments. However, other embodiments may be practiced without some of these specific details. Various modifications and changes may be made to this disclosure without departing from the broader spirit and scope of the disclosure as set forth in the claims. Therefore, the specification and drawings should be considered illustrative rather than restrictive. The scope of the invention is not intended to be determined by the specific examples provided above, but only by the appended claims. In other instances, well-known circuits, structures, devices, and operations have been shown in block diagram form and / or without detail to avoid obscuring the understanding of the specification.

[0340] Embodiments of the present invention may include the steps already described above. These steps may be embodied as machine-executable instructions that can be used to cause a general-purpose or special-purpose processor to perform these steps. Alternatively, these steps may be performed by a specific hardware component containing hard-wired logic for performing these steps, or by any combination of programmed computer components and custom hardware components.

[0341] Example

[0342] The following are example implementations of different embodiments of the present invention.

[0343] Example 1. A processor includes: a first plurality of vector registers for storing a first plurality of submatrices corresponding to a first source matrix; a second plurality of vector registers for storing a second plurality of submatrices corresponding to a second source matrix; a third plurality of vector registers for storing a third plurality of submatrices corresponding to a result matrix; a decoder for decoding an instance of a multi-register cross product matrix multiplication instruction having a field for indicating the first source matrix, the second source matrix, and the result matrix, and an operand field for indicating that a multi-register cross product matrix multiplication is to be performed using the first plurality of submatrices and the second plurality of submatrices to generate a third plurality of submatrices; and execution circuitry for executing the multi-register cross product matrix multiplication instruction, the execution circuitry being configured to generate each submatrix in the third plurality of vector registers by multiplying the submatrices in the corresponding vector registers of the first plurality of vector registers with the submatrices in the corresponding vector registers of the second plurality of vector registers.

[0344] Example 2. According to the processor of Example 1, the execution circuit is used to add the corresponding submatrix in the corresponding submatrix in the corresponding vector register stored in the third plurality of vector registers to generate the submatrix in the third plurality of submatrix in the corresponding vector register.

[0345] Example 3. The processor according to Example 1 or 2 further includes: a plurality of accumulator register arrays for storing accumulator submatrices of matrix multiplication instructions, wherein a third plurality of vector registers are allocated from the corresponding accumulator register arrays to store a fourth plurality of submatrices and a third plurality of submatrices.

[0346] Example 4. A processor according to any of Examples 1-3, wherein the decoder is used to decode one or more register block move instructions, and the execution circuitry is used to execute one or more register block move instructions to move a submatrix between a vector register in the corresponding accumulator array and a corresponding vector register not in the accumulator array.

[0347] Example 5. A processor according to any of Examples 1-4, wherein an instance of a multi-register cross product matrix multiplication instruction is used to include one or more fields for indicating the matrix dimensions of a first source matrix, a second source matrix, and a result matrix.

[0348] Example 6. A processor according to any of Examples 1-5, wherein one or more fields are used to indicate the matrix dimensions as M×K dimensions for the first source matrix, K×N dimensions for the second source matrix, and M×K×N values ​​for the M×N dimensions of the resulting matrix, where M = 4, 8, or 16, and N is a positive integer.

[0349] Example 7. A processor according to any of Examples 1-6, wherein an instance of a multi-register cross product matrix multiplication instruction is used to include one or more fields for indicating the data type of each of the first source matrix, the second source matrix, and the result matrix.

[0350] Example 8. A processor according to any of Examples 1-7, wherein the data types of the first source matrix and the second source matrix include one or more of the following: 16-bit floating-point (FP16), Bfloat16 (BF16), unsigned 8-bit integer, signed 8-bit integer, unsigned INT8, and 8-bit, 6-bit, and 4-bit micro-scaled floating-point (MXFP).

[0351] Example 9. For a processor according to any of Examples 1-8, where the data type of the first or second source matrix includes 8-bit, 6-bit, or 4-bit MXFP, one or more fields of an instance of a multi-register cross product matrix multiplication instruction are used to indicate one or more block scaling factors.

[0352] Example 10. A processor according to any one of Examples 1-9 further includes: one or more operation mask registers or one or more vector registers for storing one or more corresponding block scaling factors, as indicated by one or more fields of an instance of a multi-register cross product matrix multiplication instruction.

[0353] Example 11. A processor according to any of Examples 1-10, wherein one or more corresponding block scaling factors comprise a complex number of block scaling factors, each of the complex number of block scaling factors corresponding to a different row or column of a data element in a first source matrix or a second source matrix.

[0354] Example 12. A processor according to any one of Examples 1-11, wherein one or more fields of an instance of a multi-register cross product matrix multiplication instruction are used to indicate a first plurality of block scaling factors stored in a first or more vector registers, the processor further comprising: a control register for storing an indication of a second plurality of block scaling factors stored in a second or more vector registers.

[0355] Example 13. A method comprising: loading a first plurality of submatrices of a first source matrix into corresponding first plurality of vector registers; loading a second plurality of submatrices of a second source matrix into a second plurality of vector registers; decoding an instance of a multi-register cross product matrix multiplication instruction having fields for indicating the first source matrix, the second source matrix, and the result matrix, the multi-register cross product matrix multiplication instruction having operand fields for indicating that a multi-register cross product matrix multiplication is to be performed using the first plurality of submatrices and the second plurality of submatrices to generate a third plurality of submatrices; and executing the multi-register cross product matrix multiplication instruction; generating each of the third plurality of submatrices of the result matrix in the corresponding vector registers of the third plurality of vector registers by multiplying the submatrices in the corresponding vector registers of the first plurality of vector registers with the submatrices in the corresponding vector registers of the second plurality of vector registers.

[0356] Example 14. The method of Example 13 further includes: generating a submatrix in the third plurality of submatrixes in the corresponding vector register by adding the corresponding submatrix in the corresponding vector register stored in the third plurality of vector registers through execution circuitry.

[0357] Example 15. According to the method of Example 13 or 14, a third plurality of vector registers are allocated from the corresponding accumulator register array to store a fourth plurality of submatrices and a third plurality of submatrices.

[0358] Example 16. The method according to any one of Examples 13-15 further includes: decoding and executing one or more register block move instructions to move a submatrix between a vector register in the corresponding accumulator array and a corresponding vector register not in the accumulator array.

[0359] Example 17. According to any of Examples 13-16, an instance of a multi-register cross product matrix multiplication instruction is used to include one or more fields for indicating the matrix dimensions of the first source matrix, the second source matrix, and the result matrix.

[0360] Example 18. According to the method of any of Examples 13-17, one or more fields are used to indicate the matrix dimensions as M×K dimensions for the first source matrix, K×N dimensions for the second source matrix, and M×K×N values ​​for the M×N dimensions of the result matrix, where M=4, 8, or 16, and N is a positive integer.

[0361] Example 19. According to any of Examples 13-18, an instance of a multi-register cross product matrix multiplication instruction is used to include one or more fields for indicating the data type of each of the first source matrix, the second source matrix, and the result matrix.

[0362] Example 20. A machine-readable medium having program code stored thereon, which, when executed by a machine, causes the machine to perform operations including: loading a first plurality of submatrices of a first source matrix into corresponding first plurality of vector registers; loading a second plurality of submatrices of a second source matrix into a second plurality of vector registers; decoding an instance of a multi-register cross product matrix multiplication instruction having fields for indicating the first source matrix, the second source matrix, and the result matrix, the multi-register cross product matrix multiplication instruction having fields for indicating the operands to be used to perform multi-register cross product matrix multiplication with the first plurality of submatrices and the second plurality of submatrices to generate a third plurality of submatrices; and executing the multi-register cross product matrix multiplication instruction; generating each of the third plurality of submatrices of the result matrix in the corresponding vector registers of the third plurality of vector registers by multiplying the submatrices in the corresponding vector registers of the first plurality of vector registers with the submatrices in the corresponding vector registers of the second plurality of vector registers.

[0363] As described herein, instructions can refer to a specific configuration of hardware such as an application-specific integrated circuit (ASIC) configured to perform certain operations or have a predetermined function, or software instructions stored in memory embodied in a non-transitory computer-readable medium. Thus, the techniques illustrated in the figures can be implemented using code and data stored on and executed on one or more electronic devices (e.g., terminal stations, network elements, etc.). Such electronic devices use computer-machine-readable media (internal and / or via a network with other electronic devices) to store and transmit code and data, such as non-transitory computer-machine-readable storage media (e.g., disks; optical disks; random access memory; read-only memory; flash memory devices; phase-change memory) and transient computer-machine-readable communication media (e.g., electrical, optical, acoustic, or other forms of propagation signals—such as carrier waves, infrared signals, digital signals, etc.). Additionally, such electronic devices typically include a collection of one or more processors coupled to one or more other components, such as one or more storage devices (non-transitory machine-readable storage media), user input / output devices (e.g., keyboards, touchscreens, and / or displays), and network connections. The coupling of the processor set with other components is typically via one or more buses and bridges (also referred to as bus controllers). Storage devices and signals carrying network traffic represent one or more machine-readable storage media and machine-readable communication media, respectively. Thus, the storage devices of a given electronic device typically store code and / or data for execution on one or more processor sets of that electronic device. Of course, one or more portions of embodiments of the invention can be implemented using different combinations of software, firmware, and / or hardware. Throughout this detailed description, numerous specific details have been set forth for illustrative purposes in order to provide a thorough understanding of the invention. However, it will be apparent to those skilled in the art that the invention can be practiced without some of these specific details. In some instances, well-known structures and functions have not been described in detail to avoid obscuring the subject matter of the invention. Accordingly, the scope and spirit of the invention should be determined based on the appended claims.

Claims

1. A processor, comprising: The first plurality of vector registers are used to store the first plurality of sub-matrices corresponding to the first source matrix; The second plurality of vector registers are used to store the second plurality of sub-matrices corresponding to the second source matrix; The third and more vector registers are used to store the corresponding third and more sub-matrices of the result matrix; A decoder is used to decode an instance of a multi-register cross product matrix multiplication instruction having a field for indicating a first source matrix, a second source matrix, and a result matrix, and an operation object field for indicating that a multi-register cross product matrix multiplication is to be performed using the first plurality of submatrices and the second plurality of submatrices to generate the third plurality of submatrices; as well as An execution circuit is configured to execute the multi-register cross product matrix multiplication instruction, wherein the execution circuit is configured to generate each of the third plurality of submatrices in the corresponding vector register of the third plurality of vector registers by multiplying the submatrices in the corresponding vector registers of the first plurality of vector registers with the submatrices in the corresponding vector registers of the second plurality of vector registers.

2. The processor according to claim 1, wherein, The execution circuit is used to add the corresponding submatrix in the corresponding submatrix in the corresponding vector register stored in the third plurality of vector registers, so as to generate the submatrix in the third plurality of submatrix in the corresponding vector register.

3. The processor according to claim 1, further comprising: Multiple accumulator register arrays are used to store accumulator submatrices for matrix multiplication instructions, wherein the third plurality of vector registers are allocated from the corresponding accumulator register arrays to store the fourth plurality of submatrices and the third plurality of submatrices.

4. The processor according to claim 3, wherein, The decoder is used to decode one or more register block move instructions, and the execution circuit is used to execute the one or more register block move instructions to move a submatrix between a vector register in the corresponding accumulator array and a corresponding vector register not in the accumulator array.

5. The processor according to any one of claims 1 to 4, wherein, The instance of the multi-register cross product matrix multiplication instruction is used to include one or more fields for indicating the matrix dimensions of the first source matrix, the second source matrix, and the result matrix.

6. The processor according to claim 5, wherein, The one or more fields are used to indicate the matrix dimensions as M×K dimensions for the first source matrix, K×N dimensions for the second source matrix, and M×N dimensions for the result matrix, where M = 4, 8, or 16, and N is a positive integer.

7. The processor according to any one of claims 1 to 6, wherein, The instance of the multi-register cross product matrix multiplication instruction is used to include one or more fields indicating the data type of each of the first source matrix, the second source matrix, and the result matrix.

8. The processor according to claim 7, wherein, The data types of the first source matrix and the second source matrix include one or more of the following: 16-bit floating-point FP16, Bfloat16 BF16, unsigned 8-bit integer, signed 8-bit integer, unsigned INT8, and 8-bit, 6-bit and 4-bit micro-scaled floating-point MXFP.

9. The processor according to claim 8, wherein, When the data type of the first source matrix or the second source matrix includes 8-bit, 6-bit, or 4-bit MXFP, one or more fields of the instance of the multi-register cross product matrix multiplication instruction are used to indicate one or more block scaling factors.

10. The processor of claim 9, further comprising: One or more operation mask registers or one or more vector registers, as indicated by one or more fields of the instance of the multi-register cross product matrix multiplication instruction, to store the one or more corresponding block scaling factors.

11. The processor according to claim 9 or 10, wherein, The one or more corresponding block scaling factors include a plurality of block scaling factors, each of which corresponds to a different row or column of a data element in the first source matrix or the second source matrix.

12. The processor according to any one of claims 9 to 11, wherein, The one or more fields of the instance of the multi-register cross product matrix multiplication instruction are used to indicate a first plurality of block scaling factors stored in a first or more vector registers, and the processor further includes: A control register is used to store indications of a second plurality of block scaling factors stored in a second or more vector registers.

13. A method comprising: Load the first and a dozen sub-matrices of the first source matrix into the corresponding first and a dozen vector registers; Load the second and more sub-matrices of the second source matrix into the second and more vector registers; Decoding an instance of a multi-register cross product matrix multiplication instruction, the multi-register cross product matrix multiplication instruction having a field for indicating the first source matrix, the second source matrix, and the result matrix, the multi-register cross product matrix multiplication instruction having an operation object field for indicating that multi-register cross product matrix multiplication is to be performed using the first plurality of sub-matrices and the second plurality of sub-matrices to generate the third plurality of sub-matrices; as well as Execute the multi-register cross product matrix multiplication instruction; Each submatrix in the third plurality of submatrixes of the result matrix is ​​generated in the corresponding vector register of the third plurality of vector registers by multiplying the submatrix in the corresponding vector register of the first plurality of vector registers with the submatrix in the corresponding vector register of the second plurality of vector registers.

14. The method of claim 13, further comprising: The execution circuit adds the corresponding submatrix in the fourth submatrix stored in the third plurality of vector registers to generate the submatrix in the third plurality of submatrix in the corresponding vector register.

15. The method according to claim 13, wherein, The third plurality of vector registers are allocated from the corresponding accumulator register array to store the fourth plurality of submatrices and the third plurality of submatrices.

16. The method of claim 15, further comprising: Decode and execute one or more register block move instructions to move a submatrix between a vector register in the corresponding accumulator array and a corresponding vector register not in the accumulator array.

17. The method according to any one of claims 13 to 16, wherein, The instance of the multi-register cross product matrix multiplication instruction is used to include one or more fields for indicating the matrix dimensions of the first source matrix, the second source matrix, and the result matrix.

18. The method according to claim 17, wherein, The one or more fields are used to indicate the matrix dimensions as M×K dimensions for the first source matrix, K×N dimensions for the second source matrix, and M×N dimensions for the result matrix, where M = 4, 8, or 16, and N is a positive integer.

19. The method according to any one of claims 13 to 18, wherein, The instance of the multi-register cross product matrix multiplication instruction is used to include one or more fields indicating the data type of each of the first source matrix, the second source matrix, and the result matrix.

20. A machine-readable medium having program code stored thereon, the program code, when executed by a machine, causing the machine to perform operations, the operations including: Load the first and a dozen sub-matrices of the first source matrix into the corresponding first and a dozen vector registers; Load the second and more sub-matrices of the second source matrix into the second and more vector registers; Decoding an instance of a multi-register cross product matrix multiplication instruction, the multi-register cross product matrix multiplication instruction having a field for indicating the first source matrix, the second source matrix, and the result matrix, the multi-register cross product matrix multiplication instruction having an operation object field for indicating that multi-register cross product matrix multiplication is to be performed using the first plurality of sub-matrices and the second plurality of sub-matrices to generate the third plurality of sub-matrices; as well as Execute the multi-register cross product matrix multiplication instruction; Each submatrix in the third plurality of submatrixes of the result matrix is ​​generated in the corresponding vector register of the third plurality of vector registers by multiplying the submatrix in the corresponding vector register of the first plurality of vector registers with the submatrix in the corresponding vector register of the second plurality of vector registers.

Images