Systems, methods, and apparatus for slice matrix multiplication and accumulation

Abstract

The invention discloses a system, method and apparatus for slice matrix multiplication and accumulation. In particular, matrix (slice) multiply accumulations and negative version matrix (slice) multiply accumulations are discussed. For example, in some embodiments, decode circuitry is detailed to decode an instruction having fields for an opcode, an identifier of a first source matrix operand, an identifier of a second source matrix operand, and an identifier of a source / destination matrix operand; and execution circuitry to execute the decoded instruction to: multiply the identified first source matrix operand by the identified second source matrix operand; adding a result of the multiplication to the identified source / destination matrix operand; and storing a result of the addition in the identified source / destination matrix operand and zeroing an unconfigured column of the identified source / destination matrix operand.

Description

This application is a divisional application of the invention patent application with PCT international application number PCT / US2017 / 040548, international application date July 1, 2017, application number 201780086978.6 entering the Chinese national phase, and entitled "System, method and device for slice matrix multiplication and accumulation". Technical Field

[0001] The field of the invention relates generally to computer processor architecture and more particularly to matrix manipulation. Background Art

[0002] Matrices are becoming increasingly important in many computing tasks such as machine learning and other batch data processing. BRIEF DESCRIPTION OF THE DRAWINGS

[0003] The present invention is illustrated by way of example and not by way of limitation in the accompanying figures, in which like reference numerals indicate similar elements, and in which:

[0004] Figure 1 An embodiment of a configured tile is shown;

[0005] Figure 2 Several examples of illustrated matrix storage are shown;

[0006] Figure 3 An embodiment of a system for operating an accelerator using a matrix (slice) is illustrated;

[0007] Figure 4 and Figure 5 Different embodiments showing how to use a matrix operation accelerator to share memory;

[0008] Figure 6 An embodiment of a matrix multiply-accumulate operation ("TMMA") using a tile is illustrated;

[0009] Figure 7 An embodiment illustrating a subset of execution of an iteration of a chained fused multiply-accumulate instruction;

[0010] Figure 8 An embodiment illustrating a subset of execution of an iteration of a chained fused multiply-accumulate instruction;

[0011] Fig. 9 An embodiment illustrating a subset of execution of an iteration of a chained fused multiply-accumulate instruction;

[0012] Fig.10 An embodiment illustrating a subset of execution of an iteration of a chained fused multiply-accumulate instruction;

[0013] Fig.11illustrates a SIMD implementation of a power of two size, wherein the accumulator uses an input size larger than the size of the input to the multiplier, according to an embodiment;

[0014] Fig.12 An embodiment of a system utilizing a matrix operation circuit is illustrated;

[0015] Fig.13 illustrates an embodiment of a processor core pipeline that supports matrix operations using slices;

[0016] Fig.14 illustrates an embodiment of a processor core pipeline that supports matrix operations using slices;

[0017] Fig.15 illustrates examples of matrices expressed in row-major format and column-major format;

[0018] Fig.16 An example of the use of a graphical matrix (tile);

[0019] Fig.17 An embodiment of a method of illustrating the use of a matrix (tile);

[0020] Fig.18 FIGURE 1 illustrates an exemplary execution of a TILECONFIG instruction;

[0021] Figure 19(A)-Figure 19(D) illustrates an example of register(s);

[0022] Fig. 20 An embodiment of the description of the matrices (tiles) to be supported is illustrated;

[0023] Fig.21 An embodiment of a method executed by a processor to process a TILECONFIG instruction is illustrated;

[0024] Fig. 22 Figure 1 shows a more detailed description of using memory addressing to execute the TILECONFIG instruction;

[0025] Fig.23 Example pseudo code illustrating execution of the TILECONFIG instruction;

[0026] Fig.24 illustrates an exemplary execution of a TMMA instruction using a memory source operand;

[0027] Fig.25 An embodiment of a method performed by a processor to process a TMMA instruction is illustrated;

[0028] Fig.26 FIGURES A more detailed description of the execution of the TMMA instruction using register addressing;

[0029] Fig. 27 Pseudo code illustrating a method of implementing the TMMPS instruction;

[0030] Fig.28 illustrates an exemplary execution of a TNNMA instruction using a memory source operand;

[0031] Fig.29 An embodiment of a method executed by a processor to process a TNNMA instruction is illustrated;

[0032] Fig.30 FIGURES A more detailed description of the execution of the TNNMA instruction using register addressing;

[0033] Figures 31(A)-31(C) illustrate exemplary instruction formats;

[0034] Fig.32 is a block diagram of a register architecture according to one embodiment of the present invention;

[0035] Figure 33A-Figure 33B Diagram of in-order pipeline and in-order core;

[0036] Figure 34A-Figure 34B A block diagram illustrating a more specific exemplary in-order core architecture, which would be one logic block among several logic blocks in a chip (including other cores of the same type and / or different types);

[0037] Fig.35 is a block diagram of a processor 3500 that may have more than one core, may have an integrated memory controller, and may have integrated graphics according to an embodiment of the present invention;

[0038] Figure 36-Figure 39 is a block diagram of an exemplary computer architecture; and

[0039] Fig.40 is a block diagram comparing the use of a software instruction converter to convert binary instructions in a source instruction set into binary instructions in a target instruction set according to an embodiment of the present invention. DETAILED DESCRIPTION

[0040] In the following description, numerous specific details are set forth. However, it should be understood that embodiments of the present invention may be practiced without these specific details. In other instances, well-known circuits, structures, and techniques are not shown in detail to avoid obscuring the understanding of this description.

[0041] References in the specification to "one embodiment," "an embodiment," "an example embodiment," etc. indicate that the described embodiment may include a particular feature, structure, or characteristic, but not every embodiment may include the particular feature, structure, or characteristic. Furthermore, such phrases do not necessarily refer to the same embodiment. Furthermore, when a particular feature, structure, or characteristic is described in conjunction with an embodiment, it is considered to be within the knowledge of those skilled in the art to affect such feature, structure, or characteristic in conjunction with other embodiments, whether or not explicitly described.

[0042] In many mainstream processors, handling matrices is a difficult and / or instruction-intensive task. For example, multiple rows of a matrix can be placed in multiple compact data (e.g., SIMD or vector) registers, and then multiple rows of the matrix can be operated individually. For example, depending on the data size, two 8x2 matrices may be added and may require loading or gathering into four compact data registers. Subsequently, the first addition to the compact data register corresponding to the first row from each matrix is ​​performed, and the second addition to the compact data register corresponding to the second row from each matrix is ​​performed. Subsequently, the resulting compact data register is dispersed back to the memory. Although this scenario may be acceptable for small matrices, this is usually unacceptable for larger matrices.

[0043] I. High-level discussion

[0044] Described herein are mechanisms for supporting matrix operations in computer hardware such as central processing units (CPUs), graphics processing units (GPUs), and accelerators. Matrix operations utilize 2-dimensional (2-D) data structures that represent one or more compact regions of memory (such as registers). Throughout this specification, these 2-D data structures are referred to as slices. Note that a matrix can be smaller than a slice (using less than all of a slice), or multiple slices can be utilized (a matrix is ​​larger than the size of any slice). Throughout this specification, matrix (slice) language is used to indicate operations performed using slices that affect a matrix; whether that matrix is ​​larger than any slice is generally irrelevant.

[0045] Each tile may be acted upon by a different operation, such as those detailed herein, including, but not limited to: matrix (tile) multiplication, tile addition, tile subtraction, tile diagonal, tile zeroing, tile transpose, tile dot product, tile broadcast, tile row broadcast, tile column broadcast, tile multiplication, tile multiplication and accumulation, tile move, etc. Additionally, support for operators such as the use of scaling and / or biasing may be used in the future with these operations or to support non-numerical applications, such as OpenCL "local memory", data compression / decompression, etc.

[0046] Portions of storage (such as (non-volatile and volatile) memory, registers, cache, etc.) are arranged into slices with different horizontal and vertical dimensions. For example, a slice may have a horizontal dimension of 4 (e.g., four rows of a matrix) and a vertical dimension of 8 (e.g., 8 columns of a matrix). Typically, the horizontal dimension is related to the element size (e.g., 2 bits, 4 bits, 8 bits, 16 bits, 32 bits, 64 bits, 128 bits, etc.). Multiple data types may be supported (single precision floating point, double precision floating point, integer, etc.).

[0047] A. Exemplary Uses of Configured Sheets

[0048] Figure 1 An embodiment of a configured slice is illustrated. As shown, there are four slices 111, 113, 115, and 117 loaded from application memory 101. In this example, slices T0 111 and T1 113 have M rows and N columns with 4 element bytes (e.g., single precision data). Slices T2 115 and T3 117 have M rows and N / 2 columns with 8 element bytes (e.g., double precision data). Since double precision operands are twice as wide as single precision operands, this configuration is consistent with providing a palette of slice options, providing at least 16*N*M bytes of total storage for at least 4 names. Depending on the instruction encoding scheme used, the number of available slices varies.

[0049] In some embodiments, slice parameters are definable. For example, a "palette" is used to provide slice options. Exemplary options include, but are not limited to: number of slice names, number of bytes in a row stored, number of rows and columns in a slice, etc. For example, the maximum "height" (number of rows) of a slice may be defined as:

[0050] Maximum number of rows in a slice = constructed storage / (number of palette names * number of bytes per row)

[0051] Thus, applications can be written so that fixed use of names will be able to take advantage of different storage sizes across implementations.

[0052] Configuration of tiles is accomplished using a tile configuration ("TILECONFIG") instruction, in which specific tile usage is defined within a selected palette. The declaration includes the number of tile names to be used, the requested number of rows and columns for each name (tile), and in some embodiments, the requested data type for each tile. In some embodiments, a consistency check is performed during execution of the TILECONFIG instruction to determine that it matches the constraints of the palette entries.

[0053] B. Exemplary Chunk Storage Types

[0054] Figure 2Several examples of matrix storage are shown. In (A), slices are stored in memory. As shown, each "row" consists of four packed data elements. To get to the next "row", a stride value is used. Note that rows can be stored consecutively in memory. When slice storage does not map the underlying memory array row width, strided memory access allows access to one row to the subsequent next row.

[0055] Loading tiles from memory and storing tiles to memory are typically strided accesses from application memory to packed rows of data. Exemplary TILELOAD and TILESTORE instructions or other instruction references to application memory as TILE operands in load operation instructions are restartable in some embodiments to handle (up to) 2*rows of page faults, unmasked floating point exceptions, and / or interrupts per instruction.

[0056] In (B), the matrix is ​​stored in slices consisting of multiple registers, such as packed data registers (Single Instruction Multiple Data (SIMD) or vector registers). In this example, the slice is superimposed on three physical registers. Typically, contiguous registers are used, however, this need not be the case.

[0057] In (C), the matrix is ​​stored in a slice in non-register storage accessible by a fused multiply-accumulate (FMA) circuit used in a slice operation. The storage may be inside the FMA, or adjacent to the FMA. Furthermore, in some embodiments, as discussed below, the storage may be for data elements rather than entire rows or slices.

[0058] The supported parameters of the TMMA architecture are reported via CPUID. In some embodiments, the information list includes the maximum height and the maximum SIMD scale. Configuring the TMMA architecture requires specifying the scale of each tile, the element size of each tile, and the palette identifier. This configuration is completed by executing the TILECONFIG instruction.

[0059] Successful execution of the TILECONFIG instruction enables subsequent TILE operators. The TILERELEASEALL instruction clears the tile configuration and disables TILE operations (until the next TILECONFIG instruction is executed). In some embodiments, XSAVE, XSTORE, etc. are used in context switches that use tiles. In some embodiments, 2 XCR0 bits are used in XSAVE, one for TILECONFIF metadata and one corresponding to the actual tile payload data.

[0060] TILECONFIG not only configures tile usage, but also sets a state variable that indicates that the program is in the code region if the tile is configured. An implementation may enumerate restrictions on other instructions that may be used with tile regions, such as no use of existing register sets, and so on.

[0061] Exiting a tile region is typically accomplished using the TILERELEASEALL instruction. This instruction takes no parameters and immediately invalidates all tiles (indicating that data no longer requires any preservation or restoration) and clears the internal state corresponding to being in the tile region.

[0062] In some embodiments, slice operations will zero any rows and any columns that exceed the dimensions specified by the slice configuration. For example, as each row is written, the slice operation will zero the data beyond the configured number of columns (taking the size of the element into account). For example, for 64-byte rows and a slice configured with 10 rows and 12 columns, an operation that writes an FP32 element will write the output / result data to each of the first 10 rows with 12*4 bytes and zero the remaining 4*4 bytes in each row. The slice operation also completely zeroes any rows after the first 10 configured rows. When using a 1K slice with 64-byte rows, there will be 16 rows, so in this example, the last 6 rows will also be zeroed.

[0063] In some embodiments, when loading data, context restore (e.g., XRSTOR) forces that data beyond the configured rows of the slice will be maintained as zeros. If there is no valid configuration, all rows are zeroed. An XRSTOR on the slice data can load garbage information in columns beyond those configured. It should not be possible for the XRSTOR to clear beyond the configured number of columns because there is no element width associated with the slice configuration.

[0064] A context save (e.g., XSAVE) exposes the entire TILE memory area when it is written to memory. If XRSTOR loads useless data into the rightmost portion of a tile, that data will be saved by XSAVE. For rows exceeding the number specified for each tile, XSAVE will write zeros.

[0065] In some embodiments, slice instructions are restartable. Operations accessing memory allow restarting after a page fault. Compute instructions that process floating point operations also allow unmasked floating point exceptions, by virtue of masking of exceptions controlled by control and / or status registers.

[0066] To support restarting the instruction after these events, the instruction stores information in the start register described in detail below.

[0067] II. Matrix (chip) operating system

[0068] A. Exemplary Hardware Support

[0069] Figure 3 An embodiment of a system utilizing a matrix (slice) operation accelerator is illustrated. In the illustration, a host processor / processing system 301 passes a command 311 (e.g., a matrix manipulation operation, such as an arithmetic or matrix manipulation operation, or a load and store operation) to a matrix operation accelerator 307. However, this is shown in this manner for discussion purposes only. As described in detail later, the accelerator 307 may be part of a processing core. Typically, the command 311, which is a slice manipulation operator instruction, references the slice in a register-register ("reg-reg") or register-memory ("reg-mem") format. Other commands such as TILESTORE, TILELOAD, TILECONFIG, etc. do not perform data operations on the slice. The command may be a decoded instruction (e.g., a micro-operation) or a macro-instruction for the accelerator 307 to handle.

[0070] In this example, the consistent memory interface 303 is coupled to the host processor / processing system 301 and the matrix operation accelerator 405 so that they can share memory. Figure 4 and Figure 5 Different embodiments of how to use a matrix operation accelerator to share memory are shown. Figure 4 As shown in FIG. 4 , the host processor 401 and the matrix operation accelerator circuit 405 share the same memory 403 . Figure 5 An embodiment is shown in which the host processor 501 and the matrix operation accelerator 505 do not share memory, but can access each other's memory. For example, the processor 501 can access the slice memory 507 and utilize its host memory 503 as usual. Similarly, the matrix operation accelerator 505 can access the host memory 503, but more typically uses its own memory 507. Note that these memories can be of different types.

[0071] The matrix operation accelerator 307 includes a plurality of FMAs 309 coupled to a data buffer 305 (in some implementations, one or more of these buffers 305 are stored in a grid of FMAs as shown). The data buffer 305 buffers slices loaded from memory (e.g., using a slice load or slice store instruction) and / or slices stored to memory. The data buffer may be, for example, a plurality of registers. Typically, these FMAs are arranged as a grid of chained FMAs 309 capable of reading and writing slices. In this example, the matrix operation accelerator 307 is used to perform matrix multiplication operations using slices T0, T1, and T2. At least one of the slices is accommodated in the FMA grid 309. In some embodiments, all slices in the operation are stored in the FMA grid 309. In other embodiments, only a subset is stored in the FMA grid 309. As shown, T1 is accommodated, while T0 and T2 are not accommodated. Note that A, B, and C refer to matrices of these slices, which may or may not occupy the entire space of the slice.

[0072] Figure 6 An embodiment using a tiled matrix multiply-accumulate operation ("TMMA") is illustrated.

[0073] The number of rows in the matrix (slice A 601) matches the number of serially connected (chained) FMAs including the latency of the computation. The implementation is free to recycle on a grid of smaller height, but the computation remains the same.

[0074] The source / destination vectors come from a slice of N rows (slice C 605), and the grid of FMAs 611 performs N vector-matrix operations, resulting in a complete instruction to perform the matrix multiplication of the slice. Slice B 603 is another vector source, and provides a "broadcast" term to the FMA in each stage.

[0075] In operation, in some embodiments, the elements of matrix B (stored in slice B 603) are spread across the rectangular grid of FMAs. Matrix B (stored in slice A 601) has its row elements transposed to match the column dimensions of the rectangular grid of FMAs. At each FMA in the grid, the elements of A and B are multiplied and added to the incoming augend (from above in the figure), and the outgoing sum is passed to the next row (or final output) of the FMA.

[0076] The latency of a single step is proportional to K (the row height of matrix B), and the dependent TMMAs typically have enough source-destination rows (in a single chip or across chips) to hide this latency. Implementations can also split the SIMD (packed data elements) dimension M (row height of matrix A) across time steps, but this only changes the constant by which K is multiplied. When the program specifies a K that is smaller than the maximum enumerated by the TMACC, the implementation is free to implement this using "masking" or "early-out".

[0077] The waiting time of the entire TMMA is proportional to N*K. The repetition rate is proportional to N. The number of MACs per TMMA instruction is N*K*M.

[0078] Figure 7 An embodiment of a subset of the execution of an iteration of a chained fused multiply-accumulate instruction is shown. Specifically, Figure 7 The execution circuitry for an iteration of one packed data element position of the destination is shown. In this embodiment, a chained fused multiply-accumulate operates on multiple signed sources, where the accumulator is twice the input data size.

[0079] The first signed source (source 1 701) and the second signed source (source 2 703) each have four packed data elements. Each of these packed data elements stores signed data such as floating point data. The third signed source (source 3 709) has two packed data elements, each of which stores signed data. The size of the first signed source 701 and the size of the second signed source 703 are half the size of the third signed source (initial value or previous result) 709. For example, the first signed source 701 and the second signed source 703 may have 32-bit packed data elements (e.g., single precision floating point), while the third signed source 709 may have 64-bit packed data elements (e.g., double precision floating point).

[0080] In this illustration, only the most significant two packed data element positions of the first signed source 701 and the second signed source 703 are shown, as well as the most significant packed data element position of the third signed source 709. Of course, other packed data element positions will also be processed.

[0081] As shown, packed data elements are processed in pairs. For example, multiplier circuit 705 is used to multiply the data of the most significant packed data element position of the first signed source 701 and the second signed source 703, and multiplier circuit 707 is used to multiply the data of the second most significant packed data element position from the first signed source 701 and the second signed source 703. In some embodiments, these multiplier circuits 705 and 707 are reused for other packed data element positions. In other embodiments, additional multiplier circuits are used so that the packed data elements are processed in parallel. In some contexts, parallel execution is completed using a channel of size that is the size of the signed third source 709. The results of each multiplication in these multiplications are added using addition circuit 711.

[0082] The result of the addition of these multiplication results is added to the data from the most significant packed data element position from Signed Source 3 709 (using either a different adder 713 or the same adder 711).

[0083] Finally, the result of the second addition is stored in the packed data element position of the signed destination 715 corresponding to the used packed data element position from the signed third source 709, or is passed on to the next iteration if there is one. In some embodiments, a write mask is applied to this storage, so that if the corresponding write mask (bit) is set, the storage occurs, and if the corresponding write mask (bit) is not set, the storage does not occur.

[0084] Figure 8 An embodiment of a subset of the execution of an iteration of a chained fused multiply-accumulate instruction is shown. Specifically, Figure 8 The execution circuitry for an iteration of one packed data element position of the destination is shown. In this embodiment, a chained fused multiply-accumulate is operating on a signed source, where the accumulator is twice the size of the input data.

[0085] The first signed source (source 1 801) and the second signed source (source 2 803) each have four packed data elements. Each of these packed data elements stores signed data such as integer data. The third signed source (source 3 809) has two packed data elements, each of which stores signed data. The size of the first signed source 801 and the size of the second signed source 803 are half the size of the third signed source 809. For example, the first signed source 801 and the second signed source 803 may have 32-bit packed data elements (e.g., single-precision floating point), while the third signed source 809 may have 64-bit packed data elements (e.g., double-precision floating point).

[0086] In this illustration, only the most significant two packed data element positions of the first signed source 801 and the second signed source 803 are shown, as well as the most significant packed data element position of the third signed source 809. Of course, other packed data element positions will also be processed.

[0087] As shown, packed data elements are processed in pairs. For example, multiplier circuit 805 is used to multiply the data of the most effective packed data element position of the first signed source 801 and the second signed source 803, and multiplier circuit 807 is used to multiply the data of the second most effective packed data element position from the first signed source 801 and the second signed source 803. In some embodiments, these multiplier circuits 805 and 807 are reused for other packed data element positions. In other embodiments, additional multiplier circuits are used so that the packed data elements are processed in parallel. In some contexts, parallel execution is completed using a channel of the size of the signed third source (initial value or previous iteration result) 809. The result of each multiplication in multiple multiplications is added to the signed third source 809 using addition / saturation circuit 811.

[0088] Addition / saturation (accumulator) circuit 811 preserves the sign of the operand when an addition results in a value that is too large. Specifically, saturation evaluation occurs for infinite precision results between multi-way addition and writing to a destination or next iteration. When accumulator 811 is floating point and the input terms are integers, the sum of the products and the floating point accumulator input value are converted to infinite precision values ​​(hundreds of digits of fixed point), addition of the multiplication result and the third input is performed, and a single rounding to the actual accumulator type is performed.

[0089] Unsigned saturation means that the output value is limited to the maximum unsigned number of that element width (all ones). Signed saturation means that the value is limited to the range between the smallest negative number and the largest positive number of that element width (e.g., for bytes, the range is from -128 (=-2^7) to 127 (=2^7-1)).

[0090] The result of the addition and saturation check is stored in the packed data element position of signed result 815 corresponding to the used packed data element position from signed third source 809, or is passed on to the next iteration, if there is one. In some embodiments, a write mask is applied to this storage, so that if the corresponding write mask (bit) is set, the storage occurs, and if the corresponding write mask (bit) is not set, the storage does not occur.

[0091] Fig. 9 An embodiment of a subset of the execution of an iteration of a chained fused multiply-accumulate instruction is shown. Specifically, Fig. 9The execution circuitry for an iteration of one packed data element position of the destination is shown. In this embodiment, a chained fused multiply-accumulate is operating on both signed and unsigned sources, where the accumulator is 4 times the size of the input data.

[0092] The first signed source (source 1 901) and the second unsigned source (source 2 903) each have four packed data elements. Each of these packed data elements has data such as floating point data or integer data. The third signed source (initial value or result 915) has packed data elements that store signed data. The size of the first source 901 and the size of the second source 903 are one-fourth the size of the third signed source 915. For example, the first source 901 and the second source 903 may have 16-bit packed data elements (e.g., words), while the third signed source 915 may have 64-bit packed data elements (e.g., double precision floating point or 64-bit integer).

[0093] In this illustration, only the most significant four packed data element positions of the first source 901 and the second source 903 are shown, as well as the most significant packed data element position of the third signed source 915. Of course, if there are any other packed data element positions, they will also be processed.

[0094] As shown, packed data elements are processed in quads. For example, data at the most significant packed data element position of the first source 901 and the second source 903 are multiplied using multiplier circuit 907, data at the second most significant packed data element position from the first source 901 and the second source 903 are multiplied using multiplier circuit 907, data at the third most significant packed data element position from the first source 901 and the second source 903 are multiplied using multiplier circuit 909, and data at the least significant packed data element position from the first source 901 and the second source 903 are multiplied using multiplier circuit 911. In some embodiments, the signed packed data elements of the first source 901 are sign-extended, and the unsigned packed data elements of the second source 903 are zero-extended prior to multiplication.

[0095] In some embodiments, these multiplier circuits 905-911 are reused for other packed data element positions. In other embodiments, additional multiplier circuits are used so that the packed data elements are processed in parallel. In some contexts, parallel execution is accomplished using lanes of the size of the signed third source 915. The results of each of these multiplications are added using an adder circuit 911.

[0096] The result of the addition of these multiplication results is added to the data from the most significant packed data element position from Signed Source 3 915 (using a different adder 913 or the same adder 911).

[0097] Finally, the result 919 of the second addition is stored in the signed destination in a packed data element position corresponding to the used packed data element position from the signed third source 915, or passed to the next iteration. In some embodiments, a write mask is applied to this storage, so that if the corresponding write mask (bit) is set, the storage occurs, and if the corresponding write mask (bit) is not set, the storage does not occur.

[0098] Fig.10 An embodiment of a subset of the execution of an iteration of a chained fused multiply-accumulate instruction is shown. Specifically, Fig.10 The execution circuitry for an iteration of one packed data element position of the destination is shown. In this embodiment, a chained fused multiply-accumulate is operating on both signed and unsigned sources, where the accumulator is 4 times the size of the input data.

[0099] The first signed source (source 1 1001) and the second unsigned source (source 2 1003) each have four packed data elements. Each of these packed data elements stores data such as floating point data or integer data. The third signed source (initial value or previous result 1015) has packed data elements that store signed data. The size of the first source 1001 and the size of the second source 1003 are one-fourth the size of the third signed source 1015. For example, the first source 1001 and the second source 1003 may have 16-bit packed data elements (e.g., words), while the third signed source 1015 may have 64-bit packed data elements (e.g., double precision floating point or 64-bit integer).

[0100] In this illustration, only the most significant four packed data element positions of the first source 1001 and the second source 1003 are shown, as well as the most significant packed data element position of the third signed source 1015. Of course, if there are any other packed data element positions, they will also be processed.

[0101] As shown, packed data elements are processed in quads. For example, data at the most significant packed data element positions of the first source 1001 and the second source 1003 are multiplied using multiplier circuit 1007, data at the second most significant packed data element positions from the first source 1001 and the second source 1003 are multiplied using multiplier circuit 1007, data at the third most significant packed data element positions from the first source 1001 and the second source 1003 are multiplied using multiplier circuit 1009, and data at the least significant packed data element positions from the first source 1001 and the second source 1003 are multiplied using multiplier circuit 1011. In some embodiments, the signed packed data elements of the first source 1001 are sign-extended, and the unsigned packed data elements of the second source 1003 are zero-extended prior to multiplication.

[0102] In some embodiments, these multiplier circuits 1005-1011 are reused for other packed data element positions. In other embodiments, additional multiplier circuits are used so that the packed data elements are processed in parallel. In some contexts, parallel execution is accomplished using channels of the size of the signed third source 1015. The result of the addition of these multiplication results is added to the data from the most significant packed data element position of the signed source 3 1015 using the addition / saturation circuit 1013.

[0103] Addition / saturation (accumulator) circuit 1013 preserves the sign of the operand when an addition results in a value that is too large or too small for signed saturation. Specifically, saturation evaluation occurs for infinite precision results between multi-way additions and writes to the destination. When accumulator 1013 is floating point and the input terms are integers, the sum of the products and the floating point accumulator input value are converted to infinite precision values ​​(hundreds of digits of fixed point), addition of the multiplication result and the third input is performed, and a single rounding to the actual accumulator type is performed.

[0104] The result 1019 of the addition and saturation check is stored in the signed destination in a packed data element position corresponding to the used packed data element position from the signed third source 1015, or passed to the next iteration. In some embodiments, a write mask is applied to this storage so that if the corresponding write mask (bit) is set, the storage occurs, and if the corresponding write mask (bit) is not set, the storage does not occur.

[0105] Fig.11The diagram illustrates a SIMD implementation according to an embodiment with a size that is a power of 2, wherein the accumulator uses an input size that is larger than the size of the input to the multiplier. Note that the source (to the multiplier) and accumulator values ​​can be signed or unsigned values. For an accumulator with a 2X input size (in other words, the size of the accumulator input value is twice the size of the packed data elements of the source), Table 1101 illustrates different configurations. For a byte-sized source, the accumulator uses a word or half-precision floating point (HPFP) value of size 16 bits. For a word-sized source, the accumulator uses a 32-bit integer or single-precision floating point (SPFP) value of size 32 bits. For a SPFP or 32-bit integer-sized source, the accumulator uses a 64-bit integer or double-precision floating point (DPFP) value of size 64 bits.

[0106] Table 1103 illustrates different configurations for an accumulator with a 4X input size (in other words, the size of the accumulator input value is 4 times the size of the packed data elements of the source). For byte-sized sources, the accumulator uses 32-bit integers or single-precision floating point (SPFP) values ​​with a size of 32 bits. In some embodiments, for word-sized sources, the accumulator uses 64-bit integers or double-precision floating point (DPFP) values ​​with a size of 64 bits.

[0107] For an accumulator with an 8X input size (in other words, the size of the accumulator input value is 8 times the size of the packed data elements of the source), table 1105 illustrates the configuration. For a byte-sized source, the accumulator uses 64-bit integers.

[0108] As previously noted, the matrix manipulation circuitry may be included in the core, or may be provided as an external accelerator. Fig.12 An embodiment of a system utilizing a matrix operating circuit is illustrated. In this illustration, multiple entities are coupled to a ring interconnect 1245.

[0109] Multiple cores 1201 , 1203 , 1205 , and 1207 provide non-slice-based instruction support. In some embodiments, matrix operation circuitry is provided in core 1203 , and in other embodiments, matrix operation circuitry 1211 and 1213 are accessible over ring interconnect 1245 .

[0110] Additionally, one or more memory controllers 1223 - 1225 are provided to communicate with memories 1233 and 1231 on behalf of the core and / or matrix operation circuitry.

[0111] Fig.13The embodiment of the processor core pipeline is shown, which supports matrix operations using slices. Branch prediction and decoding circuit 1303 performs branch prediction of instructions from instructions stored in instruction storage 1301, decoding of these instructions and / or both branch prediction and decoding. For example, the instructions described in detail herein may be stored in the instruction storage. In some implementations, separate circuits are used for branch prediction, and in some embodiments, at least some instructions are decoded into one or more micro-operations, microcode entry points, microinstructions, other instructions or other control signals using microcode 1305. Branch prediction and decoding circuit 1303 can be implemented using various different mechanisms. Examples of suitable mechanisms include, but are not limited to, lookup tables, hardware implementations, programmable logic arrays (PLA), microcode read-only memories (ROM), etc.

[0112] Branch prediction and decode circuitry 1303 is coupled to rename / allocator circuitry 1307, which in some embodiments is coupled to scheduler circuitry 1309. In some embodiments, these circuits provide register renaming, register allocation, and / or scheduling functionality by performing one or more of the following steps: 1) renaming logical operand values ​​to physical operand values ​​(e.g., a register alias table in some embodiments); 2) assigning status bits and flags to decoded instructions; and 3) (e.g., using a reservation station in some embodiments) scheduling decoded instructions for execution on execution circuitry external to an instruction pool.

[0113] The scheduler circuit 1309 represents any number of different schedulers, including reservation stations, central instruction windows, etc. Scheduler unit (s) Scheduler circuit 1309 is coupled to or includes (s) physical register file (s) 1315. Each of the (s) physical register files 1315 represents one or more physical register files, where different physical register files store one or more different data types, such as scalar integer, scalar floating point, packed integer, packed floating point, vector integer, vector floating point, status (e.g., instruction pointer as the address of the next instruction to be executed), slices, etc. In one embodiment, the (s) physical register file 1315 includes vector register circuits, write mask register circuits, and scalar register circuits. These register circuits may provide architectural vector registers, vector mask registers, and general purpose registers. The physical register file(s) 1315 are overlaid with retirement circuitry 1317 to illustrate various ways in which register renaming and out-of-order execution may be implemented (such as using reorder buffer(s) and retirement register file(s), using future file(s), history buffer(s), and retirement register file(s), using register maps and register pools, etc.). The retirement circuitry 1317 and physical register file(s) 1315 are coupled to the execution circuitry 1311.

[0114] Although register renaming is described in the context of out-of-order execution, it should be understood that register renaming can be used in an in-order architecture. Although the illustrated embodiment of the processor may also include separate instruction and data cache units and a shared L2 cache unit, alternative embodiments may also have a single internal cache for both instructions and data, such as, for example, a first level (L1) internal cache, or multiple levels of internal cache. In some embodiments, the system may include a combination of internal caches and external caches external to the core and / or processor. Alternatively, all caches may be external to the core and / or processor.

[0115] Execution circuitry 1311 includes a set of one or more execution circuits 1321, 1323, and 1327 and a set of one or more memory access circuits 1325. Execution circuitry 1321, 1323, and 1327 perform various operations (e.g., shifts, additions, subtractions, multiplications) and on various data types (e.g., scalar floating point, packed integers, packed floating point, vector integers, vector floating point). Although some embodiments may include multiple execution units dedicated to a particular function or set of functions, other embodiments may include only one execution unit or multiple execution units that all perform all functions. Scalar circuitry 1321 performs scalar operations, vector / SIMD circuitry 1323 performs vector / SIMD operations, and matrix operation circuitry 1327 performs matrix (slice) operations as described in detail herein.

[0116] A set of memory access units 1364 is coupled to a memory unit 1370, which includes a data TLB unit 1372, which is coupled to a data cache unit 1374, which is coupled to a second level (L2) cache unit 1376. In an exemplary embodiment, the memory access unit 1364 may include a load unit, a store address unit, and a store data unit, each of which is coupled to the data TLB unit 1372 in the memory unit 1370. The instruction cache unit 1334 is further coupled to a level 2 (L2) cache unit 1376 in the memory unit 1370. The L2 cache unit 1376 is coupled to one or more other levels of cache and ultimately to the main memory.

[0117] As an example, an exemplary register renaming, out-of-order issue / execution core architecture may implement a pipeline as follows: 1) instruction fetch circuitry performs a fetch and length decoding stage; 2) branch and decode circuitry 1303 performs a decode stage; 3) rename / allocator circuitry 1307 performs an allocation stage and a rename stage; 4) scheduler circuitry 1309 performs a scheduling stage; 5) (multiple) physical register files (coupled to or included in scheduler circuitry 1307 and rename / allocator circuitry 1307 and memory units) perform a register read / memory read stage; execution circuitry 1311 performs an execution stage; 6) memory units and (multiple) physical register file units perform a write back / memory write stage; 7) various units may be involved in an exception handling stage; and 8) retirement units and (multiple) physical register file units perform a commit stage.

[0118] The core may support one or more instruction sets (e.g., the x86 instruction set (with some extensions added with newer versions); the MIPS instruction set of MIPS Technologies, Inc. of Sunnyvale, California; the ARM instruction set of ARM Holdings, Inc. of Sunnyvale, California (with optional additional extensions such as NEON)), including the instruction(s) described herein. In one embodiment, core 1390 includes logic to support packed data instruction set extensions (e.g., AVX1, AVX2), thereby allowing operations used by many multimedia applications to be performed with packed data.

[0119] It should be understood that a core may support multithreading (executing a set of two or more operations or threads in parallel) and may accomplish this in a variety of ways, including time-division multithreading, simultaneous multithreading (where a single physical core provides a logical core for each of the threads that the physical core is simultaneously multithreading), or a combination thereof (e.g., time-division fetching and decoding and thereafter such as with Hyper-threading technology to synchronize multithreading).

[0120] Fig.14 The embodiment of the processor core pipeline is shown, which supports matrix operations using slices. Branch prediction and decoding circuit 1403 performs branch prediction of instructions from instructions stored in instruction storage 1401, decoding of these instructions and / or both branch prediction and decoding. For example, the instructions described in detail herein may be stored in the instruction storage. In some implementations, separate circuits are used for branch prediction, and in some embodiments, at least some instructions are decoded into one or more micro-operations, microcode entry points, microinstructions, other instructions or other control signals using microcode 1405. Branch prediction and decoding circuit 1403 can be implemented using various different mechanisms. Examples of suitable mechanisms include, but are not limited to, lookup tables, hardware implementations, programmable logic arrays (PLA), microcode read-only memories (ROM), etc.

[0121] Branch prediction and decode circuitry 1403 is coupled to rename / allocator circuitry 1407, which in some embodiments is coupled to scheduler circuitry 1409. In some embodiments, these circuits provide register renaming, register allocation, and / or scheduling functionality by performing one or more of the following steps: 1) renaming logical operand values ​​to physical operand values ​​(e.g., a register alias table in some embodiments); 2) assigning status bits and flags to decoded instructions; and 3) (e.g., using a reservation station in some embodiments) scheduling decoded instructions for execution on execution circuitry external to an instruction pool.

[0122] The scheduler circuit 1409 represents any number of different schedulers, including reservation stations, central instruction windows, etc. The (multiple) scheduler units The scheduler circuit 1409 is coupled to or includes (multiple) physical register files 1415. Each of the (multiple) physical register files 1415 represents one or more physical register files, where different physical register files store one or more different data types, such as scalar integers, scalar floating points, packed integers, packed floating points, vector integers, vector floating points, status (e.g., instruction pointers as addresses of next instructions to be executed), slices, etc. In one embodiment, the (multiple) physical register files 1415 include vector register circuits, write mask register circuits, and scalar register circuits. These register circuits may provide architectural vector registers, vector mask registers, and general purpose registers. The physical register file(s) 1415 are overlaid with retirement circuitry 1417 to illustrate various ways in which register renaming and out-of-order execution may be implemented (such as using reorder buffer(s) and retirement register file(s), using future file(s), history buffer(s), and retirement register file(s), using register maps and register pools, etc.). The retirement circuitry 1417 and physical register file(s) 1415 are coupled to the execution circuitry 1411.

[0123] Although register renaming is described in the context of out-of-order execution, it should be understood that register renaming can be used in an in-order architecture. Although the illustrated embodiment of the processor may also include separate instruction and data cache units and a shared L2 cache unit, alternative embodiments may also have a single internal cache for both instructions and data, such as, for example, a first level (L1) internal cache, or multiple levels of internal cache. In some embodiments, the system may include a combination of internal caches and external caches external to the core and / or processor. Alternatively, all caches may be external to the core and / or processor.

[0124] Execution circuitry 1411 includes a set of one or more execution circuits 1427 and a set of one or more memory access circuits 1425. Execution circuitry 1427 performs the matrix (slice) operations detailed herein.

[0125] A set of memory access units 1464 is coupled to a memory unit 1470, which includes a data TLB unit 1472, which is coupled to a data cache unit 1474, which is coupled to a second level (L2) cache unit 1476. In an exemplary embodiment, the memory access unit 1464 may include a load unit, a store address unit, and a store data unit, each of which is coupled to the data TLB unit 1472 in the memory unit 1470. The instruction cache unit 1434 is further coupled to a second level (L2) cache unit 1476 in the memory unit 1470. The L2 cache unit 1476 is coupled to one or more other levels of cache and ultimately to the main memory.

[0126] As an example, an exemplary register renaming, out-of-order issue / execution core architecture may implement a pipeline as follows: 1) instruction fetch circuitry performs a fetch and length decoding stage; 2) branch and decode circuitry 1403 performs a decode stage; 3) rename / allocator circuitry 1407 performs an allocation stage and a rename stage; 4) scheduler circuitry 1409 performs a scheduling stage; 5) (multiple) physical register files (coupled to or included in scheduler circuitry 1407 and rename / allocator circuitry 1407 and memory units) perform a register read / memory read stage; execution circuitry 1411 performs an execution stage; 6) memory units and (multiple) physical register file units perform a write back / memory write stage; 7) various units may be involved in an exception handling stage; and 8) retirement units and (multiple) physical register file units perform a commit stage.

[0127] The core may support one or more instruction sets (e.g., the x86 instruction set (with some extensions added with newer versions); the MIPS instruction set of MIPS Technologies, Inc. of Sunnyvale, California; the ARM instruction set of ARM Holdings, Inc. of Sunnyvale, California (with optional additional extensions such as NEON), including the instruction(s) described herein. In one embodiment, core 1490 includes logic to support packed data instruction set extensions (e.g., AVX1, AVX2), thereby allowing operations used by many multimedia applications to be performed with packed data.

[0128] It should be understood that a core may support multithreading (executing a set of two or more operations or threads in parallel) and may accomplish this in a variety of ways, including time-division multithreading, simultaneous multithreading (where a single physical core provides a logical core for each of the threads that the physical core is simultaneously multithreading), or a combination thereof (e.g., time-division fetching and decoding and thereafter such as with Hyper-threading technology to synchronize multithreading).

[0129] B. Layout

[0130] Throughout this specification, row-major data layouts are used to represent data. Column-major users should transform items according to their orientation. Fig.15 The figure shows examples of matrices expressed in row-major format and column-major format. As shown in the figure, matrix A is a 2x3 matrix. When the matrix is ​​stored in row-major format, the data elements of the rows are continuous. When the matrix is ​​stored in column-major format, the data elements of the columns are continuous. T *B T =(BA) T is a well-known property of matrices, where the superscript T represents the transpose. Reading column-major data as row-major data results in a matrix that looks like a transposed matrix.

[0131] In some embodiments, row-major semantics are utilized in hardware, and column-major data will swap the operand order and make the result the transposed of the matrix, but for subsequent column-major reads from memory it is the correct non-transposed matrix.

[0132] For example, if you have two column-major matrices that you want to multiply:

[0133] abgik ag+bh ai+bj ak+bl

[0134] cd*hjl=cg+dhci+djck+dl

[0135] ef eg+fh ei+fj ek+fl

[0136] (3x2)(2x3)(3x3)

[0137] The input matrix will be stored in linear memory (column-major) as follows:

[0138] acebdf

[0139] as well as

[0140] ghijkl.

[0141] Reading those matrices as row-major with scale 2x3 and 3x2, they will appear as:

[0142] ace and gh

[0143] bdfij

[0144] k

[0145] Swap the order and multiply the matrices:

[0146] ghace ag+bh cg+dh eg+fh

[0147] ij*bdf=ai+bj ci+dj ei+fj

[0148] kl ak+bl ck+dl ek+fl

[0149] The transposed matrix is ​​shifted out, and can then be stored in row-major order:

[0150] ag+bh cg+dh eg+fh ai+bj ci+dj ei+fj ak+bl ck+dl ek+fl

[0151] And it is used in subsequent column-major calculations as the correct untransposed matrix:

[0152] ag+bh ai+bj ak+bl

[0153] cg+dh ci+dj ck+dl

[0154] eg+fh ei+fj ek+fl

[0156] III. Exemplary Uses

[0157] Fig.16 An example of the use of matrices (slices) is illustrated. In this example, matrix C 1601 includes two slices, matrix A 1603 includes one slice, and matrix B 1605 includes two slices. The figure shows an example of the inner loop of an algorithm for calculating matrix multiplication. In this example, two result slices tmm0 and tmm1 from matrix C 1601 are used to accumulate intermediate results. When one slice (tmm2) from matrix A 1603 is multiplied by two slices from matrix B 1605, this slice is reused 2 times. Pointers are used to load a new A slice and two new B slices from the direction indicated by the arrows. The outer loop not shown adjusts the pointer for the C slice.

[0158] The exemplary code as shown includes the use of slice configuration instructions and is executed to configure slice usage, load a slice, a loop for processing the slice, store the slice to memory, and release the slice usage.

[0159] Fig.17 An embodiment of the use of matrices (tiles) is illustrated. At 1701, tile use is configured. For example, a TILECONFIG instruction is executed to configure tile use, including setting the number of rows and columns of each tile. Typically, at 1703, at least one matrix (tile) is loaded from memory.

[0160] IV. Example Instructions

[0161] A. Slice Configuration

[0162] As discussed above, slice usage typically requires configuration before use. For example, it may not be necessary to fully use all rows and columns. Not configuring these rows and columns not only saves power in some embodiments, but also can use configuration to determine whether an operation will generate an error. For example, if M and L are not the same, a matrix multiplication of the form (NxM)*(L*N) will typically not work.

[0163] Detailed herein are embodiments of a matrix (tile) configuration ("TILECONFIG") instruction and its execution. Prior to using a matrix that utilizes tiles, in some embodiments, tile support will be configured. For example, how many rows and columns each tile has, which tiles will be used, etc. The TILECONFIG instruction is an improvement to the computer itself, as it provides support for configuring the computer to use a matrix accelerator (either as part of a processor core or as an external device). Specifically, execution of the TILECONFIG instruction causes the configuration to be retrieved from memory and applied to the matrix (tile) settings within the matrix accelerator.

[0164] i. Exemplary Implementation

[0165] Fig.18 An exemplary execution of a TILECONFIG instruction is illustrated. The TILECONFIG instruction format includes fields for an opcode and a memory address.

[0166] As illustrated, the TILECONFIG instruction uses the address as a pointer to a memory 1801 location containing a description 1803 of the matrices (tiles) to be supported.

[0167] Execution circuitry 1811 of processor / core 1805 performs TILECONFIG by: retrieving description 1803 from memory 1801 via memory controller 1815; configuring tiles for the palette in tile configuration 1817 (setting the number of rows and columns); and marking matrix support as in use. Specifically, instruction execution resources 1811 are configured to use tiles as specified by setting tile configuration 1817. Instruction execution resources may also include machine-specific registers or configuration registers for indicating tile usage.

[0168] The tile configuration 1817 is set to indicate parameters for each tile as indicated by the tile description 1803 via the execution of the TILECONFIG instruction. The parameters set are the number of rows and columns for each tile. Additional values ​​are also set, such as in-use values ​​and start values. The tile configuration 1817 utilizes one or more registers 1819 to store tile usage and configuration information.

[0169] ii. Exemplary Chunk Storage

[0170] Figure 19(A)-Figure 19(D) An example of register(s) 1819 is illustrated. FIG. 19(A) illustrates a plurality of registers 1819. As shown, each tile (TMM0 1901 ... TMMN 1903) has separate registers, where each register stores the row and column dimensions for that particular tile. StartK and StartM are stored in separate registers 1911 and 1913. One or more status registers 1915 are set (e.g., TILES_CONFIGURED = 1) to indicate that the tile is configured for use.

[0171] FIG. 19(B) illustrates a plurality of registers 1819. As shown, each tile has separate registers for its rows and its columns. For example, TMM0 row configuration 1921, TMM0 column configuration 1923, StartK, and StartM are stored in separate registers 1911 and 1913. One or more status registers 1915 are set (e.g., TILES_CONFIGURED=1) to indicate that the tile is configured for use.

[0172] 19(C) illustrates a single register 1819. As shown, this register stores the tile configuration (rows and columns per tile) 1931, StartK 1933, and StartM 1933 in a single register that is a packed data register. One or more status registers 1915 are set (e.g., TILES_CONFIGURED=1) to indicate that the tile is configured for use.

[0173] 19(D) illustrates multiple registers 1819. As shown, a single register stores the tile configuration (rows and columns per tile) 1931. StartK and StartM are stored in separate registers 1911 and 1913. One or more status registers 1915 are set (e.g., TILES_CONFIGURED = 1) to indicate that the tile is configured for use.

[0174] Other combinations are contemplated, such as combining the start registers into a single register in which they are displayed separately, and so forth.

[0175] iii. Example stored matrix (slice) description

[0176] Fig. 20 An embodiment of the description of matrices (tiles) to be supported is illustrated. In this example, each field is a byte. In byte [0], the palette ID 2001 is stored. The palette ID is used to index the palette table 1813, which stores the number of bytes in the slice and the bytes per row of the slice associated with the ID according to the palette ID as defined by the configuration. Bytes 1-7 are reserved and are typically zero.

[0177] Byte 8-9 stores the value for "startM" register 2003, and byte 10-11 stores the value for "startK" register 2005. In order to support restarting instructions after these events, these instructions store information in these registers. The startM indication should be used for the row to restart. The startK indication is for the position in the inner product of the related operation. This position in the row (column) is not needed. Two-dimensional operations like element-by-element addition / subtraction / multiplication only use startM. Three-dimensional operations use the values ​​from both startM and startK. Typically, the operation that only needs startM will make startK zero when writing startM.

[0178] At any time when the interrupted slice instruction is not restarted, in some embodiments, it is the responsibility of the software to zero the startM value and the startK value. For example, an unmasked floating point exception handler may decide to complete the operation in software and change the program counter value to another instruction, typically the next instruction. In this case, before resuming the program, the software exception handler must zero the startM value and the startK value in the exception frame presented to the software exception handler by the operating system. The operating system will subsequently reload those values.

[0179] Bytes 16-17 store the number of rows 2013 and columns 2015 for slice 0, bytes 18-19 store the number of rows and columns for slice 1, and so on. In other words, each 2-byte group specifies the number of rows and columns for a slice. If the 2-byte group is not used to specify slice parameters, they should have the value zero. Specifying slice parameters for more slices than the implementation limit or palette limit results in an error. Unconfigured slices are set to the INIT state with 0 rows and 0 columns.

[0180] Finally, the configuration in memory typically ends with a final description such as all zeros for several consecutive bytes.

[0181] iv. Example Format(s)

[0182] An embodiment of a format for a TILECONFIG instruction is a TILECONFIG address. In some embodiments, TILECONFIG is the opcode mnemonic for the instruction. The address is a pointer to a matrix (tile) description in memory. In some embodiments, the address field is an R / M value (such as 2446).

[0183] In an embodiment, the encoding of the instruction includes a scale-index-base (SIB) type memory addressing operand that indirectly identifies a plurality of indexed destination locations in memory (e.g., field 2450). In one embodiment, the SIB type memory operand may include an encoding that identifies a base register. The contents of the base register may represent a base address in memory from which the address of a particular destination location in memory is calculated. For example, the base address may be the address of the first location in a block of potential destination locations for an extended vector instruction. In one embodiment, the SIB type memory operand may include an encoding that identifies an index register. Each element of the index register may specify an index or offset value that can be used to calculate the address of a corresponding destination location within the block of potential destination locations from the base address. In one embodiment, the SIB type memory operand may include an encoding that specifies a scale factor that is applied to each index value when calculating the corresponding destination address. For example, if a scale factor value of 4 is encoded in the SIB type memory operand, each index value obtained from an element of the index register may be multiplied by 4 and then added to the base address to calculate the destination address.

[0184] In one embodiment, a SIB-type memory operand of the form vm32{x,y,z} may identify a vector array of memory operands specified using SIB-type memory addressing. In this example, a common base address register, a constant scaling factor, and a vector index register containing each element of an index value that is each 32 bits are used to specify an array of memory addresses. The vector index register may be a 128-bit register (e.g., XMM) register (vm32x), a 256-bit (e.g., YMM) register (vm32y), or a 512-bit (e.g., ZMM) register (vm32z). In another embodiment, a SIB-type memory operand of the form vm64{x,y,z} may identify a vector array of memory operands specified using SIB-type memory addressing. In this example, a common base address register, a constant scaling factor, and a vector index register containing each element of an index value that is each 64 bits are used to specify an array of memory addresses. The vector index register may be a 128-bit (eg, XMM) register (vm64x), a 256-bit (eg, YMM) register (vm64y), or a 512-bit (eg, ZMM) register (vm64z).

[0185] v.(Various) Exemplary Implementation Methods

[0186] Fig.21 An embodiment of a method performed by a processor to process a TILECONFIG instruction is illustrated.

[0187] At 2101, an instruction is fetched. For example, a TILECONFIG instruction is fetched. An embodiment of the TILECONFIG instruction includes fields for an opcode and a memory address operand.

[0188] At 2103, the fetched instruction is decoded. For example, the fetched TILECONFIG instruction is decoded by a decoding circuit such as described in detail herein.

[0189] At 2105, the description found at the memory address of the memory address operand is retrieved and the decoded instruction is dispatched (as needed).

[0190] At 2107, the decoded instruction is executed by execution circuitry (hardware) such as described in detail herein. For a TILECONFIG instruction, execution will cause the execution circuitry to configure the use of tiles in the tile configuration (set the number of rows and columns) and mark the matrix (tile) support in use (active). For example, configure one or more registers 1819. Tile support use (e.g., "TILES_CONFIGURED") is typically indicated by setting a bit in a status, control, or machine-specific register. Specifically, the instruction execution resources 1811 are configured to use the tiles as specified by the retrieved configuration.

[0191] In some embodiments, at 2109, the instruction is committed or retired.

[0192] Fig. 22 The diagram shows a more detailed description of performing the TILECONFIG instruction using memory addressing. Typically, this is performed by execution circuitry such as that detailed above after the description has been retrieved from memory. Although not shown, in some embodiments, a check is first performed to determine whether the tile is supported. Support is typically discovered through a CPUID check.

[0193] At 2201, a determination is made whether the palette ID is supported. For example, does the CPUID state that the ID is supported? If not, at 2203, a general protection fault occurs.

[0194] At 2205, the first slice specific packet is read. For example, the row number and column number of slice 0 (T0) are read.

[0195] At 2207, a determination is made whether the packet read is valid. For example, if one of the number of rows or the number of columns (not both) is set to zero, then the packet is not valid and at 2203, configuration stops and the tile is not considered in use. An invalid group occurs, for example, when one of the number of rows or columns (not both) is zero. In addition, an error occurs when the value of the number of rows is greater than the maximum number of rows supported (which is found by dividing the tile byte size of the palette ID by the number of bytes per row found in the palette table for that palette ID). Another potential error is when there are more names than supported.

[0196] If the read packet is valid, then at 2211, the tile associated with the read packet is configured to use the number of rows and columns specified by the packet in the tile configuration. The size of the elements in the tile is set by the palette table entry for the palette ID.

[0197] At 2213, a determination is made whether all tiles of the retrieved configuration have been configured. For example, have all possible tile names been processed? In some embodiments, when both the row and column for a particular tile are 0, then all tiles have been processed.

[0198] When not all slices have been configured, at 2215, the slice number is incremented so that the next slice in the configuration will be evaluated.

[0199] At 2217, the incremented grouping of the slice is read. For example, the row and column numbers of slice 1 (T1) are read. At 2207, a determination is made whether the read grouping is valid, and so on.

[0200] When all tiles have been configured, then the instruction completes at 2209. The tiles will be marked as being in use for matrix operations, for example by setting an in-use indicator in a register.

[0201] vi. Exemplary Pseudocode

[0202] Fig.23 Example pseudo code illustrating execution of the TILECONFIG instruction.

[0203] B. Slice Multiplication and Accumulation

[0204] Detailed herein are embodiments of the matrix (slice) multiply-accumulate ("TMMA") instruction and its execution. The TMMA instruction is an improvement to the computer itself because it provides support for performing matrix-matrix multiplication and accumulation (addition) using a single instruction. Specifically, execution of the TMMA instruction causes data from a first source matrix (slice) to be multiplied with data from a second source matrix (slice) and added to data from a destination matrix (slice), and the result of the multiplication-addition is stored in the destination matrix (slice). The size of the data values ​​to be stored varies depending on the instruction and slice support. Exemplary sizes include, but are not limited to, 16 bits, 32 bits, 64 bits, 128 bits, 256 bits, etc.

[0205] i. Exemplary Implementation

[0206] Fig.24 An exemplary execution of a TMMA instruction using a memory source operand is illustrated. The TMMA instruction format includes fields for the following: an opcode, a destination matrix (slice) operand (shown as "slice destination"), an identifier of a first source matrix (slice) operand (shown as "first slice source"), an identifier of a second source matrix (slice) operand (shown as "second slice source"), and an identifier of a counter register in some embodiments. In some implementations, fields for registers used in progress tracking are also included when the second source matrix (slice) operand is in memory.

[0207] When one of the sources of the instruction is a memory, the memory is accessed according to a scale-index-base (SIB) type memory addressing operand, which indirectly identifies multiple indexed destination locations in the memory, however, other memory addressing schemes may be used. As described in detail, each "row" of a matrix (slice) source or destination is a group of elements. In the memory, these groups are separated by a "stride" value. As will be described in detail, the stride can be specified using an "index" of the SIB. Depending on the implementation, the stride is from the address corresponding to the initial data element of the group to the initial data element of the subsequent group in the memory, or from the address corresponding to the last data element of the group to the initial data element of the subsequent group in the memory. Typically, strides are used to describe rows, however, that is not necessarily true.

[0208] One or both of the sources of the instruction is a matrix (slice) stored in multiple registers or stored in a matrix (slice) data structure. The source is encoded in the instruction as if it were a single register. When there are two such matrices, both are encoded as if they were both single registers.

[0209] The final source is the destination matrix (slice) 2409.

[0210] Although the illustrated matrix (tile) is shown as a 2x2 matrix, this is an arbitrary illustration to aid understanding, and different matrix (tile) sizes may be used. The TMMA operation is source 1 * source 2 + destination. Thus, (NxM) * (MxK) + (NxK) matrices are supported.

[0211] Matrix (slice) sources 2401 and 2403 and destination matrix (slice) 2409 are provided to execution circuit 2405 for TMMA operation. In some embodiments, the operation is performed position by position of data element of the matrix (slice) using grid 2407 of FMA. Grid 2407 of FMA has been described previously. In some implementations, one or more of matrix (slice) source 2401 and destination matrix (slice) 2409 are stored in grid 2407 of FMA.

[0212] The execution circuit 2405 performs TMMA by performing multiplication on the source data element by data element (using row x column matrix multiplication) and adding the data from the corresponding data element position of the destination matrix. As shown in 2411, the result of the TMMA is stored in the corresponding data element position of the destination matrix.

[0213] because Fig.24 is simplified, so it does not show the use of a counter register that acts as a progress tracker. As each "row" of destinations is written, the counter is updated. This allows the TMMA operation to be restarted if necessary by using the counter to determine where the operation stopped. Also note that in some embodiments, the counter function is not utilized.

[0214] ii.(Various) Example Formats

[0215] An embodiment of the format for the TMMA instruction is TMMAP{S,H}TMM1,TMM2,TMM3. In some embodiments, TMMAP{S,H} is the opcode mnemonic of the instruction, where S,H represents single precision (S) floating point data elements and half precision floating point data elements. TMM1 is a field for the identifier of the source / destination matrix (slice), and TMM3 is a field for the identifier of the second source matrix (slice), and TMM2 is a field for the identifier of the first source matrix (slice). In some embodiments, the TMM2 identifier is a field R / M value (such as, 3146), TMM3 is a field VVVV 3120, and the source / destination matrix (slice) identifier is field 3144. Note that if the counter is not used, SRC3 is not included in the instruction format.

[0216] iii. Exemplary Method(s) of Implementation

[0217] Fig.25An embodiment of a method performed by a processor to process a TMMA instruction is illustrated.

[0218] At 2501, an instruction is fetched. For example, a TMMA instruction is fetched. An embodiment of the TMMA instruction includes fields for: an opcode, a destination matrix (slice) operand identifier, a first source matrix (slice) operand identifier, and a second source matrix (slice) operand identifier (e.g., stored in a memory, or accessed as a register). In some embodiments, a register for storing a counter value identifier is also included.

[0219] At 2503, the fetched instruction is decoded. For example, the fetched TMMA instruction is decoded by a decoding circuit such as described in detail herein.

[0220] At 2505, data values ​​associated with the sources are retrieved and decoded instructions are dispatched (as needed).

[0221] At 2507, the decoded instruction is executed by an execution circuit (hardware) such as described in detail herein. For a TMMA instruction, the execution will cause the execution circuit to perform: 1) a matrix multiplication of the identified first source matrix (slice) operand (from the accessed memory or register) and the identified second source matrix (slice); and 2) adding the result of the matrix multiplication to the corresponding data element positions of the identified destination matrix (slice) operand. In some embodiments, the data element positions of the identified destination matrix (slice) operand that are not subject to addition are zeroed (unconfigured columns).

[0222] In some embodiments, at 2509, the instruction is committed or retired.

[0223] Fig.26 A more detailed description of the execution of a TMMA instruction using register addressing is illustrated. Typically, this is performed by an execution circuit such as that detailed above.

[0224] At 2601, the values ​​in the first, second and third counters are set. For example, startK and startM are set. Typically this is done during configuration, but on the first instance of this instruction, these values ​​are usually set to 0.

[0225] At 2603, a determination is made whether the first counter value (eg, startM) is less than the number of configured rows of the destination. If not, at 2605, the instruction is complete and all unconfigured rows are reset to zero.

[0226] If so, then the row from the destination is written into the temporary location at 2607. The row is at position [first counter] (or the value of startM).

[0227] At 2609, a determination is made whether the second counter value (eg, startK) is less than the number of configured columns of the first source.

[0228] If not, then at 2611, the row at the temporary location is written into the corresponding row location at the destination. Typically, the unconfigured columns of that row are also zeroed. At 2617, the second and third counters are reset, and the first counter is incremented. Essentially, the next row is set to start processing again at 2603.

[0229] If so, there is potentially more work to do for that row. At 2612, a determination is made whether the third counter value (e.g., n) is less than the number of configured columns of the destination. If so, at 2613, a multiplication of the data element of the first source at position row [first counter, second counter] by the data element of the second source at position row [second counter, third counter] is performed, and the result of that multiplication is added to the temporary value at position temporary [third counter]. At 2615, the third counter is incremented to move along the loop and the determination at 2612 is made again.

[0230] If not, then at 2616, the second counter is incremented and the determination at 2609 is made again.

[0231] iv. Example Pseudocode

[0232] Fig. 27 Pseudocode illustrating a method of implementing the TMMPS instruction. FMAOP may be a negative version of the opcode when it is called (TNMMPS).

[0233] v.Example

[0234] Example 1 A processor comprises: a decoding circuit for decoding an instruction having fields for an opcode, an identifier of a first source matrix operand, an identifier of a second source matrix operand, and an identifier of a source / destination matrix operand; and an execution circuit for executing the decoded instruction to: multiply the identified first source matrix operand by the identified second source matrix operand; add the result of the multiplication to the identified source / destination matrix operand; and store the result of the addition in the identified source / destination matrix operand.

[0235] Example 2 A processor as in Example 1, wherein the execution circuit comprises a grid of fused multiply-accumulators.

[0236] Example 3 The processor of any of Examples 1-2, wherein the identified second source matrix operand is stored in a memory.

[0237] Example 4 The processor of any of Examples 1-3, wherein the multiplication is performed row by row of the identified first source matrix operand and column by column of the identified second source matrix operand.

[0238] Example 5 The processor of any of Examples 1-4, wherein at least one of the operands is a plurality of registers configured to represent a matrix.

[0239] Example 6: A processor as in any of Examples 1-5, wherein the data elements are single precision floating point data elements.

[0240] Example 7 The processor of any of Examples 1-5, wherein the data elements are half-precision floating point data elements.

[0241] Example 8 A method comprising: decoding an instruction having fields for an opcode, an identifier of a first source matrix operand, an identifier of a second source matrix operand, and an identifier of a source / destination matrix operand; and executing the decoded instruction to: multiply the identified first source matrix operand by the identified second source matrix operand; add the result of the multiplication to the identified source / destination matrix operand; and store the result of the addition in the identified source / destination matrix operand.

[0242] Example 9 The method of Example 8, wherein the grid using a fused multiply-accumulator is performed.

[0243] Example 10 The method of any of Examples 8-9, wherein the identified second source matrix operand is stored in a memory.

[0244] Example 11 The method of any of Examples 8-10, wherein the multiplication is performed row by row of the identified first source matrix operand and column by column of the identified second source matrix operand.

[0245] Example 12 The method of any of Examples 8-11, wherein at least one of the operands is a plurality of registers configured to represent a matrix.

[0246] Example 13 The method of any of Examples 8-12, wherein the data elements are single precision floating point data elements.

[0247] Example 14 The method of any of Examples 8-12, wherein the data elements are half-precision floating point data elements.

[0248] Example 15 A non-transitory machine-readable medium storing instructions that cause a processor to perform a method comprising: decoding an instruction having fields for an opcode, an identifier of a first source matrix operand, an identifier of a second source matrix operand, and an identifier of a source / destination matrix operand; and executing the decoded instruction to: multiply the identified first source matrix operand by the identified second source matrix operand; add the result of the multiplication to the identified source / destination matrix operand; and store the result of the addition in the identified source / destination matrix operand.

[0249] Example 16 The non-transitory machine-readable medium of Example 15, wherein the grid using a fused multiply-accumulator is performed.

[0250] Example 17 The non-transitory machine-readable medium of any of Examples 15-16, wherein the identified second source matrix operand is stored in a memory.

[0251] Example 18 A non-transitory machine readable medium as any of Examples 15-17, wherein the multiplication is performed row by row of the identified first source matrix operand and column by column of the identified second source matrix operand.

[0252] Example 19 The non-transitory machine readable medium of any of Examples 15-18, wherein at least one of the operands is a plurality of registers configured to represent a matrix.

[0253] Example 20 The non-transitory machine readable medium of any of Examples 15-19, wherein the data element is a single precision floating point data element.

[0254] Example 21 The non-transitory machine-readable medium of any of Examples 15-19, wherein the data element is a half-precision floating point data element.

[0255] Example 22 A system comprising: a processor; and an accelerator coupled to the processor, the accelerator comprising: a decoding circuit for decoding an instruction having fields for an opcode, an identifier of a first source matrix operand, an identifier of a second source matrix operand, and an identifier of a source / destination matrix operand; and an execution circuit for executing the decoded instruction to: multiply the identified first source matrix operand by the identified second source matrix operand; add the result of the multiplication to the identified source / destination matrix operand; and store the result of the addition in the identified source / destination matrix operand and zero unconfigured columns of the identified source / destination matrix operand.

[0256] Example 23 The system of Example 22, wherein the execution circuit comprises a grid of fused multiply-accumulators.

[0257] Example 24 The system of any of Examples 22-23, wherein at least one of the operands is a plurality of registers configured to represent a matrix.

[0258] C. Negated multiplication and accumulation

[0259] Detailed herein are embodiments of the matrix (slice) negative version multiply-accumulate ("TNMMA") instruction and its execution. The TNNMA instruction is an improvement to the computer itself because it provides support for performing matrix-matrix multiplication and negative version accumulation (subtraction) using a single instruction. Specifically, the execution of the TNNMA instruction causes data from a first source matrix (slice) to be multiplied with data from a second source matrix (slice) and subtracted from data from a destination matrix (slice), and the result of the multiplication-subtraction is stored in the destination matrix (slice). The size of the data values ​​to be stored varies depending on the instruction and slice support. Exemplary sizes include, but are not limited to, 16 bits, 32 bits, 64 bits, 128 bits, 256 bits, etc.

[0260] i. Exemplary Implementation

[0261] Fig.28 An exemplary execution of a TMMMA instruction using a memory source operand is illustrated. The TMMA instruction format includes fields for the following: an opcode, a source / destination matrix (slice) operand (shown as "slice destination"), an identifier of a first source matrix (slice) operand (shown as "first slice source"), an identifier of a second source matrix (slice) operand (shown as "second slice source"), and an identifier of a counter register in some embodiments. In some implementations, fields for registers used in progress tracking are also included when the second source matrix (slice) operand is in memory.

[0262] When one of the sources of the instruction is a memory, the memory is accessed according to a scale-index-base (SIB) type memory addressing operand, which indirectly identifies multiple indexed destination locations in the memory, however, other memory addressing schemes may be used. As described in detail, each "row" of a matrix (slice) source or destination is a group of elements. In the memory, these groups are separated by a "stride" value. As will be described in detail, the stride can be specified using an "index" of the SIB. Depending on the implementation, the stride is from the address corresponding to the initial data element of the group to the initial data element of the subsequent group in the memory, or from the address corresponding to the last data element of the group to the initial data element of the subsequent group in the memory. Typically, strides are used to describe rows, however, that is not necessarily true.

[0263] One or both of the sources of the instruction is a matrix (slice) stored in multiple registers or stored in a matrix (slice) data structure. The source is encoded in the instruction as if it were a single register. When there are two such matrices, both are encoded as if they were both single registers.

[0264] The final source is the destination matrix (slice) 2809.

[0265] Although the illustrated matrix (tile) is shown as a 2x2 matrix, this is an arbitrary illustration to aid understanding, and different matrix (tile) sizes may be used. The TNMMA operation is source 1 * source 2 + destination. Thus, (NxK)-(NxM)*(MxK) matrices are supported.

[0266] Matrix (slice) sources 2801 and 2803 and destination matrix (slice) 2809 are provided to execution circuit 2805 for TNNMA operation. In some embodiments, the operation is performed position by position of data element of the matrix (slice) using grid 2807 of FMA. Grid 2807 of FMA has been described previously. In some implementations, one or more of matrix (slice) source 2801 and destination matrix (slice) 2809 are stored in grid 2807 of FMA.

[0267] Execution circuit 2805 performs TNNMA by performing multiplication on the source data element by data element (using row x column matrix multiplication) and subtracting from the corresponding data element position of the destination matrix. The result of TNNMA is stored in the corresponding data element position of the destination matrix as shown in 2811.

[0268] because Fig.28 is simplified, so it does not show the use of a counter register that acts as a progress tracker. As each "row" of destinations is written, the counter is updated. This allows the TNNMA operation to be restarted if necessary by using the counter to determine where the operation stopped. Also note that in some embodiments, the counter function is not utilized.

[0269] ii.(Various) Example Formats

[0270] An embodiment of the format for the TMMMA instruction is TMMMA{S,H}TMM1,TMM2,TMM3. In some embodiments, TNMMAP{S,H} is the opcode mnemonic of the instruction, where S,H represent single precision (S) floating point data elements and half precision floating point data elements. TMM1 is a field for the identifier of the source / destination matrix (slice), and TMM3 is a field for the identifier of the second source / matrix (slice), and TMM2 is a field for the identifier of the first source matrix (slice). In some embodiments, the TMM2 identifier is a field R / M value (such as, 3146), TMM3 is a field VVVV 3120, and the source / destination matrix (slice) identifier is field 3144. Note that if the counter is not used, SRC3 is not included in the instruction format.

[0271] iii. Exemplary Method(s) of Implementation

[0272] Fig.29 An embodiment of a method performed by a processor to process a TNNMA instruction is illustrated.

[0273] At 2901, an instruction is fetched. For example, a TNNMA instruction is fetched. An embodiment of the TNNMA instruction includes fields for: an opcode, a source / destination matrix (slice) operand identifier, a first source matrix (slice) operand identifier, and a second source matrix (slice) operand identifier (e.g., stored in a memory, or accessed as a register). In some embodiments, a register for storing a counter value identifier is also included.

[0274] At 2903, the fetched instruction is decoded. For example, the fetched TNNMA instruction is decoded by a decoding circuit such as described in detail herein.

[0275] At 2905, data values ​​associated with the sources are retrieved and decoded instructions are dispatched (as needed).

[0276] At 2907, the decoded instruction is executed by an execution circuit (hardware) such as described in detail herein. For a TNNMA instruction, the execution will cause the execution circuit to perform: 1) a matrix multiplication of the identified first source matrix (slice) operand (from the accessed memory or register) and the identified second source matrix (slice); and 2) subtracting the result of the matrix multiplication from the corresponding data element positions of the identified destination matrix (slice) operand. In some embodiments, the data element positions of the identified source / destination matrix (slice) operands that are not subject to the subtraction are zeroed (unconfigured columns).

[0277] In some embodiments, at 2909, the instruction is committed or retired.

[0278] Fig.30 A more detailed description of the execution of a TNNMA instruction using register addressing is illustrated. Typically, this is performed by an execution circuit such as that detailed above.

[0279] At 3001, the values ​​in the first, second and third counters are set. For example, startK and startM are set. Typically this is done during configuration, but at the first instance of this instruction, these values ​​are usually set to 0.

[0280] At 3003, a determination is made whether the first counter value (eg, startM) is less than the number of configured rows of the destination. If not, at 3005, the instruction is complete and all unconfigured rows are reset to zero.

[0281] If yes, then the row from the destination is written into the temporary location at 3007. The row is at position [first counter] (or the value of startM).

[0282] At 3009, a determination is made whether the second counter value (eg, startK) is less than the number of configured columns of the first source.

[0283] If not, then at 3011, the row at the temporary location is written into the corresponding row location at the destination. Typically, the unconfigured columns of that row are also zeroed. At 3017, the second and third counters are reset, and the first counter is incremented. Essentially, the next row is set to start processing again at 3003.

[0284] If so, there is potentially more work to do for that row. At 3012, a determination is made whether the third counter value (e.g., n) is less than the number of configured columns of the destination. If so, at 3013, a multiplication of the data element of the first source at position row [first counter, second counter] by the data element of the second source at position row [second counter, third counter] is performed, and the result of that multiplication is subtracted from the temporary value at position temporary [third counter]. At 3015, the third counter is incremented to move along the loop and the determination at 3012 is made again.

[0285] If not, then at 3016, the second counter is incremented and the decision at 3009 is made again.

[0286] iv. Example

[0287] Example 1 A processor comprises: a decoding circuit for decoding an instruction having fields for an opcode, an identifier of a first source matrix operand, an identifier of a second source matrix operand, and an identifier of a source / destination matrix operand; and an execution circuit for executing the decoded instruction to: multiply the identified first source matrix operand by the identified second source matrix operand; subtract the result of the multiplication from the identified source / destination matrix operand; and store the result of the subtraction in the identified source / destination matrix operand.

[0288] Example 2 A processor as in Example 1, wherein the execution circuit comprises a grid of fused multiply-accumulators.

[0289] Example 3 The processor of any of Examples 1-2, wherein the identified second source matrix operand is stored in a memory.

[0290] Example 4 The processor of any of Examples 1-3, wherein the multiplication is performed row by row of the identified first source matrix operand and column by column of the identified second source matrix operand.

[0291] Example 5 The processor of any of Examples 1-4, wherein at least one of the operands is a plurality of registers configured to represent a matrix.

[0292] Example 6: A processor as in any of Examples 1-5, wherein the data elements are single precision floating point data elements.

[0293] Example 7 The processor of any of Examples 1-5, wherein the data elements are half-precision floating point data elements.

[0294] Example 8 A method comprising: decoding an instruction having fields for an opcode, an identifier of a first source matrix operand, an identifier of a second source matrix operand, and an identifier of a source / destination matrix operand; and executing the decoded instruction to: multiply the identified first source matrix operand by the identified second source matrix operand; subtract the result of the multiplication from the identified source / destination matrix operand; and store the result of the subtraction in the identified source / destination matrix operand.

[0295] Example 9 The method of Example 8, wherein the grid using a fused multiply-accumulator is performed.

[0296] Example 10 The method of any of Examples 8-9, wherein the identified second source matrix operand is stored in a memory.

[0297] Example 11 The method of any of Examples 8-10, wherein the multiplication is performed row by row of the identified first source matrix operand and column by column of the identified second source matrix operand.

[0298] Example 12 The method of any of Examples 8-11, wherein at least one of the operands is a plurality of registers configured to represent a matrix.

[0299] Example 13 The method of any of Examples 8-12, wherein the data elements are single precision floating point data elements.

[0300] Example 14 The method of any of Examples 8-12, wherein the data elements are half-precision floating point data elements.

[0301] Example 15 A non-transitory machine-readable medium storing instructions that cause a processor to perform a method comprising: decoding an instruction having fields for an opcode, an identifier of a first source matrix operand, an identifier of a second source matrix operand, and an identifier of a source / destination matrix operand; and executing the decoded instruction to: multiply the identified first source matrix operand by the identified second source matrix operand; subtract the result of the multiplication from the identified source / destination matrix operand; and store the result of the subtraction in the identified source / destination matrix operand.

[0302] Example 16 The non-transitory machine-readable medium of Example 15, wherein the grid using a fused multiply-accumulator is performed.

[0303] Example 17 The non-transitory machine-readable medium of any of Examples 15-16, wherein the identified second source matrix operand is stored in a memory.

[0304] Example 18 A non-transitory machine readable medium as any of Examples 15-17, wherein the multiplication is performed row by row of the identified first source matrix operand and column by column of the identified second source matrix operand.

[0305] Example 19 The non-transitory machine readable medium of any of Examples 15-18, wherein at least one of the operands is a plurality of registers configured to represent a matrix.

[0306] Example 20 The non-transitory machine readable medium of any of Examples 15-19, wherein the data element is a single precision floating point data element.

[0307] Example 21 The non-transitory machine-readable medium of any of Examples 15-19, wherein the data element is a half-precision floating point data element.

[0308] Example 22 A system comprising: a processor; and an accelerator coupled to the processor, the accelerator comprising: a decoding circuit for decoding an instruction having fields for an opcode, an identifier of a first source matrix operand, an identifier of a second source matrix operand, and an identifier of a source / destination matrix operand; and an execution circuit for executing the decoded instruction to: multiply the identified first source matrix operand by the identified second source matrix operand; subtract the result of the multiplication from the identified source / destination matrix operand; and store the result of the subtraction in the identified source / destination matrix operand and zero unconfigured columns of the identified source / destination matrix operand.

[0309] Example 23 The system of Example 22, wherein the execution circuit comprises a grid of fused multiply-accumulators.

[0310] Example 24 The system of any of Examples 22-23, wherein at least one of the operands is a plurality of registers configured to represent a matrix.

[0311] V. Detailed Exemplary Systems, Processors, and Simulations

[0312] Detailed herein are examples of hardware, software, etc. for executing the instructions described above. For example, the following description details various aspects of instruction execution, including various pipeline stages such as fetch, decode, schedule, execute, retire, etc.

[0313] The instruction set includes one or more instruction formats. A given instruction format defines various fields (number of bits, position of bits) to specify the operation to be performed (opcode) and (multiple) operands on which the operation will be performed, etc. Some instruction formats are further decomposed by the definition of instruction templates (or sub-formats). For example, the instruction templates of a given instruction format can be defined as having different subsets of the fields of the instruction format (the included fields are usually in the same order, but at least some fields have different bit positions because fewer fields are included), and / or defined as having given fields interpreted in different ways. Thus, each instruction of the ISA is expressed using a given instruction format (and if defined, according to a given one of the instruction templates of the instruction format) and includes fields for specifying operations and operands. For example, an exemplary ADD (addition) instruction has a specific opcode and instruction format, and the specific instruction format includes an opcode field for specifying the opcode and an operand field for selecting operands (source 1 / destination and source 2); and the ADD instruction appears in the instruction stream so that the operand field has specific content for selecting specific operands.

[0314] A. Exemplary Instruction Format

[0315] Embodiments of the (multiple) instructions described herein can be embodied in different formats. In addition, exemplary systems, architectures, and pipelines are described in detail below. Embodiments of the (multiple) instructions can be executed on such systems, architectures, and pipelines, but are not limited to those systems, architectures, and pipelines described in detail.

[0316] VEX instruction format

[0317] VEX encoding allows instructions to have more than two operands and allows SIMD vector registers to be longer than 128 bits. The use of the VEX prefix provides a three-operand (or more operand) syntax. For example, the previous two-operand instructions performed operations such as A=A+B that overwrote the source operands. The use of the VEX prefix enables operands to perform non-destructive operations such as A=B+C.

[0318] Fig.31A The figure shows an exemplary instruction format, including VEX prefix 3102, real opcode field 3130, Mod R / M byte 3140, SIB byte 3150, displacement field 3162 and IMM8 3172. Fig.31B The figure shows the Fig.31A Which fields constitute the complete opcode field 3174 and the basic operation field 3141. Fig. 31C The figure shows the Fig.31A Which fields constitute the register index field 3144.

[0319] The VEX prefix (bytes 0-2) 3102 is encoded in three-byte form. The first byte is the format field 3190 (VEX byte 0, bits [7:0]), which contains an explicit C4 byte value (a unique value used to distinguish the C4 instruction format). The second-third bytes (VEX bytes 1-2) include several bit fields that provide special capabilities. Specifically, the REX field 3105 (VEX byte 1, bits [7-5]) consists of a VEX.R bit field (VEX byte 1, bits [7]-R), a VEX.X bit field (VEX byte 1, bits [6]-X), and a VEX.B bit field (VEX byte 1, bits [5]-B). The other fields of these instructions encode the lower three bits of the register index (rrr, xxx, and bbb) as known in the art, so that Rrrr, Xxxx, and Bbbb can be formed by adding VEX.R, VEX.X, and VEX.B. The opcode map field 3115 (VEX byte 1, bits [4:0]–mmmmm) includes content for encoding the implied leading opcode byte. W field 3164 (VEX byte 2, bits [7]–W) - represented by the notation VEX.W, and provides different functions depending on the instruction. The effects of VEX.vvvv 3120 (VEX byte 2, bits [6:3]-vvvv) may include the following: 1) VEX.vvvv encodes the first source register operand specified in inverted (1's complement) form and is valid for instructions with two or more source operands; 2) VEX.vvvv encodes the destination register operand specified in 1's complement form for certain vector displacements; or 3) VEX.vvvv does not encode any operand, this field is reserved and should contain 1111b. If the VEX.L 3168 size field (VEX byte 2, bit [2]-L) = 0, it indicates a 318-bit vector; if VEX.L = 1, it indicates a 256-bit vector. The prefix encoding field 3125 (VEX byte 2, bits [1:0]-pp) provides additional bits for the base operation field 3141.

[0320] The real opcode field 3130 (byte 3) is also called the opcode byte. A portion of the opcode is specified in this field.

[0321] The MOD R / M field 3140 (byte 4) includes a MOD field 3142 (bits [7-6]), a Reg field 3144 (bits [5-3]), and an R / M field 3146 (bits [2-0]). The role of the Reg field 3144 may include the following: encoding a destination register operand or a source register operand (Rrrr's rrr); or being treated as an opcode extension and not used to encode any instruction operand. The role of the R / M field 3146 may include the following: encoding an instruction operand that references a memory address; or encoding a destination register operand or a source register operand.

[0322] Scale, Index, Base (SIB) - The contents of the scale field 3150 (byte 5) include SS 3152 (bits [7-6]), which is used for memory address generation. The contents of SIB.xxx 3154 (bits [5-3]) and SIB.bbb 3156 (bits [2-0]) have been previously mentioned for register indexes Xxxx and Bbbb.

[0323] The displacement field 3162 and the immediate field (IMM8) 3172 contain data.

[0324] B. Exemplary Register Architecture

[0325] Fig.32 is a block diagram of a register architecture 3200 according to one embodiment of the present invention. In the illustrated embodiment, there are 32 512-bit wide vector registers 3210; these registers are referenced as zmm0 through zmm31. The lower order 256 bits of the lower 35 zmm registers are overlaid on registers ymm0-15. The lower order 128 bits of the lower 35 zmm registers (the lower order 128 bits of the ymm registers) are overlaid on registers xmm0-15.

[0326] General purpose registers 3225 - In the embodiment shown, there are sixteen 64-bit general purpose registers that are used with existing x86 addressing modes to address memory operands. These registers are referenced by the names RAX, RBX, RCX, RDX, RBP, RSI, RDI, RSP, and R8 through R15.

[0327] A scalar floating point stack register file (x87 stack) 3245 on which an MMX packed integer flat register file 3250 is overlaid - in the illustrated embodiment, the x87 stack is an eight-element stack used to perform scalar floating point operations on 32 / 64 / 80-bit floating point data using the x87 instruction set extension; while MMX registers are used to perform operations on 64-bit packed integer data, as well as to hold operands for some operations performed between MMX and XMM registers.

[0328] In some embodiments, an overlay structure above the physical registers is used to support the slice 3220. For example, a slice may utilize 16 1024-bit registers, 32 512-bit registers, etc., depending on the implementation.

[0329] Alternative embodiments of the present invention may use wider or narrower registers. In addition, alternative embodiments of the present invention may use more, fewer or different register files and registers.

[0330] Exemplary Core Architectures, Processors, and Computer Architectures

[0331] Processor cores can be implemented in different ways, for different purposes, and in different processors. For example, implementations of such cores may include: 1) a general-purpose in-order core intended for general-purpose computing; 2) a high-performance general-purpose out-of-order core intended for general-purpose computing; 3) a special-purpose core intended primarily for graphics and / or scientific (throughput) computing. Implementations of different processors may include: 1) a CPU that includes one or more general-purpose in-order cores intended for general-purpose computing and / or one or more general-purpose out-of-order cores intended for general-purpose computing; and 2) a coprocessor that includes one or more special-purpose cores intended primarily for graphics and / or scientific (throughput). Such different processors lead to different computer system architectures, which may include: 1) a coprocessor on a separate chip from the CPU; 2) a coprocessor in the same package as the CPU but on a separate die; 3) a coprocessor on the same die as the CPU (in which case such a coprocessor is sometimes referred to as dedicated logic or as a dedicated core, such as integrated graphics and / or scientific (throughput) logic); and 4) a system on a chip, which may include the described CPU (sometimes referred to as (multiple) application cores or (multiple) application processors), the coprocessor described above, and additional functionality on the same die. An exemplary core architecture is then described, followed by an exemplary processor and computer architecture. Circuits (units) including exemplary cores, processors, etc. are described in detail herein.

[0332] C. Exemplary Core Architecture

[0333] In-order and out-of-order core block diagram

[0334] Fig.33A is a block diagram illustrating an exemplary in-order pipeline and an exemplary register-renaming out-of-order issue / execution pipeline according to various embodiments of the invention. Fig.33B is a block diagram illustrating an exemplary embodiment of an in-order architecture core and an exemplary register renaming, out-of-order issue / execution architecture core to be included in a processor according to embodiments of the present invention. Figure 33A-Figure 33BThe solid line boxes in illustrate the in-order pipeline and in-order core, while the optional addition of dashed line boxes illustrates the register renaming, out-of-order issue / execution pipeline and core. Considering that the in-order aspect is a subset of the out-of-order aspect, the out-of-order aspect will be described.

[0335] exist Fig.33A , the processor pipeline 3300 includes a fetch stage 3302, a length decode stage 3304, a decode stage 3306, an allocation stage 3308, a rename stage 3310, a scheduling (also known as dispatch or issue) stage 3312, a register read / memory read stage 3314, an execution stage 3316, a write back / memory write stage 3318, an exception handling stage 3322, and a commit stage 3324.

[0336] Fig.33B A processor core 3390 is shown, which includes a front end unit 3330, which is coupled to an execution engine unit 3350, and both the front end unit 3330 and the execution engine unit 3350 are coupled to a memory unit 3370. The core 3390 can be a reduced instruction set computing (RISC) core, a complex instruction set computing (CISC) core, a very long instruction word (VLIW) core, or a hybrid or alternative core type. As another option, the core 3390 can be a special-purpose core, such as, for example, a network or communication core, a compression engine, a coprocessor core, a general-purpose computing graphics processing unit (GPGPU) core, a graphics core, and the like.

[0337] The front end unit 3330 includes a branch prediction unit 3332, which is coupled to an instruction cache unit 3334, which is coupled to an instruction translation lookaside buffer (TLB) 3336, which is coupled to an instruction fetch unit 3338, which is coupled to a decode unit 3340. The decode unit 3340 (or decoder) can decode the instruction and generate one or more micro-operations, microcode entry points, microinstructions, other instructions, or other control signals decoded from the original instruction, or otherwise reflecting the original instruction, or derived from the original instruction as output. The decode unit 3340 can be implemented using a variety of different mechanisms. Examples of suitable mechanisms include, but are not limited to, lookup tables, hardware implementations, programmable logic arrays (PLA), microcode read-only memories (ROMs), etc. In one embodiment, the core 3390 includes a microcode ROM or other medium (e.g., in the decode unit 3340, or otherwise within the front end unit 3330) that stores microcode for certain macroinstructions. The decode unit 3340 is coupled to the rename / allocator unit 3352 in the execution engine unit 3350 .

[0338] The execution engine unit 3350 includes a rename / allocator unit 3352 coupled to a retirement unit 3354 and a set 3356 of one or more scheduler units. The scheduler unit(s) 3356 represent any number of different schedulers, including reservation stations, central instruction windows, etc. The scheduler unit(s) 3356 are coupled to physical register file(s) units 3358. Each of the physical register file(s) units 3358 represents one or more physical register files, where different physical register files store one or more different data types, such as scalar integers, scalar floating points, packed integers, packed floating points, vector integers, vector floating points, status (e.g., an instruction pointer that is the address of the next instruction to be executed), etc. In one embodiment, the physical register file(s) units 3358 include vector register units and scalar register units. These register units may provide architectural vector registers, vector mask registers, and general purpose registers. The physical register file(s) units 3358 are overlapped by the retirement unit 3354 to illustrate various ways in which register renaming and out-of-order execution may be implemented (e.g., using reorder buffer(s) and retirement register file(s); using future files(s), history buffer(s), retirement register file(s); using register maps and register pools, etc.). The retirement unit 3354 and the physical register file(s) units 3358 are coupled to the execution cluster(s) 3360. The execution cluster(s) 3360 include a set of one or more execution units 3362 and a set of one or more memory access units 3364. The execution unit 3362 may perform various operations (e.g., shifts, additions, subtractions, multiplications) and may perform on various data types (e.g., scalar floating point, packed integer, packed floating point, vector integer, vector floating point). While some embodiments may include multiple execution units dedicated to a particular function or set of functions, other embodiments may include only one execution unit or multiple execution units that all perform all functions. The (multiple) scheduler units 3356, (multiple) physical register file units 3358, and (multiple) execution clusters 3360 are shown as possibly being multiple, as certain embodiments create separate pipelines for certain types of data / operations (e.g., scalar integer pipelines, scalar floating point / packed integer / packed floating point / vector integer / vector floating point pipelines, and / or memory access pipelines each with its own scheduler unit, (multiple) physical register file units, and / or execution clusters - and in the case of separate memory access pipelines, certain embodiments are implemented in which only the execution cluster of that pipeline has (multiple) memory access units 3364). It should also be understood that where separate pipelines are used, one or more of these pipelines may be out-of-order issue / execution, and the rest may be in-order.

[0339] A set of memory access units 3364 is coupled to a memory unit 3370, which includes a data TLB unit 3372, which is coupled to a data cache unit 3374, which is coupled to a second level (L2) cache unit 3376. In an exemplary embodiment, the memory access unit 3364 may include a load unit, a store address unit, and a store data unit, each of which is coupled to the data TLB unit 3372 in the memory unit 3370. The instruction cache unit 3334 is also coupled to a second level (L2) cache unit 3376 in the memory unit 3370. The L2 cache unit 3376 is coupled to one or more other levels of cache and ultimately to the main memory.

[0340] As an example, an exemplary register renaming out-of-order issue / execution core architecture may implement pipeline 3300 as follows: 1) instruction fetch 3338 executes fetch stage 3302 and length decoding stage 3304; 2) decode unit 3340 executes decode stage 3306; 3) rename / allocator unit 3352 executes allocation stage 3308 and rename stage 3310; 4) (multiple) scheduler units 3356 execute scheduling stage 3312; 5) (multiple) physical register file units 3358 and memory units 3370 execute register read / memory read stage 3314; execution cluster 3360 executes execution stage 3316; 6) memory unit 3370 and (multiple) physical register file units 3358 execute write back / memory write stage 3318; 7) each unit may be involved in exception handling stage 3322; and 8) retirement unit 3354 and (multiple) physical register file units 3358 execute commit stage 3324.

[0341] Core 3390 may support one or more instruction sets (e.g., the x86 instruction set (with some extensions that have been added with newer versions); the MIPS instruction set of MIPS Technologies, Inc. of Sunnyvale, California; the ARM instruction set of ARM Holdings, Inc. of Sunnyvale, California (with optional additional extensions such as NEON)), including the instruction(s) described herein. In one embodiment, core 3390 includes logic to support packed data instruction set extensions (e.g., AVX1, AVX2), thereby allowing operations used by many multimedia applications to be performed using packed data.

[0342] It should be understood that a core may support multithreading (executing a set of two or more operations or threads in parallel), and that this multithreading may be accomplished in a variety of ways, including time-division multithreading, simultaneous multithreading (where a single physical core provides a logical core for each of the threads that the physical core is simultaneously multithreading), or a combination thereof (e.g., time-division fetching and decoding and subsequent operations such as Hyper-Threading (Simultaneous Multithreading).

[0343] Although register renaming is described in the context of out-of-order execution, it should be understood that register renaming can be used in an in-order architecture. Although the illustrated embodiment of the processor also includes separate instruction and data cache units 3334 / 3374 and a shared L2 cache unit 3376, alternative embodiments may have a single internal cache for both instructions and data, such as, for example, a first level (L1) internal cache or multiple levels of internal cache. In some embodiments, the system may include a combination of internal caches and external caches external to the core and / or processor. Alternatively, all caches may be external to the core and / or processor.

[0344] Specific Exemplary In-Order Core Architecture

[0345] Figure 34A-Figure 34B A block diagram illustrating a more specific exemplary in-order core architecture, which would be one logic block among several logic blocks in a chip (including other cores of the same type and / or different types). Depending on the application, the logic block communicates with some fixed function logic, memory I / O interfaces, and other necessary I / O logic via a high bandwidth interconnect network (e.g., a ring network).

[0346] Fig.34A 3404 and its local subset 3404 of the second level (L2) cache according to an embodiment of the present invention. In one embodiment, the instruction decoder 3400 supports the x86 instruction set with the compressed data instruction set extension. The L1 cache 3406 allows low latency access to the cache memory into the scalar and vector units. Although in one embodiment (to simplify the design), the scalar unit 3408 and the vector unit 3410 use separate register sets (scalar registers 3412 and vector registers 3414, respectively), and data transferred between these registers is written to the memory and then read back from the first level (L1) cache 3406, alternative embodiments of the present invention may use different approaches (e.g., using a single register set or including a communication path that allows data to be transferred between the two register files without being written and read back).

[0347] The local subset 3404 of L2 cache is part of the global L2 cache, which is divided into multiple separate local subsets, one local subset for each processor core. Each processor core has a direct access path to its own local subset 3404 of L2 cache. The data read by the processor core is stored in its L2 cache subset 3404, and can be quickly accessed in parallel with other processor cores accessing its own local L2 cache subset. The data written by the processor core is stored in its own L2 cache subset 3404, and is dumped and cleared from other subsets when necessary. The ring network ensures the consistency of shared data. The ring network is bidirectional to allow agents such as processor cores, L2 caches and other logic blocks to communicate with each other within the chip. In some embodiments, each ring data path is 1024 bits wide for each direction.

[0348] Fig.34B According to an embodiment of the present invention Fig.34A An expanded view of a portion of a processor core in FIG. Fig.34B Includes the L1 data cache 3406A portion of the L1 cache 3404, as well as more details about the vector unit 3410 and vector registers 3414. Specifically, the vector unit 3410 is a 35-wide vector processing unit (VPU) (see 16-wide ALU 3428) that executes one or more of integer, single-precision floating point, and double-precision floating point instructions. The VPU supports blending of register inputs through blending unit 3420, supports numerical conversion through numerical conversion units 3422A-B, and supports copying of memory inputs through copy unit 3424.

[0349] Processor with integrated memory controller and graphics device

[0350] Fig.35 is a block diagram of a processor 3500 that may have more than one core, may have an integrated memory controller, and may have integrated graphics according to an embodiment of the present invention. Fig.35 The solid line box in the figure illustrates a processor 3500 having a single core 3502A, a system agent 3510, a set 3516 of one or more bus controller units, while the optional addition of the dashed line box illustrates an alternative processor 3500 having multiple cores 3502A-N, a set 3514 of one or more integrated memory controller units in the system agent unit 3510, and dedicated logic 3508.

[0351] Thus, different implementations of processor 3500 may include: 1) a CPU, where the dedicated logic 3508 is integrated graphics and / or scientific (throughput) logic (which may include one or more cores), and cores 3502A-N are one or more general purpose cores (e.g., general purpose in-order cores, general purpose out-of-order cores, combinations of the two); 2) a coprocessor, where cores 3502A-N are a large number of dedicated cores intended primarily for graphics and / or scientific (throughput); and 3) a coprocessor, where cores 3502A-N are a large number of general purpose in-order cores. Thus, processor 3500 may be a general purpose processor, a coprocessor, or a dedicated processor, such as, for example, a network or communication processor, a compression engine, a graphics processor, a GPGPU (general purpose graphics processing unit), a high throughput integrated many-core (MIC) coprocessor (including 30 or more cores), an embedded processor, and the like. The processor may be implemented on one or more chips. Processor 3500 may be part of one or more substrates, and / or may be implemented on one or more substrates using any of a variety of process technologies (such as, for example, BiCMOS, CMOS, or NMOS).

[0352] The memory hierarchy includes one or more levels of cache within the cores 3504A-N, a set of one or more shared cache units 3506, and external memory (not shown) coupled to a set of integrated memory controller units 3514. The set of shared cache units 3506 may include one or more intermediate levels of cache, such as a second level (L2), a third level (L3), a fourth level (L4), or other levels of cache, a last level cache (LLC), and / or a combination of the above. Although in one embodiment, a ring-based interconnect unit 3512 interconnects the integrated graphics logic 3508, the set of shared cache units 3506, and the system agent unit 3510 / (multiple) integrated memory controller units 3514, alternative embodiments may use any number of well-known techniques to interconnect such units. In one embodiment, coherency is maintained between the one or more cache units 3506 and the cores 3502A-N.

[0353] In some embodiments, one or more of the cores 3502A-N can be multithreaded. The system agent 3510 includes those components that coordinate and operate the cores 3502A-N. The system agent unit 3510 may include, for example, a power control unit (PCU) and a display unit. The PCU may be the logic and components required to regulate the power state of the cores 3502A-N and the integrated graphics logic 3508, or may include these logic and components. The display unit is used to drive one or more externally connected displays.

[0354] The cores 3502A-N may be homogeneous or heterogeneous in terms of the architectural instruction set; that is, two or more of the cores 3502A-N may be capable of executing the same instruction set, while other cores may be capable of executing only a subset of the instruction set or a different instruction set.

[0355] D. Exemplary Computer Architecture

[0356] Figure 36-Figure 39 is a block diagram of an exemplary computer architecture. Other system designs and configurations known in the art for laptops, desktops, handheld PCs, personal digital assistants, engineering workstations, servers, network devices, network hubs, switches, embedded processors, digital signal processors (DSPs), graphics devices, video game devices, set-top boxes, microcontrollers, cellular phones, portable media players, handheld devices, and various other electronic devices are also suitable. In general, a wide variety of systems or electronic devices that can contain processors and / or other execution logic as disclosed herein are generally suitable.

[0357] Reference now Fig.36 , shown is a block diagram of a system 3600 according to one embodiment of the present invention. The system 3600 may include one or more processors 3610, 3615 coupled to a controller hub 3620. In one embodiment, the controller hub 3620 includes a graphics memory controller hub (GMCH) 3690 and an input / output hub (IOH) 3650 (which may be on separate chips); the GMCH 3690 includes memory and a graphics controller to which a memory 3640 and a coprocessor 3645 are coupled; and the IOH 3650 couples an input / output (I / O) device 3660 to the GMCH 3690. Alternatively, one or both of the memory and the graphics controller are integrated within a processor (as described herein), the memory 3640 and the coprocessor 3645 are directly coupled to the processor 3610, and the controller hub 3620 and the IOH 3650 are in a single chip.

[0358] The optional addition of processor 3615 is Fig.36 Each processor 3610 , 3615 may include one or more of the processing cores described herein and may be a version of processor 3500 .

[0359] The memory 3640 may be, for example, dynamic random access memory (DRAM), phase change memory (PCM), or a combination of the two. For at least one embodiment, the controller hub 3620 communicates with the processor(s) 3610, 3615 via a multi-drop bus such as a front-side bus (FSB), a point-to-point interface, or a similar connection 3695.

[0360] In one embodiment, coprocessor 3645 is a special purpose processor such as, for example, a high throughput MIC processor, a network or communication processor, a compression engine, a graphics processor, a GPGPU, an embedded processor, etc. In one embodiment, controller hub 3620 may include an integrated graphics accelerator.

[0361] There may be various differences between the physical resources 3610 , 36155 in terms of a range of quality metrics including architectural, microarchitectural, thermal, and power consumption characteristics.

[0362] In one embodiment, processor 3610 executes instructions that control general types of data processing operations. Embedded within these instructions may be coprocessor instructions. Processor 3610 recognizes these coprocessor instructions as being of a type that should be executed by an attached coprocessor 3645. Therefore, processor 3610 issues these coprocessor instructions (or control signals representing coprocessor instructions) to coprocessor 3645 on a coprocessor bus or other interconnect. Coprocessor(s) 3645 accept and execute the received coprocessor instructions.

[0363] See now Fig.37 , shown is a block diagram of a first more specific exemplary system 3700 according to an embodiment of the present invention. Fig.37 As shown in FIG. 3 , multiprocessor system 3700 is a point-to-point interconnect system and includes a first processor 3770 and a second processor 3780 coupled via a point-to-point interconnect 3750. Each of processors 3770 and 3780 may be a version of processor 3500. In one embodiment of the present invention, processors 3770 and 3780 are processors 3610 and 3615, respectively, and coprocessor 3738 is coprocessor 3645. In another embodiment, processors 3770 and 3780 are processor 3610 and coprocessor 3645, respectively.

[0364] Processors 3770 and 3780 are shown as including integrated memory controller (IMC) units 3772 and 3782, respectively. Processor 3770 also includes point-to-point (PP) interfaces 3776 and 3778 as part of its bus controller unit; similarly, second processor 3780 includes PP interfaces 3786 and 3788. Processors 3770, 3780 can exchange information via PP interface 3750 using point-to-point (PP) interface circuits 3778, 3788. Fig.37 As shown in FIG. 4 , IMCs 3772 and 3782 couple the processors to respective memories, namely, memory 3732 and memory 3734, which may be portions of main memory locally attached to the respective processors.

[0365] The processors 3770, 3780 may each exchange information with the chipset 3790 via respective PP interfaces 3752, 3754 using point-to-point interface circuits 3776, 3794, 3786, 3798. The chipset 3790 may optionally exchange information with a coprocessor 3738 via a high performance interface 3792. In one embodiment, the coprocessor 3738 is a special purpose processor such as, for example, a high throughput MIC processor, a network or communication processor, a compression engine, a graphics processor, a GPGPU, an embedded processor, or the like.

[0366] A shared cache (not shown) may be included in either processor, or external to both processors but connected to the processors via the PP interconnect, so that if the processors are placed in a low power mode, local cache information of either or both processors may be stored in the shared cache.

[0367] Chipset 3790 may be coupled to first bus 3716 via interface 3796. In one embodiment, first bus 3716 may be a peripheral component interconnect (PCI) bus or a bus such as a PCI Express bus or another I / O interconnect bus, but the scope of the present invention is not limited in this regard.

[0368] like Fig.37 As shown in , various I / O devices 3714 may be coupled to the first bus 3716 along with a bus bridge 3718 that couples the first bus 3716 to a second bus 3720. In one embodiment, one or more additional processors 3715 such as a coprocessor, a high throughput MIC processor, a GPGPU, an accelerator (such as, for example, a graphics accelerator or a digital signal processing (DSP) unit), a field programmable gate array, or any other processor are coupled to the first bus 3716. In one embodiment, the second bus 3720 may be a low pin count (LPC) bus. In one embodiment, various devices may be coupled to the second bus 3720 including, for example, a keyboard and / or mouse 3722, communication devices 3727, and a storage unit 3728 such as a disk drive or other mass storage device which may include instructions / code and data 3730. Additionally, an audio I / O 3724 may be coupled to the second bus 3716. Note that other architectures are possible. For example, instead of Fig.37 Instead of a point-to-point architecture, the system may implement a multi-drop bus or other such architecture.

[0369] Reference now Fig.38 , shown is a block diagram of a second more specific exemplary system 3800 according to an embodiment of the present invention. Fig.37 and38 Similar elements in the same reference numerals are used, and Fig.38 Omitted Fig.37 to avoid confusion Fig.38 other aspects.

[0370] Fig.38 The illustrated processors 3770, 3780 may include integrated memory and I / O control logic ("CL") 3872 and 3882, respectively. Thus, the CL 3872, 3882 includes an integrated memory controller unit and includes I / O control logic. Fig.38 The illustration shows that not only the memories 3732, 3734 are coupled to the CL 3872, 3882, but also the I / O devices 3814 are coupled to the control logic 3772, 3782. The legacy I / O devices 3815 are coupled to the chipset 3790.

[0371] Reference now Fig.39 , shown is a block diagram of a SoC 3900 according to an embodiment of the present invention. Fig.35 Similar elements in the same figure are marked with similar reference numerals. In addition, the dashed boxes are optional features on more advanced SoCs. Fig.39 , the interconnect unit(s) 3902 are coupled to: an application processor 3910, which includes a set of one or more cores 392A-N, cache units 3504A-N, and shared cache unit(s) 3506; a system agent unit 3510; a bus controller unit(s) 3516; an integrated memory controller unit(s) 3514; a set of one or more coprocessors 3920, which may include integrated graphics logic, an image processor, an audio processor, and a video processor; a static random access memory (SRAM) unit 3930; a direct memory access (DMA) unit 3932; and a display unit 3940 for coupling to one or more external displays. In one embodiment, the coprocessor(s) 3920 include a special purpose processor, such as, for example, a network or communication processor, a compression engine, a GPGPU, a high throughput MIC processor, or an embedded processor, or the like.

[0372] The various embodiments of the mechanisms disclosed herein may be implemented in hardware, software, firmware, or a combination of such implementations. Embodiments of the present invention may be implemented as a computer program or program code executed on a programmable system, the programmable system comprising at least one processor, a storage system (including volatile and non-volatile memory and / or storage elements), at least one input device, and at least one output device.

[0373] Program code (such as Fig.37The code 3730 illustrated in FIG. 37 is applied to input instructions to perform the functions described herein and generate output information. The output information can be applied to one or more output devices in a known manner. For the purposes of this application, a processing system includes any system having a processor, such as, for example, a digital signal processor (DSP), a microcontroller, an application specific integrated circuit (ASIC), or a microprocessor.

[0374] The program code can be implemented with a high-level process-oriented programming language or an object-oriented programming language to communicate with the processing system. If necessary, the program code can also be implemented with assembly language or machine language. In fact, the mechanism described herein is not limited to the scope of any specific programming language. In any case, the language can be a compiled language or an interpreted language.

[0375] One or more aspects of at least one embodiment may be implemented by representative instructions stored on a machine-readable medium, which represent various logic in a processor, which when read by a machine causes the machine to fabricate logic for performing the techniques described herein. Such representations, known as "IP cores," may be stored on a tangible, machine-readable medium and supplied to various customers or manufacturing facilities to load into a manufacturing machine that actually manufactures the logic or processor.

[0376] Such machine-readable storage media may include, but are not limited to, a non-transitory, tangible arrangement of an article manufactured or formed by a machine or apparatus, including storage media such as a hard disk; any other type of disk, including a floppy disk, an optical disk, a compact disk-read only memory (CD-ROM), a compact disk-rewritable (CD-RW), and a magneto-optical disk; semiconductor devices, such as read-only memory (ROM), random access memory (RAM) such as dynamic random access memory (DRAM) and static random access memory (SRAM), erasable programmable read-only memory (EPROM), flash memory, electrically erasable programmable read-only memory (EEPROM); phase change memory (PCM); a magnetic card or an optical card; or any other type of medium suitable for storing electronic instructions.

[0377] Therefore, embodiments of the present invention also include non-transitory tangible machine-readable media containing instructions or containing design data, such as hardware description language (HDL), which defines the structures, circuits, devices, processors and / or system features described herein. These embodiments are also referred to as program products.

[0378] E. Simulation (including binary conversion, code deformation, etc.)

[0379] In some cases, the instruction converter can be used to convert instructions from a source instruction set to a target instruction set. For example, the instruction converter can convert (e.g., using static binary conversion, dynamic binary conversion including dynamic compilation), deform, simulate or otherwise convert instructions into one or more other instructions to be processed by the core. The instruction converter can be implemented in software, hardware, firmware, or a combination thereof. The instruction converter can be on the processor, outside the processor, or partially on the processor and partially outside the processor.

[0380] Fig.40 1 is a block diagram of a software instruction converter that converts binary instructions in a source instruction set into binary instructions in a target instruction set according to an embodiment of the present invention. In the illustrated embodiment, the instruction converter is a software instruction converter, but alternatively, the instruction converter can be implemented in software, firmware, hardware, or various combinations thereof. Fig.40 A program in a high-level language 4002 is shown to be compilable using a first compiler 4004 to generate a first binary code (e.g., x86) 4006 that can be natively executed by a processor 4016 having at least one first instruction set core. In some embodiments, the processor 4016 having at least one first instruction set core represents any processor that performs substantially the same functions as an Intel processor having at least one x86 instruction set core by compatibly executing or otherwise processing: 1) a substantial portion of the instruction set of an Intel x86 instruction set core, or 2) an object code version of an application or other software that is targeted to run on an Intel processor having at least one x86 instruction set core to achieve substantially the same results as an Intel processor having at least one x86 instruction set core. The first compiler 4004 represents a compiler that is operable to generate binary code 4006 (e.g., object code) in a first instruction set that can be executed on a processor 4016 having at least one first instruction set core with or without additional linking processing. Similarly, Fig.40It is shown that a program in a high-level language 4002 can be compiled using an alternative instruction set compiler 4008 to generate an alternative instruction set binary code 4010 that can be natively executed by a processor 4014 that does not have at least one first instruction set core (e.g., a processor that has a core that executes the MIPS instruction set of MIPS Technologies, Inc. of Sunnyvale, California, and / or executes the ARM instruction set of ARM Holdings, Inc. of Sunnyvale, California). The instruction converter 4012 is used to convert the first binary code 4006 into code that can be natively executed by the processor 4014 that does not have the first instruction set core. The converted code is unlikely to be the same as the alternative instruction set binary code 4010 because an instruction converter that can do so is difficult to manufacture; however, the converted code will complete the general operation and be composed of instructions from the alternative instruction set. Therefore, the instruction converter 4012 represents software, firmware, hardware, or a combination thereof that allows a processor or other electronic device that does not have a first instruction set processor or core to execute the first binary code 4006 through simulation, emulation, or any other process.

Claims

1. A processor, comprising: a fetch circuit for fetching a single instruction having an operation code and one or more operands, the one or more operands being used to indicate a first source matrix slice comprising a first plurality of source matrix data elements, a second source matrix slice comprising a second plurality of source matrix data elements, and a third source matrix slice comprising a third plurality of source matrix data elements; as well as a plurality of vector registers, each vector register for storing a plurality of packed matrix data elements, wherein a first set of vector registers is allocated to store the first source matrix slice and a second set of vector registers is allocated to store the second source matrix slice; An execution circuit is used to execute an operation corresponding to the fetched single instruction, wherein the execution circuit is used to: multiplying the first plurality of source matrix data elements of the first source matrix slice from the first set of vector registers with corresponding ones of the second plurality of source matrix data elements of the second source matrix slice from the second set of vector registers to generate a corresponding plurality of products; Each data element of the third plurality of source matrix data elements is added to one or more corresponding products of the plurality of products to generate a corresponding result data element of a result matrix slice, each result data element being used to be stored in a data element position of a corresponding data element of the third plurality of source matrix data elements in a third set of vector registers.

2. The processor of claim 1, wherein: The execution circuit comprises a plurality of lanes in which parallel multiplications of the first plurality of source matrix data elements with corresponding ones of the second plurality of source matrix data elements are performed.

3. The processor of claim 2, wherein: Each channel of the plurality of channels has a width equal to a width of the plurality of result matrix data elements and the third plurality of source matrix data elements.

4. A processor as claimed in any one of claims 1 to 3, wherein: The single instruction specifies data element sizes and formats of 8-bit integers for the first and second pluralities of source data elements, and specifies data element sizes and formats of 32-bit integers for the third plurality of source matrix data elements.

5. The processor according to any one of claims 1 to 4, wherein: The single instruction specifies a 16-bit floating point FP16 data element size and format for the first and second plurality of source data elements, and specifies a 32-bit floating point FP32 data element size and format for the third plurality of source matrix data elements.

6. The processor of any one of claims 1 to 5, further comprising: A register file includes the plurality of vector registers.

7. A processor as claimed in any one of claims 1 to 6, wherein: The execution circuitry includes a plurality of multiply-accumulate circuits coupled to the plurality of vector registers, the plurality of multiply-accumulate circuits being used to perform the operation on a per data element position basis of a matrix tile.

8. The processor of any one of claims 1 to 7, wherein: A third plurality of vector registers are allocated to store the third source matrix slice and result matrix slice.

9. The processor of claim 8, wherein: The execution circuit is used to read the first source matrix slice from the first set of vector registers, read the second source matrix slice from the second set of vector registers, and read the third source matrix slice from the third set of vector registers, and wherein each result data element is used to be stored in the third set of vector registers in the data element position of the corresponding data element in the third plurality of source matrix data elements.

10. A method comprising: fetching a single instruction having an operation code and one or more operands indicating a first source matrix slice comprising a first plurality of source matrix data elements, a second source matrix slice comprising a second plurality of source matrix data elements, and a third source matrix slice comprising a third plurality of source matrix data elements; storing a plurality of packed matrix data elements in each vector register of a plurality of vector registers, wherein a first set of vector registers is allocated to store the first source matrix tile and a second set of vector registers is allocated to store the second source matrix tile; and Perform the operations corresponding to the single instruction fetched, including: multiplying the first plurality of source matrix data elements of the first source matrix slice from the first set of vector registers with corresponding ones of the second plurality of source matrix data elements of the second source matrix slice from the second set of vector registers to generate a corresponding plurality of products; Each data element of the third plurality of source matrix data elements is added to one or more corresponding products of the plurality of products to generate corresponding result data elements of a result matrix slice, each result data element being configured to be stored in a data element position of a corresponding data element of the third plurality of source matrix data elements.

11. A machine-readable medium storing instructions, which, when executed by a processor, cause the processor to perform the method of claim 10.

12. A processor core, comprising: Instruction decoder; a scalar unit coupled to the instruction decoder, the scalar unit using a scalar register; a vector unit coupled to the instruction decoder, the vector unit using vector registers; and an L1 cache that allows low latency access to the scalar registers and the vector registers, The processor core uses a local subset of the global L2 cache and has a direct access path to the local subset.

13. The processor core according to claim 12, wherein: The vector unit is a 16-wide vector processing unit (VPU) that executes one or more of integer, single-precision floating point, and double-precision floating point instructions.

14. The processor core according to claim 13, wherein: The VPU supports mixing of register inputs through a mixing unit, value conversion through a value conversion unit, and copying of memory inputs through a copy unit.

15. A method comprising: Compiling a program in a high-level programming language using an x86 compiler to generate x86 binary code that is natively executed by a first processor having at least one x86 instruction set core; Using an instruction converter, the x86 binary code is converted into an alternative binary code that can be natively executed by a second processor that does not have an x86 instruction set core.

Images