Systems, methods, and apparatus for dot product operations

Abstract

Systems, methods, and apparatuses for dot product operations are disclosed. Embodiments detailed herein relate to matrix operations. For example, embodiments are detailed that support instructions for matrix (tile) dot product operations. Exemplary instructions include: compute dot product of signed words, with saturating accumulation into double words; compute dot product of bytes, with saturating accumulation into double words, where input bytes can be signed or unsigned, and double word accumulation has output saturation; and so on.

Description

[0001] This application is a divisional application of the invention patent application entitled "System, method and apparatus for dot product operation" with PCT international application number PCT / US2017 / 040534, international application date July 1, 2017, and Chinese national phase application number 201780086894.2. Technical Field

[0002] The field of this invention generally relates to computer processor architecture, and more specifically to matrix manipulation. Background Technology

[0003] Matrices are becoming increasingly important in many computational tasks such as machine learning and other batch data processing. Attached Figure Description

[0004] The invention is illustrated by way of example rather than limitation in the accompanying drawings, in which similar reference numerals indicate similar elements, wherein:

[0005] Figure 1 The illustration shows an example of a configured tile;

[0006] Figure 2 Several examples of matrix storage are illustrated;

[0007] Figure 3 The illustration shows an example of a system that utilizes a matrix (slice) to operate an accelerator;

[0008] Figure 4 and Figure 5 Different embodiments of how to use a matrix operation accelerator to share memory are shown;

[0009] Figure 6 The illustration shows an example of using slice matrix multiplication accumulation operation (“TMMA”);

[0010] Figure 7 This illustration shows an example of an iterative execution subset of chained fused multiplication-accumulation instructions;

[0011] Figure 8 This illustration shows an example of an iterative execution subset of chained fused multiplication-accumulation instructions;

[0012] Figure 9 This illustration shows an example of an iterative execution subset of chained fused multiplication-accumulation instructions;

[0013] Figure 10 This illustration shows an example of an iterative execution subset of a chained fused multiplication-accumulation instruction;

[0014] Figure 11The illustration shows a SIMD implementation of a power of 2 according to an embodiment, wherein the accumulator uses an input size larger than the size of the input to the multiplier;

[0015] Figure 12 The diagram illustrates an embodiment of a system utilizing a matrix manipulation circuit;

[0016] Figure 13 The illustration shows an embodiment of a processor core pipeline that supports matrix operations using chips;

[0017] Figure 14 The illustration shows an embodiment of a processor core pipeline that supports matrix operations using chips;

[0018] Figure 15 The diagram shows examples of matrices expressed in row-major and column-major formats.

[0019] Figure 16 Illustrated examples of the use of matrices (slices);

[0020] Figure 17 An example of how to use a matrix (piece) is illustrated;

[0021] Figure 18 The illustration shows an example of the execution of the TILECONFIG instruction;

[0022] Figures 19(A)-19(D) An example of (multiple) registers;

[0023] Figure 20 The illustration shows an embodiment of the description of the supported matrix (piece);

[0024] Figure 21 The illustration shows an embodiment of a method executed by a processor to process the TILECONFIG instruction;

[0025] Figure 22 The illustration provides a more detailed description of how the TILECONFIG instruction is executed using memory addressing;

[0026] Figure 23 The illustration shows exemplary pseudocode for the execution of the TILECONFIG instruction;

[0027] Figure 24 The illustration shows an exemplary execution of the TILEDOTPRODUCT instruction;

[0028] Figure 25 The illustration shows an embodiment of a method executed by a processor to process matrix (slice) dot product instructions;

[0029] Figure 26The illustration shows additional details related to an example method executed by the processor to perform the TILEDOTPRODUCT instruction;

[0030] Figures 27A-27G The diagram illustrates an example method for performing the TILEDOTPRODUCT operation;

[0031] Figures 28(A)-28(C) illustrate exemplary instruction formats;

[0032] Figure 29 This is a block diagram of a register architecture according to an embodiment of the present invention;

[0033] Figures 30A-30B The diagram illustrates an ordered pipeline and an ordered core.

[0034] Figures 31A-31B The diagram illustrates a more specific example of an ordered core architecture, which will be one of several logical blocks in a chip (including other cores of the same type and / or different types);

[0035] Figure 32 This is a block diagram of a processor according to an embodiment of the present invention, which may have more than one core, may have an integrated memory controller, and may have an integrated graphics device;

[0036] Figures 33-36 It is a block diagram of an exemplary computer architecture; and

[0037] Figure 37 This is a block diagram illustrating the use of a software instruction converter to convert binary instructions in a source instruction set into binary instructions in a target instruction set, according to an embodiment of the present invention. Detailed Implementation

[0038] Numerous specific details are set forth in the following description. However, it should be understood that embodiments of the invention may be practiced without these specific details. In other instances, well-known circuits, structures, and techniques have not been shown in detail so as not to obscure the understanding of this description.

[0039] References to "an embodiment," "an embodiment," "an exemplary embodiment," etc., in the specification indicate that the described embodiment may include a specific feature, structure, or characteristic; however, not every embodiment necessarily includes that specific feature, structure, or characteristic. Furthermore, such phrases do not necessarily refer to the same embodiment. Additionally, when a specific feature, structure, or characteristic is described in connection with an embodiment, it is believed that the influence of such feature, structure, or characteristic on such feature, structure, or characteristic in conjunction with other embodiments, whether explicitly described or not, is within the knowledge of those skilled in the art.

[0040] In many mainstream processors, handling matrices is a difficult and / or instruction-intensive task. For example, multiple rows of a matrix may be placed into multiple compact data (e.g., SIMD or vector) registers, and then operations can be performed on those rows individually. For instance, depending on the data size, adding two 8x2 matrices might require loading or pooling into four compact data registers. A first addition is then performed on the compact data register corresponding to the first row of each matrix, and a second addition is performed on the compact data register corresponding to the second row of each matrix. The resulting compact data registers are then distributed back into memory. While this scenario may be acceptable for small matrices, it is generally unacceptable for larger matrices.

[0041] I. High-level Discussion

[0042] This document describes mechanisms for supporting matrix operations in computer hardware such as central processing units (CPUs), graphics processing units (GPUs), and accelerators. Matrix operations utilize 2-dimensional (2-D) data structures representing one or more compact regions of memory, such as registers. Throughout this specification, these 2-D data structures are referred to as slices. Note that a matrix can be smaller than a slice (using less than the entirety of a slice), or multiple slices can be utilized (the matrix is ​​larger than any single slice). Throughout this specification, the matrix (slice) language is used to indicate operations performed using slices that affect the matrix; whether that matrix is ​​larger than any single slice is generally irrelevant.

[0043] Each slice can be operated on by different operations, such as those detailed herein, including but not limited to: matrix (slice) multiplication, slice addition, slice subtraction, slice diagonal, slice zeroing, slice transpose, slice dot product, slice broadcast, slice row broadcast, slice column broadcast, slice multiplication, slice multiplication and accumulation, slice movement, etc. Furthermore, in the future, support for operators using scaling and / or biasing can be used in conjunction with these operations or to support non-numerical applications such as OpenCL "local memory," data compression / decompression, etc.

[0044] Multiple portions of storage (such as (non-volatile and volatile) memory, registers, caches, etc.) are arranged as slices with different horizontal and vertical scales. For example, a slice may have a horizontal scale of 4 (e.g., four rows of a matrix) and a vertical scale of 8 (e.g., eight columns of a matrix). Typically, the horizontal scale is the associated element size (e.g., 2-bit, 4-bit, 8-bit, 16-bit, 32-bit, 64-bit, 128-bit, etc.). Multiple data types (single-precision floating-point, double-precision floating-point, integers, etc.) can be supported.

[0045] A. Exemplary use of a configured chip

[0046] Figure 1 The illustration shows an embodiment of the configured chips. As shown, there are four chips 111, 113, 115, and 117 loaded from application memory 101. In this example, chips T0 111 and T1 113 have M rows and N columns with 4-element bytes (e.g., single-precision data). Chips T2 115 and T3 117 have M rows and N / 2 columns with 8-element bytes (e.g., double-precision data). Since the width of a double-precision operand is twice that of a single-precision operand, this configuration aligns with the palette used to provide chip options, providing at least 16*N*M bytes of total storage for at least four names. The number of chips available varies depending on the instruction encoding scheme used.

[0047] In some embodiments, slice parameters are definable. For example, a "palette" is used to provide slice options. Exemplary options include, but are not limited to, the number of slice names, the number of bytes in the stored rows, the number of rows and columns in the slice, and so on. For example, the maximum "height" (number of rows) of a slice can be defined as:

[0048] Maximum number of rows per slice = Total storage space / (Number of palette names * Number of bytes per row)

[0049] This allows applications to write to names so that fixed usage can take advantage of different storage sizes across implementations.

[0050] The slice configuration (“TILECONFIG”) directive is used to configure slices, where specific slice usage is defined within the selected palette. This declaration includes the number of slice names to be used, the requested number of rows and columns for each name (slice), and in some embodiments, the requested data type for each slice. In some embodiments, a consistency check is performed during the execution of the TILECONFIG directive to determine its compliance with palette entries.

[0051] B. Exemplary slice storage types

[0052] Figure 2 Several examples of matrix storage are illustrated. In (A), slices are stored in memory. As shown, each "row" consists of four compact data elements. To reach the next "row," a stride value is used. Note that rows can be stored contiguously in memory. When slice storage does not map the row width of the underlying memory array, stride memory access allows access from one row to the next.

[0053] Loading a slice from memory and accessing a slice of memory typically involves step-by-step access from application memory to compressed data lines. Exemplary TILELOAD and TILESTORE instructions, or other instruction references to application memory as TILE (slice) operands in load operation instructions, are in some embodiments reversible to handle page faults, unmasked floating-point exceptions, and / or interrupts for up to 2* lines per instruction.

[0054] In (B), the matrix is ​​stored in a slice of registers, such as compact data registers (Single Instruction Multiple Data (SIMD) or vector registers). In this example, the slice is stacked on three physical registers. Typically, contiguous registers are used; however, this is not always necessary.

[0055] In (C), the matrix is ​​stored in-chip in a non-register memory accessible by fused multiply-accumulate (FMA) circuitry used in on-chip operations. This memory may be within or adjacent to the FMA. Furthermore, in some embodiments, as discussed below, this memory may be used for data elements rather than entire rows or slices.

[0056] The CPUID reports the supported parameters of the TMMA architecture. In some embodiments, the information list includes the maximum height and the maximum SIMD scale. Configuring the TMMA architecture requires specifying the scale of each slice, the element size of each slice, and the palette identifier. This configuration is accomplished by executing the TILECONFIG directive.

[0057] Successful execution of the TILECONFIG instruction enables subsequent TILE operations. The TILERELEASEALL instruction clears the chip configuration and disables TILE operations (until the next TILECONFIG instruction is executed). In some embodiments, XSAVE, XSTORE, etc., are used during chip context switching. In some embodiments, XSAVE uses two XCR0 bits, one for TILECONFIG metadata and one bit corresponding to the actual chip payload data.

[0058] TILECONFIG not only configures slice usage but also sets status variables that indicate whether the program is in the code region when the slice is configured. The implementation can enumerate restrictions on other instructions that can be used with the slice region, such as no use of existing register sets, etc.

[0059] Exiting a slice region is typically accomplished using the TILERELEASEALL command. This command takes no parameters and quickly invalidates all slices (indicating that no more data needs to be saved or recovered) and clears the internal state corresponding to the slice region.

[0060] In some embodiments, a slice operation will zero out any rows and columns that exceed the scale specified by the slice configuration. For example, as each row is written, the slice operation will zero out data exceeding the configured number of columns (taking into account the element size). For instance, for a 64-byte row and a slice configured with 10 rows and 12 columns, a write operation to an FP32 element will write output / result data in 12*4 bytes per row for the first 10 rows and zero out the remaining 4*4 bytes in each row. The slice operation also completely zeroes out any rows after the first 10 configured rows. When using a 1K slice with 64-byte rows, there will be 16 rows, so in this example, the last 6 rows will also be zeroed out.

[0061] In some embodiments, when data is loaded, context recovery (e.g., XRSTOR) forces that data exceeding the configured number of rows in the slice be maintained at zero. If no valid configuration exists, all rows are zeroed out. XRSTOR on slice data can load useless information from columns exceeding those configured columns. It should not be possible for XRSTOR to clear data exceeding the configured number of columns because there is no element width associated with the slice configuration.

[0062] When writing an entire TILE memory area to memory, a context save (e.g., XSAVE) exposes the entire TILE memory area. If XRSTOR loads useless data into the rightmost part of the slice, that data will be saved by XSAVE. For lines exceeding the number specified for each slice, XSAVE will write zeros.

[0063] In some embodiments, slice instructions are resumable. Memory access operations can be restarted after a page fault. Calculation instructions handling floating-point operations also allow unmasked floating-point exceptions, thanks to exception masks controlled by control and / or status registers.

[0064] To support restarting instructions after these events, the instructions store information in the start register, which is detailed below.

[0065] II. Matrix (Slice) Operating System

[0066] A. Exemplary Hardware Support

[0067] Figure 3The illustration depicts an embodiment of a system utilizing a matrix (slice) manipulation accelerator. In this illustration, a host processor / processing system 301 passes commands 311 (e.g., matrix manipulation operations, such as arithmetic or matrix manipulation operations, or load and store operations) to a matrix manipulation accelerator 307. However, this is shown in this manner for discussion purposes only. As detailed later, the accelerator 307 may be part of a processing core. Typically, commands 311, as slice manipulation operator instructions, refer to the slice in register-register (“reg-reg”) or register-memory (“reg-mem”) format. Other commands, such as TILESTORE, TILELOAD, TILECONFIG, etc., do not perform data manipulation on the slice. Commands may be decoded instructions (e.g., micro-operations) or macro instructions to be processed by the accelerator 307.

[0068] In this example, the coherent memory interface 303 is coupled to the host processor / processing system 301 and the matrix operation accelerator 405, enabling them to share memory. Figure 4 and Figure 5 Different embodiments of how to use a matrix operation accelerator to share memory are shown. For example... Figure 4 As shown, the host processor 401 and the matrix operation accelerator circuit 405 share the same memory 403. Figure 5 The illustration shows an embodiment where the host processor 501 and matrix operation accelerator 505 do not share memory but can access each other's memory. For example, processor 501 can access on-chip memory 507 and utilizes its host memory 503 as usual. Similarly, matrix operation accelerator 505 can access host memory 503, but more typically uses its own memory 507. Note that these memories can be of different types.

[0069] Matrix operation accelerator 307 includes a plurality of FMAs 309 coupled to data buffer 305 (in some implementations, one or more of these buffers 305 are stored in an FMA of a grid as shown). Data buffer 305 buffers slices loaded from memory and / or stored to memory (e.g., using slice load or slice store instructions). Data buffers may be, for example, multiple registers. Typically, these FMAs are arranged as a grid of linked FMAs 309s capable of reading and writing slices. In this example, matrix operation accelerator 307 is used to perform matrix multiplication operations using slices T0, T1, and T2. At least one slice is accommodated in an FMA grid 309. In some embodiments, all slices in the operation are stored in the FMA grid 309. In other embodiments, only a subset is stored in the FMA grid 309. As shown, T1 is accommodated, while T0 and T2 are not. Note that A, B, and C refer to the matrices of these slices, which may or may not occupy the entire space of the slice.

[0070] Figure 6 The illustration shows an example of using slice matrix multiplication accumulation operation (“TMMA”).

[0071] The number of rows in the matrix (slice A 601) matches the number of cascaded (chained) FMAs, which include computational latency. The implementation is free to recycle on grids of smaller heights, but the computation remains the same.

[0072] The source / destination vectors come from a slice of N rows (slice C 605), and the grid 611 of the FMA performs N vector-matrix operations, resulting in a complete instruction to perform matrix multiplication of the slice. Slice B 603 is another vector source and provides "broadcast" items to the FMA at each level.

[0073] In operation, in some embodiments, the elements of matrix B (stored in slice B 603) are scattered across a rectangular grid of the FMA. Matrix B (stored in slice A 601) has its row elements transposed to match the column scale of the rectangular grid of the FMA. At each FMA in the grid, the elements of A and B are multiplied and added to the incoming addend (from above in the figure), and output and passed to the next row of the FMA (or the final output).

[0074] The latency of a single step is proportional to K (the row height of matrix B), and the dependent TMMA typically (in a single slice or across slices) has enough source-destination rows to hide this latency. Implementations can also segment the SIMD (compact data element) scale M (the row height of matrix A) across time steps, but this only changes the constant multiplied by K. When the program specifies K smaller than the maximum value enumerated by TMACC, the implementation is free to use either a "mask" or "early out".

[0075] The latency of the entire TMMA is proportional to N*K. The repetition rate is proportional to N. The number of MACs per TMMA instruction is N*K*M.

[0076] Figure 7 The illustration shows an example of a subset of the iterative execution of chained fused multiplication-accumulation instructions. Specifically, Figure 7 The diagram illustrates the execution circuitry for iteratively accessing a compressed data element location at the destination. In this embodiment, chained fused multiplication and accumulation operates on multiple signed sources, where the accumulator is twice the size of the input data.

[0077] The first signed source (source 1 701) and the second signed source (source 2 703) each have four compact data elements. Each of these compact data elements stores signed data such as floating-point data. The third signed source (source 3 709) has two compact data elements, each of which stores signed data. The size of the first signed source 701 and the size of the second signed source 703 are half the size of the third signed source (initial value or previous result) 709. For example, the first signed source 701 and the second signed source 703 may have 32-bit compact data elements (e.g., single-precision floating-point), while the third signed source 709 may have 64-bit compact data elements (e.g., double-precision floating-point).

[0078] This diagram only shows the most valid two compressed data element positions of the first signed source 701 and the second signed source 703, and the most valid compressed data element position of the third signed source 709. Of course, other compressed data element positions will also be processed.

[0079] As shown, compacted data elements are processed in pairs. For example, multiplier circuit 705 multiplies the data from the most valid compacted data element positions of the first signed source 701 and the second signed source 703, and multiplier circuit 707 multiplies the data from the second most valid compacted data element positions of the first signed source 701 and the second signed source 703. In some embodiments, these multiplier circuits 705 and 707 are reused for other compacted data element positions. In other embodiments, additional multiplier circuitry is used to enable parallel processing of compacted data elements. In some contexts, parallel execution is accomplished using channels of the same size as the signed third source 709. The results of each of these multiplications are added using adder circuit 711.

[0080] (Using different adders 713 or the same adder 711) add the result of the addition of these multiplication results to the data from the highest effective compact data element position from the signed source 3709.

[0081] Finally, the result of the second addition is stored in the signed destination 715 in the compacted data element position corresponding to the used compacted data element position from the signed third source 709, or if there is a next iteration, the result of the second addition is passed to that next iteration. In some embodiments, a write mask is applied to this storage such that storage occurs if the corresponding write mask (bit) is set, and storage does not occur if the corresponding write mask (bit) is not set.

[0082] Figure 8 The illustration shows an example of a subset of the iterative execution of chained fused multiplication-accumulation instructions. Specifically, Figure 8The diagram illustrates the execution circuitry for iteratively executing the location of a compressed data element at the destination. In this embodiment, the chained fused multiplication-accumulation operates against a signed source, where the accumulator is twice the size of the input data.

[0083] The first signed source (source 1 801) and the second signed source (source 2 803) each have four compact data elements. Each of these compact data elements stores signed data such as integer data. The third signed source (source 3 809) has two compact data elements, each of which stores signed data. The size of the first signed source 801 and the size of the second signed source 803 are half the size of the third signed source 809. For example, the first signed source 801 and the second signed source 803 may have 32-bit compact data elements (e.g., single-precision floating-point), while the third signed source 809 may have 64-bit compact data elements (e.g., double-precision floating-point).

[0084] This diagram only shows the most valid two compressed data element positions of the first signed source 801 and the second signed source 803, as well as the most valid compressed data element position of the third signed source 809. Other compressed data element positions will, of course, be processed as well.

[0085] As shown, compacted data elements are processed in pairs. For example, multiplier circuit 805 multiplies the data from the most valid compacted data element positions of the first signed source 801 and the second signed source 803, and multiplier circuit 807 multiplies the data from the second most valid compacted data element positions of the first signed source 801 and the second signed source 803. In some embodiments, these multiplier circuits 805 and 807 are reused for other compacted data element positions. In other embodiments, additional multiplier circuitry is used to enable parallel processing of compacted data elements. In some contexts, parallel execution is accomplished using a channel of size 809, the size of the signed third source (initial value or result of previous iterations). Addition / saturation circuit 811 adds the result of each of the multiple multiplications to the signed third source 809.

[0086] When addition results in an excessively large value, the adder / saturation (accumulator) circuit 811 preserves the sign of the operand. Specifically, saturation evaluation occurs for infinite-precision results between multiplexed additions and writes to the destination or the next iteration. When the accumulator 811 is floating-point and the input is an integer, the sum of the product and the floating-point accumulator input value are converted to an infinite-precision value (a fixed-point number with hundreds of digits), the multiplication result is added to the third input, and a single rounding to the actual accumulator type is performed.

[0087] Unsigned saturation means that the output value is limited to the maximum unsigned number (all 1s) of that element width. Signed saturation means that the value is limited to the range between the minimum negative number and the maximum positive number of that element width (e.g., for a byte, the range is from -128 (=-2^7) to 127 (=2^7-1)).

[0088] The result of addition and saturation check is stored in the signed result 815 in the corresponding compacted data element position from the signed third source 809, or the result is passed to the next iteration if there is one. In some embodiments, a write mask is applied to this storage such that storage occurs if the corresponding write mask (bit) is set, and storage does not occur if the corresponding write mask (bit) is not set.

[0089] Figure 9 The illustration shows an example of a subset of the iterative execution of chained fused multiplication-accumulation instructions. Specifically, Figure 9 The diagram illustrates the execution circuitry for iteratively executing the location of a compressed data element at the destination. In this embodiment, the chained fused multiplication-accumulation operates on both signed and unsigned sources, where the accumulator is four times the size of the input data.

[0090] The first signed source (source 1 901) and the second unsigned source (source 2 903) each have four compacted data elements. Each of these compacted data elements contains data such as floating-point or integer data. The third signed source (initial value or result 915) has compacted data elements that store signed data. The size of the first source 901 and the second source 903 is one-quarter the size of the third signed source 915. For example, the first source 901 and the second source 903 may have 16-bit compacted data elements (e.g., words), while the third signed source 915 may have 64-bit compacted data elements (e.g., double-precision floating-point or 64-bit integer).

[0091] This diagram only shows the four most valid compressed data element positions of the first source 901 and the second source 903, as well as the most valid compressed data element position of the third signed source 915. Of course, if there are any other compressed data element positions, those will also be processed.

[0092] As shown in the figure, compacted data elements are processed as quadruples. For example, multiplier circuit 907 multiplies the data from the most significant compacted data element positions of the first source 901 and the second source 903, multiplies the data from the second most significant compacted data element positions of the first source 901 and the second source 903, multiplies the data from the third most significant compacted data element positions of the first source 901 and the second source 903, and multiplier circuit 911 multiplies the data from the least significant compacted data element positions of the first source 901 and the second source 903. In some embodiments, before multiplication, the signed compacted data elements of the first source 901 are sign-extended, and the unsigned compacted data elements of the second source 903 are zero-extended.

[0093] In some embodiments, these multiplier circuits 905-911 are reused at other compacted data element locations. In other embodiments, additional multiplier circuitry is used to enable parallel processing of compacted data elements. In some contexts, parallel execution is accomplished using channels of size 915 of the signed third source. The results of each multiplication in these multiplications are added using the adder circuit 911.

[0094] (Using different adders 913 or the same adder 911) add the result of the addition of these multiplication results to the data from the highest effective compact data element position from the signed source 3 915.

[0095] Finally, the result 919 of the second addition is either stored in the signed destination at the compacted data element position corresponding to the position of the compacted data element used from the signed third source 915, or passed to the next iteration. In some embodiments, a write mask is applied to this storage such that storage occurs if the corresponding write mask (bit) is set, and storage does not occur if the corresponding write mask (bit) is not set.

[0096] Figure 10 The illustration shows an example of a subset of the iterative execution of chained fused multiplication-accumulation instructions. Specifically, Figure 10 The diagram illustrates the execution circuitry for iteratively executing the location of a compressed data element at the destination. In this embodiment, the chained fused multiplication-accumulation operates on both signed and unsigned sources, where the accumulator is four times the size of the input data.

[0097] The first signed source (source 1 1001) and the second unsigned source (source 2 1003) each have four compact data elements. Each of these compact data elements stores data such as floating-point or integer data. The third signed source (initial value or previous result 1015) has compact data elements that store signed data. The size of the first source 1001 and the second source 1003 is one-quarter the size of the third signed source 1015. For example, the first source 1001 and the second source 1003 may have 16-bit compact data elements (e.g., words), while the third signed source 1015 may have 64-bit compact data elements (e.g., double-precision floating-point or 64-bit integer).

[0098] This diagram only shows the four most valid compressed data element positions of the first source 1001 and the second source 1003, as well as the most valid compressed data element position of the third signed source 1015. Of course, if there are any other compressed data element positions, those will also be processed.

[0099] As shown in the figure, compacted data elements are processed as quadruples. For example, multiplier circuit 1007 multiplies the data from the most significant compacted data element positions of the first source 1001 and the second source 1003, multiplies the data from the second most significant compacted data element positions of the first source 1001 and the second source 1003, multiplies the data from the third most significant compacted data element positions of the first source 1001 and the second source 1003, and multiplier circuit 1011 multiplies the data from the least significant compacted data element positions of the first source 1001 and the second source 1003. In some embodiments, before multiplication, the signed compacted data elements of the first source 1001 are sign-extended, and the unsigned compacted data elements of the second source 1003 are zero-extended.

[0100] In some embodiments, these multiplier circuits 1005-1011 are reused for other compacted data element locations. In other embodiments, additional multiplier circuitry is used to enable parallel processing of compacted data elements. In some contexts, parallel execution is accomplished using channels of the same size as the signed third source 1015. The result of the addition of these multiplication results is added to the data from the most valid compacted data element location of the signed source 3 1015 using the addition / saturation circuit 1013.

[0101] When addition results in a value that is too large or too small for signed saturation, the adder / saturation (accumulator) circuit 1013 preserves the sign of the operand. Specifically, saturation evaluation occurs for infinite-precision results between multiplexed additions and writes to the destination. When the accumulator 1013 is floating-point and the input is an integer, the sum of the product and the floating-point accumulator input value are converted to an infinite-precision value (a fixed-point number of several hundred digits), the multiplication result is added to the third input, and a single rounding to the actual accumulator type is performed.

[0102] The result 1019 of the addition and saturation check is either stored in the signed destination at the compacted data element position corresponding to the used compacted data element position from the signed third source 1015, or passed to the next iteration. In some embodiments, a write mask is applied to this storage such that storage occurs if the corresponding write mask (bit) is set, and storage does not occur if the corresponding write mask (bit) is not set.

[0103] Figure 11 The illustration shows a SIMD implementation of a power of 2 according to an embodiment, where the accumulator uses an input size larger than the size of the input to the multiplier. Note that the source and accumulator values ​​(to the multiplier) can be signed or unsigned. Table 1101 illustrates different configurations for an accumulator with an input size of 2X (in other words, the size of the accumulator input value is twice the size of the compacted data element of the source). For a byte-sized source, the accumulator uses a 16-bit word or half-precision floating-point (HPFP) value. For a word-sized source, the accumulator uses a 32-bit integer or single-precision floating-point (SPFP) value. For an SPFP or 32-bit integer-sized source, the accumulator uses a 64-bit integer or double-precision floating-point (DPFP) value.

[0104] For an accumulator with a 4X input size (in other words, the size of the accumulator input value is four times the size of the compacted data element of the source), Table 1103 illustrates different configurations. For byte-sized sources, the accumulator uses 32-bit integer or single-precision floating-point (SPFP) values. In some embodiments, for word-sized sources, the accumulator uses 64-bit integer or double-precision floating-point (DPFP) values.

[0105] For an accumulator with an input size of 8X (in other words, the size of the accumulator input value is 8 times the size of the compressed data element of the source), Table 1105 illustrates the configuration. For a byte-sized source, the accumulator uses a 64-bit integer.

[0106] As previously mentioned, the matrix operation circuitry can be included in the core or used as an external accelerator. Figure 12The illustration shows an embodiment of a system utilizing matrix operation circuitry. In this illustration, multiple entities are coupled to a ring interconnect 1245.

[0107] Multiple cores 1201, 1203, 1205, and 1207 provide non-chip-based instruction support. In some embodiments, matrix operation circuitry is located in core 1203; in other embodiments, matrix operation circuitry 1211 and 1213 are accessible on a ring interconnect 1245.

[0108] In addition, one or more memory controllers 1223-1225 are provided to communicate with memories 1233 and 1231 on behalf of the core and / or matrix operation circuitry.

[0109] Figure 13 The illustration depicts an embodiment of a processor core pipeline that supports matrix operations using chips. Branch prediction and decoding circuitry 1303 performs branch prediction, decoding of instructions stored in instruction store 1301, and / or both branch prediction and decoding. For example, the instructions detailed herein may be stored in the instruction store. In some implementations, separate circuitry is used for branch prediction, and in some embodiments, at least some instructions are decoded into one or more micro-operations, microcode entry points, microinstructions, other instructions, or other control signals using microcode 1305. Branch prediction and decoding circuitry 1303 can be implemented using a variety of different mechanisms. Examples of suitable mechanisms include, but are not limited to, lookup tables, hardware implementations, programmable logic arrays (PLAs), microcode read-only memory (ROM), etc.

[0110] Branch prediction and decoding circuitry 1303 is coupled to rename / allocator circuitry 1307, which in some embodiments is coupled to scheduler circuitry 1309. In some embodiments, these circuits provide register renaming, register allocation, and / or scheduling functionality by performing one or more of the following steps: 1) renaming logical operand values ​​to physical operand values ​​(e.g., a register alias table in some embodiments); 2) assigning status bits and flags to decoded instructions; and 3) (e.g., using a reserved station in some embodiments) scheduling the decoded instructions for execution on execution circuitry outside the instruction pool.

[0111] Scheduler circuitry 1309 represents any number of different schedulers, including reserved stations, central command windows, etc. Scheduler circuitry 1309 is coupled to or includes (multiple) physical register files 1315. Each of the (multiple) physical register files 1315 represents one or more physical register files, where different physical register files store one or more different data types, such as scalar integers, scalar floating-point numbers, compressed integers, compressed floating-point numbers, vector integers, vector floating-point numbers, status (e.g., an instruction pointer as the address of the next instruction to be executed), slices, etc. In one embodiment, physical register files 1315 include vector register circuitry, write mask register circuitry, and scalar register circuitry. These register circuitry can provide architectural vector registers, vector mask registers, and general-purpose registers. Multiple physical register files 1315 are covered by retirement circuitry 1317 to illustrate various ways in which register renaming and out-of-order execution can be implemented (such as using multiple reordering buffers and multiple retirement register files, using multiple future files, multiple history buffers, and multiple retirement register files, using register mapping and register pools, etc.). Retirement circuitry 1317 and multiple physical register files 1315 are coupled to multiple execution circuitry 1311.

[0112] Although register renaming is described in the context of out-of-order execution, it should be understood that register renaming can also be used in ordered architectures. While the illustrated embodiment of the processor may also include separate instruction and data cache units as well as a shared L2 cache unit, alternative embodiments may also have a single internal cache for both instructions and data, such as, for example, a Level 1 (L1) internal cache, or multiple levels of internal caches. In some embodiments, the system may include a combination of internal caches and external caches located outside the core and / or processor. Alternatively, all caches may be located outside the core and / or processor.

[0113] Execution circuitry 1311 includes a collection of one or more execution circuits 1321, 1323, and 1327, and a collection of one or more memory access circuits 1325. Execution circuits 1321, 1323, and 1327 perform various operations (e.g., shift, addition, subtraction, multiplication) and various data types (e.g., scalar floating-point, compressed integer, compressed floating-point, vector integer, vector floating-point). While some embodiments may include multiple execution units dedicated to a particular function or set of functions, other embodiments may include only one execution unit or multiple execution units that perform all functions. Scalar circuitry 1321 performs scalar operations, vector / SIMD circuitry 1323 performs vector / SIMD operations, and matrix operation circuitry 1327 performs the matrix (slice) operations detailed herein.

[0114] A set of memory access units 1364 is coupled to a memory unit 1370, which includes a data TLB unit 1372, which is coupled to a data cache unit 1374, which is coupled to a Level 2 (L2) cache unit 1376. In one exemplary embodiment, the memory access unit 1364 may include a load unit, a memory address unit, and a memory data unit, each of which is coupled to the data TLB unit 1372 in the memory unit 1370. An instruction cache unit 1334 is further coupled to the Level 2 (L2) cache unit 1376 in the memory unit 1370. The L2 cache unit 1376 is coupled to one or more other levels of cache and ultimately to main memory.

[0115] As an example, an exemplary register-renaming, out-of-order release / execution core architecture can be pipelined as follows: 1) an instruction fetch circuit performs the fetch and length decoding stage; 2) a branch and decode circuit 1303 performs the decoding stage; 3) a rename / allocator circuit 1307 performs the allocation and rename stages; 4) a scheduler circuit 1309 performs the scheduling stage; 5) (multiple) physical register files coupled to or included in the scheduler circuit 1307, the rename / allocation circuit 1307, and memory units perform the register read / memory read stage; execution circuit 1311 performs the execution stage; 6) memory units and (multiple) physical register file units perform the write-back / memory write stage; 7) each unit may involve an exception handling stage; and 8) a retirement unit and (multiple) physical register file units perform the commit stage.

[0116] The core may support one or more instruction sets (e.g., the x86 instruction set (with some extensions added with newer versions); the MIPS instruction set of MIPS Technologies, Inc., Sunnyvale, California; the ARM instruction set of ARM Holdings, Inc., Sunnyvale, California (with optional additional extensions such as NEON)), including the instructions described herein. In one embodiment, core 1390 includes logic for supporting compressed data instruction set extensions (e.g., AVX1, AVX2), thereby allowing operations used by many multimedia applications to be performed using compressed data.

[0117] It should be understood that a core can support multithreading (execution of two or more parallel operations or a set of threads), and this multithreading can be accomplished in various ways, including time-division multithreading, synchronous multithreading (where a single physical core provides a logical core for each of the threads in which the physical core is synchronously multithreading), or combinations thereof (e.g., time-division fetching and decoding, and subsequent operations such as using...). Hyper-threading technology for synchronous multithreading.

[0118] Figure 14 The illustration depicts an embodiment of a processor core pipeline that supports matrix operations using chips. Branch prediction and decoding circuitry 1403 performs branch prediction, decoding of instructions stored in instruction store 1401, and / or both branch prediction and decoding. For example, the instructions detailed herein may be stored in the instruction store. In some implementations, separate circuitry is used for branch prediction, and in some embodiments, at least some instructions are decoded into one or more micro-operations, microcode entry points, microinstructions, other instructions, or other control signals using microcode 1405. Branch prediction and decoding circuitry 1403 can be implemented using a variety of different mechanisms. Examples of suitable mechanisms include, but are not limited to, lookup tables, hardware implementations, programmable logic arrays (PLAs), microcode read-only memory (ROM), etc.

[0119] Branch prediction and decoding circuitry 1403 is coupled to rename / allocator circuitry 1407, which in some embodiments is coupled to scheduler circuitry 1409. In some embodiments, these circuits provide register renaming, register allocation, and / or scheduling functionality by performing one or more of the following steps: 1) renaming logical operand values ​​to physical operand values ​​(e.g., a register alias table in some embodiments); 2) assigning status bits and flags to decoded instructions; and 3) (e.g., using a reserved station in some embodiments) scheduling the decoded instructions for execution on execution circuitry outside the instruction pool.

[0120] Scheduler circuitry 1409 represents any number of different schedulers, including reserved stations, central command windows, etc. Scheduler circuitry 1409 is coupled to or includes (multiple) physical register files 1415. Each of the (multiple) physical register files 1415 represents one or more physical register files, where different physical register files store one or more different data types, such as scalar integers, scalar floating-point numbers, compressed integers, compressed floating-point numbers, vector integers, vector floating-point numbers, status (e.g., an instruction pointer as the address of the next instruction to be executed), slices, etc. In one embodiment, physical register files 1415 include vector register circuitry, write mask register circuitry, and scalar register circuitry. These register circuitry can provide architectural vector registers, vector mask registers, and general-purpose registers. Multiple physical register files 1415 are covered by retirement circuitry 1417 to illustrate various ways in which register renaming and out-of-order execution can be implemented (such as using multiple reordering buffers and multiple retirement register files, using multiple future files, multiple history buffers, and multiple retirement register files, using register mappings and register pools, etc.). Retirement circuitry 1417 and multiple physical register files 1415 are coupled to multiple execution circuitry 1411.

[0121] Although register renaming is described in the context of out-of-order execution, it should be understood that register renaming can also be used in ordered architectures. While the illustrated embodiment of the processor may also include separate instruction and data cache units as well as a shared L2 cache unit, alternative embodiments may also have a single internal cache for both instructions and data, such as, for example, a Level 1 (L1) internal cache, or multiple levels of internal caches. In some embodiments, the system may include a combination of internal caches and external caches located outside the core and / or processor. Alternatively, all caches may be located outside the core and / or processor.

[0122] The execution circuitry 1411 includes a set of one or more execution circuits 1427 and a set of one or more memory access circuits 1425. The execution circuits 1427 perform the matrix (slice) operations detailed herein.

[0123] A set of memory access units 1464 is coupled to a memory unit 1470, which includes a data TLB unit 1472, which is coupled to a data cache unit 1474, which is coupled to a Level 2 (L2) cache unit 1476. In one exemplary embodiment, the memory access unit 1464 may include a load unit, a memory address unit, and a memory data unit, each of which is coupled to the data TLB unit 1472 in the memory unit 1470. An instruction cache unit 1434 is further coupled to the Level 2 (L2) cache unit 1476 in the memory unit 1470. The L2 cache unit 1476 is coupled to one or more other levels of cache and ultimately to main memory.

[0124] As an example, an exemplary register-renaming, out-of-order release / execution core architecture can be pipelined as follows: 1) an instruction fetch circuit performs the fetch and length decoding stage; 2) a branch and decode circuit 1403 performs the decoding stage; 3) a rename / allocator circuit 1407 performs the allocation and rename stages; 4) a scheduler circuit 1409 performs the scheduling stage; 5) (multiple) physical register files coupled to or included in the scheduler circuit 1407 and the rename / allocation circuit 1407 and memory units) perform the register read / memory read stage; execution circuit 1411 performs the execution stage; 6) memory units and (multiple) physical register file units perform the write-back / memory write stage; 7) each unit may involve an exception handling stage; and 8) a retirement unit and (multiple) physical register file units perform the commit stage.

[0125] The core may support one or more instruction sets (e.g., the x86 instruction set (with some extensions added with newer versions); the MIPS instruction set of MIPS Technologies, Inc., Sunnyvale, California; the ARM instruction set of ARM Holdings, Inc., Sunnyvale, California (with optional additional extensions such as NEON)), including the instructions described herein. In one embodiment, core 1490 includes logic for supporting compressed data instruction set extensions (e.g., AVX1, AVX2), thereby allowing operations used by many multimedia applications to be performed using compressed data.

[0126] It should be understood that a core can support multithreading (execution of two or more parallel operations or a set of threads), and this multithreading can be accomplished in various ways, including time-division multithreading, synchronous multithreading (where a single physical core provides a logical core for each of the threads in which the physical core is synchronously multithreading), or combinations thereof (e.g., time-division fetching and decoding, and subsequent operations such as using...). Hyper-threading technology for synchronous multithreading.

[0127] B. Layout

[0128] Throughout this specification, a row-based data layout is used to represent data. Column-based users should modify these items according to their orientation. Figure 15 The diagram illustrates examples of matrices expressed in row-major and column-major formats. As shown, matrix A is a 2x3 matrix. When this matrix is ​​stored in row-major format, the data elements in the rows are consecutive. When this matrix is ​​stored in column-major format, the data elements in the columns are consecutive. T *B T =(BA) T This is a well-known property of matrices, where the superscript T denotes transpose. Reading data as row-major data results in a matrix that looks like a transpose matrix.

[0129] In some embodiments, the semantics of column-major are utilized in the hardware, and column-major data will swap the operand order and result in a transpose of the matrix, but for subsequent column-major reads from memory, it is a correct untranspose matrix.

[0130] For example, if you have two columns-matrixes to be multiplied:

[0131]

[0132] The input matrix will be stored in linear memory (column-major) as follows:

[0133] acebdf

[0134] as well as

[0135] gh ij k 1.

[0136] If those matrices are read as row-major matrices at scales of 2x3 and 3x2, they will be represented as follows:

[0137] ace and gh

[0138] bdfij

[0139] k1

[0140] Commutation and matrix multiplication:

[0141]

[0142] The transpose matrix is ​​shifted out and can then be stored in row-major order:

[0143] ag+bh cg+dh eg+fh ai+bj ci+dj ei+fj ak+bl ck+dl ek+fl

[0144] And it is used in subsequent column-major calculations, and it is the correct untranspose matrix:

[0145]

[0146] III. Exemplary Use

[0147] Figure 16 This diagram illustrates an example of the use of matrices (slices). In this example, matrix C 1601 comprises two slices, matrix A 1603 comprises one slice, and matrix B 1605 comprises two slices. The diagram shows an example of the inner loop of an algorithm used to compute matrix multiplication. In this example, the two resulting slices tmm0 and tmm1 from matrix C 1601 are used to accumulate intermediate results. When a slice (tmm2) from matrix A 1603 is multiplied by the two slices from matrix B 1605, this slice is reused twice. Pointers are used to load a new slice A and two new slices B from the direction indicated by the arrow. The outer loop, not shown, adjusts the pointer for slice C.

[0148] The exemplary code shown in the figure includes the use of slice configuration instructions, which are executed to configure slice usage, load the slice, perform a slice loop, store the slice in memory, and release the slice usage.

[0149] Figure 17 The illustration shows an example of the use of a matrix (slice). At 1701, slice usage is configured. For example, the TILECONFIG instruction is executed to configure slice usage, including setting the number of rows and columns for each slice. Typically, at 1703, at least one matrix (slice) is loaded from memory.

[0150] IV. Exemplary Instructions

[0151] A. Chip Configuration

[0152] As discussed above, slice usage typically requires configuration before use. For example, it may not be necessary to use all rows and columns. Not configuring these rows and columns not only saves power in some embodiments but also allows the configuration to determine whether an operation will generate an error. For example, matrix multiplication of the form (NxM)*(L*N) will typically not work if M and L are not the same.

[0153] This document details the matrix (slice) configuration (“TILECONFIG”) instructions and embodiments thereof. In some embodiments, slice support is configured before using a matrix that utilizes slices. For example, the number of rows and columns per slice, the slices to be used, and so on are configured. The TILECONFIG instructions are an improvement to the computer itself because they provide support for configuring the computer to use a matrix accelerator (either as part of a processor core or as an external device). Specifically, execution of the TILECONFIG instructions causes the configuration to be retrieved from memory and applied to the matrix (slice) settings within the matrix accelerator.

[0154] i. Exemplary Execution

[0155] Figure 18 The illustration shows an exemplary execution of the TILECONFIG instruction. The TILECONFIG instruction format includes fields for the opcode and memory address.

[0156] As illustrated, the TILECONFIG instruction uses the address as a pointer to memory location 1801, which contains the description 1803 of the matrix (slice) to be supported.

[0157] The execution circuitry 1811 of the processor / core 1805 executes TILECONFIG through the following steps: retrieving description 1803 from memory 1801 via memory controller 1815; configuring the chip for the palette in chip configuration 1817 (setting the number of rows and columns); and ensuring the tag matrix support is in use. Specifically, the instruction execution resource 1811 is configured to use the chip as specified in the chip configuration 1817. The instruction execution resource may also include machine-specific registers or configuration registers for indicating chip usage.

[0158] Chip configuration 1817 is set via the execution of the TILECONFIG instruction, as indicated by chip description 1803, to specify parameters for each chip. The parameters set are the number of rows and columns for each chip. Additional values, such as in-use and start values, are also set. Chip configuration 1817 utilizes one or more registers 1819 to store chip usage and configuration information.

[0159] ii. Exemplary slice storage

[0160] Figures 19(A)-19(D)The illustration shows an example of multiple registers 1819. Figure 19(A) illustrates multiple registers 1819. As shown, each slice (TMM0 1901...TMMN 1903) has separate registers, where each register stores the row and column dimensions of that particular slice. StartK and StartM are stored in separate registers 1911 and 1913. One or more status registers 1915 are set (e.g., TILES_CONFIGURED = 1) to indicate that the slice is configured for use.

[0161] Figure 19(B) illustrates multiple registers 1819. As shown, each slice has separate registers for its rows and columns. For example, TMM0 row configuration 1921, TMM0 column configuration 1923, StartK, and StartM are stored in separate registers 1911 and 1913. One or more status registers 1915 are set (e.g., TILES_CONFIGURED = 1) to indicate that the slice is configured for use.

[0162] Figure 19(C) illustrates a single register 1819. As shown, this register stores the slice configuration (rows and columns per slice) 1931, StartK 1933, and StartM 1933 in a single register that serves as a compact data register. One or more status registers 1915 are set (e.g., TILES_CONFIGURED = 1) to indicate that the slice is configured for use.

[0163] Figure 19(D) illustrates multiple registers 1819. As shown, a single register stores slice configurations (rows and columns per slice) 1931. StartK and StartM are stored in separate registers 1911 and 1913. One or more status registers 1915 are set (e.g., TILES_CONFIGURED = 1) to indicate that the slice is configured for use.

[0164] Other combinations were conceived, such as combining the start registers into a single register in which they are displayed separately, and so on.

[0165] iii. Description of an exemplary stored matrix (slice)

[0166] Figure 20 The illustration shows an embodiment of the description of the supported matrix (slice). In this example, each field is a byte. In byte [0], the palette ID 2001 is stored. The palette ID is used to index the palette table 1813, which, as defined by the configuration, stores the number of bytes in the slice and the bytes of each row of the slice associated with that ID. Bytes 1-7 are reserved and are typically zero.

[0167] Bytes 8-9 store the value for the "startM" register 2003, and bytes 10-11 store the value for the "startK" register 2005. These registers are used to support restart instructions after these events. `startM` indicates the row to be used for restarting. `startK` indicates the position in the inner product for the relevant operation. This position in the row (column) is not required. Two-dimensional operations like element-wise addition / subtraction / multiplication use only `startM`. Three-dimensional operations use values ​​from both `startM` and `startK`. Typically, operations that only require `startM` will zero out `startK` when `startM` is written.

[0168] In some embodiments, it is the software's responsibility to zero out the startM and startK values ​​at any point without restarting the interrupted slice of instructions. For example, an unmasked floating-point exception handler might decide to perform the operation in software and change the program counter value for another instruction, typically the next instruction. In this case, the software exception handler must zero out the startM and startK values ​​in the exception frame presented to it by the operating system before resuming the program. The operating system will subsequently reload those values.

[0169] Bytes 16-17 store the number of rows (2013) and columns (2015) of slice 0, bytes 18-19 store the number of rows and columns of slice 1, and so on. In other words, each 2-byte group specifies the number of rows and columns of the slice. If 2-byte groups are not used to specify slice parameters, they should have a value of zero. Specifying slice parameters for more slices than the implementation limits or palette limits will result in an error. Unconfigured slices are set to the INIT (initial) state with 0 rows and 0 columns.

[0170] Ultimately, the configuration in memory typically ends with a description such as all zeros for several consecutive bytes.

[0171] iv. (Multiple) Exemplary formats

[0172] An embodiment of the format used for the TILECONFIG instruction is a TILECONFIG address. In some embodiments, TILECONFIG is the opcode mnemonic for the instruction. The address is a pointer to a matrix (slice) description in memory. In some embodiments, the address field is an R / M value (such as 2446).

[0173] In one embodiment, the instruction encoding includes a Scale-Index-Base (SIB) type memory addressing operand that indirectly identifies multiple indexed destination locations in memory (e.g., field 2450). In one embodiment, the SIB type memory operand may include encoding identifying a base address register. The contents of the base address register may represent a base address in memory through which the address of a specific destination location in memory is calculated. For example, the base address may be the address of a first location within a block of potential destination locations for extended vector instructions. In one embodiment, the SIB type memory operand may include encoding identifying an index register. Each element of the index register may specify an index or offset value that can be used to calculate the address of the corresponding destination location within a block of potential destination locations using the base address. In one embodiment, the SIB type memory operand may include encoding specifying a scaling factor that will be applied to each index value when calculating the corresponding destination address. For example, if a scaling factor value of 4 is encoded in the SIB type memory operand, each index value obtained from an element of the index register may be multiplied by 4 and then added to the base address to calculate the destination address.

[0174] In one embodiment, a SIB memory operand of the form vm32{x,y,z} can identify a vector array of memory operands specified using SIB memory addressing. In this example, an array of memory addresses is specified using a common base register, a constant scaling factor, and a vector index register containing elements, each with a 32-bit index value. The vector index register can be a 128-bit register (e.g., XMM) register (vm32x), a 256-bit register (e.g., YMM) register (vm32y), or a 512-bit register (e.g., ZMM) register (vm32z). In another embodiment, a SIB memory operand of the form vm64{x,y,z} can identify a vector array of memory operands specified using SIB memory addressing. In this example, an array of memory addresses is specified using a common base register, a constant scaling factor, and a vector index register containing elements, each with a 64-bit index value. The vector index register can be a 128-bit register (e.g., XMM) register (vm64x), a 256-bit register (e.g., YMM) register (vm64y), or a 512-bit register (e.g., ZMM) register (vm64z).

[0175] v. (Multiple) Exemplary execution methods

[0176] Figure 21 The illustration shows an embodiment of a method executed by a processor to process the TILECONFIG instruction.

[0177] At position 2101, fetch the instruction. For example, fetch the TILECONFIG instruction. An embodiment of the TILECONFIG instruction includes fields for the opcode and the memory address operand.

[0178] At point 2103, the retrieved instruction is decoded. For example, the retrieved TILECONFIG instruction is decoded by a decoding circuit such as that detailed herein.

[0179] At 2105, the description found at the memory address of the memory address operand is retrieved, and the decoded instruction is scheduled (as needed).

[0180] At 2107, the decoded instructions are executed by execution circuitry (hardware) such as those detailed herein. For the TILECONFIG instruction, execution causes the execution circuitry to configure slice usage in the slice configuration (setting the number of rows and columns) and mark the matrix (slice) support as in use (active). For example, one or more registers 1819 are configured. Slice support usage (e.g., “TILES_CONFIGURED(slice_configured)”) is typically indicated by setting bits in status, control, or machine-specific registers. Specifically, instruction execution resource 1811 is configured to use the slice as specified in the retrieved configuration.

[0181] In some embodiments, at 2109, a submit or retire instruction is given.

[0182] Figure 22 The illustration provides a more detailed description of how the TILECONFIG instruction is executed using memory addressing. Typically, this is performed by execution circuitry, such as the execution circuitry detailed above, after the description has been retrieved from memory. Although not illustrated, in some embodiments, a check is first performed to determine if the chip is supported. Support is typically discovered via a CPUID check.

[0183] At point 2201, a determination is made as to whether the palette ID is supported. For example, does CPUID declare that the ID is supported? If not, a general protection error occurs at point 2203.

[0184] At position 2205, read the first specific block. For example, read the row and column numbers of block 0 (T0).

[0185] At 2207, a determination is made as to whether the read group is valid. For example, if either the number of rows or columns (not both) is set to 0, the group is not valid, and at 2203, configuration stops and the slice is not considered to be in use. For example, an invalid group occurs when either the number of rows or columns (not both) is zero. Furthermore, an error occurs when the number of rows is greater than the maximum supported number of rows (which is determined by dividing the slice byte size of the palette ID by the number of bytes per row for that palette ID found in the palette table). Another potential error occurs when there are more names than are supported.

[0186] If the read group is valid, then at 2211, the slice associated with the read group is configured to use the number of rows and columns specified by the group in the slice configuration. The size of the elements in the slice is set by the palette table entry for the palette ID.

[0187] At 2213, a determination is made as to whether all the slices that have been configured have been configured. For example, have all possible slice names been processed? In some embodiments, all slices have been processed when both the row and column for a particular slice are 0.

[0188] When not all chips have been configured, at 2215, the chip number is incremented so that the next chip in the configuration will be evaluated.

[0189] At 2217, read the incremented slice's data block. For example, read the row and column numbers of slice 1 (T1). At 2207, determine whether the read data block is valid, and so on.

[0190] Once all chips have been configured, the instruction completes at 2209. For example, by setting the in-use indicator in the register, a chip will be marked as in use for matrix operations.

[0191] vi. Exemplary pseudocode

[0192] Figure 23 The illustration shows exemplary pseudocode for the execution of the TILECONFIG instruction.

[0193] B. Piece product

[0194] This document details the matrix (slice) dot product (“TILEDOTPRODUCT”) instruction and its execution embodiments. The TILEDOTPRODUCT instruction is an improvement to the computer itself because it provides support for performing a dot product operation on two matrices (slices) involving data values ​​using a single instruction. Specifically, execution of the TILEDOTPRODUCT instruction results in the execution of a dot product operation on the elements of two source matrices (slices) from the data values ​​and the result is accumulated in the corresponding data element position of the destination matrix (slice). The size of the data elements in the source matrix (slice) varies depending on the instruction and slice support. Exemplary sizes of the data elements contained in the source matrix (slice) include, but are not limited to, 4-bit, 8-bit, 16-bit, 32-bit, 64-bit, 128-bit, 256-bit, etc. In some embodiments, elements in the destination matrix (slice) that do not have corresponding elements in the source matrix (slice) are zeroed out.

[0195] i. Exemplary Execution

[0196] Figure 24 The illustration shows an exemplary execution of the TILEDOTPRODUCT instruction. The TILEDOTPRODUCT instruction format includes fields for the opcode (e.g., shown as "TDP" in the illustration), the destination accumulator operand (e.g., shown as "DESTINATION MATRIX(TILE)" in the illustration), and two source operands (e.g., shown as "FIRST SOURCE MATRIX(TILE)" and "SECOND SOURCE MATRIX(TILE)" in the illustration). In this embodiment, the destination accumulator operand is used to accumulate data obtained by performing a dot product operation on the elements of the first and second source matrix (piece) operands. Figure 24 The example destination matrix (slice) operand 2401 is shown, which initially stores a matrix of data elements of double word size.

[0197] The two source matrix (slice) operand fields represent the first source matrix (slice) operand 2403 and the second source matrix (slice) operand 2405, respectively. As previously detailed, the matrix (slice) may be stored in a set of registers, in a location in memory (e.g., as a straddle row), or in memory accessible by execution circuitry.

[0198] exist Figure 24 In the process, each of the destination matrix (slice) accumulator operand 2401, the first source matrix (slice) operand 2403, and the second source matrix (slice) operand 2405 includes a 2x2 data element matrix. Figure 24The scale of the matrices in the text is for illustrative purposes only; generally, the TILEDOTPRODUCT instruction can operate on any two source matrix (slice) operands in which the number of columns associated with the first matrix (slice) operand is the same as the number of rows of the second matrix (slice) operand (that is, as...). Figure 24 As shown, the first matrix (slice) operand has a scale of M rows x K columns, and the second matrix (slice) operand has a scale of K rows x N columns. In this example, the destination matrix (slice) accumulator operand has M rows x N columns, such that the number of rows in the destination matrix (slice) is the same as the number of rows in the first matrix (slice) operand, and the number of columns in the destination matrix (slice) is the same as the number of columns in the second matrix (slice) operand.

[0199] As shown in the figure, the execution circuit 2407 uses the grid of the fused multiply-adder (FMA) 2409 to execute the decoded TILEDOTPRODUCT instruction by performing a dot product operation on the elements of the two source matrix (slice) operands 2403 and 2405 and accumulating the result into the corresponding data element position of the destination matrix (slice) accumulator operand 2401.

[0200] Referring to example destination matrix (slice) accumulator operand 2401 and source matrix (slice) operands 2403 and 2405, execution circuit 2407 uses the first row of the first source matrix (slice) operand 2403 and the first column of the second source matrix (slice) operand 2405 to generate a dot product value, and accumulates the result in the [0,0] data element position of destination matrix (slice) operand 2401. Figure 24 For example, the [0,0] data element position of the destination matrix (slice) operand 2401 will accumulate the initially stored value W with the dot product calculated using the first row (elements [A,B] and [C,D]) of the first source matrix (slice) operand 2403 and the first column (elements [I,J] and [M,N]) of the second source matrix (slice) operand 2405, that is, W+DP([A,B],[I,J])+DP([C,D],[M,N]).

[0201] The execution circuit 2407 further calculates the dot product value using the first row of the first source matrix (slice) operand 2403 and the second column of the second source matrix (slice) operand 2405, and accumulates the result in the [0,1] data element position of the destination matrix (slice) operand 2401. The execution circuit 2407 further generates a dot product value using the second row of the first source matrix (slice) operand 2403 and the first column of the second matrix (slice) operand 2405, and accumulates the result in the [1,0] data element position of the destination matrix (slice) operand 2401. The execution circuit 2407 further generates a dot product value using the second row of the first source matrix (slice) operand 2403 and the second column of the second source matrix (slice) operand 2405, and accumulates the result in the [1,1] data element position of the destination matrix (slice) operand 2401.

[0202] ii. (Multiple) Exemplary formats

[0203] One embodiment of the format for the TILEDOTPRODUCT instruction is TDPWSSDS TMM1, TMM2, TMM3. In some embodiments, TDPWSSDS is the opcode mnemonic for the instruction, wherein the “TDP” portion of the mnemonic indicates the TILEDOTPRODUCT operation, and the “WSSDS” portion of the mnemonic indicates that the instruction calculates the dot product of the source matrix (piece) operands including elements of signed word size, and accumulates the result to the destination matrix (piece) including elements of double word size using saturation. In this instruction format and the instruction formats described below, TMM1 is the field for the destination matrix (piece) operand. TMM2 and TMM3 are the fields for the source matrix (piece) operands. In some embodiments, the TMM3 field is an R / M value (e.g., 2846), the TMM1 field is REG 2844, and the data element size is found in 2865.

[0204] Another embodiment of the format for the TILEDOTPRODUCT instruction is TDPWSSQS TMM1,TMM2,TMM3. In some embodiments, TDPWSSQS is the opcode mnemonic for the instruction, wherein the "WSSQS" portion of the mnemonic instructs the instruction to compute the dot product of source matrix (slice) operands including elements of signed word size, and to accumulate the result into a destination matrix (slice) including elements of four word size using saturation.

[0205] Another embodiment of the format for the TILEDOTPRODUCT instruction is TDPB[SS / UU / US / SU]DS TMM1,TMM2,TMM3. In some embodiments, TDPB[SS / UU / US / SU]DS is the opcode mnemonic for the instruction, wherein the “B” and “D” portions of the mnemonic indicate that the instruction calculates the dot product of the source matrix (slice) operands, which include elements of the byte size, and accumulates the result to the destination matrix (slice) accumulator operand, which includes elements of the double-word size, using saturation.

[0206] In this embodiment, the [SS / UU / US / SU] portion of the mnemonic in the preceding and similarly following instructions indicates whether each of the two source matrix (slice) operands includes signed or unsigned data values. The first letter of the [SS / UU / US / SU] mnemonic portion corresponds to the first source matrix (slice) operand, and the second letter corresponds to the second source matrix (slice) operand. For example, "SS" indicates that both source matrix (slice) operands are signed, "UU" indicates that both source matrix (slice) operands are unsigned, "US" indicates that the first source matrix (slice) operand is unsigned and the second source matrix (slice) operand is signed, and "SU" indicates that the first source matrix (slice) operand is signed and the second source matrix (slice) operand is unsigned. If either the first source matrix (slice) operand or the second source matrix (slice) operand is signed, then the destination matrix (slice) operand is signed; otherwise, if both source matrix (slice) operands are unsigned, then the destination matrix (slice) is unsigned. If either source matrix (slice) operand is signed, then the result output saturation is signed saturation; otherwise, the result output saturation is unsigned saturation.

[0207] Another embodiment of the format used for the TILEDOTPRODUCT instruction is TDP8B[SS / UU / US / SU]4BITDSTMM1,TMM2,TMM3. In some embodiments, TDP8B[SS / UU / US / SU]4BITDS is the opcode mnemonic for the instruction, where the identifiers "8B" and "4BITD" indicate that the instruction calculates the dot product of two-word source matrix (slice) operands (one operand containing byte-sized elements and the other operand containing 4-bit (half-byte)-sized elements) and accumulates the result into a destination matrix (slice) that includes double-word-sized elements.

[0208] Another embodiment of the format used for the TILEDOTPRODUCT instruction is TDP4BIT[S,U][S,U]DS TMM1,TMM2,TMM3. In some embodiments, TDP4BIT[S,U][S,U]DS is the opcode mnemonic for the instruction, wherein the “4BIT” and “D” portions of the mnemonic indicate that the instruction calculates the dot product of two-word source matrix (slice) operands (each source matrix (slice) operand comprising 4-bit (half-byte) elements) and accumulates the result into a destination matrix (slice) comprising double-word elements.

[0209] In one embodiment, the instruction encoding includes a scale-index-base (SIB) type memory addressing operand that indirectly identifies multiple indexed destination locations in memory. In one embodiment, the SIB type memory operand may include encoding that identifies a base address register. The contents of the base address register may represent a base address in memory through which the address of a particular destination location in memory is calculated. For example, the base address may be the address of a first location within a block of potential destination locations for extended vector instructions. In one embodiment, the SIB type memory operand may include encoding that identifies an index register. Each element of the index register may specify an index or offset value that can be used to calculate the address of the corresponding destination location within a block of potential destination locations using the base address. In one embodiment, the SIB type memory operand may include encoding that specifies a scaling factor to be applied to each index value when calculating the corresponding destination address. For example, if a scaling factor value of 4 is encoded in the SIB type memory operand, each index value obtained from an element of the index register may be multiplied by 4 and then added to the base address to calculate the destination address.

[0210] In one embodiment, a SIB memory operand of the form vm32{x,y,z} can identify a vector array of memory operands specified using SIB memory addressing. In this example, an array of memory addresses is specified using a common base register, a constant scaling factor, and vector index registers containing elements, each with a 32-bit index value. The vector index register can be a 128-bit register (e.g., XMM) register (vm32x), a 256-bit register (e.g., YMM) register (vm32y), or a 512-bit register (e.g., ZMM) register (vm32z). In another embodiment, a SIB memory operand of the form vm64{x,y,z} can identify a vector array of memory operands specified using SIB memory addressing. In this example, an array of memory addresses is specified using a common base register, a constant scaling factor, and vector index registers containing elements, each with a 64-bit index value. The vector index register can be a 128-bit register (e.g., XMM) register (vm64x), a 256-bit register (e.g., YMM) register (vm64y), or a 512-bit register (e.g., ZMM) register (vm64z).

[0211] iii. (Multiple) Exemplary Execution Methods

[0212] Figure 25 The illustration shows an embodiment of a method executed by a processor to process matrix (slice) dot product instructions.

[0213] At position 2501, an instruction is fetched. For example, the TILEDOTPRODUCT instruction is fetched. The TILEDOTPRODUCT instruction includes fields for the opcode, a first source matrix (slice) operand, a second source matrix (slice) operand, and a destination matrix (slice) operand. In some embodiments, the instruction further includes a field for a write mask. In some embodiments, the instruction is fetched from the instruction cache. The source and destination operands consist of compressed data. The opcode of the TILEDOTPRODUCT instruction indicates that a dot product operation will be performed on the source matrix (slice) operand. In some embodiments, the opcode further indicates whether each of the first and second source matrix (slice) operands consists of signed or unsigned values. In some embodiments, the opcode further indicates the size (e.g., a specified number of bits, bytes, quadwords, doublewords, etc.) of the matrix (slice) data values ​​including each of the first, second, and destination matrix (slice) operands.

[0214] At position 2503, the retrieved instruction is decoded. For example, the retrieved TILEDOTPRODUCT instruction is decoded by a decoding circuit such as that detailed herein.

[0215] At position 2505, the data value associated with the source matrix (slice) operand of the decoded instruction is retrieved, and the decoded instruction is scheduled (as needed). For example, when one or more of the source matrix (slice) operands are memory operands, data is retrieved from the indicated memory location.

[0216] At point 2507, the decoded instruction is executed by execution circuitry (hardware) such as that detailed herein. For the TILEDOTPRODUCT instruction, execution causes the execution circuitry to perform a dot product operation on the source data. In some embodiments, execution of the decoded matrix dot product instruction causes the execution circuitry to: compute the result by performing a dot product operation on the elements of the first source matrix operand and the second source matrix operand; and accumulate the result into the elements of the destination matrix operand.

[0217] In some embodiments, an error is generated when one or more of the following conditions are true: the number of columns associated with the first source matrix operand is different from the number of rows associated with the second source matrix operand; the number of rows associated with the destination matrix (slice) operand is different from the number of rows associated with the first source matrix (slice) operand; and the number of columns associated with the destination matrix (slice) operand is different from the number of columns associated with the second source matrix (slice) operand.

[0218] In some embodiments, at 2509, a submit or retire instruction is given.

[0219] Figure 26 The illustration shows additional details related to an example method of executing the TILEDOTPRODUCT instruction by a processor, wherein the instruction has fields for a first source matrix (slice) operand, a second source matrix (slice) operand, and a destination matrix (slice) accumulator operand.

[0220] At 2601, the execution circuit sets the first counter to a value of 0. At 2602, it determines whether the first counter is less than the number of rows configured in the destination matrix (slice) operand. If the first counter is not less than the number of rows configured in the destination matrix (slice) operand, the process ends.

[0221] At 2603, if the first counter is less than the number of rows configured in the destination matrix (slice) operand, the second counter is set to the value 0. At 2604, it is determined whether the second counter is less than the number of columns configured in the first source matrix (slice) operand. If not, at 2612, the first counter is incremented, and the process returns to 2602.

[0222] At 2605, if the second counter is less than the number of columns configured for the first source matrix (slice) operand, the row from the destination matrix (slice) operand identified by the first counter is written to a temporary location. For example, if the first counter is currently set to 0, the row [first counter] identifies the first row of the destination matrix (slice) operand. Similarly, if the first counter is currently set to 1, the row [first counter] identifies the second row of the destination matrix (slice) accumulator operand, and so on. At 2606, the third counter is set with the value 0.

[0223] At 2607, it is determined whether the third counter is less than the number of columns configured for the destination matrix operand. If the third counter is not less than the number of columns configured for the destination matrix (slice) accumulator operand, then at 2610, the data value stored at the temporary location is written to the row of the destination matrix (slice) operand identified by the first counter (i.e., the row [first counter] of the destination matrix (slice) operand). At 2611, the second counter is incremented, and the process returns to 2604.

[0224] If the third counter is less than the number of columns configured for the destination matrix (slice) accumulator operand, then at 2608, the execution circuit performs a dot product operation involving the data elements of the first source matrix (slice) operand at position row [first counter, second counter] and the data elements of the second source matrix (slice) operand at position row [second counter, third counter], and accumulates the result at the element position [third counter] of the temporary location. (Reference) Figure 24 Assuming that each of the first, second, and third counters is currently set to 0, a dot product operation involving the data element (element value [A,B]) at position [0,0] of the first source matrix (slice) operand 2402 and the data element (element value [I,J]) at position [0,0] of the second source matrix (slice) operand 2403 is performed at position [0,0], and the result is accumulated at element position [0] of the temporary position (currently storing the value W from row [0,0] of the destination matrix (slice) accumulator operand 2401).

[0225] At 2609, the third counter is incremented, and the process returns to 2607. Figure 26 The process described herein results in performing a dot product operation on the elements of the first source matrix (slice) operand and the second source matrix (slice) operand, and accumulating the result into the elements of the destination matrix (slice) accumulator operand, as in Figure 24 The destination matrix (slice) operand 2401 is illustrated in the diagram.

[0226] Figures 27A-27GThe illustrations depict example methods for performing the TILEDOTPRODUCT operation as described above. For example, the steps shown in 2701 and 2703 illustrate example procedures for performing the TILEDOTPRODUCT operation, which involves saturating a source matrix of signed word-sized elements into double-word-sized elements using saturation (e.g., instructions based on the example format TDPWSSDS TMM1,TMM2,TMM3). Specifically, 2701 illustrates an example helper procedure DWSS(c,x,y) for performing a multiply-add operation on double-word arguments. As shown in the accompanying procedure 2703, the multiply-add operation illustrated in 2701 is used as part of a dot product calculation performed on the rows and columns of the source matrix (slice) operands.

[0227] Step 2703 Instructions: An error is generated if any of the following is true: the matrix (slice) architecture is not currently configured; the number of columns in the first source matrix (slice) is different from the number of rows in the second source matrix (slice); the number of rows in the destination matrix (slice) is different from the number of rows in the first source matrix (slice); the number of columns in the destination matrix (slice) is different from the number of columns in the second source matrix (slice); or the number of rows or columns in the second source matrix (slice) exceeds the configured limit.

[0228] The process in 2703 continues iteratively through the rows and columns of the destination matrix (slice) and the source matrix (slice). Specifically, the example process computes the result by performing a dot product operation on the elements of the first source matrix (slice) operand ("tsrc1") and the second source matrix (slice) operand ("tsrc2"), and then accumulating the result into the elements of the destination matrix (slice) accumulator operand ("tsrcdest").

[0229] Figures 27B-27C The example process shown illustrates the procedure executed to implement other example TILEDOTPRODUCT instruction formats. For example, Figures 27B-27C The procedures illustrated in 2705 and 2707 illustrate an example helper function DPWSSQ(c,x,y) that performs a multiply-add operation on a four-word argument. The example shown in 2707 illustrates a procedure for performing a TILEDOTPRODUCT operation involving the accumulation of a matrix of signed word-sized elements into a matrix of double-word-sized elements (e.g., based on instructions with the format TDPWSSQS TMM1,TMM2,TMM3, as described above).

[0230] Figure 27DThe procedure illustrated in 2709 is an example helper function DPBD(c,x,y) for performing multiply-add operations on double-word arguments. The example illustrated in 2711 is a procedure for performing the TILEDOTPRODUCT operation, which involves the matrix accumulation of byte-sized elements into double-word-sized elements (e.g., based on instructions with the format TDPB[SS,UU,US,SU]DS TMM1,TMM2,TMM3, as described above).

[0231] Figures 27E-27F The process illustrated in 2713 illustrates the procedure for performing the TILEDOTPRODUCT operation, which involves two word source matrix (slice) operands, one operand containing byte-sized elements and the other operand containing 4-bit (half-byte)-sized elements, and the result is accumulated into a destination matrix (slice) containing double-word-sized elements (e.g., based on an instruction with the format TDP8B[SS,UU,US,SU]4BITDSTMM1,TMM2,TMM3, as described above).

[0232] Figure 27G The process illustrated in 2715 illustrates the procedure for performing the TILEDOTPRODUCT operation, which involves two-word source matrix (slice) operands, each operand containing 4-bit (half-byte) elements, and the results are accumulated into a destination matrix (slice) containing double-word elements (e.g., based on instructions with the format TDP4BIT[SS,UU,US,SU]DS TMM1,TMM2,TMM3, as described above).

[0233] iv. Example

[0234] Example 1 A processor includes: a decoding circuit for decoding an instruction having fields for a first source matrix operand, a second source matrix operand, and a destination matrix operand; and an execution circuit for executing the decoded instruction to: compute a result by performing a dot product operation on elements from the first and second source matrix operands; and accumulate the result into a data element position of the destination matrix operand.

[0235] Example 2 The processor of Example 1, wherein the elements of the first source matrix operand and the second source matrix operand are signed word elements, and wherein the elements of the destination matrix operand are signed double words.

[0236] Example 3The processor of Example 1, wherein the elements of the first source matrix operand and the second source matrix operand are signed word-sized elements, and wherein the elements of the destination matrix operand are signed four-word-sized elements.

[0237] Example 4 The processor of Example 1, wherein the elements of the first source matrix operand and the second source matrix operand are byte-sized elements, and wherein the elements of the destination matrix operand are double-word-sized elements.

[0238] Example 5 The processor of Example 1, wherein the elements of the first source matrix operand are byte-sized elements and the elements of the second source matrix operand are 4-bit-sized elements, and wherein the elements of the destination matrix are double-word-sized elements.

[0239] Example 6 The processor of Example 1, wherein the elements of the first source matrix operand and the elements of the second source matrix operand are 4-bit elements, and wherein the elements of the destination matrix are double-word elements.

[0240] Example 7 The processor for any of the examples 1-6, where the result is calculated using saturation.

[0241] Example 8 The processor of any of Examples 1-7, wherein the execution circuitry includes multiple fused multiply-adder units.

[0242] Example 9 A processor for any of Examples 1-8, wherein the instruction indicates that at least one of the first source matrix operand and the second source matrix operand contains an unsigned data value.

[0243] Example 10 A processor for any of Examples 1-9, wherein an error is generated when the number of columns of the first source matrix operand is different from the number of rows of the second source matrix operand.

[0244] Example 11 A processor for any of Examples 1-10, wherein an error is generated when the number of rows of the destination matrix operand differs from the number of rows of the first source matrix operand.

[0245] Example 12 A processor for any of the examples 1-11, wherein an error is generated when the number of columns of the destination matrix operand is different from the number of columns of the second source matrix operand.

[0246] Example 13A method includes: decoding an instruction having fields for a first source matrix operand, a second source matrix operand, and a destination matrix operand; and executing the decoded instruction to: compute a result by performing a dot product operation on elements from the first and second source matrix operands; and accumulating the result into a data element position of the destination matrix operand.

[0247] Example 14 The method of Example 13, wherein the elements of the first source matrix operand and the second source matrix operand are signed word elements, and wherein the elements of the destination matrix operand are signed double words.

[0248] Example 15 The method of Example 13, wherein the elements of the first source matrix operand and the second source matrix operand are signed word-sized elements, and wherein the elements of the destination matrix operand are signed four-word-sized elements.

[0249] Example 16 The method of Example 13, wherein the elements of the first source matrix operand and the second source matrix operand are byte-sized elements, and wherein the elements of the destination matrix operand are double-word-sized elements.

[0250] Example 17 The method in Example 13, wherein the elements of the first source matrix operand are byte-sized elements and the elements of the second source matrix operand are 4-bit-sized elements, and wherein the elements of the destination matrix are double-word-sized elements.

[0251] Example 18 The method in Example 13, wherein the elements of the first source matrix operand and the elements of the second source matrix operand are 4-bit elements, and wherein the elements of the destination matrix are double-word elements.

[0252] Example 19 The method for any of the examples 13-18, where the result is calculated using saturation.

[0253] Example 20 The method of any of Examples 13-19, wherein the execution circuitry includes multiple fused multiply-adder units.

[0254] Example 21 The method of any of Examples 13-20, wherein the instruction indicates that at least one of the first source matrix operand and the second source matrix operand contains an unsigned data value.

[0255] Example 22 The method of any of Examples 13-21, wherein an error is generated when the number of columns of the first source matrix operand is different from the number of rows of the second source matrix operand.

[0256] Example 23 The method of any of Examples 13-22, wherein an error is generated when the number of rows of the destination matrix operand is different from the number of rows of the first source matrix operand.

[0257] Example 24 The method of any of Examples 13-23, wherein an error is generated when the number of columns of the destination matrix operand is different from the number of columns of the second source matrix operand.

[0258] Example 25 A non-transitory machine-readable medium is provided, storing instructions that, when executed by a processor, cause the processor to perform a method comprising: decoding the instructions, the instructions having fields for a first compressed data source operand, a second compressed data source operand, and a compressed data destination operand; and executing the decoded instructions to: compute a result by performing a dot product operation on elements from the first source matrix operand and the second source matrix operand; and accumulating the result into a data element position of the destination matrix operand.

[0259] Example 26 Example 25 is a non-transient machine-readable medium, wherein the elements of the first source matrix operand and the second source matrix operand are signed word elements, and wherein the elements of the destination matrix operand are signed double words.

[0260] Example 27 Example 25 is a non-transient machine-readable medium, wherein the elements of the first source matrix operand and the second source matrix operand are signed word-sized elements, and wherein the elements of the destination matrix operand are signed four-word-sized elements.

[0261] Example 28 Example 25 is a non-transient machine-readable medium in which the elements of the first source matrix operand and the second source matrix operand are byte-sized elements, and the elements of the destination matrix operand are double-word-sized elements.

[0262] Example 29 Example 25 is a non-transient machine-readable medium in which the elements of the first source matrix operand are byte-sized elements and the elements of the second source matrix operand are 4-bit-sized elements, and the elements of the destination matrix are double-word-sized elements.

[0263] Example 30 Example 25 is a non-transient machine-readable medium in which the elements of the first source matrix operand and the elements of the second source matrix operand are 4-bit elements, and the elements of the destination matrix are double-word elements.

[0264] Example 31For any of the items in Examples 25-30, the non-transient machine-readable medium, where the result is calculated using saturation.

[0265] Example 32 The non-transient machine-readable medium of any of Examples 25-31, wherein the execution circuitry includes a plurality of fused multipliers and adders.

[0266] Example 33 A non-transient machine-readable medium for any of Examples 25-32, wherein the instructions indicate that at least one of the first source matrix operand and the second source matrix operand contains an unsigned data value.

[0267] Example 34 A non-transient machine-readable medium for any of Examples 25-33, wherein an error is generated when the number of columns of the first source matrix operand differs from the number of rows of the second source matrix operand.

[0268] Example 35 A non-transient machine-readable medium for any of Examples 25-34, wherein an error is generated when the number of rows of the destination matrix operand differs from the number of rows of the first source matrix operand.

[0269] Example 36 A non-transient machine-readable medium for any of Examples 25-35, wherein an error is generated when the number of columns of the destination matrix operand differs from the number of columns of the second source matrix operand.

[0270] Example 37 A system is provided, comprising: a processor; and an accelerator coupled to the processor, the accelerator comprising: decoding circuitry for decoding an instruction having fields for a first source matrix operand, a second source matrix operand, and a destination matrix operand; and execution circuitry for executing the decoded instruction to: compute a result by performing a dot product operation on elements from the first and second source matrix operands; and accumulate the result into a data element position of the destination matrix operand.

[0271] Example 38 The system of Example 37, wherein the elements of the first source matrix operand and the second source matrix operand are signed word elements, and wherein the elements of the destination matrix operand are signed double words.

[0272] Example 39 The system of Example 37, wherein the elements of the first source matrix operand and the second source matrix operand are signed word-sized elements, and wherein the elements of the destination matrix operand are signed four-word-sized elements.

[0273] Example 40The system of Example 37, wherein the elements of the first source matrix operand and the second source matrix operand are byte-sized elements, and wherein the elements of the destination matrix operand are double-word-sized elements.

[0274] Example 41 The system of Example 37, wherein the elements of the first source matrix operand are byte-sized elements and the elements of the second source matrix operand are 4-bit-sized elements, and wherein the elements of the destination matrix are double-word-sized elements.

[0275] Example 42 The system of Example 37, wherein the elements of the first source matrix operand and the elements of the second source matrix operand are 4-bit elements, and wherein the elements of the destination matrix are double-word elements.

[0276] Example 43 The system of any of the terms in Examples 37-42, where the result is calculated using saturation.

[0277] Example 44 A system of any of Examples 37-43, wherein the execution circuitry includes multiple fused multiply-adder units.

[0278] Example 45 A system of any of Examples 37-44, wherein the instruction indicates that at least one of the first source matrix operand and the second source matrix operand contains an unsigned data value.

[0279] Example 46 The system of any of Examples 37-45, wherein an error is generated when the number of columns of the first source matrix operand is different from the number of rows of the second source matrix operand.

[0280] Example 47 The system of any of Examples 37-46, wherein an error is generated when the number of rows of the destination matrix operand is different from the number of rows of the first source matrix operand.

[0281] Example 48 The system of any of Examples 37-47, wherein an error is generated when the number of columns of the destination matrix operand is different from the number of columns of the second source matrix operand.

[0282] V. Detailed Exemplary Systems, Processors, and Simulations

[0283] This article details examples of hardware, software, etc., used to execute the instructions described above. For example, the following description details various aspects of instruction execution, including various pipeline stages such as fetch, decode, schedule, execute, and retire.

[0284] An instruction set includes one or more instruction formats. A given instruction format defines various fields (number of bits, bit positions) to specify the operation to be performed (opcode) and the operand(s) to which the operation will be performed, etc. Some instruction formats are further decomposed by defining instruction templates (or subformats). For example, an instruction template for a given instruction format can be defined as a different subset of the fields of that instruction format (the included fields are generally in the same order, but at least some fields have different bit positions because fewer fields are included), and / or defined as a given field that is interpreted in a different way. Thus, each instruction of the ISA is expressed using a given instruction format (and, if defined, according to a given instruction template in the instruction template of that instruction format) and includes fields for specifying the operation and operands. For example, the exemplary ADD (addition) instruction has a specific opcode and instruction format, which includes an opcode field for specifying the opcode and an operand field for selecting operands (source 1 / destination and source 2); and the appearance of the ADD instruction in the instruction stream will cause specific content in the operand field to select a specific operand.

[0285] A. Exemplary instruction format

[0286] Examples of the instructions described herein can be embodied in different formats. Furthermore, exemplary systems, architectures, and pipelines are detailed below. Examples of the instructions can be executed on such systems, architectures, and pipelines, but are not limited to those detailed herein.

[0287] VEX command format

[0288] VEX encoding allows instructions to have more than two operands and allows SIMD vector registers to be longer than 128 bits. The use of the VEX prefix provides three-operand (or more) syntax. For example, the previous two-operand instructions performed operations such as A = A + B, which overwrite the source operand. The use of the VEX prefix allows operands to perform non-destructive operations such as A = B + C.

[0289] Figure 28A The diagram illustrates an exemplary instruction format, including the VEX prefix 2802, the actual opcode field 2830, the Mod R / M byte 2840, the SIB byte 2850, the offset field 2862, and the IMM8 2872. Figure 28B The diagram shows the source Figure 28A Which fields constitute the complete opcode field 2874 and the basic operation field 2841? Figure 28C The diagram shows the source Figure 28A Which fields constitute register index field 2844?

[0290] The VEX prefix (bytes 0-2) 2802 is encoded in three-byte form. The first byte is the format field 2890 (VEX byte 0, bits [7:0]), which contains the explicit C4 byte value (a unique value used to distinguish the C4 instruction format). The second and third bytes (VEX bytes 1-2) include several bit fields that provide special capabilities. Specifically, the REX field 2805 (VEX byte 1, bits [7-5]) consists of the VEX.R bit field (VEX byte 1, bits [7]-R), the VEX.X bit field (VEX byte 1, bits [6]-X), and the VEX.B bit field (VEX byte 1, bits [5]-B). The other fields of these instructions encode the lower three bits (rrr, xxx, and bbb) of the register index as known in the art, so that Rrrr, Xxxx, and Bbbb can be formed by adding VEX.R, VEX.X, and VEX.B. The opcode mapping field 2815 (VEX byte 1, bits [4:0] – mmmmmm) includes the content used to encode the implicit preamble opcode byte. The W field 2864 (VEX byte 2, bits [7] – W) is represented by the notation VEX.W and provides different functions depending on the instruction. The function of VEX.vvvv 2820 (VEX byte 2, bits [6:3] – vvvv) can include: 1) VEX.vvvv encodes the first source register operand specified in reverse (two's complement) form and is valid for instructions with two or more source operands; 2) VEX.vvvv encodes the destination register operand specified in two's complement form for some vector shift; or 3) VEX.vvvv does not encode any operand, this field is reserved and should contain 1111b. If the VEX.L 2868 size field (VEX byte 2, bit [2]-L) = 0, it indicates a 128-bit vector; if VEX.L = 1, it indicates a 256-bit vector. The prefix encoding field 2825 (VEX byte 2, bit [1:0]-pp) provides additional bits for the basic operation field 2841.

[0291] The actual opcode field 2830 (byte 3) is also known as the opcode byte. Part of the opcode is specified in this field.

[0292] The MOD R / M field 2840 (byte 4) includes the MOD field 2842 (bits [7-6]), the Reg field 2844 (bits [5-3]), and the R / M field 2846 (bits [2-0]). The Reg field 2844 may encode the destination register operand or the source register operand (rrr of Rrrr); or it may be treated as an opcode extension and not used to encode any instruction operand. The R / M field 2846 may encode the instruction operand that references a memory address; or it may encode the destination register operand or the source register operand.

[0293] The scale, index, base address (SIB) field 2850 (byte 5) contains SS2852 (bits [7-6]), which is used for memory address generation. The contents of SIB.xxx 2854 (bits [5-3]) and SIB.bbb 2856 (bits [2-0]) have been previously mentioned for register indices Xxxx and Bbbb.

[0294] The displacement field 2862 and the immediate numeric field (IMM8) 2872 contain data.

[0295] B. Exemplary Register Architecture

[0296] Figure 29 This is a block diagram of a register architecture 2900 according to an embodiment of the present invention. In the illustrated embodiment, there are 32 vector registers 2910, each 512 bits wide; these registers are referenced as zmm0 to zmm31. The lower-order 256 bits of the lower 32 zmm registers are overlaid on registers ymm0-15. The lower-order 128 bits of the lower 32 zmm registers (the lower-order 128 bits of the ymm registers) are overlaid on registers xmm0-15.

[0297] General Purpose Register 2925 — In the illustrated embodiment, there are sixteen 64-bit general purpose registers that are used in conjunction with the existing x86 addressing mode for addressing memory operands. These registers are referenced by the names RAX, RBX, RCX, RDX, RBP, RSI, RDI, RSP, and R8 through R15.

[0298] A scalar floating-point stack register file (x87 stack) 2945 is overlaid with an MMX compact integer flat register file 2950. In the illustrated embodiment, the x87 stack is an octal stack used to perform scalar floating-point operations on 32 / 64 / 80-bit floating-point data using x87 instruction set extensions; while the MMX registers are used to perform operations on 64-bit compact integer data and to store operands for some operations performed between the MMX and XMM registers.

[0299] In some embodiments, a superimposed structure above the physical registers is used to support the chip 2920. For example, depending on the implementation, the chip may utilize 16 1024-bit registers, 32 512-bit registers, and so on.

[0300] Alternative embodiments of the invention may use wider or narrower registers. Furthermore, alternative embodiments of the invention may use more, fewer, or different register files and registers.

[0301] Exemplary core architectures, processors, and computer architectures

[0302] Processor cores can be implemented in different ways, for different purposes, and in different processors. For example, implementations of such cores may include: 1) general-purpose ordered cores intended for general-purpose computing; 2) high-performance general-purpose out-of-order cores intended for general-purpose computing; and 3) dedicated cores intended primarily for graphics and / or scientific (throughput) computing. Implementations of different processors may include: 1) CPUs, which include one or more general-purpose ordered cores and / or one or more general-purpose out-of-order cores intended for general-purpose computing; and 2) coprocessors, which include one or more dedicated cores intended primarily for graphics and / or scientific (throughput) computing. These different processors result in different computer system architectures, which may include: 1) coprocessors on a separate chip from the CPU; 2) coprocessors in the same package as the CPU but on a separate die; 3) coprocessors on the same die as the CPU (in this case, such coprocessors are sometimes referred to as dedicated logic or dedicated cores, such as integrated graphics and / or scientific (throughput) logic); and 4) systems-on-a-chip (SoCs) that can include the described CPU (sometimes referred to as application cores or application processors), the coprocessors described above, and additional functionality on the same die. Exemplary core architectures are then described, followed by exemplary processors and computer architectures. Circuits (units) including exemplary cores, processors, etc., are described in detail herein.

[0303] C. Exemplary Core Architecture

[0304] Ordered and disordered kernel diagrams

[0305] Figure 30A This is a block diagram illustrating an exemplary ordered pipeline and an exemplary out-of-order release / execution pipeline for register renaming according to various embodiments of the present invention. Figure 30B This is a block diagram illustrating exemplary embodiments of ordered architecture cores to be included in a processor according to various embodiments of the present invention, and exemplary out-of-order release / execution architecture cores with register renaming. Figures 30A-30BThe solid-line boxes in the diagram illustrate ordered pipelines and ordered cores, while the optional additions to the dashed boxes illustrate register renaming, out-of-order release / execution pipelines, and cores. Since the ordered aspect is a subset of the out-of-order aspect, the out-of-order aspect will be described.

[0306] exist Figure 30A In the processor pipeline 3000, there are fetch stage 3002, length decoding stage 3004, decoding stage 3006, allocation stage 3008, renaming stage 3010, scheduling (also known as dispatch or issue) stage 3012, register read / memory read stage 3014, execution stage 3016, write-back / memory write stage 3018, exception handling stage 3022, and commit stage 3024.

[0307] Figure 30B A processor core 3090 is shown, which includes a front-side unit 3030 coupled to an execution engine unit 3050, and both the front-side unit 3030 and the execution engine unit 3050 are coupled to a memory unit 3070. Core 3090 can be a Reduced Instruction Set Computing (RISC) core, a Complex Instruction Set Computing (CISC) core, a Very Long Instruction Word (VLIW) core, or a hybrid or alternative core type. Alternatively, core 3090 can be a dedicated core, such as, for example, a network or communication core, a compression engine, a coprocessor core, a general-purpose computing graphics processing unit (GPGPU) core, a graphics core, etc.

[0308] Front-end unit 3030 includes branch prediction unit 3032 coupled to instruction cache unit 3034 coupled to instruction translation lookup buffer (TLB) 3036 coupled to instruction fetch unit 3038 coupled to decoding unit 3040. Decoding unit 3040 (or decoder) decodes instructions and generates one or more micro-operations, microcode entry points, microinstructions, other instructions, or other control signals as output, which are decoded from, or otherwise reflect, the original instructions or derived from them. Decoding unit 3040 can be implemented using various different mechanisms. Examples of suitable mechanisms include, but are not limited to, lookup tables, hardware implementations, programmable logic arrays (PLAs), microcode read-only memory (ROM), etc. In one embodiment, core 3090 includes microcode ROM or other media (e.g., in decoding unit 3040, or otherwise within front-end unit 3030) storing microcode for certain macro instructions. The decoding unit 3040 is coupled to the rename / allocator unit 3052 in the execution engine unit 3050.

[0309] Execution engine unit 3050 includes a renaming / allocator unit 3052 coupled to a retirement unit 3054 and a set 3056 of one or more scheduler units. The scheduler units 3056 represent any number of different schedulers, including reservation stations, central instruction windows, etc. The scheduler units 3056 are coupled to physical register file units 3058. Each physical register file unit in the physical register file units 3058 represents one or more physical register files, where different physical register files store one or more different data types, such as scalar integers, scalar floating-point numbers, compressed integers, compressed floating-point numbers, vector integers, vector floating-point numbers, status (e.g., an instruction pointer as the address of the next instruction to be executed), etc. In one embodiment, the physical register file units 3058 include vector register units and scalar register units. These register units may provide architectural vector registers, vector mask registers, and general-purpose registers. Multiple physical register file units 3058 are overlapped by retirement units 3054 to illustrate various ways in which register renaming and out-of-order execution can be implemented (e.g., using multiple reorder buffers and multiple retirement register files; using multiple future files, multiple history buffers, and multiple retirement register files; using register mappings and register pools, etc.). Retirement units 3054 and multiple physical register file units 3058 are coupled to multiple execution clusters 3060. Multiple execution clusters 3060 include a set 3062 of one or more execution units and a set 3064 of one or more memory access units. Execution units 3062 can perform various operations (e.g., shift, addition, subtraction, multiplication) and can perform operations on various data types (e.g., scalar floating-point, compressed integer, compressed floating-point, vector integer, vector floating-point). While some embodiments may include multiple execution units dedicated to a particular function or set of functions, other embodiments may include only one execution unit or multiple execution units that all perform all functions. Multiple scheduler units 3056, multiple physical register file units 3058, and multiple execution clusters 3060 are shown as potentially multiple because some embodiments create separate pipelines for certain types of data / operations (e.g., scalar integer pipelines, scalar floating-point / compact integer / compact floating-point / vector integer / vector floating-point pipelines, and / or memory access pipelines each having their own scheduler unit, multiple physical register file units, and / or execution clusters—and in the case of separate memory access pipelines, some embodiments are implemented where only the execution cluster of that pipeline has multiple memory access units 3064). It should also be understood that, in the case of using separate pipelines, one or more of these pipelines may be out-of-order deployment / execution, and the remaining pipelines may be ordered.

[0310] A set of memory access units 3064 is coupled to a memory unit 3070, which includes a data TLB unit 3072, which is coupled to a data cache unit 3074, which is coupled to a Level 2 (L2) cache unit 3076. In one exemplary embodiment, the memory access unit 3064 may include a load unit, a memory address unit, and a memory data unit, each of which is coupled to the data TLB unit 3072 in the memory unit 3070. An instruction cache unit 3034 is also coupled to the Level 2 (L2) cache unit 3076 in the memory unit 3070. The L2 cache unit 3076 is coupled to one or more other levels of cache and ultimately to main memory.

[0311] As an example, the exemplary out-of-order release / execution core architecture for register renaming can implement pipeline 3000 as follows: 1) Instruction fetch 3038 executes fetch stage 3002 and length decoding stage 3004; 2) Decoding unit 3040 executes decoding stage 3006; 3) Rename / allocator unit 3052 executes allocation stage 3008 and rename stage 3010; 4) (multiple) scheduler units 3056 execute scheduling stage 3012; 5) (multiple) physical register file units 3058 and memory unit 3070 execute register read / memory read stage 3014; execution cluster 3060 executes execution stage 3016; 6) memory unit 3070 and (multiple) physical register file units 3058 execute write-back / memory write stage 3018; 7) each unit may be involved in exception handling stage 3022; and 8) retirement unit 3054 and (multiple) physical register file units 3058 execute commit stage 3024.

[0312] Core 3090 may support one or more instruction sets (e.g., the x86 instruction set (with some extensions added with newer versions); the MIPS instruction set of MIPS Technologies, Inc., Sunnyvale, California; the ARM instruction set of ARM Holdings, Inc., Sunnyvale, California (with optional additional extensions such as NEON)), including the instructions described herein. In one embodiment, Core 3090 includes logic for supporting compressed data instruction set extensions (e.g., AVX1, AVX2), thereby allowing the use of compressed data to perform operations used by many multimedia applications.

[0313] It should be understood that a core can support multithreading (execution of two or more parallel operations or a set of threads), and this multithreading can be accomplished in various ways, including time-division multithreading, simultaneous multithreading (where a single physical core provides a logical core for each thread in the simultaneous multithreading of the physical core), or combinations thereof (e.g., time-division fetching and decoding, and subsequent operations such as...). Simultaneous multithreading in hyper-threading technology.

[0314] Although register renaming is described in the context of out-of-order execution, it should be understood that register renaming can also be used in ordered architectures. While the illustrated processor embodiment also includes separate instruction and data cache units 3034 / 3074 and a shared L2 cache unit 3076, alternative embodiments may have a single internal cache for both instructions and data, such as, for example, a Level 1 (L1) internal cache or multiple levels of internal caches. In some embodiments, the system may include a combination of internal caches and external caches located outside the core and / or processor. Alternatively, all caches may be located outside the core and / or processor.

[0315] Specific exemplary ordered core architecture

[0316] Figures 31A-31B The diagram illustrates a more specific exemplary ordered core architecture, where the core is one of several logic blocks within a chip (including other cores of the same and / or different types). Depending on the application, the logic block communicates with some fixed functional logic, memory I / O interfaces, and other necessary I / O logic via a high-bandwidth interconnect network (e.g., a ring network).

[0317] Figure 31A This is a block diagram of a single processor core according to an embodiment of the invention, its connection to an on-die interconnect network 3102, and a local subset 3104 of its second-level (L2) cache. In one embodiment, the instruction decoder 3100 supports the x86 instruction set with a compact data instruction set extension. The L1 cache 3106 allows low-latency access to cache memory into scalar and vector units. Although in one embodiment (for design simplification), scalar unit 3108 and vector unit 3110 use separate register sets (scalar register 3112 and vector register 3114, respectively), and data transferred between these registers is written to memory and subsequently read back from the first-level (L1) cache 3106, alternative embodiments of the invention may use different methods (e.g., using a single register set or including a communication path that allows data to be transferred between these two register sets without being written and read back).

[0318] The local subset 3104 of the L2 cache is part of the global L2 cache, which is divided into multiple separate local subsets, one for each processor core. Each processor core has a direct access path to its own local subset 3104 of the L2 cache. Data read by a processor core is stored in its L2 cache subset 3104 and can be accessed quickly in parallel with other processor cores accessing their own local L2 cache subsets. Data written by a processor core is stored in its own L2 cache subset 3104 and is dumped and cleared from other subsets if necessary. The ring network ensures the consistency of shared data. The ring network is bidirectional to allow agents such as processor cores, L2 caches, and other logic blocks to communicate with each other within the chip. In some embodiments, each ring data path is 1024 bits wide in each direction.

[0319] Figure 31B This is according to an embodiment of the present invention. Figure 31A An expanded diagram of a portion of the processor core. Figure 31B This includes the L1 data cache 3106A portion of the L1 cache 3104, and further details regarding the vector unit 3110 and the vector register 3114. Specifically, the vector unit 3110 is a 32-width vector processing unit (VPU) (see 16-width ALU 3128) that executes one or more of integer, single-precision floating-point, and double-precision floating-point instructions. This VPU supports mixing of register inputs via mixing unit 3120, numerical conversion via value conversion units 3122A-B, and copying of memory inputs via copy unit 3124.

[0320] Processor with integrated memory controller and graphics device

[0321] Figure 32 This is a block diagram of a processor 3200 according to an embodiment of the present invention, which may have more than one core, may have an integrated memory controller, and may have an integrated graphics device. Figure 32 The solid-line box illustration shows a processor 3200 having a single core 3202A, a system agent 3210, and a collection of one or more bus controller units 3216, while the optional additional illustration with dashed boxes shows an alternative processor 3200 having multiple cores 3202A-N, a collection of one or more integrated memory controller units 3214 from the system agent units 3210, and dedicated logic 3208.

[0322] Therefore, different implementations of processor 3200 may include: 1) a CPU, wherein dedicated logic 3208 is integrated graphics and / or scientific (throughput) logic (which may include one or more cores), and cores 3202A-N are one or more general-purpose cores (e.g., general-purpose ordered cores, general-purpose out-of-order cores, or a combination of both); 2) a coprocessor, wherein cores 3202A-N are a large number of dedicated cores designed primarily for graphics and / or scientific (throughput); and 3) a coprocessor, wherein cores 3202A-N are a large number of general-purpose ordered cores. Thus, processor 3200 may be a general-purpose processor, coprocessor, or dedicated processor, such as, for example, a network or communication processor, a compression engine, a graphics processor, a GPGPU (General-Purpose Graphics Processing Unit), a high-throughput integrated many-core (MIC) coprocessor (including 30 or more cores), an embedded processor, etc. The processor may be implemented on one or more chips. Processor 3200 may be part of one or more substrates, and / or may be implemented on one or more substrates using any of a variety of process technologies (e.g., BiCMOS, CMOS, or NMOS).

[0323] The memory hierarchy includes one or more levels of cache within cores 3204A-N, a set 3206 of one or more shared cache units, and external memory (not shown) coupled to a set 3214 of integrated memory controller units. The set 3206 of shared cache units may include one or more intermediate levels of cache, such as Level 2 (L2), Level 3 (L3), Level 4 (L4), or other levels of cache, Last Level Cache (LLC), and / or combinations thereof. While in one embodiment, ring-based interconnect units 3212 interconnect integrated graphics logic 3208, the set 3206 of shared cache units, and system proxy units 3210 / (multiple) of integrated memory controller units 3214, alternative embodiments may use any number of known techniques to interconnect such units. In one embodiment, consistency is maintained between one or more cache units 3206 and cores 3202A-N.

[0324] In some embodiments, one or more of the cores 3202A-N are capable of multithreading. System agent 3210 includes those components that coordinate and operate the cores 3202A-N. System agent unit 3210 may include, for example, a power control unit (PCU) and a display unit. The PCU may be, or may include, the logic and components required to regulate the power state of the cores 3202A-N and the integrated graphics logic 3208. The display unit is used to drive one or more externally connected displays.

[0325] The 3202A-N cores can be homogeneous or heterogeneous in terms of their instruction set architecture; that is, two or more cores in the 3202A-N cores may be able to execute the same instruction set, while other cores may be able to execute only a subset of that instruction set or a different instruction set.

[0326] D. Exemplary Computer Architecture

[0327] Figures 33-36 This is a block diagram of an exemplary computer architecture. Other system designs and configurations known in the art for laptops, desktops, handheld PCs, personal digital assistants, engineering workstations, servers, network devices, network hubs, switches, embedded processors, digital signal processors (DSPs), graphics devices, video game devices, set-top boxes, microcontrollers, cellular phones, portable media players, handheld devices, and various other electronic devices are also suitable. Generally, a wide variety of systems or electronic devices capable of incorporating processors and / or other execution logic as disclosed herein are generally suitable.

[0328] Now for reference Figure 33 The diagram illustrates a system 3300 according to an embodiment of the present invention. System 3300 may include one or more processors 3310, 3315 coupled to a controller hub 3320. In one embodiment, the controller hub 3320 includes a graphics memory controller hub (GMCH) 3390 and an input / output hub (IOH) 3350 (which may be on separate chips); the GMCH 3390 includes memory and a graphics controller, to which a memory 3340 and a coprocessor 3345 are coupled; the IOH 3350 couples an input / output (I / O) device 3360 to the GMCH 3390. Alternatively, one or both of the memory and the graphics controller may be integrated within a processor (as described herein), with the memory 3340 and the coprocessor 3345 directly coupled to the processor 3310, and the controller hub 3320 and the IOH 3350 residing on a single chip.

[0329] The optionality of the additional processor 3315 is Figure 33 The numbers are indicated by dashed lines. Each processor 3310, 3315 may include one or more of the processing cores described herein, and may be a version of processor 3200.

[0330] The memory 3340 may be, for example, dynamic random access memory (DRAM), phase-change memory (PCM), or a combination of both. In at least one embodiment, the controller hub 3320 communicates with the processor(s) 3310, 3315 via a multi-branch bus such as a front-side bus (FSB), a point-to-point interface, or a similar connection 3395.

[0331] In one embodiment, the coprocessor 3345 is a dedicated processor, such as, for example, a high-throughput MIC processor, a network or communication processor, a compression engine, a graphics processor, a GPGPU, an embedded processor, and so on. In one embodiment, the controller hub 3320 may include an integrated graphics accelerator.

[0332] There can be various differences between physical resources 3310 and 3315 in terms of a range of quality metrics, including architecture, microarchitecture, thermal and power consumption characteristics.

[0333] In one embodiment, processor 3310 executes instructions that control general-type data processing operations. Embedded within these instructions may be coprocessor instructions. Processor 3310 recognizes these coprocessor instructions as having a type that should be executed by an attached coprocessor 3345. Therefore, processor 3310 issues these coprocessor instructions (or control signals representing coprocessor instructions) to coprocessor 3345 on a coprocessor bus or other interconnect. Coprocessors 3345 (multiple) receive and execute the received coprocessor instructions.

[0334] See now Figure 34 The diagram shown is a block diagram of a first, more specific, exemplary system 3400 according to an embodiment of the present invention. Figure 34 As shown, the multiprocessor system 3400 is a point-to-point interconnect system and includes a first processor 3470 and a second processor 3480 coupled via a point-to-point interconnect 3450. Each of processors 3470 and 3480 may be a version of processor 3200. In one embodiment of the invention, processors 3470 and 3480 are processors 3310 and 3315, respectively, and coprocessor 3438 is coprocessor 3345. In another embodiment, processors 3470 and 3480 are processor 3310 and coprocessor 3345, respectively.

[0335] Processors 3470 and 3480 are shown as including integrated memory controller (IMC) units 3472 and 3482, respectively. Processor 3470 also includes point-to-point (PP) interfaces 3476 and 3478 as part of its bus controller unit; similarly, the second processor 3480 includes PP interfaces 3486 and 3488. Processors 3470 and 3480 can exchange information via PP interface 3450 using point-to-point (PP) interface circuitry 3478 and 3488. Figure 34 As shown, IMC 3472 and 3482 couple the processor to the corresponding memory, namely memory 3432 and memory 3434, which may be portions of the main memory locally attached to the corresponding processor.

[0336] Processors 3470 and 3480 can exchange information with chipset 3490 via respective PP interfaces 3452 and 3454 using point-to-point interface circuits 3476, 3494, 3486, and 3498. Chipset 3490 can optionally exchange information with coprocessor 3438 via high-performance interface 3492. In one embodiment, coprocessor 3438 is a dedicated processor, such as, for example, a high-throughput MIC processor, a network or communication processor, a compression engine, a graphics processor, a GPGPU, an embedded processor, etc.

[0337] A shared cache (not shown) may be included in either processor or external to both processors but connected to them via a PP interconnect, such that if the processors are in a low-power mode, the local cache information of either or both processors may be stored in the shared cache.

[0338] Chipset 3490 can be coupled to first bus 3416 via interface 3496. In one embodiment, first bus 3416 may be a peripheral component interconnect (PCI) bus or a bus such as PCI fast bus or another I / O interconnect bus, but the scope of the invention is not limited thereto.

[0339] like Figure 34 As shown, various I / O devices 3414 may be coupled to a first bus 3416 together with a bus bridge 3418, which couples the first bus 3416 to a second bus 3420. In one embodiment, one or more additional processors 3415, such as a coprocessor, a high-throughput MIC processor, a GPGPU, an accelerator (such as, for example, a graphics accelerator or digital signal processing (DSP) unit), a field-programmable gate array, or any other processor, are coupled to the first bus 3416. In one embodiment, the second bus 3420 may be a low pin count (LPC) bus. In one embodiment, various devices may be coupled to the second bus 3420, including, for example, a keyboard and / or mouse 3422, a communication device 3427, and a storage unit 3428, such as a disk drive that may include instruction / code and data 3430, or other mass storage devices. Additionally, audio I / O 3424 may be coupled to the second bus 3420. Note that other architectures are possible. For example, instead of… Figure 34 The point-to-point architecture allows the system to implement multi-branch buses or other similar architectures.

[0340] Now for reference Figure 35 The diagram shown is a block diagram of a second, more specific, exemplary system 3500 according to an embodiment of the present invention. Figure 34 and 35Similar elements in the figure use similar reference numerals, and from Figure 35 The middle part is omitted Figure 34 certain aspects to avoid confusion Figure 35 Other aspects.

[0341] Figure 35 The illustrated processors 3470 and 3480 may include integrated memory and I / O control logic (“CL”) 3572 and 3582, respectively. Therefore, CL 3572 and 3582 include an integrated memory controller unit and I / O control logic. Figure 35 The diagram shows that not only are the memories 3432 and 3434 coupled to CLs 3572 and 3582, but the I / O device 3514 is also coupled to the control logic 3572 and 3582. The conventional I / O device 3515 is coupled to the chipset 3490.

[0342] Now for reference Figure 36 The diagram shown is a block diagram of a SoC 3600 according to an embodiment of the present invention. Figure 32 Similar elements in the figure use similar reference numerals. Additionally, dashed boxes are optional features on more advanced SoCs. Figure 36 In this embodiment, multiple interconnect units 3602 are coupled to: an application processor 3610, which includes an assembly of one or more cores 3202A-N, cache units 3204A-N, and multiple shared cache units 3206; a system proxy unit 3210; multiple bus controller units 3216; multiple integrated memory controller units 3214; an assembly of one or more coprocessors 3620, which may include integrated graphics logic, an image processor, an audio processor, and a video processor; a static random access memory (SRAM) unit 3630; a direct memory access (DMA) unit 3632; and a display unit 3640 for coupling to one or more external displays. In one embodiment, the multiple coprocessors 3620 include dedicated processors, such as, for example, network or communication processors, compression engines, GPGPUs, high-throughput MIC processors, or embedded processors, etc.

[0343] The various embodiments of the mechanisms disclosed herein can be implemented in hardware, software, firmware, or a combination of such implementations. Embodiments of the invention can be implemented as computer programs or program code executable on a programmable system including at least one processor, a storage system (including volatile and non-volatile memories and / or storage elements), at least one input device, and at least one output device.

[0344] Program code (such as, Figure 34The code 3430 shown in the diagram is applied to input instructions to perform the functions described herein and generate output information. The output information can be applied to one or more output devices in a known manner. For the purposes of this application, the processing system includes any system having a processor, such as, for example, a digital signal processor (DSP), a microcontroller, an application-specific integrated circuit (ASIC), or a microprocessor.

[0345] The program code can be implemented using a high-level procedural or object-oriented programming language to communicate with the processing system. Assembly or machine language can also be used if needed. In fact, the mechanisms described herein are not limited to any particular programming language. In any case, the language can be a compiled or interpreted language.

[0346] One or more aspects of at least one embodiment can be implemented by representational instructions stored on a machine-readable medium, which represent various logics in a processor, and which, when read by a machine, cause the machine to manufacture logic for performing the techniques described herein. Such representations, referred to as “IP cores,” can be stored on tangible machine-readable media and can be supplied to various customers or production facilities for loading into manufacturing machines that actually manufacture the logic or processor.

[0347] Such machine-readable storage media can include, but are not limited to, non-transitory, tangible arrangements of articles made or formed by a machine or device, including storage media such as hard disks; any other type of disk, including floppy disks, optical disks, compact disc read-only memory (CD-ROM), rewritable compact discs (CD-RW), and magneto-optical disks; semiconductor devices such as read-only memory (ROM), random access memory (RAM) such as dynamic random access memory (DRAM) and static random access memory (SRAM), erasable programmable read-only memory (EPROM), flash memory, electrically erasable programmable read-only memory (EEPROM); phase-change memory (PCM); magnetic cards or optical cards; or any other type of medium suitable for storing electronic instructions.

[0348] Therefore, embodiments of the invention also include a non-transitory tangible machine-readable medium containing instructions or design data, such as a hardware description language (HDL), that defines the architectures, circuits, devices, processors, and / or system features described herein. These embodiments are also referred to as program products.

[0349] E. Simulation (including binary conversion, code transformation, etc.)

[0350] In some cases, instruction translators can be used to translate instructions from a source instruction set to a target instruction set. For example, an instruction translator can transform instructions (e.g., using static binary transformation, including dynamically compiled binary transformation), modify, emulate, or otherwise convert them into one or more other instructions to be processed by the kernel. Instruction translators can be implemented in software, hardware, firmware, or a combination thereof. Instruction translators can be on the processor, off the processor, or partially on and partially off the processor.

[0351] Figure 37 This is a block diagram illustrating the use of a software instruction converter to convert binary instructions in a source instruction set into binary instructions in a target instruction set, according to an embodiment of the present invention. In the illustrated embodiment, the instruction converter is a software instruction converter; however, alternatively, the instruction converter can be implemented using software, firmware, hardware, or various combinations thereof. Figure 37 A program of the form of a high-level language 3702 can be compiled using a first compiler 3704 to generate first binary code (e.g., x86) 3706 that can be natively executed by a processor 3716 having at least one first instruction set core. In some embodiments, a processor 3716 having at least one first instruction set core represents any processor that performs substantially the same function as an Intel processor having at least one x86 instruction set core by compatibly executing or otherwise processing: 1) a substantial portion of the instruction set of an Intel x86 instruction set core, or 2) a version of object code for an application or other software aimed at running on an Intel processor having at least one x86 instruction set core to achieve substantially the same results as an Intel processor having at least one x86 instruction set core. The first compiler 3704 represents a compiler operable to generate binary code 3706 (e.g., object code) in the first instruction set, which can be executed on a processor 3716 having at least one first instruction set core, with or without additional linking processing. Similarly, Figure 37A program of the form of a high-level language 3702 can be compiled using an alternative instruction set compiler 3708 to generate an alternative instruction set binary code 3710 that can be natively executed by a processor 3714 that does not have at least one first instruction set core (e.g., a processor with a core that executes the MIPS instruction set of MIPS Technologies, Inc., Sunnyvale, California, and / or the ARM instruction set of ARM Holdings, Inc., Sunnyvale, California). An instruction converter 3712 is used to translate the first binary code 3706 into code that can be natively executed by the processor 3714 that does not have a first instruction set core. This translated code is unlikely to be identical to the alternative instruction set binary code 3710 because an instruction converter capable of doing so would be difficult to manufacture; however, the translated code will perform general operations and consists of instructions from the alternative instruction set. Therefore, the instruction converter 3712 represents, through emulation, simulation, or any other process, software, firmware, hardware, or a combination thereof that allows a processor or other electronic device without a first instruction set processor or core to execute the first binary code 3706.

Claims

1. A processor, comprising: A decoding circuit is used to decode a single matrix instruction having fields for an opcode, multiple identifiers, and bits. The multiple identifiers correspond to a first plurality of 4-bit data elements of a first source matrix, a second plurality of 4-bit data elements of a second source matrix, a plurality of double-word source data elements of a third source matrix, and a plurality of double-word result data elements of a result matrix. The bits indicate whether one or both of the first plurality of 4-bit data elements and the second plurality of 4-bit data elements are signed or unsigned. as well as An execution circuit is configured to execute the single matrix instruction. The execution circuit includes a multiplication-accumulation circuit, which comprises: A multiplier is used to multiply each 4-bit data element in a first subset of the first plurality of 4-bit data elements with the corresponding 4-bit data element in the first subset of the second plurality of 4-bit data elements to generate multiple products. as well as An accumulator is used to add the plurality of products to the corresponding double-word source data elements among the plurality of double-word source data elements to generate the corresponding double-word result data element among the plurality of double-word result data elements.

2. The processor as claimed in claim 1, wherein, The first subset of the first plurality of 4-bit data elements includes eight 4-bit data elements, and the first subset of the second plurality of 4-bit data elements includes eight 4-bit data elements.

3. The processor as claimed in claim 2, wherein, Multiple 4-bit data elements of the first source matrix are used to be stored in a first or more source vector registers, and multiple 4-bit data elements of the second source matrix are used to be stored in a second or more source vector registers.

4. The processor as claimed in claim 3, wherein, The first source matrix includes a slice of the first matrix, and the second source matrix includes a slice of the second matrix.

5. The processor as claimed in claim 3, wherein, A first subset of the first plurality of 4-bit data elements is used to be stored in a first source vector register of the first one or more source vector registers, and a first subset of the second plurality of 4-bit data elements is used to be stored in a second source vector register of the second one or more source vector registers.

6. The processor according to any one of claims 1-5, wherein, The multiplier includes a set of multipliers for performing parallel multiplication of each 4-bit data element in a first subset of the first plurality of 4-bit data elements with a corresponding 4-bit data element in a first subset of the second plurality of 4-bit data elements to generate the plurality of products.

7. The processor according to any one of claims 1-6, wherein, The 4-bit data element includes a 4-bit integer data element, and the double-word data element includes a double-word integer element.

8. The processor according to any one of claims 1-7, wherein, The multiplication-accumulation circuit includes the multiplication-accumulation circuit of a matrix processing accelerator.

9. The processor of claim 8, further comprising: A first interface is used to couple the matrix processing accelerator to the accelerator memory; And a second interface for coupling the matrix processing accelerator to host memory.

10. A method for matrix manipulation, comprising: The processor's decoding circuit decodes a single matrix instruction, which has fields for an opcode, multiple identifiers, and bits. The multiple identifiers correspond to a first plurality of 4-bit data elements of a first source matrix, a second plurality of 4-bit data elements of a second source matrix, a plurality of double-word source data elements of a third source matrix, and a plurality of double-word result data elements of a result matrix. The bits indicate whether one or both of the first plurality of 4-bit data elements and the second plurality of 4-bit data elements are signed or unsigned. as well as The single matrix instruction is executed by the processor's execution circuit, which includes a multiplication-accumulation circuit. The multiplication-accumulation circuit includes: A multiplier is used to multiply each 4-bit data element in a first subset of the first plurality of 4-bit data elements with the corresponding 4-bit data element in the first subset of the second plurality of 4-bit data elements to generate multiple products. as well as An accumulator is used to add the plurality of products to the corresponding double-word source data elements among the plurality of double-word source data elements to generate the corresponding double-word result data element among the plurality of double-word result data elements.

11. The method of claim 10, wherein, The first subset of the first plurality of 4-bit data elements includes eight 4-bit data elements, and the first subset of the second plurality of 4-bit data elements includes eight 4-bit data elements.

12. The method of claim 11, wherein, Multiple 4-bit data elements of the first source matrix are used to be stored in a first or more source vector registers, and multiple 4-bit data elements of the second source matrix are used to be stored in a second or more source vector registers.

13. The method of claim 12, wherein, The first source matrix includes a slice of the first matrix, and the second source matrix includes a slice of the second matrix.

14. The method of claim 12, wherein, A first subset of the first plurality of 4-bit data elements is used to be stored in a first source vector register of the first one or more source vector registers, and a first subset of the second plurality of 4-bit data elements is used to be stored in a second source vector register of the second one or more source vector registers.

15. The method according to any one of claims 10-14, wherein, The multiplier includes a set of multipliers for performing parallel multiplication of each 4-bit data element in a first subset of the first plurality of 4-bit data elements with a corresponding 4-bit data element in a first subset of the second plurality of 4-bit data elements to generate the plurality of products.

16. The method according to any one of claims 10-15, wherein, The 4-bit data element includes a 4-bit integer data element, and the double-word data element includes a double-word integer element.

17. The method according to any one of claims 10-16, wherein, The multiplication-accumulation circuit includes the multiplication-accumulation circuit of a matrix processing accelerator.

18. The method of claim 17, wherein, The processor further includes: a first interface for coupling the matrix processing accelerator to an accelerator memory; and a second interface for coupling the matrix processing accelerator to a host memory.

Images