System and method for executing matrix compression and decompression instructions

Abstract

The disclosed embodiments relate to matrix compression / decompression instructions. In one example, a processor includes fetch circuitry to fetch a compression instruction having a format with fields to specify positions of an opcode and a decompressed source matrix and a compressed destination matrix; the decoding circuit is used for decoding the extracted compression instruction; and execution circuitry to, in response to the decoded compression instruction, generate a compressed result according to a compression algorithm by compressing a specified decompressed source matrix via any of: compress non-zero-value elements together and store a matrix position of each non-zero-value element in a header, or using fewer bits to represent one or more elements and using a header to identify matrix elements represented by fewer bits; and storing the compressed result to the specified compressed destination matrix.

Description

This divisional patent application is a divisional application of the patent application with application number 201910790534.1, titled "Systems and Methods for Performing Matrix Compression and Decompression Instructions", which was filed on August 26, 2019. Technical Field

[0001] The field of the present invention generally relates to computer processor architectures, and more particularly to systems and methods for performing matrix compression and decompression instructions. Background Art

[0002] In many computing tasks such as machine learning and other bulk data processing, matrices are becoming increasingly important. Deep learning is a class of machine learning algorithms. Deep learning architectures such as deep neural networks have been applied to fields including computer vision, speech recognition, natural language processing, audio recognition, social network filtering, machine translation, bioinformatics, and drug design.

[0003] Two tools for deep learning, inference and training, tend to use low-precision arithmetic. Maximizing the throughput of deep learning algorithms and computations can help meet the demands of deep learning processors, such as those that execute deep learning in data centers.

[0004] General matrix multiplication (GEMM) is an algorithm common in machine learning and also in linear algebra, statistics, and many other fields. Convolution is also commonly applied in machine learning. Instructions for performing matrix compression and decompression operations are useful in performing convolution and GEMM algorithms in a machine learning context. Brief Description of the Drawings

[0005] The present invention is illustrated by way of example and not limitation in the accompanying drawings, in which like reference numerals indicate like elements, wherein:

[0006] Figure 1A Illustrates an embodiment of a configured tile;

[0007] Figure 1B Illustrates an embodiment of a configured tile;

[0008] Figure 2 Illustrates several examples of matrix storage;

[0009] Figure 3 Illustrates an embodiment of a system utilizing a matrix (tile) operation accelerator;

[0010] Figure 4 and Figure 5 Illustrates different embodiments of how to use a matrix operation accelerator to share memory;

[0011] Figure 6Illustrates an embodiment of a matrix multiplication accumulation operation (“TMMA”) using tiles;

[0012] Figure 7 Illustrates an embodiment of a subset of an iterative execution of a chained fused multiply-accumulate instruction;

[0013] Figure 8 Illustrates an embodiment of a subset of an iterative execution of a chained fused multiply-accumulate instruction;

[0014] Figure 9 Illustrates an embodiment of a subset of an iterative execution of a chained fused multiply-accumulate instruction;

[0015] Figure 10 Illustrates an embodiment of a subset of an iterative execution of a chained fused multiply-accumulate instruction;

[0016] Figure 11 Illustrates a SIMD implementation of size power of 2 according to an embodiment, wherein the accumulator uses an input size larger than the size of the input to the multiplier;

[0017] Figure 12 Illustrates an embodiment of a system utilizing a matrix operation circuit;

[0018] Figure 13 Illustrates an embodiment of a processor core pipeline that supports matrix operations using tiles;

[0019] Figure 14 Illustrates an embodiment of a processor core pipeline that supports matrix operations using tiles;

[0020] Figure 15 Illustrates examples of matrices expressed in row-major format and column-major format;

[0021] Figure 16 Illustrates an example of the use of a matrix (tile);

[0022] Figure 17 Illustrates an embodiment of a method of using a matrix (tile);

[0023] Figure 18 Illustrates support for a configuration of the use of tiles according to an embodiment;

[0024] Figure 19 Illustrates an embodiment of a description of a supported matrix (tile);

[0025] Figure 20(A)-Figure 20(D) Illustrates an example of a register(s);

[0026] Figure 21ABlock diagram of a hardware processing component for executing TileCompress / Decompress instructions according to some embodiments;

[0027] Figure 21B Illustration of an exemplary execution of a TileCompress instruction according to some embodiments;

[0028] Figure 21C Illustration of an exemplary execution of a TileCompress instruction according to some embodiments;

[0029] Figure 21D Illustration of an exemplary execution of a TileDecompress instruction according to some embodiments;

[0030] Figure 21E Illustration of an exemplary execution of a TileCompress instruction according to some embodiments;

[0031] Figure 21F Illustration of an exemplary execution of a TileCompressQuad instruction according to some embodiments;

[0032] Figure 21G Illustration of an exemplary execution of a TileDecompress-Compress instruction according to some embodiments;

[0033] Figure 21H Illustration of an exemplary execution of a TileCompress-Decompress instruction according to some embodiments;

[0034] Figure 22 Illustration of the processing flow of a processor for executing a TileCompress instruction according to some embodiments;

[0035] Figure 23 Illustration of the processing flow of a processor for executing a TileDecompress instruction according to some embodiments;

[0036] Figure 24 Block diagram illustrating the formats of various TileCompress and TileDecompress instructions according to some embodiments;

[0037] Figure 25A-Figure 25B Block diagram illustrating a general vector-friendly instruction format and its instruction templates according to an embodiment;

[0038] Figure 25A Block diagram illustrating a general vector-friendly instruction format and its class A instruction templates according to an embodiment;

[0039] Figure 25Bis a block diagram illustrating a general vector friendly instruction format and its B - type instruction template according to an embodiment;

[0040] Figure 26A is a block diagram illustrating an exemplary special vector friendly instruction format according to an embodiment;

[0041] Figure 26B is a block diagram illustrating a field of a special vector friendly instruction format that constitutes a complete opcode field according to an embodiment;

[0042] Figure 26C is a block diagram illustrating a field of a special vector friendly instruction format that constitutes a register index field according to an embodiment;

[0043] Figure 26D is a block diagram illustrating a field of a special vector friendly instruction format that constitutes an extended operation field according to an embodiment;

[0044] Figure 27 is a block diagram of a register architecture according to an embodiment;

[0045] Figure 28A is a block diagram illustrating both an exemplary in - order pipeline and an exemplary register - renamed out - of - order issue / execution pipeline according to an embodiment;

[0046] Figure 28B is a block diagram illustrating both an exemplary embodiment of an in - order architecture core to be included in a processor and an exemplary register - renamed out - of - order issue / execution architecture core according to an embodiment;

[0047] Figure 29A-Figure 29B is a block diagram illustrating a more specific exemplary in - order core architecture that would be one of several logic blocks in a chip (including other cores of the same type and / or different types);

[0048] Figure 29A [[ID= thirty - six]]is a block diagram of a single processor core according to an embodiment and its connection to an on - die interconnect network and a local subset of its second - level (L2) cache;

[0049] Figure 29B is according to an embodiment of the Figure 29A expanded view of a part of the processor core in;

[0050] Figure 30 is a block diagram of a processor according to an embodiment that may have more than one core, may have an integrated memory controller, and may have an integrated graphics device; [[ID= forty - seven]]

[0051] Figure 31-Figure 34 is a block diagram of an exemplary computer architecture;

[0052] Figure 31 It should be noted that in the translation, the Chinese number "thirty - six" in the translation of is for better understanding of the translation process, and the actual translation should be the Arabic numeral "36" in the English text. Also, for the tags like , they are preserved exactly as they are in the original text as required.Block diagram showing a system according to an embodiment of the present invention;

[0053] Figure 32 is a block diagram of a first more specific exemplary system according to an embodiment of the present invention;

[0054] Figure 33 is a block diagram of a second more specific exemplary system according to an embodiment of the present invention;

[0055] Figure 34 is a block diagram of a system on chip (SoC) according to an embodiment of the present invention; and

[0056] Figure 35 is a block diagram of converting binary instructions in a source instruction set into binary instructions in a target instruction set using a software instruction converter according to an embodiment. Detailed Description

[0057] In the following description, numerous specific details are set forth. However, it should be understood that embodiments may be practiced without these specific details. In other instances, well-known circuits, structures, and techniques have not been shown in detail so as not to obscure the understanding of this description.

[0058] References in the specification to "an embodiment", "embodiment", "exemplary embodiment", etc., indicate that the described embodiment may include a particular feature, structure, or characteristic, but each embodiment may not necessarily include that particular feature, structure, or characteristic. Moreover, such phrases do not necessarily refer to the same embodiment. Furthermore, when a particular feature, structure, or characteristic is described in connection with an embodiment, it is considered within the knowledge of those skilled in the art to affect such feature, structure, or characteristic in connection with other embodiments whether or not explicitly described.

[0059] In many mainstream processors, processing matrices is a difficult and / or instruction-intensive task. For example, multiple rows of a matrix can be placed into multiple packed data (e.g., SIMD or vector) registers, and then multiple rows of the matrix can be operated on individually. For example, depending on the data size, multiplying two 8x2 matrices may require loading or gathering into four 8x1 packed data registers. Then, the contents of the packed data registers are multiplied. Subsequently, the resulting packed data registers are scattered back to memory. Although for small matrices, this scenario may be acceptable, gathering matrix rows in vector registers, then performing operations, and then scattering the rows back to memory is generally unacceptable for larger matrices. The disclosed embodiments provide a better solution. Discussion

[0060] Described herein are mechanisms for supporting matrix operations in computer hardware such as a central processing unit (CPU), a graphics processing unit (GPU), and an accelerator. Matrix operations utilize 2-dimensional (2-D) data structures representing one or more compact regions of memory, such as registers. Throughout this specification, these 2-D data structures are referred to as tiles. Note that a matrix can be smaller than a tile (using less than all of the tile), or can utilize multiple tiles (the matrix being larger than the size of any one tile). Throughout this specification, matrix (tile) language is used to indicate operations performed using the tiles of a matrix; whether the matrix is larger than any one tile is generally irrelevant.

[0061] Each tile can be acted upon by different operations, such as those detailed herein, including but not limited to: matrix (tile) multiplication, tile addition, tile subtraction, tile diagonal, tile zeroing, tile transpose, tile dot product, tile broadcast, tile row broadcast, tile column broadcast, tile multiplication, tile multiply and accumulate, tile shift, and so on. In addition, support for operators such as using scaling and / or biasing can be used in conjunction with these operations in the future or to support non-numerical applications, non-numerical applications such as OpenCL "local memory", data compression / decompression, and so on. Instructions for performing tile compression / decompression (TileCompress / Decompress) instructions are also described herein.

[0062] Multiple portions of storage, such as (non-volatile and volatile) memory, registers, caches, etc., are arranged as tiles having different horizontal and vertical scales. For example, a tile can have a horizontal scale of 4 (e.g., four rows of a matrix) and a vertical scale of 8 (e.g., 8 columns of a matrix). Typically, the horizontal scale is related to the element size (e.g., 2-bit, 4-bit, 8-bit, 16-bit, 32-bit, 64-bit, 128-bit, etc.). Multiple data types (single-precision floating point, double-precision floating point, integer, etc.) can be supported. Exemplary Use of Configured Slice

[0063] In some embodiments, tile parameters can be configured. For example, a given tile can be configured to provide tile options. Exemplary tile options include but are not limited to: the number of rows of the tile, the number of columns of the tile, whether the tile is valid, and whether the tile consists of a pair of tiles of equal size.

[0064] Figure 1AIllustrates an embodiment of a configured tile. As shown, 4 kB of the application memory 102 has 4 tiles of 1 kB stored thereon - tile t0 104, tile t1 106, tile t2 108, and tile t3 110. In this example, these 4 tiles are not composed of pairs, and each tile has elements arranged in rows and columns. Tiles t0 104 and t1 106 have 4-byte elements (e.g., single-precision data) with K rows and N columns, where K = 8 and N = 32. Tiles t2 108 and t3 110 have 8-byte elements (e.g., double-precision data) with K rows and N / 2 columns. Since the width of a double-precision operand is twice that of a single-precision operand, this configuration is consistent with a palette for providing tile options, providing at least 4 kB of total storage to at least 4 names. In operation, tiles can be loaded from and stored to the memory using load operations and store operations. Depending on the instruction encoding scheme used, the amount of available application memory and the size, number, and configuration of available tiles vary.

[0065] Figure 1B Illustrates an embodiment of a configured tile. As shown, 4 kB of the application memory 122 has 2 pairs of 1 kB tiles stored thereon, the first pair being tile t4L 124 and tile t4R 126, and the second pair being tile t5L 128 and tile t5R 130. As shown, the tile pairs are divided into left tiles and right tiles. In other embodiments, the tile pairs are divided into even tiles and odd tiles. In this example, each of these 4 tiles has elements arranged in rows and columns. Tiles t4L 124 and t4R 126 have 4-byte elements (e.g., single-precision floating-point data) with K rows and N columns, where K = 8 and N = 32. Tiles t5L 128 and t5R 130 have 8-byte elements (e.g., double-precision floating-point data) with K rows and N / 2 columns. Since the width of a double-precision operand is twice that of a single-precision operand, this configuration is consistent with a palette for providing tile options, providing at least 4 kB of total storage to at least 2 names. Figure 1A The four tiles use 4 names, each name naming a 1 kB tile, while Figure 1B the 2 tile pairs in can use 2 names to specify the paired tiles. In some embodiments, tile instructions accept the names of paired tiles as operands. In operation, tiles can be loaded from and stored to the memory using load operations and store operations. Depending on the instruction encoding scheme used, the amount of available application memory and the size, number, and configuration of available tiles vary.

[0066] In some embodiments, tile parameters are definable. For example, a "palette" is used to provide tile options. Exemplary options include, but are not limited to: the number of tile names, the number of bytes in a stored row, the number of rows and columns in a tile, and so on. For example, the maximum "height" (number of rows) of a tile can be defined as:

[0067] Maximum number of slices = constructed memory / (number of palette names * bytes per row).

[0068] Accordingly, the application can be written such that the fixed use of names will be able to take advantage of different memory sizes across implementation styles.

[0069] The configuration of the slice is completed using the slice configuration ("TILECONFIG") instruction, where the specific use of the slice is defined in the selected palette. The declaration includes the number of slice names to be used, the requested number of rows and columns for each name (slice), and in some embodiments, the requested data type for each slice. In some embodiments, a consistency check is performed during the execution of the TILECONFIG instruction to determine its match to the palette entry limits. Exemplary Slice Storage Type

[0070] Figure 2 Several examples of matrix storage are illustrated. In (A), the slices are stored in memory. As shown, each "row" consists of four packed data elements. To reach the next "row", a stride value is used. Note that the rows can be stored continuously in memory. Strided memory access allows access to one row and then to the next row when the slice storage does not map to the underlying memory array row width.

[0071] Loading a slice from memory and storing a slice to memory are typically strided accesses from the application memory to the packed data rows. Exemplary TILELOAD and TILESTORE instructions or other instructions for the application memory as the TILE (slice) operand in the load operation instructions are restartable in some embodiments to handle (up to) 2 * rows of page faults, unmasked floating-point exceptions, and / or interrupts for each instruction.

[0072] In (B), the matrix is stored in a slice composed of multiple registers, such as packed data registers (single instruction multiple data (SIMD) or vector registers). In this example, the slice is stacked on three physical registers. Typically, consecutive registers are used; however, this need not be the case.

[0073] In (C), the matrix is stored in a slice in non-register storage that can be accessed by a fused multiply-accumulate (FMA) circuit used in slice operations. This storage can be internal to the FMA or adjacent to the FMA. Additionally, in some embodiments, as discussed below, this storage can be used for data elements rather than for an entire row or an entire slice.

[0074] Report the supported parameters of the TMMA architecture via the CPUID. In some embodiments, the list of information includes the maximum height and the maximum SIMD scale. Configuring the TMMA architecture requires specifying the scale per tile, the element size per tile, and the palette identifier. This configuration is accomplished by executing the TILECONFIG instruction.

[0075] Successful execution of the TILECONFIG instruction enables subsequent TILE operators. The TILERELEASEALL instruction clears the tile configuration and disables TILE operations (until the next TILECONFIG instruction is executed). In some embodiments, XSAVE, XSTORE, etc. are used in context switching of using tiles. In some embodiments, 2 XCR0 bits are used in XSAVE, one for TILECONFIG metadata and one bit corresponding to the actual tile payload data.

[0076] TILECONFIG not only configures tile usage but also sets a status variable that indicates that the program is in the code region when the tile is configured. The implementation can enumerate restrictions on other instructions that can be used with the tile region, such as no use of the existing register set, etc.

[0077] Exiting the tile region typically uses the TILERELEASEALL instruction to complete. This instruction takes no parameters and quickly invalidates all tiles (indicating that the data no longer requires any saving or restoring), and clears the internal state corresponding to being in the tile region.

[0078] In some embodiments, tile operations will zero out any rows and any columns that exceed the scale specified by the tile configuration. For example, as each row is written, the tile operation will zero out the data beyond the configured number of columns (taking into account the element size). For example, for a 64 - byte row and a tile configured with 10 rows and 12 columns, the operation of writing FP32 elements will write the output / result data in 12 * 4 - byte chunks forward to each of the 10 rows, and zero out the remaining 4 * 4 - byte chunks in each row. The tile operation also completely zeros out any rows after the first 10 configured rows. When using a 1K tile with 64 - byte rows, there will be 16 rows, so in this example, the last 6 rows will also be zeroed out.

[0079] In some embodiments, when loading data, the context restore instruction (e.g., XRESTOR) forces the data beyond the configured rows of the tile to be maintained as zero. If there is no valid configuration, all rows are zeroed out. XRESTOR for tile data can load junk information in columns beyond those configured. It should not be possible for XRESTOR to clear beyond the configured number of columns because there is no element width associated with the tile configuration.

[0080] When writing the entire TILE store to memory, context save (e.g., XSAVE) exposes the entire TILE store. If XRSTOR loads useless data into the rightmost part of the tile, that data will be saved by XSAVE. For lines beyond the number specified for each tile, XSAVE will write zeros.

[0081] In some embodiments, tile instructions are restartable. Operations accessing memory allow restart after a page fault. Computational instructions handling floating-point operations also allow unmasked floating-point exceptions, by masking exceptions controlled by control and / or status registers.

[0082] To support restarting instructions after these events, these instructions store information in the start registers detailed below. Matrix (Slice) Operating System Exemplary Hardware Support

[0083] Figure 3 An embodiment of a system utilizing a matrix (tile) operation accelerator is illustrated. In this illustration, the host processor / processing system 301 passes a command 311 (e.g., a matrix manipulation operation, such as an arithmetic or matrix manipulation operation, or a load and store operation) to the matrix operation accelerator 307. However, this is shown in this way only for purposes of discussion. As detailed later, the accelerator 307 can be part of a processing core. Typically, a command 311, as a tile manipulation operator instruction, refers to the tile in register-register (“reg-reg”) or register-memory (“reg-mem”) format. Other commands such as TILESTORE, TILELOAD, TILECONFIG, etc. do not perform data operations on the tile. The command can be a decoded instruction (e.g., a micro-operation) or a macro-instruction for the accelerator 307 to handle.

[0084] In this example, the coherent memory interface 303 is coupled to the host processor / processing system 301 and the matrix operation accelerator 307 such that they can share memory. Figure 4 and Figure 5 Different embodiments showing how to use a matrix operation accelerator to share memory are illustrated. As Figure 4 shown, the host processor 401 and the matrix operation accelerator circuit 405 share the same memory 403. Figure 5Illustrate an embodiment in which the host processor 501 and the matrix operation accelerator 505 do not share memory but can access each other's memory. For example, the processor 501 can access the on-chip memory 507 and utilize its host memory 503 as normal. Similarly, the matrix operation accelerator 505 can access the host memory 503 but more typically uses its own memory 507. Note that these memories can be of different types.

[0085] In some embodiments, the matrix operation accelerator 307 includes a plurality of FMAs 309 coupled to the data buffer 305 (in some implementations, one or more of these buffers 305 are stored in the FMAs of the grid as shown). The data buffer 305 buffers the tiles loaded from memory and / or the tiles stored to memory (e.g., using tile load or tile store instructions). The data buffer can be, for example, a plurality of registers. Typically, these FMAs are arranged as a grid of chained FMAs 309 capable of reading and writing tiles. In this example, the matrix operation accelerator 307 is used to perform a matrix multiplication operation using tiles T0, T1, and T2. At least one of the tiles is accommodated in the FMA grid 309. In some embodiments, all of the tiles in the operation are stored in the FMA grid 309. In other embodiments, only a subset is stored in the FMA grid 309. As shown, T1 is accommodated while T0 and T2 are not. Note that A, B, and C refer to the matrices of these tiles, which may or may not occupy the entire space of the tile.

[0086] Figure 6 Illustrate an embodiment of a matrix multiplication accumulation operation ("TMMA") using tiles.

[0087] The number of rows in the matrix (tile A 601) matches the number of cascaded (chained) FMAs that include the latency of the computation. In some embodiments, the number of rows does not match the number of cascaded FMAs, which can be smaller or larger. Implementations can freely recycle on a smaller height grid, but the computation remains the same.

[0088] In the illustrated embodiment, the source / destination vector comes from an N-row tile (tile C 605), and the grid of FMAs 611 performs N vector-matrix operations, resulting in a complete instruction for performing the matrix multiplication of the tiles. Tile B 603 is another vector source and provides "broadcast" terms to the FMAs in each stage.

[0089] In operation, in some embodiments, elements of matrix B (stored in tile B 603) are scattered across a rectangular grid of FMAs. Matrix B (stored in tile A 601) has its row elements transposed to match the column dimension of the rectangular grid of FMAs. At each FMA in the grid, elements of A and B are multiplied and added to an incoming addend (from the figure above), and the outgoing sum is passed to the next row of the FMA (or the final output).

[0090] The latency of a single step is proportional to K (the row height of matrix B), and the dependent TMMA typically (within a single tile or across tiles) has sufficient source-destination rows to hide this latency. The implementation may also split the SIMD (packed data element) dimension M (the row height of matrix A) across time steps, but this only changes the constant by which K is multiplied. When the program specifies a K smaller than the maximum value enumerated by TMACC, the implementation uses "masking" or "early out" to freely implement this.

[0091] The latency of the entire TMMA is proportional to N*K. The throughput is proportional to N. The number of MACs per TMMA instruction is N*K*M.

[0092] Figure 7 An embodiment illustrating a subset of the iterative execution of chained fused multiply-add instructions. Specifically, this illustrates the iterative execution circuitry for one packed data element location of the destination. In this embodiment, the chained fused multiply-add operates on signed sources, where the accumulator is twice the size of the input data.

[0093] The first signed source (source 1 701) and the second signed source (source 2 703) each have four packed data elements. Each of these packed data elements stores signed data such as floating-point data. The third signed source (source 3 709) has two packed data elements, each of which stores signed data. The size of the first signed source 701 and the second signed source 703 is half the size of the third signed source (initial value or previous result) 709. For example, the first signed source 701 and the second signed source 703 may have 32-bit packed data elements (e.g., single-precision floating-point), while the third signed source 709 may have 64-bit packed data elements (e.g., double-precision floating-point).

[0094] In this illustration, only the two most significant packed data element locations of the first signed source 701 and the second signed source 703 and the most significant packed data element location of the third signed source 709 are shown. Of course, the other packed data element locations will also be processed.

[0095] As shown, the packed data elements are processed in pairs. For example, a multiplier circuit 705 multiplies the data at the most significant packed data element positions of the first signed source 701 and the second signed source 703, and a multiplier circuit 707 multiplies the data at the next most significant packed data element positions from the first signed source 701 and the second signed source 703. In some embodiments, these multiplier circuits 705 and 707 are reused for other packed data element positions. In other embodiments, additional multiplier circuits are used such that the packed data elements are processed in parallel. In some contexts, a channel sized to the size of the signed third source 709 is used to accomplish the parallel execution. An adder circuit 711 adds the results of each of these multiplications.

[0096] (Using a different adder 713 or the same adder 711), the result of the addition of the results of these multiplications is added to the data at the most significant packed data element position from the signed source 3 709.

[0097] Finally, the result of the second addition is stored in the signed destination 715 at the packed data element position corresponding to the used packed data element position from the signed third source 709, or if there is a next iteration, the result of this second addition is passed on to that next iteration. In some embodiments, a write mask is applied to this storage such that if the corresponding write mask (bit) is set, the storage occurs, and if the corresponding write mask (bit) is not set, the storage does not occur.

[0098] Figure 8 An embodiment illustrating a subset of the iterative execution of a chained fused multiply-accumulate instruction is shown. Specifically, this shows the iterative execution circuitry for one packed data element position of a destination. In this embodiment, the chained fused multiply-accumulate is operating on signed sources, where the accumulator is twice the size of the input data.

[0099] The first signed source (source 1 801) and the second signed source (source 2 803) each have four packed data elements. Each of these packed data elements stores signed data such as integer data. The third signed source (source 3 809) has two packed data elements, each of which stores signed data. The size of the first signed source 801 and the second signed source 803 is half the size of the third signed source 809. For example, the first signed source 801 and the second signed source 803 may have 32-bit packed data elements (e.g., single-precision floating point), while the third signed source 809 may have 64-bit packed data elements (e.g., double-precision floating point).

[0100] In this illustration, only the two most significant packed data element positions of the first signed source 801 and the second signed source 803 and the most significant packed data element position of the third signed source 809 are shown. Of course, other packed data element positions will also be processed.

[0100] As shown, the packed data elements are processed in pairs. For example, the data at the most significant packed data element positions of the first signed source 801 and the second signed source 803 are multiplied using multiplier circuit 805, and the data at the next most significant packed data element positions from the first signed source 801 and the second signed source 803 are multiplied using multiplier circuit 807. In some embodiments, these multiplier circuits 805 and 807 are reused for other packed data element positions. In other embodiments, additional multiplier circuits are used such that the packed data elements are processed in parallel. In some contexts, a channel sized to the size of the signed third source (initial value or previous iteration result) 809 is used to accomplish the parallel execution. The result of each of the multiple multiplications is added to the signed third source 809 using add / saturate circuit 813.

[0101] When addition results in an overly large value, the add / saturate (accumulator) circuit 813 preserves the sign of the operands. Specifically, saturation evaluation occurs for the infinite precision result between the multiple additions and the write to the destination or next iteration. When the accumulator 813 is floating point and the input terms are integers, the sum of the products and the floating point accumulator input values are converted to an infinite precision value (a fixed point number of hundreds of bits), the addition of the multiplication result and the third input is performed, and a single rounding to the actual accumulator type is performed.

[0102] Unsigned saturation means that the output value is limited to the largest unsigned number (all 1s) of that element width. Signed saturation means that the value is limited to the range between the most negative number and the most positive number of that element width (e.g., for a byte, the range is from -128 (= -2^7) to 127 (= 2^7 - 1)).

[0103] The result of the addition and saturation check is stored in the signed result 815 in the packed data element position corresponding to the packed data element position of the signed third source 809 that was used, or if there is a next iteration, the result is passed on to that next iteration. In some embodiments, a write mask is applied to this storage such that storage occurs if the corresponding write mask (bit) is set and storage does not occur if the corresponding write mask (bit) is not set.

[0104] Figure 9An embodiment of a subset of the execution of an iteration of a chained fused multiply-accumulate instruction is illustrated. Specifically, this illustrates the execution circuitry for the iteration of one packed data element location at the destination. In this embodiment, the chained fused multiply-accumulate is operating on both signed and unsigned sources, where the accumulator is four times the size of the input data.

[0105] A first signed source (Source 1 901) and a second unsigned source (Source 2 903) each have four packed data elements. Each of these packed data elements contains data such as floating-point data or integer data. A third signed source (Initial Value or Result 915) has packed data elements that store signed data. The size of the first source 901 and the size of the second source 903 are one-fourth the size of the third signed source 915. For example, the first source 901 and the second source 903 may have 16-bit packed data elements (e.g., words), while the third signed source 915 may have 64-bit packed data elements (e.g., double-precision floating-point or 64-bit integers).

[0106] In this illustration, only the most significant four packed data element positions of the first source 901 and the second source 903 are shown, as well as the most significant packed data element position of the third signed source 915. Of course, if there are any other packed data element positions, they will also be processed.

[0107] As shown, packed data elements are processed in quads. For example, data at the most significant packed data element position of first source 901 and second source 903 are multiplied using multiplier circuit 905, data at the second most significant packed data element position of first source 901 and second source 903 are multiplied using multiplier circuit 907, data at the third most significant packed data element position of first source 901 and second source 903 are multiplied using multiplier circuit 909, and data at the least significant packed data element position of first source 901 and second source 903 are multiplied using multiplier circuit 911. In some embodiments, signed packed data elements of first source 901 are sign-extended, and unsigned packed data elements of second source 903 are zero-extended prior to multiplication.

[0108] In some embodiments, these multiplier circuits 905-911 are reused for other packed data element positions. In other embodiments, additional multiplier circuits are used so that the packed data elements can be processed in parallel. In some contexts, parallel execution is accomplished using lanes the size of the signed third source 915. The results of each of these multiplications are summed using an adder circuit 913.

[0109] The result of the addition of the results of these multiplications is added (using a different adder 917 or the same adder 913) to the data at the most significant packed data element position from the signed source 3 915.

[0110] Finally, the result 919 of the second addition is stored into the packed data element position in the signed destination corresponding to the used packed data element position from the signed third source 915, or passed to the next iteration. In some embodiments, a write mask is applied to this storage such that storage occurs if the corresponding write mask (bit) is set and storage does not occur if the corresponding write mask (bit) is not set.

[0111] Figure 10 An embodiment showing a subset of the execution of iterations of a chained fused multiply - add instruction is illustrated. Specifically, this illustrates the execution circuitry for an iteration of one packed data element position of a destination. In this embodiment, the chained fused multiply - add operates on signed and unsigned sources, where the accumulator is four times the size of the input data.

[0112] The first signed source 1001 and the second unsigned source 1003 each have four packed data elements. Each of these packed data elements stores data such as floating - point data or integer data. The third signed source 1015 (initial or previous result) has a packed data element storing signed data. The size of the first source and the size of the second source are one - quarter of the size of the third signed source 1015 (initial or previous result). For example, the first source and the second source may have 16 - bit packed data elements (e.g., words), while the third signed source 1015 (initial or previous result) may have 64 - bit packed data elements (e.g., double - precision floating - point or 64 - bit integer).

[0113] In this illustration, the most significant four packed data element positions of the first signed source 1001 and the second unsigned source 1003 and the most significant packed data element position of the third signed source 1015 are shown. Of course, if there are any other packed data element positions, these would also be processed.

[0114] As shown, the packed data elements are processed in quadruples. For example, a multiplier circuit 1005 multiplies the data at the most significant packed data element positions of the first signed source 1001 and the second unsigned source 1003, a multiplier circuit 1007 multiplies the data at the next most significant packed data element positions from the first signed source 1001 and the second unsigned source 1003, a multiplier circuit 1009 multiplies the data at the third most significant packed data element positions from the first signed source 1001 and the second unsigned source 1003, and a multiplier circuit 1011 multiplies the data at the least significant packed data element positions from the first signed source 1001 and the second unsigned source 1003. In some embodiments, prior to multiplication, the signed packed data elements of the first signed source 1001 are sign extended, and the unsigned packed data elements of the second unsigned source 1003 are zero extended.

[0115] In some embodiments, these multiplier circuits 1005 - 1011 are reused for other packed data element positions. In other embodiments, additional multiplier circuits are used such that the packed data elements are processed in parallel. In some contexts, channels sized to the size of a third signed source 1015 (initial or previous result) are used to accomplish the parallel execution. An adder / saturate 1013 circuit adds the result of the addition of these multiplication results to the data at the most significant packed data element position from the third signed source 1015 (initial or previous result).

[0116] When the addition results in a value that is too large or too small for signed saturation, the adder / saturate (accumulator) circuit 1013 preserves the sign of the operands. Specifically, saturation evaluation occurs for the infinite precision result between the multiple additions and the write to the destination. When the accumulator 1013 is floating point and the input terms are integers, the sum of the products and the floating point accumulator input values are converted to an infinite precision value (a fixed point number with hundreds of bits), the addition of the multiplication result and the third input is performed, and a single rounding to the actual accumulator type is performed.

[0117] The result 1019 of the addition and saturation check is stored into the signed destination at the packed data element position corresponding to the packed data element position used from the third signed source 1015 (initial or previous result) or passed to the next iteration. In some embodiments, a write mask is applied to this storage such that if the corresponding write mask (bit) is set, the storage occurs, and if the corresponding write mask (bit) is not set, the storage does not occur.

[0118] Figure 11Illustrated is a SIMD implementation with a power-of-two size according to an embodiment, where the accumulator uses an input size larger than the size of the input to the multiplier. Note that the (to the multiplier) source and accumulator values can be signed or unsigned values. For an accumulator with a 2X input size (in other words, the size of the accumulator input value is twice the size of the packed data elements of the source), Table 1101 illustrates different configurations. For a byte-sized source, the accumulator uses a 16-bit word or half-precision floating point (HPFP) value. For a word-sized source, the accumulator uses a 32-bit integer or single-precision floating point (SPFP) value with a size of 32 bits. For a source with an SPFP or 32-bit integer size, the accumulator uses a 64-bit integer or double-precision floating point (DPFP) value with a size of 64 bits.

[0119] For an accumulator with a 4X input size (in other words, the size of the accumulator input value is four times the size of the packed data elements of the source), Table 1103 illustrates different configurations. For a byte-sized source, the accumulator uses a 32-bit integer or single-precision floating point (SPFP) value with a size of 32 bits. In some embodiments, for a word-sized source, the accumulator uses a 64-bit integer or double-precision floating point (DPFP) value with a size of 64 bits.

[0120] For an accumulator with an 8X input size (in other words, the size of the accumulator input value is eight times the size of the packed data elements of the source), Table 1105 illustrates the configuration. For a byte-sized source, the accumulator uses a 64-bit integer.

[0121] As previously noted, the matrix operation circuit can be included in the core or can be an external accelerator. Figure 12 Illustrated is an embodiment of a system utilizing a matrix operation circuit. In this illustration, multiple entities are coupled to a ring interconnect 1245.

[0122] Multiple cores, Core 0 1201, Core 1 1203, Core 2 1205, and Core N 1207 provide non-chip-based instruction support. In some embodiments, the matrix operation circuit 1251 is located in Core 1203, while in other embodiments, the matrix operation circuits 1211 and 1213 are accessible on the ring interconnect 1245.

[0123] In addition, one or more memory controllers 1223 - 1225 are provided to communicate with memories 1233 and 1231 on behalf of the cores and / or the matrix operation circuit.

[0124] Figure 13An embodiment of a processor core pipeline is illustrated that supports matrix operations using tiles. The branch prediction and decoding circuitry 1303 performs branch prediction of instructions stored in the instruction store 1301, decodes these instructions, and / or performs both branch prediction and decoding. For example, the instructions detailed herein may be stored in the instruction store. In some implementations, separate circuitry is used for branch prediction, and in some embodiments, at least some instructions are decoded into one or more micro-operations, microcode entry points, microinstructions, other instructions, or other control signals using microcode 1305. The branch prediction and decoding circuitry 1303 may be implemented using a variety of different mechanisms. Examples of suitable mechanisms include, but are not limited to, lookup tables, hardware implementations, programmable logic arrays (PLAs), microcode read-only memories (ROMs), and the like.

[0125] The branch prediction and decoding circuitry 1303 is coupled to the allocate / rename 1307 circuitry, which in some embodiments is coupled to the scheduler circuitry 1309. In some embodiments, these circuits provide register renaming, register allocation, and / or scheduling functions by performing one or more of the following steps: 1) renaming logical operand values to physical operand values (e.g., a register alias table in some embodiments); 2) assigning status bits and flags to the decoded instructions; and 3) scheduling the decoded instructions for execution on execution circuitry external to the instruction pool (e.g., using reservation stations in some embodiments).

[0126] The scheduler circuitry 1309 represents any number of different schedulers, including reservation stations, a central instruction window, and the like. The scheduler circuitry 1309 is coupled to (a) physical register file(s) 1315 or includes (a) physical register file(s) 1315. Each of (a) physical register file(s) 1315 represents one or more physical register files, where different physical register files store one or more different data types, such as scalar integer, scalar floating point, packed integer, packed floating point, vector integer, vector floating point, status (e.g., an instruction pointer as the address of the next instruction to be executed), tiles, and so on. In one embodiment, (a) physical register file(s) 1315 includes vector register circuitry, write mask register circuitry, and scalar register circuitry. These register circuits may provide architectural vector registers, vector mask registers, and general-purpose registers. (A) physical register file(s) 1315 is covered by the retirement circuitry 1317 to illustrate the various ways in which register renaming and out-of-order execution may be implemented (such as using (a) reorder buffer(s) and (a) retirement register file(s), using (a) future file(s), (a) history buffer(s), (a) retirement register file(s), using register maps and register pools, and so on). The retirement circuitry 1317 and (a) physical register file(s) 1315 are coupled to the execution circuitry 1311.

[0127] Although register renaming is described in the context of out-of-order execution, it should be understood that register renaming can be used in an in-order architecture. Although the illustrated embodiments of the processor may also include separate instruction and data cache units and a shared L2 cache unit, alternative embodiments may have a single internal cache for both instructions and data, such as, for example, a first-level (L1) internal cache, or multiple levels of internal caches. In some embodiments, the system may include a combination of an internal cache and an external cache outside of the core and / or processor. Alternatively, all caches may be outside of the core and / or processor.

[0128] The execution circuitry 1311 is a collection of one or more execution units, including a scalar circuitry 1321, a vector / SIMD circuitry 1323, and a matrix operation circuitry 1327, as well as a memory access circuitry 1325 for accessing the cache 1313. The execution circuitry performs various operations (e.g., shift, add, subtract, multiply) and operates on various data types (e.g., scalar floating point, packed integer, packed floating point, vector integer, vector floating point). Although some embodiments may include several execution units dedicated to a particular function or set of functions, other embodiments may include only one execution unit or multiple execution units that all perform all functions. The scalar circuitry 1321 performs scalar operations, the vector / SIMD circuitry 1323 performs vector / SIMD operations, and the matrix operation circuitry 1327 performs the matrix (tile) operations detailed herein.

[0129] As an example, an exemplary register-renamed, out-of-order issue / execution core architecture may implement a pipeline as follows: 1) The instruction fetch circuitry performs the fetch and length decode stage; 2) The branch and decode circuitry 1303 performs the decode stage; 3) The allocate / rename 1307 circuitry performs the allocation stage and the rename stage; 4) The scheduler circuitry 1309 performs the schedule stage; 5) The (multiple) physical register files (coupled to or included in the scheduler circuitry 1309 and the allocate / rename 1307 circuitry and the memory unit) perform the register read / memory read stage; The execution circuitry 1311 performs the execution stage; 6) The memory unit and the (multiple) physical register file units perform the write-back / memory write stage; 7) Each unit may be involved in the exception handling stage; and 8) The retirement unit and the (multiple) physical register file units perform the commit stage.

[0130] The core is approved to support one or more instruction sets (e.g., the x86 instruction set (with some extensions added with newer versions); the MIPS instruction set of MIPS Technologies, Inc. in Sunnyvale, California; the ARM instruction set of ARM Holdings, Inc. in Sunnyvale, California (with optional additional extensions such as NEON)), which includes the instruction(s) described herein. In one embodiment, the core 1390 includes logic for supporting the Advanced Vector Extensions (e.g., AVX1, AVX2), thereby allowing the use of vector data to perform operations used by many multimedia applications.

[0131] It should be understood that the core is approved to support multithreading (execution of a set of two or more parallel operations or threads), and this multithreading can be accomplished in various ways, including time division multithreading, simultaneous multithreading (where a single physical core provides a logical core for each of the threads being simultaneously multithreaded by the physical core), or a combination thereof (e.g., time division fetching and decoding and thereafter simultaneous multithreading such as in hyperthreading technology).

[0132] Figure 14 An embodiment of a processor core pipeline is illustrated, which supports matrix operations using tiles. The branch prediction and decoding circuit 1403 performs branch prediction, decoding of the instructions, and / or both branch prediction and decoding on the instructions stored in the instruction store 1401. For example, the instructions detailed herein can be stored in the instruction store. In some implementations, separate circuits are used for branch prediction, and in some embodiments, at least some of the instructions are decoded into one or more micro-operations, microcode entry points, microinstructions, other instructions, or other control signals using the microcode 1405. The branch prediction and decoding circuit 1403 can be implemented using a variety of different mechanisms. Examples of suitable mechanisms include but are not limited to lookup tables, hardware implementations, programmable logic arrays (PLAs), microcode read-only memories (ROMs), etc.

[0133] The branch prediction and decoding circuit 1403 is coupled to the allocate / rename 1407 circuit, which in some embodiments is coupled to the scheduler circuit 1409. In some embodiments, these circuits provide register renaming, register allocation, and / or scheduling functions by performing one or more of the following steps: 1) renaming logical operand values to physical operand values (e.g., register alias tables in some embodiments); 2) assigning status bits and flags to the decoded instructions; and 3) scheduling the decoded instructions for execution on execution circuits external to the instruction pool (e.g., using reservation stations in some embodiments).

[0134] The scheduler circuitry 1409 represents any number of different schedulers, including reservation stations, central instruction windows, etc. The (multiple) scheduler units scheduler circuitry 1409 is coupled to the (multiple) physical register files 1415 or includes the (multiple) physical register files 1415. Each of the (multiple) physical register files 1415 represents one or more physical register files, where different physical register files store one or more different data types, such as scalar integers, scalar floating-point, packed integers, packed floating-point, vector integers, vector floating-point, status (e.g., instruction pointer as the address of the next instruction to be executed), slices, and so on. In one embodiment, the (multiple) physical register files 1415 include vector register circuitry, write mask register circuitry, and scalar register circuitry. These register circuits may provide architectural vector registers, vector mask registers, and general-purpose registers. The (multiple) physical register files 1415 are covered by the retirement circuitry 1417 to illustrate various ways in which register renaming and out-of-order execution can be implemented (such as using the (multiple) reorder buffers and the (multiple) retirement register files, using the (multiple) future file, the (multiple) history buffers, the (multiple) retirement register files, using register maps and register pools, etc.). The retirement circuitry 1417 and the (multiple) physical register files 1415 are coupled to the execution circuitry 1411.

[0135] Although register renaming is described in the context of out-of-order execution, it should be understood that register renaming can be used in an in-order architecture. While the illustrated embodiments of the processor may also include separate instruction and data cache units as well as a shared L2 cache unit, alternative embodiments may also have a single internal cache for both instructions and data, such as, for example, a first-level (L1) internal cache, or multiple levels of internal caches. In some embodiments, the system may include a combination of internal caches and external caches outside the core and / or the processor. Alternatively, all caches may be outside the core and / or the processor.

[0136] The execution circuitry 1411 includes a set of one or more execution circuits 1427 and a set of one or more memory access circuits 1425 for accessing the cache 1413. The execution circuits 1427 perform the matrix (slice) operations detailed herein.

[0137] As an example, an exemplary register-renamed, out-of-order issue / execution core architecture may implement a pipeline as follows: 1) An instruction fetch circuit performs the fetch and length decoding stage; 2) A branch and decode circuit 1403 performs the decoding stage; 3) An allocate / rename 1407 circuit performs the allocation stage and the rename stage; 4) A scheduler circuit 1409 performs the scheduling stage; 5) (Multiple) physical register files (coupled to or included in the scheduler circuit 1409 and the allocate / rename 1407 circuit and the memory unit) perform the register read / memory read stage; An execution circuit 1411 performs the execution stage; 6) A memory unit and (multiple) physical register file units perform the write-back / memory write stage; 7) Each unit may be involved in an exception handling stage; and 8) A retirement unit and (multiple) physical register file units perform the commit stage.

[0138] The core may support one or more instruction sets (e.g., the x86 instruction set (with some extensions added with newer versions); the MIPS instruction set of MIPS Technologies, Inc., Sunnyvale, Calif.; the ARM instruction set of ARM Holdings, Inc., Sunnyvale, Calif. (with optional additional extensions such as NEON)), including the (multiple) instructions described herein. In one embodiment, the core 1490 includes logic for supporting a packed data instruction set extension (e.g., AVX1, AVX2), thereby allowing operations used by many multimedia applications to be performed using packed data.

[0139] It should be understood that the core may support multithreading (execution of two or more sets of parallel operations or threads), and this multithreading can be accomplished in various ways, including time division multithreading, simultaneous multithreading (where a single physical core provides a logical core for each of the threads for which the physical core is simultaneously multithreading), or a combination thereof (e.g., time division fetching and decoding and then simultaneous multithreading such as in hyperthreading technology). Layout

[0140] Throughout this specification, data is expressed using a row-major data layout. Column-major users should transform the items according to the orientation of the items. Figure 15 An example of a matrix expressed in row-major format and column-major format is illustrated. As shown, matrix A is a 2x3 matrix. When the matrix is stored in row-major format, the data elements of the rows are contiguous. When the matrix is stored in column-major format, the data elements of the columns are contiguous. A T *B T =(BA) T is a well-known property of matrices, where the superscript T represents the transpose. Reading column-major data as if it were row-major data results in a matrix that looks like the transposed matrix.

[0141] In some embodiments, the semantics of column-major are exploited in hardware, and the data that is column-major will swap the operand order and make the result the transpose of the matrix, but for subsequent column-major reads from memory, it is the correct non-transposed matrix.

[0142] For example, if there are two column-major matrices to be multiplied:

[0143] The input matrices will be stored in linear memory as follows (column-major): a c e b d f And g hij kl.

[0144] If those matrices are read row-major with dimensions 2x3 and 3x2, they will appear as: a c e and g h b d f i j k l

[0145] Swapping order and matrix multiplication: <l

[0146] The transposed matrix comes out and can then be stored in row-major order: ag+bh cg+dh eg+fh ai+bj ci+dj ei+fj ak+bl ck+dl ek+fl

[0147] And is used in subsequent column-major calculations, which is the correct non-transposed matrix: Exemplary Use

[0148] Figure 16 Example of the use of illustrated matrices (tiles). In this example, matrix C 1601 includes two tiles, matrix A 1603 includes one tile, and matrix B 1605 includes two tiles. This illustration shows an example of the inner loop of the algorithm for calculating matrix multiplication. In this example, two result tiles tmm0 and tmm1 from matrix C 1601 are used to accumulate the intermediate results. When one tile (tmm2) from matrix A 1603 is multiplied by two tiles from matrix B 1605, this tile is reused 2 times. Pointers are used to load new A matrix (tiles) and two new B matrix (tiles) from the directions indicated by the arrows. The outer loop adjustment for the C tiles is not shown.

[0149] The exemplary code shown includes the use of tile configuration instructions and is executed to configure tile usage, load tiles, loop for processing tiles, store tiles to memory, and release tile usage.

[0150] Figure 17 An embodiment of the use of a depicted matrix (tile) is shown. At 1701, tile usage is configured. For example, the TILECONFIG instruction is executed to configure tile usage, including setting the number of rows and columns for each tile. Typically, at 1703, at least one matrix (tile) is loaded from memory. At 1705, the matrix (tile) is used to perform at least one matrix (tile) operation. At 1707, at least one matrix (tile) is stored out to memory, and at 1709, a context switch may occur. Exemplary Configuration Slice Configuration Hardware Support

[0151] As discussed above, tile usage typically requires configuration before use. For example, it may not be necessary to fully use all rows and columns. Not configuring these rows and columns in some embodiments not only saves power but also allows the configuration to determine whether an operation will generate an error. For example, matrix multiplication in the form of (N x M)*(L x N) will generally not work if M and L are not the same.

[0152] Before using a matrix that utilizes tiles, in some embodiments, tile support will be configured. For example, configuring how many rows and columns each tile will have, which tiles will be used, etc. The TILECONFIG instruction is an improvement to the computer itself because it provides support for configuring the computer to use a matrix accelerator (either as part of a processor core or as an external device). Specifically, the execution of the TILECONFIG instruction causes the configuration to be fetched from memory and applied to the matrix (tile) settings within the matrix accelerator. Slice Use Configuration

[0153] Figure 18 Support for the configuration of tile usage according to an embodiment is shown. Memory 1801 contains tile description 1803 of the matrix (tile) to be supported.

[0154] The instruction execution resources 1811 of the processor / core 1805 store multiple aspects of the tile description 1803 into the tile configuration 1817. The tile configuration 1817 includes a palette table 1813 for specifying what tiles are configured for the palette (number of rows and columns in each tile) and a flag indicating that the matrix is in use. Specifically, the instruction execution resources 1811 are configured to use the tiles as specified by the tile configuration 1817. The instruction execution resources 1811 may also include machine-specific registers or configuration registers for indicating tile usage. Additional values are also set, such as the in-use value and the start value. The tile configuration 1817 uses the register(s) 1819 to store tile usage and configuration information.

[0155] Figure 19 An example of a description of the matrix (tile) to be supported is shown. This is the description that should be stored upon execution of the STTILECFG instruction. In this example, each field is a byte. In byte [0], the palette ID 1901 is stored. The palette ID is used to index the palette table 1813, which stores the number of bytes in the tile and the number of bytes per row of the tile associated with that ID as defined by the configuration.

[0156] Byte 1 stores the value that will be stored in the "startRow" register 1903, and byte 2 stores the value that will be stored in the register startP 1905. To support restarting the instruction after these events, these instructions store information in these registers. To support restarting the instruction after interrupt events such as those detailed above, these instructions store information in these registers. The startRow value indicates the row that should be used for restarting. The startP value indicates the position within the row that is used for storing operations when in use, and in some embodiments, the startP value indicates the lower half of the row (in the lower tile of the pair) or the upper half of the row (in the higher tile of the pair). Generally, this position within the row (column) is not required.

[0157] Successfully executing the matrix (tile) instruction will set both startRow and startP to zero, with the exceptions of TILECONFIG and STTILECFG.

[0158] It is the responsibility of the software to zero out the startRow and startP values at any time without restarting an interrupted matrix (tile) instruction. For example, an unmasked floating-point exception handler may decide to complete the operation in software and change the program counter value to another instruction, typically the next instruction. In this case, the software exception handler must zero out the startRow and startP values in the exception presented to the software exception handler by the operating system before resuming the program. The operating system will then use a resume instruction to reload those values.

[0159] Byte 3 stores an indication (1b per tile) of the pairs of tiles 1907.

[0160] Bytes 16 - 17 store the number of rows 1913 and columns 1915 of tile 0, bytes 18 - 19 store the number of rows and columns of tile 1, and so on. In other words, each 2-byte group specifies the number of rows and columns of a tile. If the 2-byte group is not used to specify tile parameters, they shall have the value zero. Specifying tile parameters for more tiles than implementation limits or palette limits results in an error. Unconfigured tiles are set to an initial state with 0 rows and 0 columns.

[0161] Finally, the configuration in memory typically ends with an end-of-description such as all zeros for several consecutive bytes. Exemplary Slice and Slice Configuration Storage

[0162] Figure An example of the (multiple) registers 1819 is illustrated. Figure 20(A) illustrates the multiple registers 1819. As shown, each tile (TMM0 2001...TMMN 2003) has separate registers, where each register stores the row size and column size of that particular tile. StartP 2011 and StartRow 2013 are stored in separate registers. One or more status registers 2015 are set (e.g., TILES_CONFIGURED = 1) to indicate that the tiles are configured for use.

[0163] Figure 20(B) illustrates the multiple registers 1819. As shown, each tile has separate registers for its rows and its columns. For example, TMM0 row configuration 2021, TMM0 column configuration 2023, StartP 2011, and StartRow 2013 are stored in separate registers. One or more status registers 2015 are set (e.g., TILES_CONFIGURED = 1) to indicate that the tiles are configured for use.

[0164] Figure 20(C) illustrates a single register 1819. As shown, the register stores the tile configuration (rows and columns per tile) 2031, StartP 2011, and StartRow 2013 in a single register as a packed data register. One or more status registers 2015 are set (e.g., TILES_CONFIGURED = 1) to indicate that the tiles are configured for use.

[0165] Figure 20(D) illustrates multiple registers 1819. As shown, a single register stores the tile configuration (rows and columns per tile) 2031. StartP and StartRow are stored in separate registers 2011 and 2013. One or more status registers 2015 are set (e.g., TILES_CONFIGURED = 1) to indicate that the tiles are configured for use.

[0166] Other combinations are envisioned, such as combining the start registers into a single register in which the start registers are shown separately, etc. ​

[0167] The disclosed embodiments describe instructions for performing compression or decompression of matrices (tiles), sometimes referred to as TileCompress or TileDecompress instructions. The disclosed embodiments compress and decompress matrices while loading them into registers or while storing them to memory. Several compression algorithms are described.

[0168] The disclosed TileCompress and TileDecompress instructions improve processor performance by reducing the size of the data transferred and maintained. With less data to transfer, memory access takes less time. Since memory access is typically a system-wide performance bottleneck, the improvement in memory access performance is expected to also improve overall system performance.

[0169] In some embodiments, the matrix is compressed before being stored in a memory such as an L1 data cache. When the compressed matrix data is stored in a lower-level cache, it advantageously maintains its smaller size throughout the different levels of the memory hierarchy, effectively increasing the size of the available memory.

[0170] In some embodiments, the matrix is compressed by replacing zero-valued elements with non-zero-valued elements, leaving only non-zero-valued elements to multiply, and avoiding the power cost of multiplying by zero.

[0171] Reference ​Illustrate and describe the processing of TileCompress / Decompress instructions according to some embodiments. Refer to ​ Illustrate and describe the format of TileCompress / Decompress instructions according to the disclosed embodiments. Refer to FIG. 28- ​ Further illustrate and describe a processor having an execution circuit for processing TileCompress / Decompress instructions. Refer to ​ Further illustrate and describe a computing system for processing TileCompress / Decompress instructions.

[0172] In operation, the result of the compression operation is generated for each element of the specified decompressed source matrix. In some embodiments, the result of the compression operation is generated for two or more elements of the specified decompressed source matrix. Some of the generated results are moved to different relative positions within the specified compressed destination matrix, e.g., to replace zero-valued elements with non-zero-valued elements. Some of the generated results maintain the same relative position within the specified destination matrix, but use fewer bits for the values, e.g., a double-precision floating-point value of the decompressed source matrix can be replaced with a few bits for specifying a dictionary entry. The elements of the specified compressed source matrix and decompressed destination matrix can be a set of values in any of floating-point registers, vector registers, tile registers, and memory.

[0173] In some embodiments, an overlay structure on physical registers is used to support tile registers. For example, depending on the implementation, a tile can utilize 16 1024-bit registers, 32 512-bit registers, etc. In some embodiments, matrix operations utilize tile registers that include a two-dimensional (2-D) data structure representing one or more compact regions of memory, such as registers. Throughout this specification, these 2-D data structures are referred to as tiles or tile registers.

[0174] Some compression techniques use headers, which can be stored in registers or memory. Refer to ​ Further illustrate and describe an exemplary processor register file for use with the disclosed embodiments. ​

[0175] The disclosed embodiments support several different compression / decompression algorithms. ​

[0176] As mentioned above, one of the disclosed compression algorithms calls a search matrix to find zero-valued elements and replaces them with non-zero-valued elements. In doing so, the disclosed compression algorithm continuously reserves all non-zero-valued elements, thereby allowing them to be read without wasting memory bandwidth reading zero-valued elements. For example, see ​ and ​ . In some embodiments, a header is stored in memory or a register and contains the logical matrix positions of the non-zero-valued elements that have been shifted. A slice may be accompanied by bit masks that specify the destinations to which elements should be written. ​

[0177] A dictionary method may also be applied for cases where most of the values are a small subset of the entire possible range. For example, refer to ​ which shows and describes such an embodiment. ​ The embodiment shown in ​

[0178] uses bit masks to indicate which elements will use the dictionary. Alternatively, each data element may include a prefix to indicate whether the dictionary is used. ​ Although not shown in any figure, a similar method to the dictionary replacement described above and ​

[0179] shown in ​ is used to replace elements within a value range with approximations. For example, assume that floating-point numbers between 4.9 and 5.1 may be replaced with a pointer to an approximate equivalent value 5.0 in a list or dictionary of approximations. ​ In some embodiments, for example, as ​ shown, a compressed matrix is associated with a bit mask header, such as

[0180] the bit mask header 2168B of

[0181] . The header has one bit for each element and is used to represent the logical matrix position of each non-zero-valued element. For example, refer to ​ , the compressed destination matrix 2168 is a 64-entry matrix with 15 non-zero-valued elements, and the 64-bit non-zero element position bit mask 2168B includes 1� bits set to '1' to represent the logical matrix positions of each non-zero-valued element. In other embodiments, the value '0' may be used to represent non-zero-valued elements. The header may be in the compressed slice, or in a different location in memory, or in a register. In some embodiments, the header and / or bit mask are compressed to obtain even more efficient compression.

[0180] In some embodiments, the matrix position is specified as an index and value to the logical position of the compressed element in its associated matrix.

[0181] In some embodiments, such as ​ the compressed matrix 2144, non-zero values and their position indices are interleaved with the values. In other embodiments, the position indices are stored in a different header or memory location.

[0182] As described above, in the dictionary method, common values are replaced with indices into a list or dictionary of values. ​

[0183] As described herein, the decompressed matrix is an array of elements, e.g., an array having M rows and N columns, and the elements have various different formats including, for example, single-precision floating point or double-precision floating point of 32-bit single-precision and 64-bit double-precision, and also including 4-bit, 8-bit, 16-bit, and 32-bit integers.

[0184] However, when compressed, the M x N matrix has fewer non-zero elements, or fewer bits per element on average, or both fewer non-zero elements and fewer bits per element on average. Specifically, in some embodiments, compressing the source matrix (slice) requires moving non-zero elements to replace zero-valued elements. The disclosed embodiments describe how to track the original positions of such elements that have been moved. ​

[0185] ​ An exemplary compressed matrix 2128 of matrix elements 2128A having 8 rows and 8 columns is illustrated, where the non-zero elements are packed to the left. Also shown is an 8-bit header non-zero position 2128B, where one header for each row indicates the original position in the original uncompressed matrix 2124 of each of the packed non-zero elements. For example, the element 'A', which is the only non-zero element in the first row of the uncompressed matrix, is packed to the left in the compressed matrix, and the 8-bit header for the first row is 0100-0000, thus indicating that 'A' occupies logical position 1 of the decompressed source matrix (slice).

[0186] As shown, each row of the compressed destination 2128 consists of an 8-bit header and up to eight non-zero elements. In some embodiments, for example, when each row of the compressed destination is stored in a 128-bit register and assuming that each of the elements is a 16-bit word, there is no remaining space in the register to store the 8-bit header. In some such embodiments, one of the register elements that would otherwise be used to store data instead stores the header. For example, in some embodiments, each row of the compressed destination consists of 7 data elements and the element used for the header.

[0187] ​Illustrated is an exemplary compressed matrix 2138 of matrix elements 2138A having 8 rows and 8 columns, where non-zero elements are compacted into the first two rows. Advantageously, the third through eighth rows of matrix elements 2138A can subsequently be ignored as they contain only zeros. Also shown is an 8-bit header non-zero position 2138B, where one header per row indicates the original position in the original uncompressed matrix 2134 of each of the compacted non-zero elements. For example, the compacted matrix elements 2138A include fourteen (14) non-zero value elements compacted into the first two rows, and the eight headers together have fourteen (14) bits set to indicate the original positions of those non-zero value elements. ​

[0188] ​ Illustrated is another exemplary format of a compressed matrix (slice). Here, the compressed source matrix 2144 has 8 rows and 8 columns and includes 9-bit matrix positions, where each of the zero-valued elements has a matrix position equal to 64. For illustrative purposes, all zero-valued elements have been shown as having a matrix position equal to 64. However, in operation, a single value 64 can be applied to the zero-valued elements. ​

[0189] ​ Illustrated is another exemplary format of a compressed matrix (slice). Here, the compressed matrix 2158A has 64 entries, 12 of which will be replaced by elements from a dictionary 2158C. The destination elements not using the dictionary are shown with 'x' to represent don't care values. The use of the dictionary 2158B is a sequence of eight headers, one header per row, which are multi-bit selectors to indicate which elements of that row will be replaced by dictionary entries. Advantageously, the elements to be replaced (otherwise consisting of 32 bits or 64 bits) only need two bits to select one of four dictionary entries. ​

[0190] In some embodiments, the compressed format requires replacing one or more elements with approximations. Here, the compressed format can follow ​ the example where one or more values of the compressed matrix will be replaced with approximations taken from a dictionary 2158C.

[0191] In some embodiments, compression requires replacing numbers within a range of values with approximation substitutions. For example, in some embodiments, referring to ​ , an execution circuit 2156 is used to replace any element having a value between 4.9 and 5.3 with 5.1. Using approximations may sacrifice some accuracy, but it can also allow more 32 / 64-bit elements to be replaced by 2-bit dictionary indices. ​

[0192] ​ is a block diagram of a hardware processing component for executing TileCompress / Decompress instructions according to some embodiments. As shown, the computing system 2100 includes a code storage 2102 (for storing TileCompress / Decompress instructions), a fetch circuit 2104 (for fetching TileCompress / Decompress instructions), and a decode circuit 2106 (for decoding the fetched instructions). The computing system 2100 further includes an execution circuit 2108, a decompression circuit 2110, a compression circuit 2112, registers 2114, a buffer 2116, a memory 2118, and optionally a retirement or commit circuit 2119. The decompression circuit 2110 and the compression circuit 2112 are shown surrounded by a dashed border, which is used to indicate that those circuits may be incorporated into the execution circuit 2108 in some embodiments.

[0193] In the context of the system shown, the decode circuit is similar to at least the decode circuits referenced ​ , ​ and ​ -B shown and described.

[0194] As shown, the TileCompress / Decompress instruction 2101 includes fields for specifying an opcode 2101A (e.g., TileCompress or TileDecompress), a destination matrix location 2101B, a compression or decompression algorithm 2101C, and a source matrix (tile) location 2101D. As disclosed herein, the source and destination matrices identified by the TileCompress / Decompress instruction may be located in any of the registers 2114, the buffer 2116, and the memory 2118.

[0195] In operation, the computing system 2100 is used to fetch the TileCompress / Decompress instruction from the code storage 2102 and decode the TileCompress / Decompress instruction using the fetch circuit 2104 and the decode circuit 2106. The execution circuit is used to respond to the decoded TileCompress / Decompress instruction by using the compression circuit 2112 to respond to a TileCompress instruction, performing arithmetic and / or logical operations on a matrix, and using the decompression circuit 2110 to respond to a TileDecompress instruction.

[0196] Refer to​ The execution circuitry is further shown and described. In some embodiments, the execution circuitry is a matrix operation accelerator, such as the accelerator shown and described as accelerator 307( ​ ). In some embodiments, the execution circuitry is a matrix operation circuit, such as matrix operation circuits 405( ​ ), 505( ​ ), or 1213( ​ ) and 1327( ​ ).

[0197] ​ The execution of the TileCompress instruction according to some embodiments is illustrated. As shown, the TileCompress instruction 2122 includes fields for specifying an opcode 2122A (e.g., TileCompress), a compressed destination matrix location 2122B, a compression algorithm 2122C, and a decompressed source matrix location 2122D. Also shown is the specified source matrix 2124, the execution circuitry 2126 including a compression circuit, and the specified compressed destination matrix 2128.

[0198] In the context of the system shown, the decoding circuitry is similar to the decoding circuitry shown and described at least with reference to ​ , ​ and ​ -B.

[0199] Reference ​ The execution circuitry is further shown and described. In some embodiments, the execution circuitry is a matrix operation accelerator, such as the accelerator shown and described as accelerator 307( ​ ). In some embodiments, the execution circuitry is a matrix operation circuit, such as matrix operation circuits 405( ​ ), 505( ​ ), or 1213( ​ ) and 1327( ​ ).

[0200] In operation, the computing system 2120 is used to process the TileCompress instruction 2122 by generating a result matrix that requires less storage than the specified decompressed source matrix by, for example, packing non-zero elements across zero-valued elements as done here, or by reducing the average bit size of matrix elements by replacing one or more floating-point values with dictionary pointers to one or more entries in a list of floating-point values, as shown in ​ . Using such a dictionary approach allows the matrix to be represented with fewer bits because each dictionary pointer includes fewer bits than its associated matrix element.

[0201] ​The figure illustrates the execution of a TileCompress instruction according to some embodiments. As shown, the TileCompress instruction 2132 includes fields for specifying an opcode 2132A (e.g., TileCompress), a compressed destination matrix location 2132B, a compression algorithm 2132C, and a decompressed source matrix location 2132D. Also shown is the specified source matrix 2134, an execution circuit 2136 including a compression circuit, and the specified compressed destination matrix 2138. Here, the non-zero value elements of the source matrix 2134 have been compacted into the first two rows of the destination matrix 2138. In operation, the processor is used to minimize the utilization of memory bandwidth by reading only the first two rows and ignoring the remaining rows.

[0202] In the context of the system shown, the decoding circuit is similar to the decoding circuits shown and described at least with reference to ​ , ​ and ​ -B.

[0203] Reference ​ further shows and describes the execution circuit. In some embodiments, the execution circuit is a matrix operation accelerator, such as the accelerator shown and described as accelerator 307 ( ​ ). In some embodiments, the execution circuit is a matrix operation circuit, such as matrix operation circuits 405 ( ​ ), 505 ( ​ ) or 1213 ( ​ ) and 1327 ( ​ ).

[0204] In operation, the computing system 2130 is used to process the TileCompress instruction 2132 by generating a result matrix that requires fewer stored bits than the specified decompressed source matrix: by compacting non-zero value elements past zero value elements as done here, or by reducing the average bit size of matrix elements by replacing one or more floating-point values with a dictionary pointer to one or more items in a list of floating-point values, as shown in ​ . Using this dictionary method allows the matrix to be represented with fewer bits because each dictionary pointer includes fewer bits than its associated matrix element.

[0205] ​Illustrated is the execution of a TileDecompress instruction according to some embodiments. As shown, the TileDecompress instruction 2142 includes fields for specifying an opcode 2142A (e.g., TileDecompress), a decompressed destination matrix location 2142B, a decompression algorithm 2142C, and a compressed source matrix location 2142D. Also shown is the specified compressed source matrix 2144, an execution circuit 2146 including a decompression circuit, and the specified decompressed destination matrix 2148.

[0206] In the context of the system shown, the decoding circuit is similar to the decoding circuit shown and described at least with reference to ​ , ​ and ​ -B.

[0207] Reference ​ further shows and describes the execution circuit. In some embodiments, the execution circuit is a matrix operation accelerator, such as the accelerator shown and described as accelerator 307 ( ​ ). In some embodiments, the execution circuit is a matrix operation circuit, such as matrix operation circuits 405 ( ​ ), 505 ( ​ ) or 1213 ( ​ ) and 1327 ( ​ ).

[0208] In operation, the computing system 2140 is used to fetch the TileDecompress instruction 2142 using a fetch circuit, the TileDecompress instruction 2142 specifying a decompression algorithm 2142C and the location of the compressed source matrix 2144 specified by 2142D, and the decompressed destination matrix 2148 specified by 2142B, where the specified compressed source matrix is generated as in ​ by packing non-zero elements by skipping zero-valued elements and storing the associated matrix position for each non-zero element moved during the packing, and the TileDecompress instruction writes each non-zero element to its associated matrix position within the specified destination matrix 2148 and writes any remaining values of the specified destination matrix 2148 as zero.

[0209] ​Illustrated is the execution of a TileCompress instruction according to some embodiments. As shown, the TileCompress instruction 2152 includes fields for specifying an opcode 2152A (e.g., TileCompress), a compressed destination matrix location 2152B, a compression algorithm 2152C, and a decompressed source matrix location 2152D. Also shown is the specified decompressed source matrix 2154, an execution circuit 2156 including a compression circuit, and the specified compressed destination matrix 2158, the specified compressed destination matrix 2158 including matrix elements 2158A and a dictionary 2158B header consisting of an 8-bit header for each row.

[0210] In the context of the system shown, the decoding circuit is similar to the decoding circuit shown and described with at least reference ​ , ​ and ​ -B.

[0211] Reference ​ further shows and describes the execution circuit. In some embodiments, the execution circuit is a matrix operation accelerator, such as the accelerator shown and described as accelerator 307 ( ​ ). In some embodiments, the execution circuit is a matrix operation circuit, such as matrix operation circuit 405 ( ​ ), 505 ( ​ ) or 1213 ( ​ ) and 1327 ( ​ ).

[0212] In operation, the computing system 2150 is used to process a TileCompress instruction by generating a result matrix that requires fewer stored bits than the specified decompressed source matrix by reducing the average bit size of matrix elements by replacing one or more floating-point values with a dictionary pointer to a dictionary or list of floating-point values, each dictionary pointer including fewer bits than its associated matrix element. Here, each dictionary pointer only requires 2 bits.

[0213] ​Illustrated is the execution of the TileCompressQuad instruction according to some embodiments. As shown, the TileCompressQuad instruction 2162 includes fields for specifying an opcode 2162A (e.g., TileCompressQuad), a compressed destination matrix location 2162B, a compression algorithm 2162C, and a decompressed source matrix location 2162D. Also shown is the specified decompressed source matrix 2164, an execution circuit 2166 including a compression circuit, and the specified compressed destination matrix 2168, the specified compressed destination matrix 2168 including matrix elements 2168A and a non-zero element position bitmask 2168B.

[0214] In the context of the system shown, the decoding circuit is similar to the decoding circuit shown and described at least with reference to ​ , ​ and ​ -B.

[0215] Reference ​ further shows and describes the execution circuit. In some embodiments, the execution circuit is a matrix operation accelerator, such as, the accelerator shown and described as accelerator 307 ( ​ ). In some embodiments, the execution circuit is a matrix operation circuit, such as, matrix operation circuit 405 ( ​ ), 505 ( ​ ), or 1213 ( ​ ) and 1327 ( ​ ).

[0216] In operation, the computing system 2160 is used to process the TileCompressQuad instruction 2162 by generating a result matrix that requires less storage than the specified decompressed source matrix by packing non-zero elements across zero-valued elements, as shown and described with reference to ​ . However, here, the uncompressed source matrix 2164 has been divided into four quadrants, and the non-zero values of each quadrant are written to different rows of the specified destination matrix.

[0217] ​The figure illustrates the execution of the TileDecompressOpCompress instruction according to some embodiments. As shown, the TileDecompressOpCompress instruction 2172 includes fields for specifying an opcode 2172A (e.g., TileDecompressOpCompress), a compressed destination matrix location 2172B, a decompression algorithm 2172C, a compressed source matrix location 2172D, an arithmetic or logical operation 2172E, and a compression algorithm 2172F. Also shown is the specified compressed source matrix 2174 including matrix elements 2174A and non-zero selections 2174B, an execution circuit 2176 including a decompression circuit, an arithmetic or logical operation circuit, and a compression circuit, and the specified compressed destination matrix 2178 including matrix elements 2178A and non-zero element position bitmasks 2178B.

[0218] Reference ​ The execution circuit is further shown and described. In some embodiments, the execution circuit is a matrix operation accelerator, such as the accelerator shown and described as accelerator 307( ​ ). In some embodiments, the execution circuit is a matrix operation circuit, such as matrix operation circuits 405( ​ ), 505( ​ ) or 1213( ​ ) and 1327( ​ ).

[0219] In operation, the computing system 2170 is used to respond to a decoded decompression instruction by copying each element of the specified source matrix to its associated matrix location in the specified destination matrix, the decoded decompression instruction specifying a compressed source matrix location, a decompression algorithm, and a decompressed destination matrix location. The processor is then used to generate a result of the operation on the specified destination matrix. Finally, the processor is used to respond to a decoded compression instruction specifying a compressed destination matrix location, a compression algorithm, and an uncompressed source matrix location including the generated result by: compressing the specified uncompressed source matrix and writing the compressed matrix and its associated matrix location to the specified compressed destination matrix.

[0220] ​Illustrated is the execution of the TileCompressOpDecompress instruction according to some embodiments. As shown, the TileCompressOpDecompress instruction 2182 includes fields for specifying an opcode 2182A (e.g., TileCompressOpDecompress), a decompressed destination matrix location 2182B, a decompression algorithm 2182C, a decompressed source matrix location 2182D, an arithmetic or logical operation 2182E, and a compression algorithm 2182F. Also shown is the specified decompressed source matrix 2184, an execution circuit 2186 including a compression circuit, an arithmetic or logical operation circuit, and a decompression circuit, and the specified decompressed destination matrix 2188.

[0221] In operation, the computing system 2 is used to respond to a decoded compression instruction specifying a compressed destination matrix location, a compression algorithm, and an uncompressed source matrix location by processing the TileCompressOpDecompress instruction: by compressing the specified uncompressed source matrix and writing the compressed matrix and its associated matrix location to the specified compressed destination matrix. The processor is then used to generate a result of an operation on the specified compressed destination matrix and then respond to a decoded decompression instruction specifying a compressed source matrix location set to the generated result, a decompression algorithm, and a decompressed destination matrix location: by copying each element of the specified source matrix to its associated matrix location in the specified destination matrix and zeroing the remaining elements of the specified destination matrix. ​

[0222] ​ Illustrated is an embodiment of a processor that executes a TileCompress instruction. At 2201, the processor is used to fetch a compression instruction using a fetch circuit, the compression instruction specifying a compression algorithm and the locations of a decompressed source matrix and a compressed destination matrix. At 2203, the fetched compression instruction is decoded. For example, the fetched TileCompress instruction is decoded by a decoding circuit such as detailed herein.

[0223] At 2205, the processor is configured to respond to a decoded compressed instruction by using an execution circuit to generate a result matrix that requires less storage than the specified decompressed source matrix by performing the following operations: packing non-zero elements by skipping zero-valued elements; or reducing the average bit size of matrix elements by replacing one or more floating-point values with a dictionary pointer to a list of floating-point values, each dictionary pointer including fewer bits than its associated matrix element. In some embodiments, at 2207, committing or retiring the executed instruction is optional (as indicated by its dashed border), which is manifested in that it can occur at different times, or not at all.

[0224] ​ FIG. illustrates an embodiment of a processor that executes a TileDecompress instruction. At 2301, the processor is configured to use a fetch circuit to fetch a compressed instruction having a format that has fields for specifying an opcode and the locations of a decompressed source matrix and a compressed destination matrix. At 2303, the fetched compressed instruction is decoded. For example, the fetched TileDecompress instruction is decoded by a decoding circuit such as at least ​ , ​ and ​ -B as detailed herein.

[0225] At 2305, the processor is configured to respond to a decoded decompression instruction by using an execution circuit to generate a compressed result according to a compression algorithm and store the compressed result into the specified compressed destination matrix, the generating the compressed result according to the compression algorithm by compressing the specified decompressed source matrix, the compressing the specified decompressed source matrix by any of the following operations: packing non-zero elements together and storing the matrix positions of each non-zero element in a header; or representing one or more elements with fewer bits and using a header to identify the matrix elements represented by the fewer bits. In some embodiments, at 2307, committing or retiring the executed instruction is optional (as indicated by its dashed border), which is manifested in that it can occur at different times, or not at all. ​

[0226] ​ is a block diagram illustrating the format of a TileCompress / Decompress instruction according to some embodiments.

[0227] As shown, the TileCompress instruction 2402 includes fields for specifying an opcode 2402A, a compressed destination location 2402B, a compression algorithm 2402C, and a decompressed source location 2402D. The compression algorithm 2402C is optional, as indicated by its dashed border, which reflects that the processor uses a predetermined default compression algorithm in some embodiments. The source location 2402D and the destination location 2402B can specify a matrix (tile) in either a set of vector registers or a set of tile registers. In some embodiments, the destination specifies a matrix (tile) in memory. The opcode 2402A is shown as including an asterisk, which is used to convey that additional prefixes and / or suffixes can be added to specify instruction behavior. Although not shown, in some embodiments, the TileCompress instruction 2402 optionally includes additional fields for specifying instruction behavior, including element size (8b, 16b, 32b, and 64b), source and destination matrix rows (M) and columns (N). Refer to ​ -B and ​ -D further shows and describes the format of the TileCompress instruction 2402.

[0228] The load-decompress and store-compress variants of the TileCompress / TileDecompress instructions are expected to be common.

[0229] It is also shown that the TileDecompress instruction 2412 includes fields for specifying an opcode 2412A, a decompressed destination location 2412B, a decompression algorithm 2412C, and a compressed source location 2412D. The decompression algorithm 2412C is optional, as indicated by its dashed border, which reflects that the processor uses a predetermined default decompression algorithm in some embodiments. The source location 2412D and the destination location 2412B can specify a matrix (tile) in either a set of vector registers or a set of tile registers. In some embodiments, the destination specifies a matrix (tile) in memory. The opcode 2412A is shown as including an asterisk, which is used to convey that additional prefixes and / or suffixes can be added to specify instruction behavior. Although not shown, in some embodiments, the TileDecompress instruction 2412 optionally includes additional fields for specifying instruction behavior, including element size (8b, 16b, 32b, and 64b), source and destination matrix rows (M) and columns (N). Refer to ​ -B and ​ -D further shows and describes the format of the TileDecompress instruction 2412.

[0230] As shown, the TileDecompressOpCompress instruction 2422 includes fields for specifying an opcode 2422A, a compressed destination location 2422B, a decompression algorithm 2422C, a compressed source location 2422D, an arithmetic or logical operation 2422E, and a compression algorithm 2422F. The source 2422D and destination location 2422B can specify a matrix (tile) in either a set of vector registers or a set of tile registers. In some embodiments, the destination specifies a matrix (tile) in memory. The opcode 2422A is shown as including an asterisk, which is used to convey that additional prefixes and / or suffixes can be added to specify instruction behavior. Although not shown, in some embodiments, the TileDecompressOpCompress instruction 2422 optionally includes additional fields for specifying instruction behavior, including element size (8b, 16b, 32b, and 64b), source and destination matrix rows (M) and columns (N). Refer to ​ -B and ​ -D further shows and describes the format of the TileDecompressOpCompress instruction 2422.

[0231] As shown, the TileCompressOpDecompress instruction 2432 includes fields for specifying an opcode 2432A, a decompressed destination location 2432B, a decompression algorithm 2432C, a decompressed source 2432D, an arithmetic or logical operation 2432E, and a compression algorithm 2432F. The source 2432D and destination location 2432B can specify a matrix (tile) in either a set of vector registers or a set of tile registers. In some embodiments, the destination specifies a matrix (tile) in memory. The opcode 2432A is shown as including an asterisk, which is used to convey that additional prefixes and / or suffixes can be added to specify instruction behavior. Although not shown, in some embodiments, the TileCompressOpDecompress instruction 2432 optionally includes additional fields for specifying instruction behavior, including element size (8b, 16b, 32b, and 64b), source and destination matrix rows (M) and columns (N). Refer to ​ -B and ​ -D further shows and describes the format of the TileCompressOpDecompress instruction 2432.

[0232] Examples of the hardware, software, etc. for executing the instructions described above are detailed herein. For example, the following description details various aspects of instruction execution, including various pipeline stages such as fetch, decode, schedule, execute, retire, etc. ​

[0233] An instruction set may include one or more instruction formats. A given instruction format may define various fields (e.g., number of bits, position of bits) to specify the operation to be performed (e.g., opcode) and the operand(s) and / or other data field(s) (e.g., mask) on which the operation is to be performed, and so on. Some instruction formats are further decomposed by the definition of instruction templates (or sub-formats). For example, an instruction template of a given instruction format may be defined as different subsets of the fields of that instruction format (the included fields are generally in the same order, but at least some fields have different bit positions as fewer fields are included), and / or defined as having a given field interpreted in a different way. Thus, each instruction of the ISA is expressed using a given instruction format (and if defined, according to a given one of the instruction templates in that instruction format), and includes fields for specifying the operation and operands. For example, an exemplary ADD (addition) instruction has a specific opcode and instruction format, the specific instruction format including an opcode field for specifying the opcode and operand fields for selecting the operands (source 1 / destination and source 2); and the appearance of the ADD instruction in the instruction stream will cause specific contents for selecting specific operands to be in the operand fields. Advanced Vector Extensions (AVX) (AVX1 and AVX2) and a set of SIMD extensions using the Vector Extensions (VEX) encoding scheme have been introduced and / or published (see, for example, the September 2014 64 and IA-32 Architectures Software Developer's Manual; and see the October 2014 Advanced Vector Extensions Programming Reference). ​

[0234] Embodiments of the (multiple) instructions described herein can be embodied in different formats. Additionally, exemplary systems, architectures, and pipelines are detailed below. Embodiments of the (multiple) instructions can be executed on such systems, architectures, and pipelines, but are not limited to those detailed.

[0235] General Vector Friendly Instruction Format

[0236] A vector friendly instruction format is an instruction format suitable for vector instructions (e.g., there are specific fields dedicated to vector operations). Although embodiments are described in which both vector and scalar operations are supported by the vector friendly instruction format, alternative embodiments use only vector operations via the vector friendly instruction format.

[0237] ​ is a block diagram illustrating a general vector friendly instruction format and its instruction templates according to an embodiment. ​is a block diagram illustrating a general vector friendly instruction format and its Class A instruction templates according to an embodiment; and ​ is a block diagram illustrating a general vector friendly instruction format and its Class B instruction templates according to an embodiment. Specifically, Class A and Class B instruction templates are defined for the general vector friendly instruction format 2500, both of which include instruction templates for memoryless access 2505 and instruction templates for memory access 2520. The term "general" in the context of a vector friendly instruction format refers to an instruction format that is not tied to any particular instruction set.

[0238] Although embodiments will be described in which the vector friendly instruction format supports the following: a 64-byte vector operand length (or size) with a 32-bit (4-byte) or 64-bit (8-byte) data element width (or size) (and thus, a 64-byte vector consists of 16 double-word sized elements, or alternatively consists of 8 quad-word sized elements); a 64-byte vector operand length (or size) with a 16-bit (2-byte) or 8-bit (1-byte) data element width (or size); a 32-byte vector operand length (or size) with a 32-bit (4-byte), 64-bit (8-byte), 16-bit (2-byte) or 8-bit (1-byte) data element width (or size); and a 16-byte vector operand length (or size) with a 32-bit (4-byte), 64-bit (8-byte), 16-bit (2-byte), or 8-bit (1-byte) data element width (or size); alternative embodiments may support larger, smaller, and / or different vector operand sizes (e.g., 256-byte vector operands) with larger, smaller, or different data element widths (e.g., 128-bit (16-byte) data element width).

[0239] ​ The Class A instruction templates in include: 1) within the instruction template for memoryless access 2505, an instruction template showing a fully rounded control type operation 2510 without memory access and an instruction template for a data transformation type operation 2515 without memory access; and 2) within the instruction template for memory access 2520, an instruction template showing the timeliness 2525 of memory access and an instruction template for the non-timeliness 2530 of memory access. ​ The Class B instruction templates in include: 1) within the instruction template for memoryless access 2505, an instruction template showing a write mask controlled partially rounded control type operation 2512 without memory access and an instruction template for a write mask controlled vsize type operation 2517 without memory access; and 2) within the instruction template for memory access 2520, an instruction template showing the write mask control 2527 of memory access.

[0240] The general vector friendly instruction format 2500 includes the following fields listed in the order illustrated in ​ as follows.

[0241] Format field 2540 - The specific value (instruction format identifier value) in this field uniquely identifies the vector-friendly instruction format and thereby identifies that the instruction appears in the instruction stream in the vector-friendly instruction format. Thus, this field is not required for an instruction set that has only a general vector-friendly instruction format, and in this sense, this field is optional.

[0242] Base operation field 2542 - Its content differentiates different base operations.

[0243] Register index field 2544 - Its content directly or through address generation specifies the location of source or destination operands in registers or in memory. These fields include a sufficient number of bits to select N registers from a PxQ (e.g., 32x512, 16x128, 32x1024, 64x1024) register file. Although in one embodiment N can be up to three source registers and one destination register, alternative embodiments can support more or fewer source and destination registers (e.g., can support up to two sources, where one of these sources also serves as the destination; can support up to three sources, where one of these sources also serves as the destination; can support up to two sources and one destination).

[0244] Modifier field 2546 - Its content differentiates instructions that appear in the general vector instruction format that specify memory access from those that do not; i.e., it differentiates between instruction templates with no memory access 2505 and instruction templates with memory access 2520. Memory access operations read and / or write to the memory hierarchy (in some cases, using values in registers to specify source and / or destination addresses), while non-memory access operations do not (e.g., sources and destinations are registers). Although in one embodiment, this field also selects between three different ways to perform memory address calculation, alternative embodiments can support more, fewer, or different ways to perform memory address calculation.

[0245] Extended operation field 2550 - Its content differentiates which one of various different operations, in addition to the base operation, is to be performed. This field is context-dependent. In one embodiment, this field is divided into a class field 2568, an alpha field 2552, and a beta field 2554. The extended operation field 2550 allows multiple sets of common operations to be performed in a single instruction rather than in 2, 3, or 4 instructions.

[0246] Scale field 2560 - Its content allows scaling of the content of the index field used for memory address generation (e.g., for address generation using (2 比例 * index + base)

[0247] Displacement field 2562A - the content of which is used as part of memory address generation (e.g., for address generation using (2 比例 * index + base address + displacement)).

[0248] Displacement factor field 2562B (note that the juxtaposition of displacement field 2562A directly on displacement factor field 2562B indicates the use of one or the other) - the content of which is used as part of address generation; it specifies the displacement factor that will scale the size (N) of the memory access - where N is the number of bytes in the memory access (e.g., for address generation using (2 比例 * index + base address + scaled displacement)). Redundant low - order bits are ignored, and thus the content of the displacement factor field is multiplied by the total size (N) of the memory operand to generate the final displacement that will be used in calculating the effective address. The value of N is determined by the processor hardware at runtime based on the full opcode field 2574 (described later in this document) and the data manipulation field 2554C. The displacement field 2562A and the displacement factor field 2562B are optional in the sense that they are not used in instruction templates without memory access 2505 and / or different embodiments may implement only one of the two or neither of the two.

[0249] Data element width field 2564 - the content of which differentiates which of multiple data element widths will be used (in some embodiments for all instructions; in other embodiments only for some of the instructions). This field is optional in the sense that it is not needed if only one data element width is supported and / or some aspect of the opcode is used to support the data element width.

[0250] Write mask field 2570 - whose content controls, on a per data element position basis, whether the data element positions in the destination vector operand reflect the results of the base and extended operations. Class A instruction templates support merge-write masks, while Class B instruction templates support both merge-write masks and zero-write masks. When merged, the vector mask allows any set of elements in the destination to be protected from update during the execution of any operation (specified by the base and extended operations); in another embodiment, the old value of each element of the destination where the corresponding mask bit has 0 is maintained. Conversely, when zeroed, the vector mask allows any set of elements in the destination to be zeroed during the execution of any operation (specified by the base and extended operations); in one embodiment, the elements of the destination are set to 0 when the corresponding mask bit has a 0 value. A subset of this functionality is the ability to control the vector length of the operation being performed (i.e., the span from the first to the last element being modified), however, the elements being modified do not have to be contiguous. Thus, the write mask field 2570 allows partial vector operations, which include loads, stores, arithmetic, logic, etc. Although embodiments have been described in which the content of the write mask field 2570 selects one of a plurality of write mask registers that contains the write mask to be used (and thus, the content of the write mask field 2570 indirectly identifies the mask to be executed), alternative embodiments alternatively or additionally allow the content of the mask write field 2570 to directly specify the mask to be executed.

[0251] Immediate field 2572 - whose content allows the specification of an immediate value. This field is optional in the sense that it does not exist in a general vector friendly format that does not support immediates and does not exist in instructions that do not use immediates.

[0252] Class field 2568 - whose content differentiates between different classes of instructions. Refer to ​ , the content of this field selects between Class A and Class B instructions. In ​ , rounded rectangles are used to indicate that a particular value exists in the field (e.g., Class A 2568A and Class B 2568B for the class field 2568 in ​ respectively). ​

[0253] In the case of the instruction template for Class A non-memory access 2505, the α field 2552 is interpreted as an RS field 2552A whose content differentiates which of different augmentation operation types is to be performed (e.g., for the instruction templates of the rounding type operation 2510 without memory access and the data transformation type operation 2515 without memory access, rounding 2552A.1 and data transformation 2552A.2 are specified respectively), and the β field 2554 differentiates which of the operations of the specified type is to be performed. In the instruction template for non-memory access 2505, the scale field 2560, the displacement field 2562A, and the displacement scale field 2562B do not exist. Instruction Template Without Memory Access - Rounding-Controlled Operations

[0254] In the instruction template for the fully rounding control type operation 2510 without memory access, the β field 2554 is interpreted as a rounding control field 2554A whose (multiple) content provides static rounding. Although in the described embodiment the rounding control field 2554A includes a suppress all floating-point exceptions (SAE) field 2556 and a rounding operation control field 2558, alternative embodiments may support both concepts, may encode both concepts into the same field, or have only one or the other of these concepts / fields (e.g., may have only the rounding operation control field 2558).

[0255] The SAE field 2556 - whose content differentiates whether to disable the reporting of exception events; when the content of the SAE field 2556 indicates enabling suppression, a given instruction does not report any kind of floating-point exception flag and does not evoke any floating-point exception handler.

[0256] The rounding operation control field 2558 - whose content differentiates which of a set of rounding operations is to be performed (e.g., round up, round down, round towards zero, and round to nearest). Thus, the rounding operation control field 2558 allows the rounding mode to be changed instruction by instruction. In one embodiment in which the processor includes a control register for specifying the rounding mode, the content of the rounding operation control field 2550 overrides the register value.

[0257] Instruction template for non-memory access - data transformation type operation

[0258] In the instruction template for the data transformation type operation 2515 without memory access, the β field 2554 is interpreted as a data transformation field 2554B whose content differentiates which of multiple data transformations is to be performed (e.g., no data transformation, mix, broadcast).

[0259] In the case of the instruction template for Class A memory access 2520, the α field 2552 is interpreted as an eviction hint field 2552B whose content differentiates which eviction hint is to be used (inFigure 25A Among them, for the instruction template of memory access timeliness 2525 and the instruction template of memory access non - timeliness 2530, the timeliness 2552B.1 and non - timeliness 2552B.2 are specified respectively), and the β field 2554 is interpreted as a data manipulation field 2554C, and its content differentiates which one of multiple data manipulation operations (also called primitives) is to be executed (for example, no manipulation, broadcast, up - conversion of the source, and down - conversion of the destination). The instruction template of memory access 2520 includes a scale field 2560 and optionally includes a displacement field 2562A or a displacement - scale field 2562B.

[0260] Vector memory instructions use conversion support to perform vector loads from memory and vector stores to memory. Like ordinary vector instructions, vector memory instructions transfer data to / from memory in a data - element - by - data - element manner, where the actually transferred elements are specified by the content of the vector mask selected as the write mask. Instruction Template with Memory Access - Timing

[0261] Timely data is data that may be reused quickly enough to benefit from cache operations. However, this is a hint, and different processors can implement it in different ways, including completely ignoring this hint.

[0262] Instruction template for memory access - non - timely

[0263] Non - timely data is data that is unlikely to be reused quickly enough to benefit from cache operations in the first - level cache and should be given eviction priority. However, this is a hint, and different processors can implement it in different ways, including completely ignoring this hint. Class B Instruction Template

[0264] In the case of the B - class instruction template, the α field 2552 is interpreted as a write - mask control (Z) field 2552C, and its content differentiates whether the write mask controlled by the write - mask field 2570 should be merged or zeroed.

[0265] In the case of the instruction template for B - type non - memory access 2505, a part of the β - field 2554 is interpreted as the RL - field 2557A, the content of which differentiates which one of different expansion operation types is to be executed (for example, the instruction template for the write - mask - controlled partial - rounding - control - type operation 2512 without memory access and the instruction template for the write - mask - controlled VSIZE - type operation 2517 without memory access specify rounding 2557A.1 and vector length (VSIZE) 2557A.2 respectively), while the remaining part of the β - field 2554 differentiates which one of the operations of the specified type is to be executed. In the instruction template for non - memory access 2505, the scale field 2560, the displacement field 2562A, and the displacement - scale field 2562B do not exist.

[0266] In the instruction template for the write - mask - controlled partial - rounding - control - type operation 2510 without memory access, the remaining part of the β - field 2554 is interpreted as the rounding - operation field 2559A, and the reporting of exception events is disabled (a given instruction does not report any kind of floating - point exception flag and does not invoke any floating - point exception handler).

[0267] The rounding - operation control field 2559A - - like the rounding - operation control field 2558, the content of which differentiates which one of a set of rounding operations is to be executed (for example, rounding up, rounding down, rounding towards zero, and rounding to the nearest). Thus, the rounding - operation control field 2559A allows the rounding mode to be changed instruction - by - instruction. In one embodiment in which the processor includes a control register for specifying the rounding mode, the content of the rounding - operation control field 2550 overrides the register value.

[0268] In the instruction template for the write - mask - controlled VSIZE - type operation 2517 without memory access, the remaining part of the β - field 2554 is interpreted as the vector - length field 2559B, the content of which differentiates which one of multiple data vector lengths is to be executed (for example, 128 bytes, 256 bytes, or 512 bytes).

[0269] In the case of the instruction template for B - type memory access 2520, a part of the β - field 2554 is interpreted as the broadcast field 2557B, the content of which differentiates whether a broadcast - type data - manipulation operation is to be executed, and the remaining part of the β - field 2554 is interpreted as the vector - length field 2559B. The instruction template for memory access 2520 includes the scale field 2560 and optionally includes the displacement field 2562A or the displacement - scale field 2562B.

[0270] For the general vector friendly instruction format 2500, it is shown that the full opcode field 2574 includes a format field 2540, a base operation field 2542, and a data element width field 2564. Although an embodiment is shown in which the full opcode field 2574 includes all of these fields, in embodiments that do not support all of these fields, the full opcode field 2574 includes less than all of these fields. The full opcode field 2574 provides an operation code (opcode).

[0271] The extended operation field 2550, the data element width field 2564, and the write mask field 2570 allow these features to be specified on a per-instruction basis in the general vector friendly instruction format.

[0272] The combination of the write mask field and the data element width field creates various types of instructions, as these instructions allow the mask to be applied based on different data element widths.

[0273] The various instruction templates that occur within classes A and B are beneficial in different scenarios. In some embodiments, different processors or different cores within a processor may support only class A, only class B, or may support both classes. For example, a high-performance general out-of-order core intended for general computing may support only class B, a core intended primarily for graphics and / or scientific (throughput) computing may support only class A, and a core intended for both general computing and graphics and / or scientific (throughput) computing may support both class A and class B (of course, with some mixing of templates and instructions from both classes, but not all templates and instructions from both classes are within the scope). Similarly, a single processor may include multiple cores, all of which support the same class, or in which different cores support different classes. For example, in a processor with separate graphics and general cores, one core in the graphics core intended primarily for graphics and / or scientific computing may support only class A, while one or more in the general core may be high-performance general cores with out-of-order execution and register renaming that support only class B for general computing. Another processor without a separate graphics core may include one or more general in-order or out-of-order cores that support both class A and class B. Of course, in different embodiments, features from one class may also be implemented in other classes. This will cause programs written in a high-level language to be (e.g., just-in-time compiled or statically compiled) into various different executable forms, which include: 1) a form with only the instructions of the (multiple) classes supported by the target processor for execution; or 2) a form with alternative routines and control flow code, where the alternative routines are written using different combinations of instructions from all classes, and the control flow code selects these routines for execution based on the instructions supported by the processor currently executing the code. Exemplary Special Vector-Friendly Instruction Format

[0274] Figure 26A is a block diagram illustrating an exemplary specialized vector-friendly instruction format according to an embodiment. Figure 26A Illustrated is a specialized vector-friendly instruction format 2600 that specifies the positions, sizes, interpretations, and order of the various fields, and the values of some of those fields, in which sense the specialized vector-friendly instruction format 2600 is specialized. The specialized vector-friendly instruction format 2600 can be used to extend the x86 instruction set, and thus some of the fields in this format are similar or identical to those used in the existing x86 instruction set and its extensions (e.g., AVX). This format is consistent with the prefix encoding field, real opcode byte field, MOD R / M field, SIB field, displacement field, and immediate number field of the existing x86 instruction set with extensions. Illustrated are the fields from FIG. 25, and the fields from Figure 26A map to the fields from FIG. 25.

[0275] It should be understood that although the embodiments are described in the context of the specialized vector-friendly instruction format 2600 with reference to the general vector-friendly instruction format 2500 for illustrative purposes, the present invention is not limited to the specialized vector-friendly instruction format 2600 unless otherwise stated. For example, the general vector-friendly instruction format 2500 contemplates various possible sizes for the various fields, while the specialized vector-friendly instruction format 2600 is shown as having fields of specific sizes. As a specific example, although the data element width field 2564 is illustrated as a one-bit field in the specialized vector-friendly instruction format 2600, the present invention is not limited thereto (i.e., the general vector-friendly instruction format 2500 contemplates other sizes for the data element width field 2564).

[0276] The general vector-friendly instruction format 2500 includes the following fields listed in the order illustrated in Figure 26A as follows. [[ID=*15]]

[0277] EVEX prefix 2602 (bytes 0-3) - encoded in a four-byte form.

[0278] Format field 2540 (EVEX byte 0, bits [7:0]) - The first byte (EVEX byte 0) is the format field 2540, and it contains 0x62 (in one embodiment, the unique value for differentiating the vector-friendly instruction format).

[0279] The second - fourth bytes (EVEX bytes 1-3) include a plurality of bit fields that provide specialized capabilities.

[0280] REX field 2605 (EVEX byte 1, bits [7-5]) -- consists of the EVEX.R bit field (EVEX byte 1, bit [7] – R), the EVEX.X bit field (EVEX byte 1, bit [6] – X), and (2557BEX byte 1, bit [5] – B). The EVEX.R, EVEX.X, and EVEX.B bit fields provide the same functionality as the corresponding VEX bit fields and are encoded in one's complement form, i.e., ZMM0 is encoded as 1111B and ZMM15 is encoded as 0000B. Other fields of these instructions encode the lower three bits (rrr, xxx, and bbb) of the register index known in the art, whereby Rrrr, Xxxx, and Bbbb can be formed by adding EVEX.R, EVEX.X, and EVEX.B.

[0281] REX’ field 2510 -- This is the first part of the REX’ field 2510 and is the EVEX.R’ bit field (EVEX byte 1, bit [4] – R’) used to encode the upper 16 or lower 16 registers of the extended set of 32 registers. In one embodiment, this bit is stored in bit-reversed format along with the other bits indicated below to (in the well-known x86 32-bit mode) distinguish from the BOUND instruction, the real opcode byte of which is 62 but does not accept the value 11 in the MOD field in the MOD R / M field (described below); alternative embodiments do not store this indicated bit and the other bits indicated below in reversed format. The value 1 is used to encode the lower 16 registers. In other words, R’Rrrr is formed by combining EVEX.R’, EVEX.R, and other RRRs from other fields.

[0282] Opcode mapping field 2615 (EVEX byte 1, bits [3:0] – mmmm) -- its content encodes the implicit leading opcode byte (0F, 0F 38, or 0F 3).

[0283] Data element width field 2564 (EVEX byte 2, bit [7] – W) -- denoted by the notation EVEX.W. EVEX.W is used to define the granularity (size) of the data type (32-bit data element or 64-bit data element).

[0284] EVEX.vvvv 2620 (EVEX byte 2, bits [6:3] - vvvv) - The functions of EVEX.vvvv can include the following: 1) EVEX.vvvv encodes the first source register operand specified in inverted (1's complement) form and is valid for instructions with two or more source operands; 2) EVEX.vvvv encodes the destination register operand specified in 1's complement form for a specific vector displacement; or 3) EVEX.vvvv does not encode any operand, this field is reserved and shall contain 1111b. Thus, the EVEX.vvvv field 2620 encodes the 4 low-order bits of the first source register specifier stored in inverted (1's complement) form. Depending on the instruction, additional different EVEX bit fields are used to extend the specifier size to 32 registers.

[0285] EVEX.U 2568 class field (EVEX byte 2, bit [2] - U) - If EVEX.U = 0, it indicates class A or EVEX.U0; if EVEX.U = 1, it indicates class B or EVEX.U1.

[0286] Prefix encoding field 2625 (EVEX byte 2, bits [1:0] - pp) - Provides additional bits for the base operation field. In addition to supporting traditional SSE instructions in EVEX prefix format, this also has the benefit of compressing the SIMD prefix (the EVEX prefix only requires 2 bits instead of a byte to express the SIMD prefix). In one embodiment, to support traditional SSE instructions using SIMD prefixes (66H, F2H, F3H) in both traditional format and EVEX prefix format, these traditional SIMD prefixes are encoded into the SIMD prefix encoding field; and are extended to the traditional SIMD prefix before being provided to the PLA at runtime (thus, without modification, the PLA can execute these traditional instructions in both traditional format and EVEX format). Although newer instructions can directly use the content of the EVEX prefix encoding field as an opcode extension, for consistency, a specific embodiment extends it in a similar way but allows different meanings specified by these traditional SIMD prefixes. Alternative embodiments can redesign the PLA to support 2-bit SIMD prefix encoding and thus do not require extension.

[0287] α field 2552 (EVEX byte 3, bit [7] – EH, also known as EVEX.EH, EVEX.rs, EVEX.RL, EVEX.write mask control, and EVEX.N; also illustrated as α) - As previously described, this field is context-specific.

[0288] β field 2554 (EVEX byte 3, bits [6:4] - SSS, also known as EVEX.s 2-0 、EVEX.r 2-0 、EVEX.rr1, EVEX.LL0, EVEX.LLB, also illustrated as βββ) - As previously mentioned, this field is context - specific.

[0289] REX’ field 2510 - This is the remainder of the REX’ field and is the EVEX.V’ bit field (EVEX byte 3, bit [3] – V’) that can be used to encode the upper 16 or lower 16 registers of the extended 32 - register set. This bit is stored in bit - reversed format. The value 1 is used to encode the lower 16 registers. In other words, V’VVVV is formed by combining EVEX.V’ and EVEX.vvvv.

[0290] Write - mask field 2570 (EVEX byte 3, bits [2:0] - kkk) - Its content specifies the index of the register in the write - mask register, as previously described. In one embodiment, the specific value EVEX.kkk = 000 has a special behavior that implies no write - mask is used for a particular instruction (this can be implemented in various ways, including using a write - mask hard - wired to all objects or hardware that bypasses the mask hardware).

[0291] The real opcode field 2630 (byte ④) is also called the opcode byte. Part of the opcode is specified in this field.

[0292] The MOD R / M field 2640 (byte ⑤) includes the MOD field 2642, the Reg field 2644, and the R / M field 2646. As previously mentioned, the content of the MOD field 2642 distinguishes memory - access operations from non - memory - access operations. The role of the Reg field 2644 can be boiled down to two cases: encoding the destination - register operand or the source - register operand; or being regarded as an opcode extension and not being used to encode any instruction operand. The role of the R / M field 2646 can include the following: encoding the instruction operand that references a memory address; or encoding the destination - register operand or the source - register operand.

[0293] Scale, Index, Base (SIB) byte (byte ⑥) - As previously mentioned, the content of the SIB 2650 is used for memory - address generation. SIB.xxx 2654 and SIB.bbb 2656 - The content of these fields has been previously mentioned for the register indices Xxxx and Bbbb.

[0294] Displacement field 2562A (bytes 7 - 10) - When the MOD field 2642 contains 10, bytes 7 - 10 are the displacement field 2562A, and it works the same as a traditional 32 - bit displacement (disp32), and works at the byte granularity.

[0295] Displacement factor field 2562B (byte 7) - When the MOD field 2642 contains 01, byte 7 is the displacement factor field 2562B. The position of this field is the same as that of the traditional x86 instruction set 8 - bit displacement (disp8) which works at the byte granularity. Since disp8 is sign - extended, it can only address between - 128 and 127 byte offsets; in terms of a 64 - byte cache line, disp8 uses 8 bits that can be set to only four really useful values: - 128, - 64, 0, and 64; since a larger range is often needed, disp32 is used; however, disp32 requires 4 bytes. In contrast to disp8 and disp32, the displacement factor field 2562B is a reinterpretation of disp8; when using the displacement factor field 2562B, the actual displacement is determined by multiplying the content of the displacement factor field by the size (N) of the memory operand access. This type of displacement is called disp8*N. This reduces the average instruction length (a single byte for the displacement but with a much larger range). Such compressed displacements assume that the effective displacement is a multiple of the granularity of the memory access, and thus the redundant low - order bits of the address offset do not need to be encoded. In other words, the displacement factor field 2562B replaces the traditional x86 instruction set 8 - bit displacement. Thus, the displacement factor field 2562B is encoded in the same way as the x86 instruction set 8 - bit displacement (so, there is no change in the ModRM / SIB encoding rules), the only difference being that disp8 is overloaded to disp8*N. In other words, there is no change in the encoding rules or the encoding length, but only a change in the hardware's interpretation of the displacement value (which requires scaling the displacement by the size of the memory operand to obtain a byte - based address offset). The immediate field 2572 operates as previously described. Full Opcode Field

[0296] Figure 26B is a block diagram showing the fields that make up the complete opcode field 2574 with a dedicated vector - friendly instruction format 2600 according to an embodiment. Specifically, the complete opcode field 2574 includes a format field 2540, a base operation field 2542, and a data element width (W) field 2564. The base operation field 2542 includes a prefix encoding field 2625, an opcode mapping field 2615, and a real opcode field 2630.

[0297] Register index field

[0298] Figure 26C FIG. is a block diagram of a field that constitutes a register index field 2544 and has a dedicated vector-friendly instruction format 2600 according to one embodiment. Specifically, the register index field 2544 includes a REX 2605 field, a REX' 2610 field, a MODR / M.reg field 2644, a MODR / M.r / m field 2646, a VVVV field 2620, an xxx field 2654, and a bbb field 2656.

[0299] Expansion operation field

[0300] Figure 26D FIG. is a block diagram of a field that constitutes an expansion operation field 2550 and has a dedicated vector-friendly instruction format 2600 according to one embodiment. When the class (U) field 2568 contains 0, it indicates EVEX.U0 (class A 2568A); when it contains 1, it indicates EVEX.U1 (class B 2568B). When U = 0 and the MOD field 2642 contains 11 (indicating no memory access operation), the α field 2552 (EVEX byte 3, bit [7] - EH) is interpreted as the rs field 2552A. When the rs field 2552A contains 1 (rounding 2552A.1), the β field 2554 (EVEX byte 3, bit [6:4] - SSS) is interpreted as a rounding control field 2554A. The rounding control field 2554A includes a one-bit SAE field 2556 and a two-bit rounding operation field 2558. When the rs field 2552A contains 0 (data transformation 2552A.2), the β field 2554 (EVEX byte 3, bit [6:4] - SSS) is interpreted as a three-bit data transformation field 2554B. When U = 0 and the MOD field 2642 contains 00, 01, or 10 (indicating a memory access operation), the α field 2552 (EVEX byte 3, bit [7] - EH) is interpreted as an eviction hint (EH) field 2552B, and the β field 2554 (EVEX byte 3, bit [6:4] - SSS) is interpreted as a three-bit data manipulation field 2554C.

[0301] When U = 1, the α field 2552 (EVEX byte 3, bit [7] - EH) is interpreted as a write mask control (Z) field 2552C. When U = 1 and the MOD field 2642 contains 11 (indicating no memory access operation), a part of the β field 2554 (EVEX byte 3, bit [4] - S0) is interpreted as an RL field 2557A; when it contains 1 (rounding 2557A.1), the rest of the β field 2554 (EVEX byte 3, bit [6 - 5] - S 2-1) is interpreted as the rounding operation field 2559A, and when the RL field 2557A contains 0 (VSIZE2557A.2), the remainder of the β field 2554 (EVEX byte 3, bits [6-5] - S 2-1 ) is interpreted as the vector length field 2559B (EVEX byte 3, bits [6-5] – L 1-0 ). When U = 1 and the MOD field 2642 contains 00, 01, or 10 (indicating a memory access operation), the β field 2554 (EVEX byte 3, bits [6:4] – SSS) is interpreted as the vector length field 2559B (EVEX byte 3, bits [6-5] – L 1-0 ) and the broadcast field 2557B (EVEX byte 3, bit [4] – B).

[0302] Exemplary Register Architecture is a block diagram of a register architecture 2700 according to one embodiment. In the illustrated embodiment, there are 32 vector registers 2710 that are 512 bits wide; these registers are referred to as zmm0 through zmm31. The lower 256 bits of the lower 16 zmm registers overlay the registers ymm0 - 16. The lower 128 bits of the lower 16 zmm registers (the lower 128 bits of the ymm registers) overlay the registers xmm0 - 15. The specialized vector-friendly instruction format 2600 operates on these overlaid register banks, as illustrated in the following table.

[0303] In other words, the vector length field 2559B selects between a maximum length and one or more other shorter lengths, where each such shorter length is half of the previous length, and instruction templates that do not have the vector length field 2559B operate on the maximum vector length. Additionally, in one embodiment, the class B instruction templates of the specialized vector-friendly instruction format 2600 operate on packed or scalar single / double-precision floating-point data and packed or scalar integer data. A scalar operation is an operation performed on the lowest-order data element position in a zmm / ymm / xmm register; depending on the embodiment, the higher-order data element positions either remain the same as before the instruction or are zeroed.

[0304] Write Mask Registers 2715 - In the illustrated embodiment, there are eight write mask registers (k0 through k7), each write mask register being 64 bits in size. In an alternative embodiment, the write mask registers 2715 are 16 bits in size. As previously described, in one embodiment, vector mask register k0 cannot be used as a write mask; when the encoding that normally indicates k0 is used as a write mask, it selects the hard - wired write mask 0xFFFF, effectively disabling the write mask for that instruction.

[0305] General - Purpose Registers 2725 - In the illustrated embodiment, there are sixteen 64 - bit general - purpose registers that are used with existing x86 addressing modes to address memory operands. These registers are referred to by the names RAX, RBX, RCX, RDX, RBP, RSI, RDI, RSP, and R8 through R15.

[0306] Scalar Floating - Point Stack Register File (x87 Stack) 2745, overlaid with the MMX Packed - Integer Flat Register File 2750 - In the illustrated embodiment, the x87 stack is an eight - element stack for performing scalar floating - point operations on 32 / 64 / 80 - bit floating - point data using the x87 instruction - set extension; while the MMX registers are used to perform operations on 64 - bit packed - integer data and to save operands for some operations performed between the MMX and XMM registers.

[0307] Alternative embodiments may use wider or narrower registers. Additionally, alternative embodiments may use more, fewer, or different register files and registers.

[0308] Exemplary Core Architecture, Processor, and Computer Architecture

[0309] Processor nuclear energy is implemented in different ways, for different purposes, and in different processors. For example, the implementation of such cores can include: 1) general-purpose in-order cores designed for general computing; 2) high-performance general-purpose out-of-order cores designed for general computing; 3) dedicated cores designed mainly for graphics and / or scientific (throughput) computing. The implementation of different processors can include: 1) a CPU that includes one or more general-purpose in-order cores designed for general computing and / or one or more general-purpose out-of-order cores designed for general computing; and 2) a coprocessor that includes one or more dedicated cores designed mainly for graphics and / or scientific (throughput). Such different processors result in different computer system architectures, which can include: 1) a coprocessor on a chip separate from the CPU; 2) a coprocessor in the same package as the CPU but on a separate die; 3) a coprocessor on the same die as the CPU (in which case, such a coprocessor is sometimes referred to as dedicated logic or as a dedicated core, such dedicated logic being, for example, integrated graphics and / or scientific (throughput) logic); and 4) a system-on-chip that can include the described CPU (sometimes referred to as the (multiple) application core or (multiple) application processor), the coprocessor described above, and additional functions on the same die. Exemplary core architectures are then described, followed by exemplary processors and computer architectures. Figure 27

[0310] In-order and out-of-order core block diagrams

[0311] Exemplary Core Architecture is a block diagram illustrating an exemplary in-order pipeline and an exemplary register-renamed out-of-order issue / execution pipeline according to various embodiments. Figure 28A is a block diagram showing an exemplary embodiment of an in-order architecture core to be included in a processor and an exemplary register-renamed out-of-order issue / execution architecture core according to various embodiments. Figure 28B The solid boxes in illustrate the in-order pipeline and in-order core, while the optional addition of the dashed boxes illustrates the register-renamed, out-of-order issue / execution pipeline and core. Given that the in-order aspects are a subset of the out-of-order aspects, the out-of-order aspects will be described.

[0312] In Figures 28A - 28B the processor pipeline 2800 includes a fetch stage 2802, a length decoding stage 2804, a decoding stage 2806, an allocation stage 2808, a renaming stage 2810, a scheduling (also referred to as dispatch or issue) stage 2812, a register read / memory read stage 2814, an execution stage 2816, a write-back / memory write stage 2818, an exception handling stage 2822, and a commit stage 2824.

[0313] Figure 28AA processor core 2890 is shown, which includes a front-end unit 2830 that is coupled to an execution engine unit 2850, and both the front-end unit 2830 and the execution engine unit 2850 are coupled to a memory unit 2870. The core 2890 can be a reduced instruction set computing (RISC) core, a complex instruction set computing (CISC) core, a very long instruction word (VLIW) core, or a hybrid or alternative core type. As another option, the core 2890 can be a specialized core, such as, for example, a network or communication core, a compression engine, a coprocessor core, a general-purpose computing graphics processing unit (GPGPU) core, a graphics core, and so on.

[0314] The front-end unit 2830 includes a branch prediction unit 2832 that is coupled to an instruction cache unit 2834, the instruction cache unit 2834 is coupled to an instruction translation lookaside buffer (TLB) 2836, the instruction translation lookaside buffer 2836 is coupled to an instruction fetch unit 2838, and the instruction fetch unit 2838 is coupled to a decoding unit 2840. The decoding unit 2840 (or decoder) can decode the instruction and generate as output one or more micro-operations, microcode entry points, microinstructions, other instructions, or other control signals that are decoded from, or otherwise reflect, or are derived from the original instruction. The decoding unit 2840 can be implemented using a variety of different mechanisms. Examples of suitable mechanisms include, but are not limited to, look-up tables, hardware implementations, programmable logic arrays (PLAs), microcode read-only memories (ROMs), etc. In one embodiment, the core 2890 includes a microcode ROM or other medium (e.g., in the decoding unit 2840, or otherwise within the front-end unit 2830) that stores microcode for certain macroinstructions. The decoding unit 2840 is coupled to a rename / allocator unit 2852 in the execution engine unit 2850.

[0315] The execution engine unit 2850 includes a rename / allocator unit 2852 that is coupled to a retirement unit 2854 and a collection 2856 of one or more scheduler units. The (multiple) scheduler units 2856 represent any number of different schedulers, including reservation stations, a central instruction window, and the like. The (multiple) scheduler units 2856 are coupled to the (multiple) physical register file units 2858. Each of the (multiple) physical register file units 2858 represents one or more physical register files, where different physical register files store one or more different data types, such as scalar integer, scalar floating point, packed integer, packed floating point, vector integer, vector floating point, a status (e.g., an instruction pointer that is the address of the next instruction to be executed), and so on. In one embodiment, the (multiple) physical register file units 2858 include a vector register unit, a write mask register unit, and a scalar register unit. These register units may provide architectural vector registers, vector mask registers, and general-purpose registers. The (multiple) physical register file units 2858 are overlapped by the retirement unit 2854 to illustrate various ways in which register renaming and out-of-order execution may be implemented (e.g., using the (multiple) reorder buffers and the (multiple) retirement register files; using the (multiple) future files, the (multiple) history buffers, the (multiple) retirement register files; using register maps and register pools, and so on). The retirement unit 2854 and the (multiple) physical register file units 2858 are coupled to the (multiple) execution clusters 2860. The (multiple) execution clusters 2860 include a collection 2862 of one or more execution units and a collection 2864 of one or more memory access units. The execution units 2862 may perform various operations (e.g., shift, add, subtract, multiply) and may operate on various data types (e.g., scalar floating point, packed integer, packed floating point, vector integer, vector floating point). Although some embodiments may include multiple execution units dedicated to a particular function or set of functions, other embodiments may include only one execution unit or multiple execution units that all perform all functions. The (multiple) scheduler units 2856, the (multiple) physical register file units 2858, and the (multiple) execution clusters 2860 are shown as potentially multiple because certain embodiments create separate pipelines for certain types of data / operations (e.g., a scalar integer pipeline, a scalar floating point / packed integer / packed floating point / vector integer / vector floating point pipeline, and / or a memory access pipeline that each has its own scheduler unit, the (multiple) physical register file units, and / or execution cluster - and in the case of a separate memory access pipeline, certain embodiments are implemented where only the execution cluster of that pipeline has the (multiple) memory access units 2864). It should also be understood that in the case of using separate pipelines, one or more of these pipelines may be out-of-order issue / execution, and the remaining pipelines may be in-order.

[0316] A set of memory access units 2864 is coupled to a memory unit 2870, which includes a data TLB unit 2872 that is coupled to a data cache unit 2874, and the data cache unit 2874 is coupled to a second-level (L2) cache unit 2876. In one exemplary embodiment, the memory access units 2864 may include a load unit, a store address unit, and a store data unit, each of which is coupled to the data TLB unit 2872 in the memory unit 2870. The instruction cache unit 2834 is also coupled to the second-level (L2) cache unit 2876 in the memory unit 2870. The L2 cache unit 2876 is coupled to one or more other levels of cache and ultimately to the main memory.

[0317] As an example, an exemplary register-renamed out-of-order issue / execution core architecture may implement a pipeline 2800 as follows: 1) Instruction fetch 2838 performs a fetch stage 2802 and a length decoding stage 2804; 2) A decode unit 2840 performs a decode stage 2806; 3) A rename / allocator unit 2852 performs an allocation stage 2808 and a rename stage 2810; 4) A (plurality of) scheduler units 2856 perform a schedule stage 2812; 5) A (plurality of) physical register file units 2858 and a memory unit 2870 perform a register read / memory read stage 2814; An execution cluster 2860 performs an execution stage 2816; 6) The memory unit 2870 and the (plurality of) physical register file units 2858 perform a write-back / memory write stage 2818; 7) Each unit may be involved in an exception handling stage 2822; and 8) A retirement unit 2854 and the (plurality of) physical register file units 2858 perform a commit stage 2824.

[0318] The core 2890 may support one or more instruction sets (e.g., the x86 instruction set (with some extensions added with more recent versions); the MIPS instruction set of MIPS Technologies, Inc. of Sunnyvale, California; the ARM instruction set of ARM Holdings plc of Sunnyvale, California (with optional additional extensions such as NEON)), including the (plurality of) instructions described herein. In one embodiment, the core 2890 includes logic for supporting SIMD (Single Instruction, Multiple Data) instruction set extensions (e.g., AVX1, AVX2), thereby allowing operations used by many multimedia applications to be performed using SIMD data.

[0319] It should be understood that the core supports multithreading (a collection of two or more parallel operations or threads), and this multithreading can be accomplished in various ways, including time-division multithreading, simultaneous multithreading (where a single physical core provides a logical core for each of the threads that the physical core is simultaneously multithreading), or a combination thereof (e.g., time-division fetching and decoding and subsequent simultaneous multithreading such as in simultaneous multithreading in hyperthreading technology).

[0320] Although register renaming has been described in the context of out-of-order execution, it should be understood that register renaming can be used in an in-order architecture. Although the illustrated embodiments of the processor also include separate instruction and data cache units 2834 / 2874 and a shared L2 cache unit 2876, alternative embodiments can have a single internal cache for both instructions and data, such as, for example, a first-level (L1) internal cache or multiple levels of internal caches. In some embodiments, the system can include a combination of an internal cache and an external cache outside the core and / or processor. Alternatively, all caches can be outside the core and / or processor. Figure 28B

[0321] Specific Exemplary In-Order Core Architecture A block diagram illustrating a more specific exemplary in-order core architecture, which would be one of several logic blocks in a chip (including other cores of the same type and / or different types). Depending on the application, the logic block communicates with some fixed-function logic, a memory I / O interface, and other necessary I / O logic via a high-bandwidth interconnect network (e.g., a ring network).

[0322] Figures 29A - 29B A block diagram of a single processor core according to an embodiment and its connection to the on-die interconnect network 2902 and a local subset 2904 of its second-level (L2) cache. In one embodiment, the instruction decoder 2900 supports the x86 instruction set with a compact data instruction set extension. The L1 cache 2906 allows low-latency access to cache memory for data entering the scalar and vector units. Although in one embodiment (for simplicity of design), the scalar unit 2908 and the vector unit 2910 use separate register sets (scalar registers 2912 and vector registers 2914 respectively), and the data transferred between these registers is written to memory and then read back from the first-level (L1) cache 2906, alternative embodiments can use different methods (e.g., using a single register set or including a communication path that allows data to be transferred between the two register banks without being written and read back).

[0323] The local subset 2904 of the L2 cache is part of a global L2 cache that is partitioned into a number of separate local subsets, one for each processor core. Each processor core has a direct access path to its own local subset 2904 of the L2 cache. Data read by a processor core is stored in its L2 cache subset 2904 and can be quickly accessed in parallel with other processor cores accessing their own local L2 cache subsets. Data written by a processor core is stored in its own L2 cache subset 2904 and flushed from other subsets as necessary. A ring network ensures the consistency of shared data. The ring network is bi-directional to allow agents such as processor cores, L2 caches, and other logic blocks to communicate with each other within the chip. Each ring data path is 1012 bits wide in each direction.

[0324] Figure 29A is according to an embodiment Figure 29B is an expanded view of a portion of the processor core in Figure 29A includes the L1 data cache 2906A portion of the L1 cache 2904, and more details regarding the vector unit 2910 and vector registers 2914. Specifically, the vector unit 2910 is a 16-wide vector processing unit (VPU) (see the 16-wide ALU 2928) that executes one or more of integer, single-precision floating-point, and double-precision floating-point instructions. The VPU supports mixing of register inputs via a mixing unit 2920, numerical conversions via numerical conversion units 2922A-B, and copying of memory inputs via a copy unit 2924. A write mask register 2926 allows masking of resulting vector writes.

[0325] Figure 29B is a block diagram of a processor 3000 that can have more than one core, can have an integrated memory controller, and can have an integrated graphics device according to an embodiment. Figure 30 The solid box in shows a processor 3000 having a single core 3002A, a system agent 3010, and a set 3016 of one or more bus controller units, while the optional addition of the dashed box shows an alternative processor 3000 having multiple cores 3002A-N, a system agent unit 3010, a set 3014 of one or more integrated memory controller units, and dedicated logic 3008.

[0326] Accordingly, different implementations of the processor 3000 may include: 1) a CPU, where the dedicated logic 3008 is integrated graphics and / or scientific (throughput) logic (which may include one or more cores), and the cores 3002A-N are one or more general-purpose cores (e.g., general-purpose in-order cores, general-purpose out-of-order cores, a combination of both); 2) a coprocessor, where the cores 3002A-N are a large number of dedicated cores intended primarily for graphics and / or scientific (throughput); and 3) a coprocessor, where the cores 3002A-N are a large number of general-purpose in-order cores. Thus, the processor 3000 can be a general-purpose processor, a coprocessor, or a special-purpose processor, such as, for example, a network or communication processor, a compression engine, a graphics processor, a GPGPU (general-purpose graphics processing unit), a high-throughput integrated many-core (MIC) coprocessor (including 30 or more cores), an embedded processor, and the like. The processor can be implemented on one or more chips. The processor 3000 can be a part of one or more substrates, and / or can be implemented on one or more substrates using any of a variety of process technologies, such as, for example, BiCMOS, CMOS, or NMOS.

[0327] The memory hierarchy includes one or more levels of cache within the cores, a set 3006 of one or more shared cache units, and external memory (not shown) coupled to a set 3014 of integrated memory controller units. The set 3006 of shared cache units can include one or more intermediate levels of cache, such as, a second level (L2), a third level (L3), a fourth level (L4), or other levels of cache, a last-level cache (LLC), and / or a combination of the above. Although in one embodiment, the ring-based interconnect unit 3012 interconnects the dedicated logic 3008 (integrated graphics logic is an example of dedicated logic and is also referred to herein as dedicated logic), the set 3006 of shared cache units, and the system agent unit 3010 / (a) set(s) 3014 of integrated memory controller units, alternative embodiments can use any number of well-known techniques to interconnect such units. In one embodiment, coherence is maintained between one or more cache units 3006 and the cores 3002A-N.

[0328] In some embodiments, one or more of the cores 3002A-N are capable of implementing multithreading. The system agent 3010 includes those components that coordinate and operate the cores 3002A-N. The system agent unit 3010 can include, for example, a power control unit (PCU) and a display unit. The PCU can be the logic and components required to regulate the power states of the cores 3002A-N and the dedicated logic 3008, or can include such logic and components. The display unit is used to drive one or more externally connected displays.

[0329] The cores 3002A-N may be homogeneous or heterogeneous in terms of the architectural instruction set; i.e., two or more of the cores 3002A-N may be capable of executing the same instruction set, while other cores may be capable of executing only a subset of the instruction set or a different instruction set.

[0330] Exemplary computer architecture

[0331] Figure 30 is a block diagram of an exemplary computer architecture. Other system designs and configurations known in the art for laptop devices, desktops, handheld PCs, personal digital assistants, engineering workstations, servers, network devices, network hubs, switches, embedded processors, digital signal processors (DSPs), graphics devices, video game devices, set-top boxes, microcontrollers, cellular telephones, portable media players, handheld devices, and various other electronic devices are also suitable. In general, a wide variety of systems or electronic devices capable of incorporating a processor and / or other execution logic as disclosed herein are generally suitable.

[0332] Now referring to Figures 31 - 34 , shown is a block diagram of a system 3100 according to an embodiment of the present invention. The system 3100 may include one or more processors 3110, 3115, which are coupled to a controller hub 3120. In one embodiment, the controller hub 3120 includes a Graphics Memory Controller Hub (GMCH) 3190 and an Input / Output Hub (IOH) 3150 (which may be on separate chips); the GMCH 3190 includes a memory and a graphics controller, to which a memory 3140 and a coprocessor 3145 are coupled; the IOH 3150 couples input / output (I / O) devices 3160 to the GMCH 3190. Alternatively, one or both of the memory and the graphics controller are integrated within the processor (as described herein), the memory 3140 and the coprocessor 3145 are directly coupled to the processor 3110, and the controller hub 3120 and the IOH 3150 are in a single chip.

[0333] The optionality of the additional processor 3115 is indicated by the dashed lines in Figure 31 . Each processor 3110, 3115 may include one or more of the processing cores described herein and may be a certain version of the processor 3000.

[0334] The memory 3140 can be, for example, a dynamic random access memory (DRAM), a phase change memory (PCM), or a combination of the two. For at least one embodiment, the controller hub 3120 communicates with the processors 3110, 3115 via a multi-branch bus such as a front side bus (FSB), a point-to-point interface such as a QuickPath Interconnect (QPI), or a similar connection 3195.

[0335] In one embodiment, the coprocessor 3145 is a specialized processor such as, for example, a high throughput MIC processor, a network or communication processor, a compression engine, a graphics processor, a GPGPU, an embedded processor, and the like. In one embodiment, the controller hub 3120 can include an integrated graphics accelerator.

[0336] There can be various differences in a series of quality metrics including architecture, microarchitecture, thermal, power consumption characteristics, etc. between the physical resources 3110, 3115.

[0337] In one embodiment, the processor 3110 executes instructions that control general types of data processing operations. Coprocessor instructions can be embedded within these instructions. The processor 3110 identifies these coprocessor instructions as being of a type that should be executed by the attached coprocessor 3145. Accordingly, the processor 3110 issues these coprocessor instructions (or control signals representing coprocessor instructions) to the coprocessor 3145 on a coprocessor bus or other interconnect. The coprocessor(s) 3145 receive and execute the received coprocessor instructions.

[0338] Now referring to Figure 31 , shown is a block diagram of a first more specific exemplary system 3200 in accordance with an embodiment of the present invention. As Figure 32 shown, the multi-processor system 3200 is a point-to-point interconnect system and includes a first processor 3270 and a second processor 3280 coupled via a point-to-point interconnect 3250. Each of the processors 3270 and 3280 can be a certain version of the processor 3000. In one embodiment, the processors 3270 and 3280 are the processors 3110 and 3115 respectively, and the coprocessor 3238 is the coprocessor 3145. In another embodiment, the processors 3270 and 3280 are the processor 3110 and the coprocessor 3145 respectively. <&

[0339] Processors 3270 and 3280 are shown as including integrated memory controller (IMC) units 3272 and 3282, respectively. Processor 3270 also includes point-to-point (P-P) interfaces 3276 and 3278 as part of its bus controller unit; similarly, second processor 3280 includes P-P interfaces 3286 and 3288. Processors 3270, 3280 may exchange information via P-P interface 3250 using point-to-point (P-P) interface circuits 3278, 3288. As Figure 32 shown, IMCs 3272 and 3282 couple the processors to respective memories, namely memories 3232 and 3234, which may be portions of the main memory locally attached to the respective processors.

[0340] Processors 3270, 3280 may each exchange information with chipset 3290 via respective P-P interfaces 3252, 3254 using point-to-point interface circuits 3276, 3294, 3286, 3298. Chipset 3290 may optionally exchange information with coprocessor 3238 via high performance interface 3292. In one embodiment, coprocessor 3238 is a special purpose processor such as, for example, a high throughput MIC processor, a network or communication processor, a compression engine, a graphics processor, a GPGPU, an embedded processor, and the like.

[0341] A shared cache (not shown) may be included in either processor or external to both processors but connected to these processors via a P-P interconnect such that if the processors are placed in a low power mode, the local cache information of either or both processors may be stored in the shared cache.

[0342] Chipset 3290 may be coupled to first bus 3216 via interface 3296. In one embodiment, first bus 3216 may be a Peripheral Component Interconnect (PCI) bus or a bus such as a PCI Express bus or another third generation I / O interconnect bus, but the scope of the present invention is not limited thereto.

[0343] As Figure 32As shown, various I / O devices 3214 can be coupled to a first bus 3216 along with a bus bridge 3218 that couples the first bus 3216 to a second bus 3220. In one embodiment, one or more additional processors 3215, such as a coprocessor, a high-throughput MIC processor, a GPGPU, an accelerator (such as, for example, a graphics accelerator or a digital signal processing (DSP) unit), a field programmable gate array, or any other processor, are coupled to the first bus 3216. In one embodiment, the second bus 3220 can be a low pin count (LPC) bus. In one embodiment, various devices can be coupled to the second bus 3220, including, for example, a keyboard and / or mouse 3222, a communication device 3227, and a storage unit 3228, such as a disk drive or other mass storage device that can include instructions / code and data 3230. Additionally, audio I / O 3224 can be coupled to the second bus 3220. Note that other architectures are possible. For example, instead of Figure 32 a point-to-point architecture, the system can implement a multi-branch bus or other such architectures.

[0344] Now referring to Figure 32 , shown is a block diagram of a second more specific exemplary system 3300 in accordance with an embodiment of the present invention. Figure 33 and 33 Similar elements in Figure 32 are designated with similar reference numerals, and certain aspects of Figure 33 are omitted from Figure 32 to avoid obscuring other aspects of

[0345] Figure 33 Illustrated processors 3270, 3280 can each include integrated memory and I / O control logic ("CL") 3272 and 3282. Thus, CL 3272, 3282 includes an integrated memory controller unit and includes I / O control logic. Figure 33 Illustrated are that not only memories 3232, 3234 are coupled to CL 3272, 3282, but also I / O devices 3314 are coupled to control logic 3272, 3282. Conventional I / O devices 3315 are coupled to a chipset 3290.

[0346] Now referring to Figure 33 , shown is a block diagram of a SoC 3400 in accordance with an embodiment of the present invention. Figure 34 Similar elements in Figure 30In [the figure], the (multiple) interconnect units 3402 are coupled to: an application processor 3410, which includes a set of one or more cores 3002A-N and (multiple) shared cache units 3006, the set of one or more cores 3002A-N including cache units 3004A-N; a system agent unit 3010; (multiple) bus controller units 3016; (multiple) integrated memory controller units 3014; a set of one or more coprocessors 3420, which may include integrated graphics logic, an image processor, an audio processor, and a video processor; a static random access memory (SRAM) unit 3430; a direct memory access (DMA) unit 3432; and a display unit 3440 for coupling to one or more external displays. In one embodiment, the (multiple) coprocessors 3420 include dedicated processors, such as, for example, a network or communication processor, a compression engine, a GPGPU, a high throughput MIC processor, or an embedded processor, and so on.

[0347] Embodiments of the mechanisms disclosed herein may be implemented in hardware, software, firmware, or a combination of such implementations. An embodiment may be implemented as a computer program or program code executing on a programmable system including at least one processor, a storage system (including volatile and non-volatile memory and / or storage elements), at least one input device, and at least one output device.

[0348] The program code (such as, Figure 34 the code 3230 illustrated in [the figure]) may be applied to input instructions to perform the functions described herein and generate output information. The output information may be applied to one or more output devices in a known manner. For the purposes of this application, a processing system includes any system having a processor, such as, for example, a digital signal processor (DSP), a microcontroller, an application specific integrated circuit (ASIC), or a microprocessor.

[0349] The program code may be implemented in a high-level procedural programming language or an object-oriented programming language in order to communicate with the processing system. If desired, the program code may also be implemented in assembly language or machine language. In fact, the mechanisms described herein are not limited to any specific programming language scope. In any case, the language may be a compiled language or an interpreted language.

[0350] One or more aspects of at least one embodiment may be implemented by representative instructions stored on a machine-readable medium, the instructions representing various logics in a processor, which when read by the machine cause the machine to fabricate the logics for performing the techniques described herein. Such representations, referred to as “IP cores,” may be stored on a tangible machine-readable medium and supplied to various customers or manufacturing facilities to be loaded into a manufacturing machine that actually fabricates the logics or processors.

[0351] Such machine-readable storage media may include, but are not limited to, non-transitory, tangible arrangements of articles manufactured or formed by a machine or device, which include storage media such as hard disks; any other type of disk, including floppy disks, optical disks, compact disk read-only memory (CD-ROM), rewritable compact disks (CD-RW), and magneto-optical disks; semiconductor devices such as read-only memory (ROM), random access memory (RAM) such as dynamic random access memory (DRAM) and static random access memory (SRAM), erasable programmable read-only memory (EPROM), flash memory, electrically erasable programmable read-only memory (EEPROM); phase change memory (PCM); magnetic or optical cards; or any other type of medium suitable for storing electronic instructions.

[0352] Accordingly, embodiments also include non-transitory, tangible machine-readable media that contain instructions or contain design data, such as a hardware description language (HDL), that define the structures, circuits, devices, processors, and / or system features described herein. These embodiments are also referred to as program products. Figure 32

[0353] In some cases, an instruction converter may be used to convert instructions from a source instruction set to a target instruction set. For example, the instruction converter may transform (e.g., using static binary translation, dynamic binary translation including dynamic compilation), morph, emulate, or otherwise convert one or more other instructions to be processed by a core. The instruction converter may be implemented in software, hardware, firmware, or a combination thereof. The instruction converter may be on the processor, outside the processor, or partly on the processor and partly outside the processor.

[0354] Emulation (including binary translation, code morphing, etc.) is a block diagram of an example of using a software instruction converter to convert binary instructions in a source instruction set to binary instructions in a target instruction set according to an embodiment. In the illustrated embodiment, the instruction converter is a software instruction converter, but alternatively, the instruction converter may be implemented in software, firmware, hardware, or various combinations thereof. Figure 35It is shown that a program in the form of a high-level language 3502 can be compiled using an x86 compiler 3504 to generate x86 binary code 3506 that can be natively executed by a processor 3516 having at least one x86 instruction set core. A processor 3516 having at least one x86 instruction set core represents any processor that performs substantially the same functions as an Intel processor having at least one x86 instruction set core by compatibly executing or otherwise processing: 1) a substantial portion of the instruction set of the Intel x86 instruction set core, or 2) a target code version of an application or other software targeted to run on an Intel processor having at least one x86 instruction set core to achieve substantially the same results as an Intel processor having at least one x86 instruction set core. The x86 compiler 3504 represents a compiler operable to generate x86 binary code 3506 (e.g., target code) that can be executed on a processor 3516 having at least one x86 instruction set core with or without additional linking processing. Similarly, Figure 35 It is shown that a program in the form of a high-level language 3502 can be compiled using an alternative instruction set compiler 3508 to generate alternative instruction set binary code 3510 that can be natively executed by a processor 3514 not having at least one x86 instruction set core (e.g., a processor having a core that executes the MIPS instruction set of MIPS Technologies, Inc. of Sunnyvale, California, and / or the ARM instruction set of ARM Holdings plc of Sunnyvale, California). An instruction converter 3512 is used to convert the x86 binary code 3506 into code that can be natively executed by a processor 3514 not having an x86 instruction set core. The converted code is not likely to be the same as the alternative instruction set binary code 3510 because an instruction converter capable of doing so is difficult to manufacture; however, the converted code will perform general operations and will consist of instructions from an alternative instruction set. Thus, the instruction converter 3512 represents software, firmware, hardware, or a combination thereof that allows a processor or other electronic device not having an x86 instruction set processor or core to execute the x86 binary code 3506 through emulation, simulation, or any other process. Figure 35 Further Examples

[0355] Example 1 provides an exemplary processor, including: a fetch circuit for fetching a compressed instruction having a format with fields for specifying an opcode and positions of a decompressed source matrix and a compressed destination matrix; a decode circuit for decoding the fetched compressed instruction; and an execution circuit for performing the following operations in response to the decoded compressed instruction: generating a compressed result according to a compression algorithm by compressing the specified decompressed source matrix by any one of the following operations: packing non-zero value elements together and storing matrix positions of each non-zero value element in a header; or representing one or more elements using fewer bits and using the header to identify matrix elements represented by the fewer bits; and storing the compressed result into the specified compressed destination matrix.

[0356] Example 2 includes the substantial content of the exemplary processor of Example 1, wherein the execution circuit is further configured to: perform an arithmetic or logical operation on elements of the compressed result before storing the compressed result into the specified compressed destination matrix.

[0357] Example 3 includes the substantial content of the exemplary processor of Example 1, wherein each of the specified decompressed source matrix and the compressed destination matrix is located in any one of a set of floating-point registers, a set of vector registers, a set of tile registers, and a memory.

[0358] Example 4 includes the substantial content of the exemplary processor of Example 1, wherein: the fetch circuit is further configured to fetch a decompression instruction that specifies positions of a compressed source matrix and a decompressed destination matrix, wherein the compressed result is specified as the compressed source matrix, and the compressed result has been generated by packing non-zero value elements together and storing matrix positions of each non-zero value element in a header; the decode circuit is further configured to decode the fetched decompression instruction, and the execution circuit is further configured to respond to the decoded decompression instruction by writing each of the non-zero value elements of the specified compressed source matrix to an associated position within the specified decompressed destination matrix for each of the non-zero value elements of the specified compressed source matrix, the associated position being determined by the header.

[0359] Example 5 includes the substantial content of the exemplary processor of Example 1, wherein when the compressed result is generated by packing non-zero value elements together, the header includes a multi-bit value having bits for each element position of the specified decompressed source matrix, the bits being used to identify non-zero value elements of the decompressed source matrix.

[0360] Example 6 includes the substantial content of the exemplary processor of Example 1, wherein when a compressed result is generated by representing one or more elements using fewer bits, the header includes a multi-bit value having a bit for each element position of the specified decompressed source matrix, the bit being used to identify the element of the compressed result represented by fewer bits.

[0361] Example 7 includes the substantial content of the exemplary processor of Example 1, wherein when an execution circuit is used to generate a compressed result by representing one or more elements using fewer bits, one or more elements are replaced with pointers to a dictionary of values, the pointers using fewer bits than the specified uncompressed source matrix elements.

[0362] Example 8 provides an exemplary processor, comprising: a fetch circuit for fetching a decompression instruction, the format of the decompression instruction having fields for specifying an opcode and the positions of a compressed source matrix and a decompressed destination matrix; a decode circuit for decoding the fetched decompression instruction; and an execution circuit for performing the following operations in response to the decoded decompression instruction: generating a decompressed result according to a decompression algorithm by: when the specified compressed source matrix includes packed non-zero value elements, copying each element of the specified source matrix to an associated position of each element of the specified source matrix within the specified decompressed destination matrix, the associated position being identified by a multi-bit header having a bit for each element of the specified decompressed destination matrix, wherein the bit corresponding to a non-zero value element is set; and storing the decompressed result in the specified decompressed destination matrix.

[0363] Example 9 includes the substantial content of the exemplary processor of Example 8, wherein the execution circuit is further for: performing an arithmetic or logical operation on the elements of the decompressed result before storing the decompressed result in the specified decompressed destination matrix.

[0364] Example 10 includes the substantial content of the exemplary processor of Example 8, wherein the specified compressed source matrix and the decompressed destination matrix are each located in any one of a set of floating-point registers, a set of vector registers, a set of tile registers, and a memory.

[0365] Example 11 includes the substantial content of the exemplary processor of Example 8, wherein: the fetch circuit is further configured to fetch a compressed instruction that specifies the positions of the decompressed source matrix and the compressed destination matrix; wherein the generated decompressed result is designated as the decompressed source matrix, the decoding circuit is further configured to decode the fetched compressed instruction, and the execution circuit is further configured to respond to the decoded compressed instruction by compressing the specified compressed source matrix according to a compression algorithm by any of the following operations: packing non-zero value elements together and storing the matrix positions of each non-zero value element in a header; or using fewer bits to represent one or more elements and using a header to identify the matrix elements represented by the fewer bits.

[0366] Example 12 includes the substantial content of the exemplary processor of Example 8, wherein when the execution circuit generates a compressed result by using fewer bits to represent one or more elements, the one or more elements are replaced with pointers to a dictionary of values, and the pointers use fewer bits than the specified uncompressed source matrix elements.

[0367] Example 13 provides an exemplary method performed by a processor, the processor being configured to: use a fetch circuit to fetch a compressed instruction having a format that has fields for specifying an opcode and the positions of a decompressed source matrix and a compressed destination matrix; use a decoding circuit to decode the fetched compressed instruction; and use an execution circuit to respond to the decoded compressed instruction by generating a compressed result according to a compression algorithm and storing the compressed result in the specified compressed destination matrix, the generating the compressed result according to the compression algorithm by compressing the specified decompressed source matrix, the compressing the specified decompressed source matrix by any of the following operations: packing non-zero value elements together and storing the matrix positions of each non-zero value element in a header; or using fewer bits to represent one or more elements and using a header to identify the matrix elements represented by the fewer bits.

[0368] Example 14 includes the substantial content of the exemplary method of Example 13, wherein the execution circuit is further configured to: perform an arithmetic or logical operation on the elements of the compressed result before storing the compressed result in the specified compressed destination matrix.

[0369] Example 15 includes the substantial content of the exemplary method of Example 13, wherein the specified decompressed source matrix and the compressed destination matrix are each located in any of a set of floating-point registers, a set of vector registers, a set of tile registers, and a memory.

[0370] Example 16 includes the substantial content of the exemplary method of Example 13, wherein: the fetch circuit is further configured to fetch a decompression instruction that specifies the positions of the compressed source matrix and the decompressed destination matrix, wherein the compressed result is designated as the compressed source matrix, and the compressed result has been generated by packing non-zero value elements together and storing the matrix positions of each non-zero value element in a header; the decoding circuit is further configured to decode the fetched decompression instruction, and the execution circuit is further configured to respond to the decoded decompression instruction by writing each of the non-zero value elements of the specified compressed source matrix to the associated position of each of the non-zero value elements of the specified compressed source matrix within the specified decompressed destination matrix, the associated position being determined by the header.

[0371] Example 17 includes the substantial content of the exemplary method of Example 13, wherein when the compressed result is generated by packing non-zero value elements together, the header includes a multi-bit value that has a bit for each element position of the specified decompressed source matrix, and the bit is used to identify the non-zero value elements of the decompressed source matrix.

[0372] Example 18 provides an exemplary non-transitory computer-readable medium containing instructions that, when executed by a processor, cause the processor to: use a fetch circuit to fetch a decompression instruction whose format has fields for specifying an opcode and the positions of a compressed source matrix and a decompressed destination matrix; use a decoding circuit to decode the fetched decompression instruction; and use an execution circuit to respond to the decoded decompression instruction by performing the following operations: generating a decompressed result according to a decompression algorithm by: when the specified compressed source matrix includes packed non-zero value elements, copying each element of the specified source matrix to the associated position of each element of the specified source matrix within the specified decompressed destination matrix, the associated position being identified by a multi-bit header that has a bit for each element of the specified decompressed destination matrix, and the bit corresponding to a non-zero value element being set; and storing the decompressed result in the specified decompressed destination matrix.

[0373] Example 19 includes the substantial content of the exemplary non-transitory computer-readable medium of Example 18, wherein the execution circuit is further configured to: perform an arithmetic or logical operation on the elements of the decompressed result before storing the decompressed result in the specified decompressed destination matrix.

[0374] Example 20 includes the substantial content of the exemplary non-transitory computer-readable medium of Example 18, wherein: the fetch circuit is further configured to fetch a compressed instruction that specifies positions of a decompressed source matrix and a compressed destination matrix; wherein the generated decompressed result is designated as the decompressed source matrix, the decoding circuit is further configured to decode the fetched compressed instruction, and the execution circuit is further configured to respond to the decoded compressed instruction by compressing the specified compressed source matrix via any one of the following operations: packing non-zero value elements together and storing matrix positions of each non-zero value element in a header; or representing one or more elements using fewer bits and using the header to identify matrix elements represented by the fewer bits.

Claims

1. An apparatus, comprising: a memory; a plurality of vector registers for storing data elements of a matrix; an execution circuit for performing a combined load and decompression operation according to one or more instructions, the execution circuit being configured to: read a compressed sparse source matrix slice from the memory, the compressed sparse source matrix slice including a first subset of 8-bit integer data elements of a corresponding first source matrix and a bitmask, the bitmask being used to indicate the positions of the first subset of 8-bit integer data elements in the corresponding first source matrix, wherein the bitmask includes a plurality of bits, each bit corresponding to a data element position in the corresponding first source matrix and being set to a first bit value to indicate the matrix position of the corresponding 8-bit integer data element of the first subset of 8-bit integer data elements, decompress the compressed sparse source matrix slice according to the bitmask to generate a decompressed sparse source matrix slice, the decompressed sparse source matrix slice including the first subset of 8-bit integer data elements, and store the decompressed sparse source matrix slice in one or more of the plurality of vector registers.

2. The apparatus according to claim 1, further comprising: a matrix processing circuit of the execution circuit for performing a matrix multiplication operation according to the one or more instructions to multiply the decompressed sparse source matrix slice by a second source matrix slice, the second source matrix slice including a plurality of 8-bit integer data elements of a second source matrix, the matrix processing circuit being configured to: multiply each 8-bit integer data element in a second subset of 8-bit integer data elements of the second source matrix slice by the corresponding 8-bit integer data element of the first subset identified based on at least a portion of the bitmask to generate a plurality of products, and add each product in the plurality of products to a corresponding 32-bit accumulated integer data element to generate a corresponding 32-bit integer result data element of a result matrix slice.

3. The device according to claim 1, wherein, The one or more instructions include a first one or more instructions, and wherein the execution circuit is configured to compress the corresponding first source matrix according to a second one or more instructions to generate the compressed sparse source matrix slice and store the compressed sparse source matrix slice in the memory, the compressed sparse source matrix slice including the bitmask.

4. The apparatus according to claim 3, wherein, A third subset of 8-bit integer data elements of the corresponding first source matrix is excluded from the compressed sparse source matrix slice based on the size of the third subset of 8-bit integer data elements.

5. The device according to claim 4, wherein The third subset of 8-bit integer data elements includes zero-valued data elements.

6. The device according to claim 3, wherein The relative positions of the first subset of 8-bit integer data elements in the corresponding first source matrix are maintained in the compressed sparse source matrix slice.

7. A system, comprising: a plurality of cores for executing program code; a memory controller for coupling the plurality of cores to a memory; an interconnect coupled to the plurality of cores and the memory controller; a matrix processing accelerator coupled to the interconnect, the matrix processing accelerator including: a plurality of vector registers for storing data elements of a matrix; An execution circuit for performing a combined load and decompression operation in accordance with one or more instructions, the execution circuit being configured to: Read a compressed sparse source matrix slice from the memory, the compressed sparse source matrix slice including a first subset of 8-bit integer data elements of a corresponding first source matrix and a bit mask for indicating the positions of the first subset of 8-bit integer data elements in the corresponding first source matrix, wherein the bit mask includes a plurality of bits, each bit corresponding to a data element position in the corresponding first source matrix and being set to a first bit value to indicate the matrix position of the corresponding 8-bit integer data element of the first subset of 8-bit integer data elements, Decompress the compressed sparse source matrix slice according to the bit mask to generate a decompressed sparse source matrix slice, the decompressed sparse source matrix slice including the first subset of 8-bit integer data elements, and Store the decompressed sparse source matrix slice in one or more of the plurality of vector registers.

8. The system according to claim 7, wherein, The matrix processing accelerator further includes: A matrix processing circuit of the execution circuit for performing a matrix multiplication operation in accordance with the one or more instructions to multiply the decompressed sparse source matrix slice by a second source matrix slice, the second source matrix slice including a plurality of 8-bit integer data elements of a second source matrix, the matrix processing circuit being configured to: Multiply each 8-bit integer data element in a second subset of 8-bit integer data elements of the second source matrix slice by the corresponding 8-bit integer data element of the first subset identified based on at least a portion of the bit mask to generate a plurality of products, and Add each of the plurality of products to a corresponding 32-bit accumulated integer data element to generate a corresponding 32-bit integer result data element of a result matrix slice.

9. The system according to claim 7, wherein, The one or more instructions include a first one or more instructions, and wherein the execution circuit is configured to compress the corresponding first source matrix according to a second one or more instructions to generate the compressed sparse source matrix slice and store the compressed sparse source matrix slice in the memory, the compressed sparse source matrix slice including the bit mask.

10. The system according to claim 9, wherein, A third subset of 8-bit integer data elements of the corresponding first source matrix is excluded from the compressed sparse source matrix slice based on the size of the third subset of 8-bit integer data elements.

11. The system according to claim 10, wherein The third subset of 8-bit integer data elements includes zero-valued data elements.

12. The system according to claim 9, wherein, The relative positions of the first subset of 8-bit integer data elements in the corresponding first source matrix are maintained in the compressed sparse source matrix slice.

13. A machine-readable medium having one or more instructions stored thereon, the one or more instructions, when executed by one or more processors, for performing a combined load and decompression operation according to one or more of the instructions, the combined load and decompression operation including: Read a compressed sparse source matrix slice from a memory, the compressed sparse source matrix slice including a first subset of 8-bit integer data elements of a corresponding first source matrix and a bitmask for indicating positions of the first subset of 8-bit integer data elements in the corresponding first source matrix, wherein the bitmask includes a plurality of bits, each bit corresponding to a data element position in the corresponding first source matrix and being set to a first bit value to indicate a matrix position of a corresponding 8-bit integer data element of the first subset of 8-bit integer data elements, Decompress the compressed sparse source matrix slice according to the bitmask to generate a decompressed sparse source matrix slice, the decompressed sparse source matrix slice including the first subset of 8-bit integer data elements, and Store the decompressed sparse source matrix slice in one or more of the plurality of vector registers.

14. The machine-readable medium according to claim 13, further comprising instructions for causing matrix processing circuitry integrated into or coupled to the one or more processors to perform the following operations: performing a matrix multiplication operation to multiply the decompressed sparse source matrix slice by a second source matrix slice, the second source matrix slice including a plurality of 8-bit integer data elements of a second source matrix, the matrix multiplication operation including: Multiplying each 8-bit integer data element in a second subset of 8-bit integer data elements of the second source matrix slice by a corresponding 8-bit integer data element of the first subset identified based on at least a portion of the bitmask to generate a plurality of products, and Adding each product in the plurality of products to a corresponding 32-bit accumulated integer data element to generate a corresponding 32-bit integer result data element of a result matrix slice.

15. The machine-readable medium according to claim 14, wherein, The one or more instructions include a first one or more instructions, and the machine-readable medium has stored thereon a second one or more instructions which, when executed by the one or more processors, are for causing the one or more processors to compress the corresponding first source matrix to generate the compressed sparse source matrix slice and store the compressed sparse source matrix slice to the memory, the compressed sparse source matrix slice including the bitmask.

16. The machine-readable medium according to claim 15, wherein, A third subset of 8-bit integer data elements of the corresponding first source matrix is excluded from the compressed sparse source matrix slice based on the size of the third subset of 8-bit integer data elements.

17. The machine-readable medium according to claim 16, wherein, The third subset of 8-bit integer data elements includes zero-valued data elements.

18. The machine-readable medium according to claim 15, wherein, The relative positions of the first subset of 8-bit integer data elements in the corresponding first source matrix are maintained in the compressed sparse source matrix slice.

19. A method, comprising: Configuring the use of slices; Loading at least one slice from a memory; Using the slice to perform at least one slice operation; Storing at least one slice out to the memory; And Performing a context switch.

20. A system, comprising: A memory for storing slice descriptions; And A processor, comprising: Instruction execution resources; and Slice configuration, the slice configuration including a palette table and registers, wherein the instruction execution resources are used to store the slice description in the slice configuration.

21. A method, comprising: Compiling a program in a high-level programming language using an x86 compiler to generate x86 binary code natively executable by a first processor having at least one x86 instruction set core; Using an instruction converter to convert the x86 binary code into alternative binary code natively executable by a second processor that does not have an x86 instruction set core.

Images