DEVICE, COMPUTER SYSTEM, METHOD AND ONE OR MORE NON-TRANITORY MACHINE-READABLE STORAGE MEDIA FOR ACCELERATING MATRIX MULTIPLICATION OF THINLY PLATED MATRICES USING COLLAR FOLDING AND COMPACTION

Abstract

Device comprising: a plurality of registers for storing a first matrix and a second matrix; and a first circuit coupled to the plurality of registers, wherein the first circuit executes a matrix multiplication and accumulation instruction to: to receive a packed matrix representing a sparse matrix, where fields identify logical positions of elements in the sparse matrix that are involved in packing the sparse matrix; to perform matrix multiplications on corresponding elements in corresponding rows and columns of the packed matrix and the first matrix in order to produce corresponding products; and to accumulate the products with corresponding elements in corresponding rows and corresponding columns of the second matrix, wherein the device further comprises a second circuit coupled to the first circuit, wherein the second circuit generates the packed matrix by packing the sparse matrix, wherein the second circuit, in order to generate the packed matrix, analyzes a window in a row of the sparse matrix and moves non-zero elements in the window to positions of zero-valued elements in the window.

Description

FIELD OF THE INVENTION

[0001] The field of the invention relates generally to a computer processor architecture, and more particularly to an apparatus, a computer system, a method, and one or more non-transitory machine-readable storage media for accelerating matrix multiplication of sparse matrices using column convolution and compaction. BACKGROUND

[0002] Many current hardware accelerators for neural networks primarily perform matrix multiplication in a dense format, without considering the fact that there is a high percentage of zeros found in one (or both) matrices. This leads to inefficient use of hardware resources (due to multiplying by zero) and wasted energy.

[0003] Today's hardware accelerators for neural networks, both for training and inference, are all competing to achieve the best raw performance figures and energy-to-performance ratios. Exploiting the native and injected sparsity in these neural networks is one way to gain an advantage in this competition.

[0004] Machine learning architectures such as deep neural networks have been applied to areas including computer vision, speech recognition, natural language processing, audio recognition, social network filtering, machine translation, bioinformatics, and drug development.

[0005] Matrix multiplication is a critical performance / energy limiter for many algorithms, including machine learning. Some conventional matrix multiplication approaches are specialized, lacking, for example, the flexibility to support a variety of data formats (signed and unsigned 8b / 16b integers, 16b floating-point) with wide accumulators and the flexibility to support both dense and sparse matrices.

[0006] The problem addressed here is to increase the performance and energy efficiency of neural network processing chips by more effectively processing matrix multiplications in the presence of sparsity in the input data set (sparse matrices have a density less than 1.0, meaning that fewer than 100% of their elements have non-zero values). In particular, this problem is addressed while maintaining performance for multiplying dense (non-sparse) matrices.

[0007] Chen, Linchuan et al.: "Exploiting recent SIMD architectural advances for irregular applications" addresses a class of applications known as irregular applications that involve indirect or data-dependent memory accesses and describes a general optimization methodology to effectively optimize various subclasses of irregular applications. Based on the observation that all applications with indirect memory accesses can be considered sparse matrix computations, an optimization methodology with three substeps is proposed: 1) improving locality through tiling, 2) identifying data access patterns, and 3) eliminating write contention at both the SIMD and MIMD levels.

[0008] US 2017 / 0371839 A1 relates to a system and method for efficiently processing a sparse matrix. A compressed representation of a sparse matrix, the sparse matrix containing one or more non-zero entries in one or more of a plurality of parts of the matrix, is obtained from a server including one or more streaming multiprocessors, each of the streaming multiprocessors including one or more graphics processing unit (GPU) processor cores. Each of the parts is assigned to one of a plurality of partitions based on the number of non-zero entries in that part. For each of the partitions, a predefined number of GPU processor cores is allocated for processing each of the parts assigned to that partition based on the number of non-zero entries in the parts assigned to that partition. For each of the partitions, each of the parts assigned to that partition is processed. SUMMARY OF THE INVENTION

[0009] It can therefore be considered an object of the present invention to propose an apparatus, a computer system, a method and one or more non-transitory machine-readable storage media for performing matrix multiplication and accumulation of sparse matrices, which has improved performance and lower energy consumption compared to conventional solutions such as in Chen, Linchuan et al.: "Exploiting recent SIMD architectural advances for irregular applications".

[0010] The above object is achieved according to the invention with the device according to the main claim 1, the computer system according to the independent claim 11, the method according to the independent claim 17 and the one or more non-transitory machine-readable storage media according to the independent claim 22. The dependent claims define further developments of the device according to the invention, the computer system according to the invention and the method according to the invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0011] The present invention is illustrated by way of example and not by way of limitation in the figures of the accompanying drawings, wherein like references indicate similar elements. Fig. 1 is a block diagram illustrating processing components for executing a sparse matrix multiplication (SMM) instruction, according to one embodiment. Fig.2 shows example execution flows for executing a sparse matrix multiplication (SMM) instruction according to some embodiments. Fig. 3 is a block diagram of a processing array for executing a tiled sparse matrix multiplication (SMM) instruction according to some embodiments. Fig. 4 is a block diagram illustrating matrix convolution for a sparse matrix multiplication (SMM) instruction, according to some embodiments. Fig. 5A is a block diagram illustrating matrix convolution for a sparse matrix multiplication (SMM) instruction, according to some embodiments. Fig. 5B is a block diagram illustrating matrix multiplication with optimized convolution opportunities, according to some embodiments. Fig. 5C is a block diagram illustrating the increase of folding opportunities, according to some embodiments. Fig.6A is a block diagram illustrating matrix compaction for a sparse matrix multiply (SMM) instruction, according to some embodiments. Fig. 6B is a block flow diagram illustrating matrix compaction by a processor performing sparse matrix multiplication (SMM), according to some embodiments. Fig. 6C is another block flow diagram illustrating matrix compaction by a processor performing sparse matrix multiplication (SMM), according to some embodiments. Fig. 6D is a block diagram illustrating circuitry for implementing matrix compaction for a sparse matrix multiply (SMM) instruction, according to some embodiments. Fig. 7 is a block diagram illustrating a multiply-accumulate circuit for a sparse matrix multiply (SMM) instruction, according to some embodiments. Fig.8 is a block flow diagram of a processor executing a sparse matrix multiplication (SMM) instruction, according to some embodiments. Fig. 9 shows a format of a sparse matrix multiplication (SMM) instruction according to some embodiments. The Fig. 10A-10B are block diagrams illustrating a generic vector-friendly instruction format and instruction templates thereof, according to some embodiments of the invention. Fig. 10A is a block diagram illustrating a generic vector-friendly instruction format and Class A instruction templates thereof, according to some embodiments of the invention. Fig. 10B is a block diagram illustrating the generic vector-friendly instruction format and Class B instruction templates thereof, according to some embodiments of the invention. Fig.11A is a block diagram illustrating an exemplary specific vector-friendly instruction format, according to some embodiments of the invention. Fig. 11B is a block diagram showing the fields of the specific vector-friendly instruction format that make up the full opcode field, according to one embodiment. Fig. 11C is a block diagram showing the fields of the specific vector-friendly instruction format that form the register index field, according to one embodiment. Fig. 11D is a block diagram showing the fields of the specific vector-friendly instruction format that form the extension operation field, according to one embodiment. Fig. 12 is a block diagram of a register architecture according to one embodiment. Fig.13A is a block diagram illustrating both an example in-order pipeline and an example register-renaming out-of-order issue / execution pipeline, according to some embodiments. Fig. 13B is a block diagram illustrating both an example embodiment of an in-order architectural core and an example register-renaming out-of-order issue / execution architectural core to be included in a processor according to embodiments. The Fig. 14A-B show a block diagram of a more specific example in-line core architecture, where the core would be one of several logic blocks (including other cores of the same type and / or different types) in a chip. Fig.14A is a block diagram of a single processor core along with its connection to the on-chip interconnect network and its local subset of the Level 2 (L2) cache, according to some embodiments. Fig. 14B is an expanded view of a portion of the processor core of Fig. 14A according to some embodiments. Fig. 15 is a block diagram of a processor that may have more than one core, may have an integrated memory controller, and may have integrated graphics, according to some embodiments. The Fig. 16 - 19 are block diagrams of example computer architectures. Fig. 16 shows a block diagram of a system according to some embodiments. Fig. 17 is a block diagram of a first more specific example system according to an embodiment. Fig.18 is a block diagram of a second, more specific example system, according to some embodiments. Fig. 19 is a block diagram of a system-on-a-chip (SoC) according to some embodiments. Fig. 20 is a block diagram illustrating the use of a software instruction converter to convert binary instructions in a source instruction set to binary instructions in a target instruction set, according to some embodiments. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0012] The disclosed embodiments provide a way to exploit the sparsity of a sparse input matrix to perform matrix multiplication more efficiently and consume less power, while reducing the number of cycles required to complete the computation. Unlike conventional symmetric matrix multiplication circuits, the disclosed embodiments are expected to improve matrix multiplication performance and energy efficiency by packing sparse matrices to avoid multiplying by zero. Packing, according to some embodiments, involves folding a sparse matrix to replace zero-valued elements with non-zero elements. According to other embodiments, packing involves compacting adjacent columns of a sparse matrix to similarly replace zero-valued elements with non-zero elements.In any case, when routing circuitry moves an element of a multiplicand matrix, it also routes the appropriate elements of the multiplier matrix to produce the result.

[0013] Here, the "sparsity" of a matrix is ​​defined as the fraction of non-zero elements, with the remaining elements being zero. For example, when performing a matrix multiplication using a sparse matrix with a sparsity of 0.875 (where only 1 / 8, or 12.5%, of its elements have non-zero elements), disclosed embodiments are expected to exhibit improved cost and performance by avoiding zero multiplications. According to some embodiments, the term "sparsity" may be used to refer to the fraction of zero-valued elements.

[0014] According to some embodiments, when performing matrix multiplication operations, a processor dynamically detects sparsity in the input matrices (detects zero-valued elements). According to some embodiments, the processor sets a bit to identify each zero-valued element. Although the processor may dynamically analyze the data elements at runtime, according to some embodiments, the input matrices may be pre-analyzed and the zero-valued elements may be marked. Based on the determined sparsity level, if appropriate, the processor uses routing circuitry to pack the matrices into a "packed" or "folded" format. The processor then uses the routing circuitry to configure a processing core to process the packed matrix using fewer cycles or less hardware.

[0015] According to some disclosed embodiments, such as with reference to Fig. As illustrated and described in Figure 3, matrix multiplication is broken down into smaller matrices that can be processed with less hardware. For example, when multiplying an A matrix by a B matrix, if A and B are logical 512x512 matrices, the disclosed embodiments break down the calculation into 32x32 submatrices.

[0016] The disclosed embodiments provide improved matrix multiplication for use in applications such as deep learning or machine learning.

[0017] Some embodiments utilize matrix sparsity by folding the sparse matrix, such as with respect to Fig. 5A - 5C, or by densifying the sparse matrix, as described, for example, with reference to the Fig. 6A - 6D shown and described.

[0018] Some previous solutions require sparsity in both matrices to be effective. However, embodiments disclosed here can provide acceleration even when sparsity occurs in only one matrix.

[0019] The disclosed embodiments are expected to improve cost and area utilization by providing a single reconfigurable execution circuit to support a variety of data formats—including integer and floating-point formats—compared to some approaches that rely on different circuits specialized for different data formats. The disclosed embodiments provide a matrix multiplication accelerator that supports both floating-point and integer data formats with accumulation. The disclosed accelerator can also be optimized to operate on sparse matrices by avoiding multiplication of the zero elements.By combining these features into a reconfigurable circuit, the disclosed embodiments accordingly enable a single matrix multiplication accelerator circuit to support multiple precision formats with wide accumulators, while efficiently reconfiguring for either dense or sparse matrices. The disclosed accelerator embodiments improve area and power efficiency while providing flexibility to support many typical machine learning-type matrix multiplication tasks.

[0020] According to some embodiments, the execution circuitry uses a dot product algorithm, by which the rows of the A matrix are loaded into a matrix multiplication processing unit (MMPU) and then the columns of the B matrix are streamed through one cycle at a time. Elements of the B matrix that have yet to be processed are sometimes referred to as "downstream" elements. This algorithm generates one column of the C matrix per cycle.

[0021] Fig. Figure 1 is a block diagram illustrating processing components for executing a sparse matrix multiplication (SMM) instruction, according to some embodiments. As illustrated, a memory 101 stores one or more SMM instructions 103 to be executed. As further described below, according to some embodiments, computing system 100 is a SIMD processor for concurrently processing multiple elements of packed data vectors, such as matrices.

[0022] In operation, the SMM instruction 103 is fetched from the memory 101 by fetch circuitry 105. The fetched SMM instruction 107 is decoded by decoding circuitry 109. The SMM instruction format, which is determined from the Fig.9, 10A-B and 11A-D, includes fields (not shown here) for specifying first, second and target matrices, wherein the specified second matrix, according to some embodiments, is a sparse matrix with a density less than one (where the density is proportional to the non-zero elements, such that the second matrix has at least some zero-valued elements). Decode circuitry 109 decodes the fetched SMM instruction 107 into one or more operations. According to some embodiments, this decoding includes generating a plurality of micro-operations to be executed by execution circuitry (such as execution circuitry 119). Decode circuitry 109 also decodes instruction suffixes and prefixes (if used). Execution circuitry 119 will be described below at least with reference to the Fig.2 - 8, 13A-B and 14A-B are further described and illustrated.

[0023] According to some embodiments, a register renaming, register allocation, and / or scheduling circuit 113 provides functionality to perform one or more of the following: 1) renaming logical operand values ​​to physical operand values ​​(e.g., a register alias table, according to some embodiments), 2) assigning status bits and flags to the decoded instruction, and 3) scheduling the decoded SMM instruction 111 for execution on the execution circuitry 119 from an instruction pool (e.g., using a reservation station, according to some embodiments).

[0024] A routing circuitry 117 supports the execution circuitry 119 by performing the matrix convolution (with respect to the Fig. 4 and 5A-C) and matrix compression (with reference to the Fig.6A-D) to enable optimized matrix multiplication and zero multiplication reduction. According to some embodiments, routing circuitry 117 is external to execution circuitry 119, as shown. According to other embodiments, routing circuitry 117 is incorporated into and is part of execution circuitry 119.

[0025] Registers (a register file) and / or memory 115 store data as operands of the decoded SMM instruction 111 to be processed by the execution circuitry 119. Example register types include write mask registers, packed data registers, general purpose registers, and floating point registers, as described below at least with reference to Fig. 12 is further described and illustrated.

[0026] According to some embodiments, the write-back circuit 120 passes the result of the execution of the decoded SMM instruction 111. The execution circuitry 119 and the system 100 are further described with reference to the Fig. 2 - 8, 13A-B and 14A-B.

[0027] Fig.2 shows example execution flows for executing a sparse matrix multiplication (SMM) instruction according to some embodiments. As illustrated, the SMM instruction 200 has fields for specifying a destination matrix, a first source matrix, and a second source matrix, sometimes referred to as a C matrix, A matrix, and B matrix, respectively, or as a DST matrix, SRC1 matrix, and SRC2 matrix, respectively, or as an output matrix, a dense source matrix, and a sparse source matrix, respectively. If the second source matrix is ​​a sparse matrix, it has a sparsity less than one (sparsity refers to the proportion of non-zero elements, which can be expressed as a percentage, fraction, or number between 0 and 1).

[0028] As illustrated, the src1 matrix 202, the src2 matrix 204, and the dst matrix 206 have (M x K), (K x N), and (M x N) elements, respectively, where M = 4, K = 4, and N = 4. Of course, M, K, and N may vary according to other embodiments. Points are used to specify non-zero elements of the A, B, and C matrices, where the A matrix 202 is a dense matrix and the B matrix 204 is a sparse matrix.

[0029] In operation, in response to SMM instruction 200, a processor shall multiply src1 matrix 202 by src2 matrix 204 to produce a product and store it in dst matrix 206. For example, the processor shall compute the top leftmost element, densely labeled 208, of dst matrix 206 (also referred to as C-matrix, output, or product) by multiplying elements in the top row of src1 matrix 202 (also referred to as A-matrix, dense matrix, multiplier, or factor) by corresponding elements in the leftmost column of src2 matrix 204 (also referred to as B-matrix, sparse matrix, multiplicand, or factor). Here, the term “corresponding” means that the relative position of the element of the A matrix is ​​the same relative position within the multiplied row as the element in the multiplied column of the B matrix.To explain the initial part of the operation, note that the result 208 stored at row 0, column 0 of the C matrix 206 is generated by multiplying and accumulating corresponding nonzero elements of the src1 matrix 202 (also called the A matrix), row 0, and the src2 matrix 204 (also called the B matrix), column 0. Expressed algebraically, Cr0c0 = Ar0c0 * Br0c0 + Ar0c1*Br1c0 + Ar0c2*Br2c0 + Ar0c3*Br3c0. The remaining nonzero products of column 0 of the dst matrix 206 are generated similarly. The elements of columns 1, 2, and 3 of the C matrix 206 are all zero because columns 1, 2, and 3 of the src2 matrix 204 are all zero. According to some embodiments, the generated products are accumulated with previous contents of the dst matrix 206. Execution of the SMM instruction by disclosed embodiments is further described with reference to FIG. Fig. 3 - 8, 13A-B and 14A-B. TILED MATRIX MULTIPLICATION

[0030] Fig. Figure 3 is a block diagram of a processing array for executing a tiled sparse matrix multiplication (SMM) instruction according to some embodiments. As illustrated, the A matrix 302 and the B matrix 304 are 16x16 matrices to be multiplied to produce a C matrix 306, which is also a 16x16 matrix.

[0031] According to some embodiments, as shown, a multiplication of a large matrix is ​​performed by breaking the operands down into smaller matrices. Here, the 16x16 matrices are broken down into smaller 4x4 matrices. For example, to calculate a submatrix c00, the submatrices a00 and b00 are multiplied using a 4x4 array of processing units, and then the results of the submatrix multiplications a01*b10, a02*b20, and a03*b30 are accumulated to the first result.

[0032] Therefore, if each of the 4x4 multiplications is performed in one cycle, four cycles would be required to complete the calculation of the submatrix c11. As further shown with reference to the Fig.5-7, some disclosed embodiments utilize a "folding" concept to speed up and improve computation efficiency without increasing the number of multipliers. For example, if matrices b00, b10, and / or b20 are sparse matrices, the disclosed embodiments avoid using hardware resources to multiply zero elements and instead apply the multipliers to performing useful work by "folding" elements from submatrix b30 to use those multipliers. If all elements of submatrix b30 can be folded, the disclosed embodiments enable execution to speed up by skipping the a03xb30 multiplication entirely. MATRIX CONVOLTATION

[0033] Fig.4 is a block diagram of another optimized implementation of a sparse matrix multiplication (SMM) instruction, according to some embodiments. To illustrate the optimization, the SMM instruction 400 is illustrated as executing in both unoptimized mode 401 and optimized mode 411. As illustrated, during unoptimized matrix multiplication 401, the dense source matrix 402 (also referred to as the A matrix) is to be multiplied by the sparse source matrix 404 (also referred to as the B matrix) to generate an output matrix 406 (also referred to as the C matrix).

[0034] The dense source matrix 402 is a 3x3 matrix with nine non-zero elements, labeled A at location (0,0) through I at location (2,2). The sparse source matrix 404 is also a 3x3 matrix and has elements labeled R at location (0,0) through Z at location (2,2). According to the illustrated embodiment, the A matrix 402 is a submatrix of a logically larger matrix, and a next column is shown with elements labeled J, K, and L. Similarly, the illustrated B matrix 404 is a submatrix of a logically larger matrix, and a next row is shown with elements labeled O, P, and Q.

[0035] However, the sparse source matrix 404 has a zero-valued element Y 408, which would result in three zero modifications in the unoptimized mode to produce products CF, FY, and IY (to optimize the multiplication, the routing circuitry, as described below, may exploit an opportunity to shift element 409 to use the multiplication resources that would otherwise be used for the zero multiplication involving the zero-valued element 408).

[0036] It should be noted that, although Fig. 4 shows that 409 is shifted from a next submatrix, during the operation also elements from the same submatrix, for example the elements V or S, can be shifted.

[0037] The elements of the output matrix 406 are represented algebraically. For example, the element at position (0,0) of the C matrix 406 is "AR+BU+CX." In the non-optimized mode 401, the elements of the middle column of the output matrix 406 would have zero-valued products C * Y, F * Y, and I * Y.

[0038] In the optimized version of operation 411, the A, B, and C matrices are denoted as 412, 414, and 416, respectively. In some embodiments, the routing circuitry shifts element P 419 to the location 418 previously occupied by the zero-valued element Y.

[0039] After optimization, the execution hardware is no longer used to multiply by zero. Instead, these multipliers are used to generate useful products JP, KP, and LP. In optimized mode, the multipliers that would be used to generate the products CY, FY, and IY in non-optimized mode are now used to generate useful non-zero products JP, KP, and LP. To implement the optimization, the zero-valued products CY, FY, and IY that might otherwise occur are no longer generated. Instead, useful (i.e., non-zero) products JP, KP, and LP are generated and accumulated with the products generated in the second column of C-matrix 416, as shown. During the operation, another element may be moved into the location vacated by element 419.

[0040] Fig.5A is a block diagram illustrating matrix convolution for a sparse matrix multiplication (SMM) instruction, according to some embodiments. As illustrated, the B submatrix b20 (i.e., at the second row and zeroth column of a larger logical array) is denoted as 502 and is a sparse 4x4 matrix containing nine (9) non-zero elements represented by circles. The B submatrix b30 (i.e., at the third row and zeroth column of a larger logical array) is denoted as 504 and is a sparse 4x4 matrix containing seven (7) non-zero elements represented by squares.

[0041] To optimize the execution of the SMM instruction 500, similar to the Fig.4, one or more non-zero elements of the B submatrix b20, designated 502, are shifted into the zero-valued elements of the B submatrix b30, designated 504. When elements are shifted as described, this is sometimes referred to as "folding" the elements of one submatrix over another submatrix. After optimization, the optimized B submatrix b30, designated 506, has a chunk of nine (9) non-zero elements from the B submatrix b20 "folded" over it. The C submatrix 512 is computed by multiplying A submatrices 508 and 510 by appropriate elements of the optimized B submatrix 506. For simplicity, only the element of the C submatrix 512 at location (0,1) is shown, with the other elements shown as "X," i.e., meaningless.

[0042] The submatrix "convolution" concept stems from the observation that if matrix B is sparse (has many non-zero values), the unoptimized matrix multiplication would involve multiple useless zero multiplications. Therefore, according to disclosed embodiments, a sparse submatrix (e.g., B submatrix b20, denoted as 502) is convolved over another sparse submatrix (e.g., B submatrix b30, denoted as 304). Given that the convolution involves shifting multiple elements, this is sometimes referred to as folding a "chunk" of elements.

[0043] The Fig.The embodiment disclosed in Figure 5A therefore avoids multiplication by zeros by avoiding the use of any of the zero-valued elements of the B submatrix b20 502 and the B submatrix b30 504, which would uselessly utilize the multiplication hardware while contributing nothing to the final result. As illustrated, the zero-valued elements of the B submatrix b30 504 are overwritten with non-zero elements of a vertically adjacent B submatrix b20 502. Despite the convolution, the final result of each element in the C matrix 510 is the sum of products of the total columns of all submatrices of A with B. According to some embodiments, one entire submatrix is ​​convolved onto another, allowing the processor to skip multiplying the now all-zero submatrix, thereby speeding up the overall computation and reducing its power consumption.

[0044] Fig.Figure 5A shows a convolution that succeeded in finding an empty (i.e., zero-valued) space in the B submatrix b30 504 for each nonzero element of the B submatrix b20 502. This might not always be the case because nonzero elements appear at random indices. In this case, the Fig. 5B and Fig. 5C two embodiments for increasing the folding opportunities.

[0045] A first embodiment for increasing the folding opportunities is shown in Fig.5B, which is a block diagram illustrating matrix multiplication with optimized convolution opportunities, according to some embodiments. As illustrated, the optimized B submatrix 522 includes only non-zero elements. Focusing on column 1 of the optimized B submatrix 522, the column consists of two element types, one represented by a circle and the other represented by a square. A submatrix a00, denoted as 524, includes non-zero elements represented by circles that are to be multiplied by appropriate elements of the B submatrix 522 to produce corresponding elements of the C matrix 528, only one element of which is shown.A submatrix a01, denoted 526, has non-zero elements represented by squares that are to be multiplied by appropriate elements of the optimized B submatrix 522 to produce corresponding elements of the C matrix 528, only one element of which is shown. By providing multiplexers to select between the elements of the submatrices a00 524 and a01 526, the embodiment of FIG. Fig. 5B accordingly illustrates the opportunities for folding elements in the B-matrix 522. By increasing the level of multiplexing, the range from which elements of the B-matrix can be selected and folded down vertically is increased. According to other embodiments not shown, even wider multiplexers are provided to select from an even wider set of A-submatrix elements.

[0046] A second embodiment for increasing the folding opportunities is shown in Fig.5C, which is a block diagram illustrating the increase in folding opportunities, according to some embodiments. As shown, the Fig.5C, the embodiment illustrated in FIG. 5C optimizes the convolution opportunities by attempting to convolve over multiple cycles while remembering the "residue" after each cycle. The remainder refers to all elements that could not be convolved in a cycle. As shown, the SMM instruction 540 requests the performance of a multiplication by a sparse B submatrix as a factor. The non-optimized B submatrix 2,1, denoted 542, has nine non-zero elements represented by circles that are to be convolved into the B submatrix 3,1, denoted 544, which has six zero-valued elements. In a first cycle 548, an optimized B submatrix 3,1, denoted 546, is formed by convolving it into six non-zero elements of the B submatrix 2,1 542.The remainder 550, which consists of the three non-zero elements of the B-submatrix 2,1 542 that were not folded during the first cycle, is folded during a second cycle, cycle 1552, to the optimized B-matrix 3,1 546.

[0047] The Fig. The convolution optimizations shown in Figures 5A-C can be repeated until every non-zero element in a B-matrix has been convolved, at which point that B-submatrix can be skipped entirely, thereby reducing power consumption and increasing performance. MATRIX COMPACTION

[0048] Fig.6A is a block diagram illustrating an optimized execution of a sparse matrix multiplication (SMM) instruction, according to some embodiments. To illustrate optimization according to some embodiments, the SMM instruction 600 is shown executing in both unoptimized mode 601 and optimized mode 611. The dense source matrix 602 is a 3x3 matrix with nine non-zero elements, labeled A at location (0,0) through I at location (2,2). The sparse source matrix 604 is also a 3x3 matrix and has elements labeled R at location (0,0) through Z at location (2,2). However, the sparse source matrix 604 has a zero-valued element U 608, which would result in zero multiplication in a unoptimized mode. The elements of the output matrix 606 are represented algebraically. For example, the element at position (0, 0) of the C matrix 606 is “AR+BU+CX”.In the non-optimized mode 601, the elements of the leftmost column of the output matrix 606 would have zero-valued products B*U, E*U and H*U.

[0049] During the non-optimized matrix multiplication 601, the dense source matrix 602 (also referred to as the A matrix) is to be multiplied by the sparse source matrix 604 (also referred to as the B matrix) to generate the output matrix 606 (also referred to as the C matrix) one column at a time. The non-optimized matrix multiplication 602 comprises three zero multiplications to generate the products BU, EU, and HU. The non-optimized operation takes three cycles.

[0050] However, to optimize multiplication according to disclosed embodiments, the routing circuitry, as described below, exploits an opportunity to shift element 609 to use the multiplication resources that would otherwise be used for the zero multiplication involving zero-valued element 608. During the operation, another element may be shifted into the location vacated by element 619.

[0051] If all elements in a column of the sparse source matrix 604 are shifted, execution skips the now-empty column. The optimized mode would then require at least one fewer clock cycle to perform the multiplication.

[0052] In the optimized version of operation 611, the A, B, and C matrices are labeled 612, 614, and 616, respectively. This time, according to some embodiments, the routing circuitry shifts the element "V" from the location (1,1) labeled 619 to the location (1,1) labeled 618.

[0053] After optimization, the execution hardware is no longer used to multiply by zero. Instead, these multipliers are used to generate useful products BV, EV, and HV. In optimized mode, the multipliers that would be used to generate BU, EU, and HU in non-optimized mode are now used to generate useful non-zero products BV, EV, and HV. To implement the optimization, the otherwise potentially occurring zero-valued products BU, EU, and HU are no longer generated. Instead, useful (i.e., non-zero) products BV, EV, and HV are generated and accumulated with the products generated in the second column of the C-matrix 616, which elements would be generated in non-optimized mode.

[0054] Fig.6B is a block flow diagram illustrating matrix compaction by a processor executing a sparse matrix multiplication (SMM) instruction, according to some embodiments. As illustrated, the Fig. 1-8, 13A-B, and 14A-B execute the SMM instruction 620 over five cycles. A portion of this execution is shown, namely, the processing of the 8x8 B-matrix over the five cycles is shown and labeled B-matrix cycles 0-4, 621A-E. Five snapshots of the B-matrix are shown as 622-630.

[0055] Without optimization, assuming one column of the B matrix is ​​processed per cycle, eight cycles would be required to process the B matrix. However, the illustrated embodiment optimizes execution by exploiting opportunities to compact non-zero elements into adjacent zero-valued slots.

[0056] The illustrated embodiment uses a pairwise column "compaction window" in operation to move non-zero elements, if possible, into adjacent zero-valued slots. The pairwise compaction window is a sliding-window approach, and any elements that were not "compactable" are carried over to the next iteration (as shown in the example). An optimization opportunity arises whenever a column consisting of all zeros is encountered, at which point the compaction window may skip that column entirely, thereby improving performance and reducing power consumption. According to some embodiments not shown, the compaction window is wider than shown. According to some embodiments, such as in the Fig.6C-D, a non-zero element identified in the compression window may be moved to an available nearby zero-valued slot according to specified move rules. According to the embodiment of the Fig. 6A-B, an element can only be moved horizontally from right to left within the window size. As with reference to the Fig. 6C-D, according to some embodiments, an element can be moved horizontally from the right column to the left column and can also be moved up or down one position, thereby increasing the opportunities for moving elements. The embodiment of Fig. 6C therefore shows a "window pane width" of three elements, designated 643. According to other embodiments, the window pane width may be different.

[0057] To discuss Fig.6B, in cycle 0, as represented by snapshot 622, the routing circuitry analyzes a pairwise window 631A on the first two columns of the B-matrix 621A to identify two non-zero elements at locations (0,1) and (5,1) and shift them to adjacent zero-valued slots, thereby producing a partially optimized B-matrix 621B.

[0058] It should be noted that the B-matrix 521B, even after being optimized, still contains zero-valued elements at the positions (3,0) and (6,0) of the first column. According to embodiments with increased routing opportunities, such as with reference to Fig.6C, the positions (3,0) and (6,0) of the first column are filled with non-zero elements, zero multiplications are avoided, performance is improved by eliminating a cycle, and power consumption is reduced. According to some embodiments, the zero-valued elements of column zero of B matrix 521B are identified, and zero multiplication is avoided. According to other embodiments, zero multiplication is allowed to continue.

[0059] In cycle 1, as illustrated by snapshot 624, the routing circuitry analyzes a pairwise window 631B on the next two columns of the B-matrix 621B to identify four non-zero elements and shift them from locations (0,2), (3,2), (4,2), and (6,2) to adjacent zero-valued slots, thereby generating a partially optimized B-matrix 621C.

[0060] An optimization opportunity 632 exists in the optimized B-matrix 621C. In particular, column 2 of the optimized B-matrix 621C has no non-zero elements and can therefore be skipped to improve performance by one cycle and reduce power consumption. Then, in cycle 2, as represented by snapshot 626, the pairwise compaction window 631C has skipped column 2 of the B-matrix and now spans columns three through four. The routing circuitry analyzes the pairwise window 631C on columns three through four of the B-matrix 621C to identify a non-zero element and move it from location (0,4) to an adjacent zero-valued slot, thereby generating a partially optimized B-matrix 621D.

[0061] In cycle 3, as illustrated by snapshot 628, the pairwise compaction window 631D is shifted by one element to now analyze columns 4-5 of the B-matrix 621D. Routing circuitry analyzes the pairwise window 631D on columns four through five of the B-matrix 621D to identify five non-zero elements and shift them from locations (2,5), (3,5), (4,5), (6,5), and (7,5) to adjacent zero-valued slots, thereby generating a partially optimized B-matrix 621E.

[0062] Another optimization opportunity 634 exists in the optimized B-matrix 621E. In particular, column 5 of the optimized B-matrix 621E has no non-zero elements and can therefore be skipped to improve performance and reduce power consumption. Then, in cycle 4, as represented by snapshot 630, the pairwise compaction window 631E has skipped column 5 of the B-matrix and now spans columns six through seven. The routing circuitry analyzes the pairwise window 631E on columns six through seven of the B-matrix 621E to identify four non-zero elements and move them from locations (3,7), (4,7), (6,7), and (7,7) to adjacent zero-valued slots, thereby creating another all-zero column, column 7, which can be skipped to optimize power consumption and performance.

[0063] Fig.6C is another block flow diagram illustrating matrix compaction by a processor executing a sparse matrix multiplication (SMM) instruction, according to some embodiments. As illustrated, the block flow diagram illustrating, for example, Fig. 1-5C, 7-8, 13A-B, and 14A-B execute the SMM instruction 640 over five cycles. A portion of this execution is shown, namely, the processing of the 8x8 B-matrix over the five cycles is shown and labeled B-matrix cycles 0-4, 641A-E. Five snapshots of the B-matrix are shown and labeled 642, 644, 646, 648, and 650.

[0064] Without optimization, assuming one column of the B matrix is ​​processed per cycle, 8 cycles would be required to process the B matrix. However, the illustrated embodiment optimizes execution by using routing circuitry to exploit opportunities to compact non-zero elements into nearby zero-valued slots.

[0065] The illustrated embodiment uses a pairwise column "compaction window" in operation to move non-zero elements, if possible, into nearby zero-valued slots. The pairwise compaction window uses a sliding-window approach, and any elements that were not "compactable" are carried over to the next iteration (as illustrated in the example). An optimization opportunity arises whenever a column consisting of all zeros is encountered, at which point the compaction window can skip that column. Here, four such opportunities are labeled 652, 654, 656, and 658.

[0066] In cycle 0, as illustrated by snapshot 642, the routing circuitry analyzes a pairwise window 651A on the first two columns of the B-matrix 641A to identify four non-zero elements and move them from locations (0,1), (2,1), (5,1), and (7,1) to nearby zero-valued slots, thereby generating a partially optimized B-matrix 641B.

[0067] It should be noted that according to the Fig. 6B, only the horizontal displacement of the elements was allowed, so that the elements at positions (2,1) and (7,1) were not displaced in cycle 0. The embodiment of Fig.However, Figure 6C has a larger "window size" of 3 elements, an example of which is designated 643. With the larger window size, the elements at locations (2,1) and (7,1) can be shifted in cycle 0. According to other embodiments not shown, the window size is different, whether larger or smaller.

[0068] It should be noted that some of the non-zero elements shifted in cycle 0 were shifted to diagonally adjacent zero-valued slots. To enable such a shift, a circuit arrangement such as that shown in Fig. 6D to forward the appropriate corresponding element of the A matrix to be multiplied by the shifted element of the B matrix. According to other embodiments, the multiplexer 662 ( Fig. 6D), for example, if the window pane size exceeds three, additional A elements as inputs.

[0069] An optimization opportunity 652 exists in the optimized B-matrix 641B. In particular, column 1 of the optimized B-matrix 641B has no non-zero elements and can therefore be skipped to improve performance and reduce power consumption. Then, in cycle 1, as represented by snapshot 644, the pairwise compaction window 651B has skipped column 1 of the B-matrix and now spans columns two through three. The routing circuitry analyzes the pairwise window 651B on columns two through three of the B-matrix 641B to identify four non-zero elements and move them from locations (1,3), (3,3), (4,3), and (7,3) to nearby zero-valued slots, thereby generating a partially optimized B-matrix 641C.

[0070] Another optimization opportunity 654 exists in the optimized B-matrix 641C. In particular, column 3 of the optimized B-matrix 641C has no non-zero elements and can therefore be skipped to improve performance and reduce power consumption. Then, in cycle 2, as represented by snapshot 646, the pairwise compaction window 651C has skipped column 3 of the B-matrix and now spans columns four through five. The routing circuitry analyzes the pairwise window 651C on columns four through five of the B-matrix 641C to identify four non-zero elements and move them from locations (2,5), (4,5), (6,5), and (7,5) to nearby zero-valued slots, thereby generating a partially optimized B-matrix 641D.

[0071] Another optimization opportunity 656 exists in the optimized B-matrix 641D. In particular, column 5 of the optimized B-matrix 641D has no non-zero elements and can therefore be skipped to improve performance and reduce power consumption. In cycle 3, the pairwise compaction window 651D has then skipped column 5 of the B-matrix and now spans columns six through seven. The routing circuitry analyzes a pairwise window 651D on columns six through seven of the B-matrix 641D to identify five non-zero elements and move them from locations (0,7), (3,7), (4,7), (6,7), and (7,7) to nearby zero-valued slots, thereby generating a partially optimized B-matrix 641E.

[0072] Another optimization opportunity 658 exists in the optimized B-matrix 641E. In particular, column 7 of the optimized B-matrix 641E has no non-zero elements and can therefore be skipped to improve performance and reduce power consumption. Then, in cycle 4, as represented by snapshot 650, there are no columns of the optimized B-matrix 641E remaining to be processed.

[0073] Fig. 6D is a block diagram illustrating circuitry for supporting matrix compaction for a sparse matrix multiplication (SMM) instruction, according to some embodiments. Circuitry 660 may be used to support matrix compaction during a compaction operation, as described with reference to FIG. Fig.6A-B to allow for more optimization opportunities. In particular, circuit 660 allows an element of the B matrix to be shifted diagonally so that it aligns with the appropriate corresponding element of the A matrix for multiplication. As shown, circuit 660 includes multiplexer 662 to route one of three elements of the A matrix to a multiplier 664 to be used in implementing a sparse matrix multiplication (SMM) instruction. According to other embodiments not shown, optimization opportunities are further enhanced by allowing selection of a greater number of elements of the A matrix. For example, an 8-to-1 multiplexer may be used, allowing the B element to be shifted to one of eight available zero-valued slots. Multiplication-accumulation circuit arrangement

[0074] Fig. Figure 7 is a block diagram illustrating a multiply-accumulate (MAC) circuit for a sparse matrix multiply (SMM) instruction, according to some embodiments. As illustrated, the MAC circuit 700 includes eight multipliers 702-716 followed by a binary tree of seven two-input adders 718-730. According to some embodiments, the result from adder 730 is accumulated with a previous value of a target. According to other embodiments, wider adders and / or multipliers are used.

[0075] When used by the execution circuitry to implement a sparse matrix multiplication (SMM) instruction, for example as described with reference to Fig.1-6D, a multiplexer (not shown) at the input of each of the multipliers 702-716 selects an element from the A matrix that matches the corresponding element of the B matrix being multiplied according to the algorithm. Further, when used with the execution circuitry to implement sparse matrix multiplication (SMM), the output of the adder 730 is passed to accumulate with the appropriate output matrix element according to the algorithm.

[0076] Of course, it should be noted that when a sparse matrix is ​​folded or compacted, as for example with respect to the Fig.4-6D, the multiplexers 702-716 select from additional elements of the A matrix. Similarly, when a convolved element is used to generate a product, an adder other than adder 730 is required to accumulate this product with the appropriate element of the target matrix (also referred to as the C matrix). EXEMPLARY METHOD FOR ACCELERATION OF sparse matrix multiplication

[0077] Fig.8 is a block flow diagram of a processor executing a sparse matrix multiplication (SMM) instruction, according to some embodiments. As illustrated, flow 800 begins at 802, where the processor is to fetch, using fetch circuitry, the SMM instruction having fields for specifying the locations of the first, second, and output matrices, the specified second matrix being a sparse matrix, and the fetch circuitry is to further retrieve elements of the specified first and second matrices from their specified locations and store them in a register file. At 804, the processor is to decode the fetched SMM instruction using decode circuitry.At 806, the processor, using the execution circuitry, is to respond to the decoded SMM instruction by packing the specified second matrix stored in the register file by replacing one or more zero-valued elements with yet-to-be-processed non-zero elements, each replaced element having a field identifying its logical position within the second matrix, and further, using the execution circuitry, to: for each non-zero element at row M and column K of the specified first matrix, generating a product of the non-zero element and each corresponding non-zero element at row K and column N of the packed second matrix, and accumulating each generated product and a previous value of a corresponding element at row M and column N of the specified output matrix.According to some embodiments, at 808, the processor shall write back execution results and retire the SMM instruction. Operation 808 is optional, as indicated by its dashed borders, in that it may occur at a different time or not at all. Example format of a sparse matrix multiplication (SMM) instruction

[0078] Fig.9 is a format of a sparse matrix multiplication (SMM) instruction according to some embodiments. As illustrated, a sparse matrix multiplication (SMM) instruction 900 includes an opcode 902 (SMM*) and fields for specifying a destination matrix 904, a source matrix 906, and a source matrix 2 908. As used herein, source matrix 1, source matrix 2, and the destination matrix are sometimes referred to as the A, B, and C matrix, respectively. The SMM instruction 900 further includes optional fields for specifying the data format 910, such as integer, half-precision floating-point, single-precision floating-point, or double-precision floating-point, and the element size 912 in terms of the number of bits per matrix element. The SMM instruction 900 sometimes has fields for specifying M 914, N 916, and K 918, where the specified A, B, and C matrices have (M x K), (K x N), and (M x N) elements, respectively.As indicated by the dashed borders, the data format 910, the element size 912, M 914, N 916, and K 918 are optional, so they may be omitted and assume predetermined default values. According to some embodiments, one or more of the data format 910, the element size 912, M 914, N 916, and K 918 are specified as part of the opcode 902, such as a selected code for the opcode, a suffix, or a prefix. For example, the opcode 902 may include a suffix such as "B," "W," "D," or "Q" to specify an element size of eight, sixteen, thirty-two, or sixty-four bits, respectively. As illustrated, the opcode 902 includes an asterisk to indicate that it may optionally include additional prefixes or suffixes to specify instruction behavior. If the SMM command 900 does not specify any of the optional parameters, predefined default values ​​are applied as required.The format of the SMM command 900 is further described with reference to the . Fig. 10A-B and the Fig. 11A-D shown and described. COMMAND SEQUENCES

[0079] An instruction set may have one or more instruction formats. A given instruction format may define various fields (e.g., the number of bits, the location of the bits) to specify, among other things, the operation to be performed (e.g., opcode) and the one or more operands on which that operation is to be performed, and / or one or more other data fields (e.g., a mask). Some instruction formats are further broken down by the definition of instruction templates (or subformats). For example, the instruction templates of a given instruction format may be defined to have different subsets of the instruction format fields (the included fields are typically in the same order, but at least some have different bit positions because fewer fields are included) and / or may be defined to have a different interpreted given field.Accordingly, each instruction of an ISA is expressed using a given instruction format (and, if defined, in a given one of that instruction format's instruction templates) and includes fields for specifying the operation and operands. For example, an exemplary ADD instruction has a specific opcode and a specific instruction format, including an opcode field for specifying that opcode and operand fields for selecting operands (Source1 / Destination and Source2), and an occurrence of that ADD instruction in an instruction stream has specific contents in the operand fields that select specific operands. A set of SIMD extensions, referred to as Advanced Vector Extensions (AVX) (AVX1 and AVX2), using the Vector Extensions (VEX) coding scheme, has been released and / or published (see, for example, Intel). ® 64 and IA-32 Architectures Software Developer's Manual, September 2014 and Intel ®Advanced Vector Extensions Programming Reference, October 2014). EXAMPLE COMMAND FORMATS

[0080] Embodiments of the one or more instructions described herein may be implemented in various formats. Additionally, systems, architectures, and pipelines described below are detailed. Embodiments of the one or more instructions may be executed on such systems, architectures, and pipelines, but are not limited to those detailed. GENERIC VECTOR-FRIENDLY COMMAND FORMAT

[0081] A vector-friendly instruction format is an instruction format suitable for vector instructions (e.g., there are certain fields specific to vector operations). While embodiments are described in which both vector and scalar operations are supported by the vector-friendly instruction format, alternative embodiments use only vector operations of the vector-friendly instruction format.

[0082] The Fig. 10A-10B are block diagrams illustrating a generic vector-friendly instruction format and instruction templates thereof, according to some embodiments of the invention. Fig. 10A is a block diagram illustrating a generic vector-friendly instruction format and Class A instruction templates thereof, according to some embodiments of the invention, while Fig.10B is a block diagram illustrating the generic vector-friendly instruction format and Class B instruction templates thereof, according to some embodiments of the invention. In particular, a generic vector-friendly instruction format 1000 is illustrated, for which Class A and Class B instruction templates are defined, both including no-memory-access instruction templates 1005 and memory-access instruction templates 1020. The term generic in the context of the vector-friendly instruction format refers to the fact that the instruction format is not tied to a specific instruction set.

[0083] Although embodiments of the invention are described in which the vector-friendly instruction format supports: a 64-byte vector operand length (or size) with 32-bit (4-byte) or 64-bit (8-byte) data element widths (or sizes) (such that a 64-byte vector consists of either 16 double-word size elements or, alternatively, 8 quad-word size elements), a 64-byte vector operand length (or size) with 16-bit (2-byte) or 8-bit (1-byte) data element widths (or sizes), a 32-byte vector operand length (or size) with 32-bit (4-byte), 64-bit (8-byte), 16-bit (2-byte) or 8-bit (1-byte) data element widths (or sizes), and a 16-byte vector operand length (or size) with 32-bit (4-byte), 64-bit (8-byte), 16-bit (2-byte), or 8-bit (1-byte) data element widths (or sizes), alternative embodiments may include more, fewer, and / or different vector operand sizes (e.g., 256-byte vector operands) with more,support fewer or different data element widths (for example, 128-bit (16-byte) data element widths).

[0084] The Class A instruction templates in Fig. 10A include the following: 1) within the no-memory access instruction templates 1005, a no-memory access full rounding control-type operation instruction template 1010 and a no-memory access data transformation-type operation instruction template 1015 are illustrated, and 2) within the memory access instruction templates 1020, a temporal memory access instruction template 1025 and a non-temporal memory access instruction template 1030 are illustrated. The Class B instruction templates in Fig.10B include the following: 1) within the no memory access instruction templates 1005, a no memory access writemask control partial rounding control type operation instruction template 1012 and a no memory access writemask control V-size type operation instruction template 1017 are illustrated, and 2) within the memory access instruction templates 1020, a memory access writemask control instruction template 1027 is illustrated.

[0085] The generic vector-friendly instruction format 1000 has the following in the order shown in the Fig. 10A - 10B in the order shown.

[0086] Format field 1040 - A specific value (an instruction format identifier value) in this field uniquely identifies the vector-friendly instruction format and, accordingly, instances of instructions in the vector-friendly instruction format in instruction streams. This field is optional in the sense that it is not required for an instruction set that only has the generic vector-friendly instruction format.

[0087] Basic operation field 1042 - its content distinguishes various basic operations.

[0088] Register index field 1044 - its contents specify, directly or through address generation, the locations of the source and destination operands, whether in registers or in memory. These have a sufficient number of bits to select N registers from a PxQ (e.g., 32x512, 16x128, 32x1024, 64x1024) register file. While, according to one embodiment, N may consist of up to three source and one destination registers, alternative embodiments may support more or fewer source and destination registers (e.g., they may support up to two sources, one of which also acts as a destination, they may support up to three sources, one of which also acts as a destination, and they may support up to two sources and one destination).

[0089] Modifier field 1046 - its content distinguishes instances of instructions in the generic vector instruction format that specify memory access from those that do not, i.e., between no-memory-access instruction templates 1005 and memory-access instruction templates 1020. Memory access operations read and / or write to the memory hierarchy (in some cases specifying the source and / or destination addresses using values ​​in registers), whereas non-memory access operations do not (e.g., the source and destinations are registers). While this field also selects between three different ways of performing memory access calculations according to one embodiment, alternative embodiments may support more, fewer, or different ways of performing memory access calculations.

[0090] Extension operation field 1050 - its content determines which of a variety of different operations should be performed in addition to the base operation. This field is context-specific. According to some embodiments, this field is divided into a class field 1068, an alpha field 1052, and a beta field 1054. The extension operation field 1050 allows common groups of operations to be performed in a single instruction instead of two, three, or four instructions.

[0091] Scaling field 1060 - its content allows scaling the contents of the index field for memory access generation (for example, for address generation that is 2 Skalierung * Index + base used).

[0092] Offset field 1062A - its content is used as part of the memory address generation (for example, for an address generation that uses 2 Skalierung * Index + Base + Offset used).

[0093] Offset factor field 1062B (note that the juxtaposition of offset field 1062A directly above offset factor field 1062B indicates that one or the other is used) - its content is used as part of the address generation: it specifies an offset factor to be scaled with the size of a memory access (N) - where N is the number of bytes in the memory access (for example, for an address generation that uses 2 Skalierung* Index + base + scaled offset used). Redundant low-order bits are ignored, so the contents of the offset factor field are multiplied by the total memory operand size (N) to produce the final offset to be used in calculating an effective address. The value of N is determined by the processor hardware at runtime based on the full opcode field 1074 (described below) and the data manipulation field 1054C. The offset field 1062A and the offset factor field 1062B are optional in the sense that they are not used for the no-memory-access instruction templates 1005 and / or various embodiments may implement only one or neither.

[0094] Data element width field 1064 - its content determines which of a number of data element widths to use (according to some embodiments for all instructions, according to other embodiments for only some of the instructions). This field is optional in the sense that it is not required if only one data element width is supported and / or data element widths are supported using some aspect of the opcodes.

[0095] Writemask field 1070 - Its content controls, on a per-data-element position basis, whether that data-element position in the destination vector operand reflects the result of the base operation and the extension operation. Class A instruction templates support merge writemasking, while Class B instruction templates support both merge and zero-out writemasking. When merged, vector masks allow a set of elements at the destination to be protected from updates during the execution of an operation (specified by the base operation and the extension operation), and according to another embodiment, the old value of each element of the destination is preserved, with the corresponding mask bit having a 0.Conversely, if zeroing vector masks allow any set of elements at the destination to be zeroed during the execution of an operation (specified by the base operation and the extend operation), according to one embodiment, an element of the destination is zeroed if the corresponding mask bit has a 0 value. A subset of this functionality is the ability to control the vector length of the operation being performed (i.e., the span of modified elements from the first to the last), but it is not required that the elements being modified be consecutive. Accordingly, the write mask field 1070 partially enables vector operations, including loads, stores, arithmetic, logical, etc.Although embodiments of the invention are described in which the contents of the writemask field 1070 select one of a number of writemask registers that includes the writemask to be used (and accordingly, the contents of the writemask field 1070 indirectly identify that the masking is to be performed), alternative embodiments instead or in addition allow the contents of the writemask field 1070 to directly specify the masking to be performed.

[0096] Immediate field 1072 – its content allows the specification of an immediate value. This field is optional in the sense that it is not present in any implementation of the generic vector-friendly format that does not support immediate, and it is not present in instructions that do not use immediate.

[0097] Class field 1068 - its content distinguishes between different classes of commands. With reference to the Fig. 10A-B, it should be noted that the contents of this field select between Class A and Class B commands. Fig. 10A-B, squares with rounded corners are used to indicate that a specific value is present in a field (for example, Class A 1068A or Class B 1068B for the class field 1068 in the Fig. 10A-B). CLASS A COMMAND TEMPLATES

[0098] In the case of class A non-memory access instruction templates 1005, the alpha field 1052 is interpreted as an RS field 1052A, whose content determines which of the various extension operation types is to be performed (for example, rounding 1052A.1 and data transformation 1052A.2 are used for the no-memory access rounding-type operation instruction template 1010 and the no-memory access data transformation-type operation instruction template 1015, respectively), while the beta field 1054 determines which of the operations of the specified type is to be performed. For the no-memory access instruction templates 1005, the scaling field 1060, the offset field 1062A, and the offset scaling field 1062B are not present. NO-MEMORY-ACCESS INSTRUCTION TEMPLATES - FULL-ROUND CONTROL TYPE OPERATION

[0099] In the no-memory-access full-rounding control-type operation instruction template 1010, the beta field 1054 is interpreted as a rounding control field 1054A, whose contents provide static rounding. Although, according to the described embodiments of the invention, the rounding control field 1054A includes a suppression of all floating-point exceptions (SAE) field 1056 and a rounding operation control field 1058, alternative embodiments may encode these two concepts in the same field or include only one or the other of these concepts / fields (e.g., include only the rounding operation control field 1058).

[0100] SAE field 1056 - its content decides whether to disable exception event reporting or not, whereby if the content of SAE field 1056 indicates that suppression is enabled, a given instruction does not specify any type of floating-point exception flag and does not invoke a floating-point exception handler.

[0101] Rounding operation control field 1058 - its contents determine which of a group of rounding operations (e.g., round up, round down, round to zero, and round to nearest neighbor) should be performed. Accordingly, the rounding operation control field 1058 allows the rounding mode to be changed on a per-instruction basis. According to some embodiments where a processor includes a control register for specifying rounding modes, the contents of the rounding operation control field 1050 override this register value. NO-MEMORY-ACCESS INSTRUCTION TEMPLATES - DATA TRANSFORMATION TYPE OPERATION

[0102] In the no-memory-access data transformation type operation instruction template 1015, the beta field 1054 is interpreted as a data transformation field 1054B, the contents of which decide which of a number of data transformations is to be performed (e.g., no data transformation, swizzle, broadcast).

[0103] In the case of a memory access instruction template 1020 of class A, the alpha field 1052 is interpreted as an eviction hint field 1052B, the content of which decides which of the eviction hints is to be used (in Fig. 10A, temporal 1052B.1 and non-temporal 1052B.2 are specified for the memory access temporal instruction template 1025 and the memory access non-temporal instruction template 1030, respectively), while the beta field 1054 is interpreted as a data manipulation field 1054C, the contents of which determine which of a number of data manipulation operations (also known as primitives) is to be performed (e.g., no manipulation, broadcast, upconversion of a source, and downconversion of a destination). The memory access instruction templates 1020 include the scaling field 1060 and, optionally, the offset field 1062A or the offset scaling field 1062B.

[0104] Vector memory instructions perform vector loads from memory and vector stores to memory with conversion support. Like regular vector instructions, vector memory instructions transfer data to or from memory one data element at a time, with the elements actually transferred being dictated by the contents of the vector mask selected as the write mask. MEMORY ACCESS COMMAND TEMPLATES - TEMPORAL

[0105] Temporal data is data that is likely to be reused soon enough to benefit from caching. However, this is a hint, and different processors can implement it in different ways, including ignoring the hint entirely. MEMORY ACCESS COMMAND TEMPLATES - NON-TEMPORAL

[0106] Non-temporal data is data that isn't likely to be reused soon enough to benefit from caching in the 1st-level cache and should be given priority for eviction. However, this is a hint, and different processors can implement it in different ways, including ignoring the hint entirely. CLASS B COMMAND TEMPLATES

[0107] In the case of Class B instruction templates, the alpha field 1052 is interpreted as a write mask control (Z) field 1052C, the content of which decides whether the write mask controlled by the write mask field 1070 should be merging or zeroing.

[0108] In the case of the non-memory access instruction templates 1005 of class B, a part of the beta field 1054 is interpreted as an RL field 1057A, the contents of which decide which of the various extension operation types are to be performed (for example, rounding 1057A.1 and vector length (V-size) 1057A.2 are specified for the no-memory access writemask control partial rounding control type operation instruction template 1012 and the no-memory access writemask control V-size type operation instruction template 1017, respectively), while the remainder of the beta field 1054 decides which of the operations of the specified type is to be performed. In the no-memory-access instruction templates 1005, the scale field 1060, the offset field 1062A, and the offset scale field 1062B are not present.

[0109] In the no memory access writemask control partial rounding control type operation instruction template 1010, the remainder of the beta field 1054 is interpreted as the rounding operation field 1059A and exception event reporting is disabled (a given instruction does not report any type of floating-point exception flag and does not invoke a floating-point exception handler).

[0110] Rounding Operation Control Field 1059A - Like Rounding Operation Control Field 1058, its contents determine which of a group of rounding operations (e.g., round up, round down, round to zero, and round to nearest neighbor) should be performed. Accordingly, Rounding Operation Control Field 1059A allows the rounding mode to be changed on a per-instruction basis. According to some embodiments where a processor includes a control register for specifying rounding modes, the contents of Rounding Operation Control Field 1050 override this register value.

[0111] In the no memory access write mask control V-size type operation instruction template 1017, the remainder of the beta field 1054 is interpreted as a vector length field 1059B, the contents of which decide which of a number of data vector lengths is to be processed (e.g., 128, 256, or 512 bytes).

[0112] In the case of class B memory access instruction template 1020, a portion of beta field 1054 is interpreted as broadcast field 1057B, the contents of which determine whether to perform the broadcast-type data manipulation operation, while the remainder of beta field 1054 is interpreted as vector length field 1059B. Memory access instruction templates 1020 include scale field 1060 and optionally offset field 1062A or offset scale field 1062B.

[0113] With reference to the generic vector-friendly instruction format 1000, a full opcode field 1074 is shown, which includes the format field 1040, the base operation field 1042, and the data element width field 1064. Although an embodiment is shown where the full opcode field 1074 includes all of these fields, the full opcode field 1074 includes fewer than all of these fields according to embodiments that do not support all of them. The full opcode field 1074 provides the operation code (opcode).

[0114] The extension operation field 1050, the data element width field 1064, and the write mask field 1070 enable the specification of these features on a per-instruction basis in the generic vector-friendly instruction format.

[0115] The combination of the write mask field and the data element width field creates typed instructions in that they allow the mask to be applied based on different data element widths.

[0116] The different instruction templates found in Class A and Class B are advantageous in different situations. According to some embodiments of the invention, different processors or different cores within a processor may support only Class A, only Class B, or both classes. For example, a high-performance, general-purpose out-of-order core intended for general computing may support only Class B, a core intended primarily for graphics and / or scientific computing (throughput) may support only Class A, and a core intended for both may support both (of course, a core having a mix of templates and instructions from both classes, but not all templates and instructions from both classes, is within the scope of the invention).Also, a single processor may have multiple cores, all supporting the same class, or where different cores support different classes. For example, in a processor with separate graphics and general-purpose cores, one of the graphics cores, primarily dedicated to graphics and / or scientific computing, may support only Class A, while one or more of the general-purpose cores may be high-performance general-purpose cores with out-of-order execution and register renaming, dedicated to general-purpose computing that supports only Class B. Another processor, not having a separate graphics core, may have one or more in-order or out-of-order general-purpose cores that support both Class A and Class B.Of course, according to various embodiments of the invention, features from one class may also be implemented in the other class. Programs written in a high-level language would be compiled into a variety of different executable forms (e.g., just-in-time compiled or statically compiled), including: 1) a form containing only instructions from the classes supported by the target processor for execution, or 2) a form with alternative routines written using various combinations of the instructions from all classes, and with control flow code that selects the routines to be executed based on the instructions supported by the processor currently executing the code. EXAMPLE SPECIFIC VECTOR-FRIENDLY COMMAND FORMAT

[0117] Fig.11A is a block diagram illustrating an exemplary specific vector-friendly instruction format, according to some embodiments of the invention. Fig. 11A shows a specific vector-friendly instruction format 1100 that is specific in the sense that it specifies the location, size, interpretation, and order of the fields, as well as values ​​for some of these fields. The specific vector-friendly instruction format 1100 can be used to extend the x86 instruction set so that some of the fields resemble or are the same as those used in the existing x86 instruction set and an extension thereof (e.g., AVX). This format remains consistent with the prefix encoding field, the real opcode byte field, the MOD R / M field, the SIB field, the offset field, and the immediate fields of the existing x86 instruction set with extensions. These are the fields of Fig. 10, in which the fields from Fig. 11A can be seen.

[0118] It should be noted that, although embodiments of the invention are described for the purposes of explanation with reference to the specific vector-friendly instruction format 1100 in conjunction with the generic vector-friendly instruction format 1000, the invention is not limited to the specific vector-friendly instruction format 1100, except where claimed. For example, the generic vector-friendly instruction format 1000 contemplates a variety of possible sizes for the various fields, while the specific vector-friendly instruction format 1100 is shown to have fields of specific sizes. In a specific example, although the data element width field 1064 in the specific vector-friendly instruction format 1100 is illustrated as a one-bit field, the invention is not so limited (i.e., the generic vector-friendly instruction format 1000 contemplates other sizes of the data element width field 1064).

[0119] The generic vector-friendly instruction format 1000 has the following in the Fig. 11A listed in the order shown.

[0120] EVEX prefix (bytes 0 - 3) 1102 - is encoded in a four-byte form.

[0121] Format field 1040 (EVEX byte 0, bits [7:0]) - the first byte (EVEX byte 0) is the format field 1040 and contains 0x62 (the unique value used to distinguish the vector-friendly instruction format, according to some embodiments).

[0122] Bytes two through four (EVEX bytes 1-3) contain a number of bit fields that provide a specific capability.

[0123] REX field 1105 (EVEX byte 1, bits [7 - 5]) - consists of an EVEX.R bit field (EVEX byte 1, bit [7] - R), an EVEX.X bit field (EVEX byte 1, bit [6] - X), and an EVEX.B bit field (EVEX byte 1, bit [5] - B). The EVEX.R, EVEX.X, and EVEX.B bit fields provide the same functionality as the corresponding VEX bit fields and are encoded using the 1s complement form, i.e., ZMM0 is encoded as 1111B, ZMM15 is encoded as 0000B. Other fields of the instructions encode the lower three bits of the register indices, as known in the art (rrr, xxx and bbb), so that Rrrr, Xxxx and Bbbb can be formed by adding EVEX.R, EVEX.X and EVEX.B.

[0124] REX' 1110A - This is the first part of the REX' field 1110 and is the EVEX.R' bit field (EVEX byte 1, bit [4] - R'), which is used to encode either the upper 16 or the lower 16 of the extended 32-register set. According to some embodiments, this bit, along with others as noted below, is stored in bit-inverted format to distinguish it from the BOUND instruction (in the well-known x86 32-bit mode), whose real opcode byte is 62, but which does not accept the value 11 in the MOD field in the MOD R / M field (described below). Alternative embodiments of the invention do not store this and the other bits noted below in inverted format. A value of 1 is used to encode the lower 16 registers. In other words, R'Rrrr is formed by combining EVEX.R', EVEX.R and the other RRR from other fields.

[0125] Opcode mapping field 1115 (EVEX byte 1, bits [3:0] - mmmm) - its content encodes an implied leading opcode byte (0F, 0F 38 or 0F 3).

[0126] Data element width field 1064 (EVEX byte 2, bit [7] - W) - is represented by the notation EVEX.W. EVEX.W is used to define the granularity (size) of the data type (either 32-bit data elements or 64-bit data elements).

[0127] EVEX.vvvv 1120 (EVEX byte 2, bits [6:3]-vvvv) - The role of EVEX.vvvv can be: 1) EVEX.vvvv encodes the first source register operand specified in inverted form (1s complement) and valid for instructions with two or more source operands, 2) EVEX.vvvv encodes the destination register operand specified in 1s complement for certain vector shifts, or 3) EVEX.vvvv encodes no operand; the field is reserved and should contain 1111b. Accordingly, the EVEX.wvv field 1120 encodes the four low-order bits of the first source register specifier stored in inverted form (1s complement). Depending on the instruction, an additional different EVEX bit field is used to extend the specifier size to 32 registers.

[0128] EVEX.U-1068 class field (EVEX byte 2, bit [2]-U) - if EVEX.U = 0, it indicates class A or EVEX.U0; if EVEX.U = 1, it indicates class B or EVEX.U1.

[0129] Prefix encoding field 1125 (EVEX byte 2, bits [1:0]-pp) – provides additional bits for the base operation field. In addition to providing support for legacy SSE instructions in EVEX prefix format, this also has the advantage of compacting the SIMD prefix (instead of requiring a single byte to express the SIMD prefix, the EVEX prefix requires only 2 bits). According to one embodiment, to support legacy SSE instructions that use a SIMD prefix (66H, F2H, F3H) in both legacy and EVEX prefix formats, these legacy SIMD prefixes are encoded into the SIMD prefix encoding field and expanded to the legacy SIMD prefix at runtime before being provided to the decoder PLA (so that the PLA can execute both the legacy and EVEX formats of these legacy instructions without modification).Although newer instructions might use the contents of the EVEX prefix encoding field directly as an opcode extension, certain embodiments similarly extend for consistency but allow different meanings to be specified by these legacy SIMD prefixes. An alternative embodiment may refactor the PLA to support the 2-bit SIMD prefix encodings, thus eliminating the need for the extension.

[0130] Alpha field 1052 (EVEX byte 3, bit [7] - EH; also known as EVEX.EH, EVEX.rs, EVEX.RL, EVEX.WriteMaskControl and EVEX.N, also denoted by α) - as previously described, this field is context-specific.

[0131] Beta field 1054 (EVEX byte 3, bits [6:4]-SSS, also known as EVEX.s 2-0 , EVEX.r 2-0 , EVEX.rr1, EVEX.LL0, EVEX.LLB; also denoted by βββ) - as previously described, this field is context-specific.

[0132] REX' 1110B - This is the remainder of the REX' field 1110 and is the EVEX.V' bit field (EVEX byte 3, bit [3] - V'), which can be used to encode either the upper 16 or the lower 16 of the extended 32-register set. This bit is stored in a bit-inverted format. A value of 1 is used to encode the lower 16 registers. In other words, V'VVVV is formed by combining EVEX.V', EVEX.vvvv.

[0133] Write mask field 1070 (EVEX byte 3, bits [2:0]-kkk) - its content specifies the index of a register in the write mask registers, as previously described. According to some embodiments, the specific value EVEX.kkk = 000 has a special behavior that implies that no write mask is used for the particular instruction (this can be implemented in a variety of ways, including using a write mask hardwired with all ones or hardware that bypasses the masking hardware).

[0134] Real opcode field 1130 (byte 4) is also known as the opcode byte. A portion of the opcode is specified in this field.

[0135] The MOD R / M field 1140 (byte 5) includes the MOD field 1142, the Reg field 1144, and the R / M field 1146. As previously described, the contents of the MOD field 1142 distinguish between memory access and non-memory access operations. The role of the Reg field 1144 can be summarized in terms of two situations: encoding either the destination register operand or a source register operand, or treating it as an opcode extension and not being used to encode an instruction operand. The role of the R / M field 1146 can include encoding the instruction operand that references a memory access, or encoding either the destination register operand or a source register operand.

[0136] Scale, Index, Base (SIB) Byte (Byte 6) - As previously described, the contents of the scale field 1050 are used to generate memory addresses. SIB.xxx 1154 and SIB.bbb 1156 - The contents of these fields were previously mentioned in relation to the register indices Xxxx and Bbbb.

[0137] Offset field 1062A (bytes 7 - 10) - if the MOD field 1142 contains 10, bytes 7 - 10 are the offset field 1062A, and it acts in the same way as the legacy 32-bit offset (disp32), and it acts with byte granularity.

[0138] Offset factor field 1062B (byte 7) - If the MOD field 1142 contains 01, byte 7 is the offset factor field 1062B. The location of this field is similar to the 8-bit offset (disp8) of the legacy x86 instruction set, and it operates at byte granularity. Because disp8 is sign-extended, it can only address offsets between -128 and 127 bytes. Regarding 64-byte cache lines, disp8 uses 8 bits, which can be set to only four truly usable values: -128, -64, 0, and 64. However, because a larger range is often required, disp32 is used, which requires 4 bytes. Unlike disp8 and disp32, the offset factor field 1062B is a reinterpretation of disp8, where, when the offset factor field 1062B is used, the actual offset is determined by the contents of the offset factor field multiplied by the size of the memory operand access (N). This offset type is referred to as disp8*N.This reduces the average instruction length (a single byte is used for the offset, but with a much larger range). This compressed offset relies on the assumption that the effective offset is several times the granularity of the memory access, so the redundant low-order bits of the address offset do not need to be encoded. In other words, the 1062B offset factor field replaces the legacy x86 instruction set's 8-bit offset. Accordingly, the 1062B offset factor field is encoded in the same way as an 8-bit x86 instruction set offset (so no changes in the ModRM / SIB encoding rules occur), with the only exception that disp8 is overloaded to disp8*N.In other words, there are no changes in the encoding rules or encoding lengths, only in the hardware's interpretation of the offset value (which must scale the offset with the size of the memory operand to obtain a byte-wise address offset). The immediate field 1072 functions as previously described. COMPLETE OPCODE FIELD

[0139] Fig. 11B is a block diagram illustrating the fields of the specific vector-friendly instruction format 1100 that comprise the full opcode field 1074, according to some embodiments. Specifically, the full opcode field 1074 includes the format field 1040, the base operation field 1042, and the data element width (W) field 1064. The base operation field 1042 includes the prefix encoding field 1125, the opcode mapping field 1115, and the real opcode field 1130. REGISTER INDEX FIELD

[0140] Fig.11C is a block diagram illustrating the fields of the specific vector-friendly instruction format 1100 that comprise the register index field 1044, according to some embodiments. Specifically, the register index field 1044 includes the REX field 1105, the REX' field 1110, the MODR / M.reg field 1144, the MODR / Mr / m field 1146, the VVVV field 1120, the xxx field 1154, and the bbb field 1156. EXPANSION FIELD OF OPERATIONS

[0141] Fig.11D is a block diagram showing the fields of the specific vector-friendly instruction format 1100 that make up the extension operation field 1050, according to some embodiments. If the class field (U) 1068 contains 0, it means EVEX.U0 (Class A 1068A); if it contains 1, it means EVEX.U1 (Class B 1068B). If U = 0 and the MOD field 1142 contains 11 (signifying a no memory access operation), the alpha field 1052 (EVEX byte 3, bit [7] - EH) is interpreted as the rs field 1052A. If the rs field 1052A contains a 1 (rounding 1052A.1), the beta field 1054 (EVEX byte 3, bits [6:4]-SSS) is interpreted as the rounding control field 1054A. The rounding control field 1054A includes a one-bit SAE field 1056 and a two-bit rounding operation field 1058. If the rs field 1052A contains a 0 (data transformation 1052A.2), the beta field 1054 (EVEX byte 3, bits [6:4]-SSS) is interpreted as a three-bit data transformation field 1054B.If U = 0 and the MOD field 1142 contains 00, 01, or 10 (signifying a memory access operation), the alpha field 1052 (EVEX byte 3, bit [7] - EH) is interpreted as the eviction notice (EH) field 1052B, and the beta field 1054 (EVEX byte 3, bits [6:4] - SSS) is interpreted as a three-bit data manipulation field 1054C.

[0142] If U = 1, the alpha field 1052 (EVEX byte 3, bit [7] - EH) is interpreted as the write mask control (Z) field 1052C. If U = 1 and the MOD field 1142 contains 11 (signifying a no memory access operation), a portion of the beta field 1054 (EVEX byte 3, bit [4] - S0) is interpreted as the RL field 1057A, and if it contains a 1 (rounding 1057A.1), the remainder of the beta field 1054 (EVEX byte 3, bit [6-5] - S 2-1 ) is interpreted as the rounding operation field 1059A, while if the RL field 1057A contains a 0 (V size 1052.A2), the remainder of the beta field 1054 (EVEX byte 3, bit [6-5]- S 2-1) as vector length field 1059B (EVEX byte 3, bit [6-5]-L 1-0 ) is interpreted. If U = 1 and the MOD field 1142 contains 00, 01, or 10 (which means a memory access operation), the beta field 1054 (EVEX byte 3, bit [6:4]- SSS) is interpreted as the vector length field 1059B (EVEX byte 3, bit [6-5]- L 1-0 ) and the broadcast field 1057B (EVEX byte 3, bit [4]- B) are interpreted. EXEMPLARY REGISTER ARCHITECTURE

[0143] Fig.12 is a block diagram of a register architecture 1200 according to some embodiments. According to the illustrated embodiment, there are 32 vector registers 1210 that are 512 bits wide, with these registers referred to as zmm0 through zmm31. The low-order 256 bits of the lower 16 zmm registers are overlaid on registers ymm0-16. The low-order 128 bits of the lower 16 zmm registers (the low-order 128 bits of the ymm registers) are overlaid on registers xmm0-15. The specific vector-friendly instruction format 1100 operates on this overlaid register file as shown in the tables below. Adjustable vector length Class Operations register Instruction templates that do not have the vector length field 1059B A ( Fig. 10A; U = 0) 1010, 1015, 1025, 1030 zmm register (the vector length is 64 bytes) B (Figure 10B; U = 1) 1012 zmm register (the vector length is 64 bytes) Instruction templates that have the vector length field 1059B B (Figure 10B; U = 1) 1017, 1027 zmm, ymm or xmm register (the vector length is 64 bytes, 32 bytes or 16 bytes), depending on the vector length field 1059B

[0144] In other words, the vector length field 1059B selects between a maximum length and one or more other lesser lengths, where each such lesser length is half the previous length, and instruction templates without the vector length field 1059B operate on the maximum vector length. Furthermore, the Class B instruction templates of the specific vector-friendly instruction format 1100 operate on packed or scalar single- or double-precision floating-point data and packed or scalar integer data. Scalar operations are operations performed on the lowest-order data element position in a zmm / ymm / xmm register, with the higher-order data element positions either left as they were before the instruction or set to zero, depending on the embodiment.

[0145] Write Mask Register 1215 - According to the illustrated embodiment, there are 8 write mask registers (k0 through k7), each 64 bits in size. According to an alternative embodiment, the write mask registers 1215 are 16 bits in size. As previously described, according to some embodiments, the vector mask register k0 cannot be used as a write mask; when the encoding that normally specifies k0 is used for a write mask, it selects a hard-wired write mask of 0xffff, effectively disabling write masking for that instruction.

[0146] General-Purpose Registers 1225 - According to the illustrated embodiment, there are sixteen 64-bit general-purpose registers used in conjunction with existing x86 addressing modes to address memory operands. These registers are labeled RAX, RBX, RCX, RDX, RBP, RSI, RDI, RSP, and R8 through R15.

[0147] Scalar floating-point stack register file (x87 stack) 1245, on which the packed integer flat MMX register file 1250 is surrogately accessed - according to the illustrated embodiment, the x87 stack is an eight-element stack used to perform scalar floating-point operations on 32 / 64 / 80-bit floating-point data using the x87 instruction set extension, while the MMX registers are used to perform operations on packed integer 64-bit data, as well as to hold operands for some operations performed between the MMX and XMM registers.

[0148] Alternative embodiments may use wider or narrower registers. Additionally, alternative embodiments may use more, fewer, or different register files and registers. EXAMPLE CORE ARCHITECTURES, PROCESSORS AND COMPUTER ARCHITECTURES

[0149] Processor cores can be implemented in different ways, for different purposes, and in different processors. For example, implementations of such cores may include: 1) an in-order general-purpose core dedicated to general-purpose computations; 2) a high-performance out-of-order general-purpose core dedicated to general-purpose computations; 3) a special-purpose core dedicated primarily to graphics and / or scientific computations (throughput).Implementations of various processors may include: 1) a CPU having one or more in-order general-purpose cores dedicated to general-purpose computations and / or one or more out-of-order general-purpose cores dedicated to general-purpose computations, and 2) a coprocessor having one or more special-purpose cores dedicated primarily to graphics and / or scientific purposes (throughput).Such different processors result in different computer system architectures, which may include: 1) the coprocessor on a separate chip from the CPU, 2) the coprocessor on a separate die in the same assembly as a CPU, 3) the coprocessor on the same die as a CPU (in which case such a coprocessor is sometimes referred to as special-purpose logic such as integrated graphics and / or scientific logic (throughput) or special-purpose cores), and 4) a system-on-a-chip, which may include, on the same die, the described CPU (sometimes referred to as application core(s) or application processor(s)), the coprocessor described above, and additional functionality. Example core architectures are described next, followed by descriptions of example processors and computer architectures. EXAMPLE CORE ARCHITECTURES IN-SERIES AND OUT-OF-SERIES CORE BLOCK DIAGRAM

[0150] Fig. 13A is a block diagram illustrating both an example in-order pipeline and an example register-renaming out-of-order issue / execution pipeline, according to some embodiments of the invention. Fig. 13B is a block diagram illustrating both an exemplary embodiment of an in-order architecture core and an exemplary register-renaming out-of-order issue / execution architecture core to be incorporated into a processor according to embodiments of the invention. The solid-line boxes in the Fig.Figures 13A-B show the in-order pipeline and the in-order core, while the optional addition of the dashed boxes shows the register-renaming out-of-order issue / execution pipeline and the register-renaming out-of-order issue / execution core. Assuming that the in-order aspect is a subset of the out-of-order aspect, the out-of-order aspect is described.

[0151] In Fig. 13A, a processor pipeline 1300 includes a fetch stage 1302, a length decode stage 1304, a decode stage 1306, an allocation stage 1308, a rename stage 1310, a scheduling stage 1312 (also referred to as a dispatch or issue stage), a register read / memory read stage 1314, an execution stage 1316, a write-back / memory write stage 1318, an exception handling stage 1322, and a commit stage 1324.

[0152] Fig.13B shows a processor core 1390 with a preprocessing unit 1330 coupled to an execution engine unit 1350, both coupled to a memory unit 1370. The core 1390 may be a reduced instruction set computing (RISC) core, a complex instruction set computing (CISC) core, a very long instruction word (VLIW) core, or a hybrid or alternative core type. As another option, the core 1390 may be a special-purpose core such as a network or communications core, a compression engine, a coprocessor core, a general-purpose graphics processing unit (GPGPU) core, a graphics core, or the like.

[0153] Preprocessing unit 1330 includes a branch prediction unit 1332 coupled to an instruction cache unit 1334, which is coupled to an instruction translation lookaside buffer (TLB) 1336, which is coupled to an instruction fetch unit 1338, which is coupled to a decode unit 1340. The decode unit 1340 (or decoder) may decode instructions and generate as an output one or more micro-operations, microcode entry points, microinstructions, other instructions, or other control signals that are decoded from, or otherwise reflect, or are derived from the original instructions. The decode unit 1340 may be implemented using several different mechanisms. Examples of suitable mechanisms include, but are not limited to, lookup tables, hardware implementations, programmable logic arrays (PLAs), microcode read-only memory (ROM), etc.According to one embodiment, core 1390 includes a microcode ROM or other medium that stores microcode for specific macroinstructions (e.g., in decode unit 1340 or elsewhere in preprocessing unit 1330). Decode unit 1340 is coupled to a rename / mapping unit 1352 in execution engine unit 1350.

[0154] The execution engine unit 1350 includes the rename / allocation unit 1352, which is coupled to a retirement unit 1354 and a set of one or more scheduler units 1356. The scheduler unit(s) 1356 represent(s) any number of different schedulers, including reservation stations, a central instruction window, etc. The scheduler unit(s) 1356 is(are) coupled to the physical register file unit(s) 1358.Each of the physical register file units 1358 represents one or more physical register files, different ones of which store one or more different data types, such as a scalar integer type, a scalar floating-point type, a packed integer type, a packed floating-point type, a vector integer type, a vector floating-point type, status (e.g., an instruction pointer, which is the address of the next instruction to be executed), etc. According to one embodiment, the physical register file unit 1358 includes a vector register unit, a write mask register unit, and a scalar register unit. These register units can provide architectural vector registers, vector mask registers, and general-purpose registers.The physical register file unit(s) 1358 is (are) overlapped by the retirement unit 1354 to illustrate various ways in which register renaming and out-of-order execution may be implemented (e.g., using one or more reorder buffers and one or more retirement register files; using one or more future files, one or more history buffers and one or more retirement register files; using a register card and a pool of registers, etc.). The retirement unit 1354 and the one or more physical register file units 1358 are coupled to the one or more execution clusters 1360. The one or more execution clusters 1360 include a set of one or more execution units 1362 and a set of one or more memory access units 1364.Execution units 1362 may perform various operations (e.g., shifts, addition, subtraction, multiplication) on various data types (e.g., a scalar floating-point type, a packed integer type, a packed floating-point type, a vector integer type, a vector floating-point type). While some embodiments may include a number of execution units dedicated to specific functions or sets of functions, other embodiments may include only one execution unit or multiple execution units that all perform all functions.The one or more scheduler units 1356, the one or more physical register file units 1358, and the one or more execution clusters 1360 are depicted as potentially multiple units because certain embodiments create separate pipelines for certain types of data / operations (e.g., a scalar integer pipeline, a scalar floating point / packed integer / packed floating point / vector integer / vector floating point pipeline, and / or a memory access pipeline, each having its own scheduler unit, its own physical register file unit, and / or its own execution cluster, and in the case of a separate memory access pipeline, certain embodiments are implemented where only the execution cluster of that pipeline includes the one or more memory access units 1364).It should also be noted that where separate pipelines are used, one or more of these pipelines may have out-of-order issue / execution and the rest may have in-order issue / execution.

[0155] The set of memory access units 1364 is coupled to the memory unit 1370, which includes a data TLB unit 1372 coupled to a data cache unit 1374, which is coupled to a Level 2 (L2) cache unit 1376. According to an example embodiment, the memory access units 1364 may include a load unit, a store address unit, and a store data unit, each coupled to the data TLB unit 1372 in the memory unit 1370. The instruction cache unit 1334 is further coupled to a Level 2 (L2) cache unit 1376 in the memory unit 1370. The L2 cache unit 1376 is coupled to one or more other cache levels and ultimately to main memory.

[0156] For example, the example register rename out-of-order issue / execute core architecture may implement pipeline 1300 as follows: 1) Instruction fetch 1338 executes fetch and length decode stages 1302 and 1304; 2) decode unit 1340 executes decode stage 1306; 3) rename / allocate unit 1352 executes allocate stage 1308 and rename stage 1310; 4) one or more scheduler units 1356 execute scheduling stage 1312; 5) one or more physical register file units 1358 and memory unit 1370 execute register read / memory read stage 1314, execution cluster 1360 executes execute stage 1316; 6) the memory unit 1370 and the one or more physical register file units 1358 execute the write-back / memory write stage 1318; 7) various units may participate in the execution handling stage 1322;and 8) the retirement unit 1354 and the one or more physical register file units 1358 execute the commit stage 1324.;

[0157] The core 1390 may support one or more instruction sets (e.g., the x86 instruction set (with some extensions added with newer versions); the MIPS instruction set from MIPS Technologies of Sunnyvale, CA; the ARM instruction set (with optional additional extensions such as NEON) from ARM Holdings of Sunnyvale, CA), including the instruction(s) described therein. According to one embodiment, the core 1390 includes logic to support a packet data instruction set extension (e.g., AVX1, AVX2), enabling the operations used by many multimedia applications to be performed using packet data.

[0158] It should be noted that the core may support multithreading (execution of two or more parallel instruction sets or threads) and may do so in a variety of ways, including time-slot multithreading, concurrent multithreading (where a single physical core provides a logical core for each of the threads that the physical core is concurrently multithreading), or a combination thereof (for example, time-slot fetching and decoding followed by concurrent multithreading as in Intel's Hyperthreading technology). ® ).

[0159] Although register renaming is described in the context of out-of-order execution, it should be understood that register renaming can also be used in an in-order architecture. While the illustrated embodiment of the processor also includes separate instruction and data cache units 1334 / 1374 and a split L2 cache unit 1376, alternative embodiments may include a single internal cache for both instructions and data, such as an internal Level 1 (L1) cache, or multiple levels of internal cache. According to some embodiments, the system may include a combination of an internal cache and an external cache external to the core and / or the processor. Alternatively, the entire cache may be external to the core and / or the processor. SPECIFIC EXEMPLARY IN-SERIES CORE ARCHITECTURE

[0160] The Fig.14A-B show a block diagram of a more specific example in-line core architecture, where the core would be one of several logic blocks (including other cores of the same type and / or different types) within a chip. The logic blocks communicate via a high-bandwidth interconnect network (e.g., a ring network) with fixed-function logic, memory I / O interfaces, and other required I / O logic, depending on the application.

[0161] Fig.14A is a block diagram of a single processor core along with its connection to on-chip interconnect network 1402 and its local subset of Level 2 (L2) cache 1404, according to some embodiments of the invention. According to one embodiment, an instruction decoder 1400 supports the x86 instruction set with a packet data instruction set extension. An L1 cache 1406 enables low-latency accesses to the cache memory in scalar and vector units. Although according to one embodiment (for design simplicity), a scalar unit 1408 and a vector unit 1410 use separate register sets (scalar registers 1412 and 1414, respectively),vector registers 1414) and data transferred between them are written to memory and then read back from a Level 1 (L1) cache 1406, alternative embodiments of the invention may use a different approach (e.g., using a single register set or including a communication path that allows data to be transferred between the two register files without being written and read back).

[0162] The local subset of the L2 cache 1404 is part of a global L2 cache, which is partitioned into separate local subsets, one per processor core. Each processor core has a direct access path to its own local subset of the L2 cache 1404. Data read by a processor core is stored in its L2 cache subset 1404 and can be quickly accessed in parallel with other processor cores accessing their own local L2 cache subsets. Data written by a processor core is stored in its own L2 cache subset 1404 and fetched from other subsets as needed. The ring network ensures coherence for shared data. The ring network is bidirectional to allow agents, such as processor cores, L2 caches, and other logic blocks, to communicate with each other within the chip. Each ring data path is 1012 bits wide per direction.

[0163] Fig.14B is an expanded view of a portion of the processor core of Fig. 14A according to some embodiments of the invention. Fig. Figure 14B includes an L1 data cache 1406A portion of L1 cache 1404, as well as further details regarding vector unit 1410 and vector registers 1414. Specifically, vector unit 1410 is a 16-wide vector processing unit (VPU) (see 16-wide ALU 1428) that executes one or more of integer instructions, single-precision floating-point instructions, and double-precision floating-point instructions. The VPU supports scrambling of register inputs with scrambling unit 1420, numeric conversion with numeric conversion units 1422A-B, and replication with replication unit 1424 at the memory input. Write mask registers 1426 enable prediction of resulting vector writes.

[0164] Fig.Figure 15 is a block diagram of a processor 1500 that may have more than one core, an integrated memory controller, and integrated graphics, according to some embodiments of the invention. The boxes shown in solid lines in Fig. 15 illustrate a processor 1500 having a single core 1502A, a system agent 1510, a set of one or more bus control units 1516, while the optional addition of the dashed-line boxes illustrates an alternative processor 1500 having multiple cores 1502A-N, a set of one or more integrated memory control units 1514 in the system agent unit 1510, and special-purpose logic 1508.

[0165] Accordingly, various implementations of processor 1500 may include: 1) a CPU with special-purpose logic 1508, which is integrated graphics and / or scientific (throughput) logic (which may have one or more cores), where cores 1502A-N are one or more general-purpose cores (e.g., general-purpose in-line cores, general-purpose out-of-line cores, a combination of the two); 2) a coprocessor, where cores 1502A-N are a large number of special-purpose cores primarily dedicated to graphics and / or scientific (throughput) purposes; and 3) a coprocessor, where cores 1502A-N are a large number of general-purpose in-line cores.Accordingly, processor 1500 may be a general-purpose processor, a coprocessor, or a special-purpose processor, such as a network or communications processor, a compression engine, a graphics processor, a GPGPU (general-purpose graphics processing unit), a multi-core high-throughput (MIC) coprocessor (with 30 or more cores), an embedded processor, or the like. The processor may be implemented on one or more chips. Processor 1500 may be part of one or more substrates and / or implemented on one or more substrates using any number of process technologies, such as BiCMOS, CMOS, or NMOS.

[0166] The memory hierarchy includes one or more cache levels within the cores, a set of one or more shared cache units 1506, and external memory (not shown) coupled to the set of integrated memory controllers 1514. The set of shared cache units 1506 may include one or more mid-level caches such as Level 2 (L2), Level 3 (L3), Level 4 (L4), or other cache levels, a last-level cache (LLC), and / or combinations thereof.Although, according to one embodiment, a ring-based interconnect unit 1512 connects the integrated graphics logic 1508 (the integrated graphics logic 1508 is an example of special-purpose logic and is referred to as such herein), the set of shared cache units 1506, and the system agent unit 1510 / the one or more integrated memory controller units 1514, alternative embodiments may use any number of well-known techniques for interconnecting such units. According to one embodiment, coherency is maintained between one or more cache units 1506 and cores 1502A-N.

[0167] According to some embodiments, one or more of cores 1502A-N are capable of multithreading. System agent 1510 includes those components that coordinate and operate cores 1502A-N. System agent unit 1510 may, for example, include a power control unit (PCU) and a display unit. The PCU may include or consist of logic and components required to regulate the power state of cores 1502A-N and integrated graphics logic 1508. The display unit is used to drive one or more externally connected displays.

[0168] The cores 1502A-N may be homogeneous or heterogeneous with respect to the architectural instruction set, such that two or more of the cores 1502A-N may be capable of executing the same instruction set, while others may be capable of executing only a subset of that instruction set or a different instruction set. EXEMPLARY COMPUTER ARCHITECTURES

[0169] The Fig. 16-19 are block diagrams of example computer architectures. Other system designs and configurations known in the art for laptops, desktops, handheld PCs, personal digital assistants, engineering workstations, servers, network devices, network hubs, switches, embedded processors, digital signal processors (DSPs), graphics devices, video game devices, set-top boxes, microcontrollers, cellular phones, portable media players, handheld devices, and various other electronic devices are also suitable. In general, a wide variety of systems or electronic devices capable of incorporating a processor and / or other execution logic as disclosed herein are generally suitable.

[0170] Fig.Figure 16 shows a block diagram of a system 1600 according to an embodiment of the present invention. System 1600 may include one or more processors 1610, 1615 coupled to a controller hub 1620. According to one embodiment, the controller hub 1620 includes a graphics memory controller hub (GMCH) 1690 and an input / output hub (IOH) 1650 (which may be on separate chips), the GMCH 1690 includes memory and graphics controllers to which a memory 1640 and a coprocessor 1645 are coupled, and the IOH 1650 couples input / output (I / O) devices 1660 to the GMCH 1690. Alternatively, one or both of the memory and graphics controllers are integrated into the processor (as described herein), and the memory 1640 and the coprocessor 1645 are coupled directly to the processor 1610 and the controller hub 1620 in a single chip with the IOH 1650.

[0171] The optional nature of additional 1615 processors is in Fig. 16 is shown with dashed lines. Each processor 1610, 1615 may include one or more of the processor cores described herein and may be a version of processor 1500.

[0172] Memory 1640 may be, for example, dynamic random access memory (DRAM), phase-change memory (PCM), or a combination of the two. For at least one embodiment, controller hub 1620 communicates with the one or more processors 1610, 1615 via a multi-drop bus such as a front-side bus (FSB), a point-to-point interface such as a QuickPath interconnect (QPI), or a similar connection 1695.

[0173] According to one embodiment, coprocessor 1645 is a special-purpose processor, such as a high-throughput MIC processor, a network or communications processor, a compression engine, a graphics processor, a GPGPU, an embedded processor, or the like. According to one embodiment, controller hub 1620 may include an integrated graphics accelerator.

[0174] There may be a variety of differences between the physical resources 1610, 1615 with respect to a spectrum of performance metrics, including architectural, microarchitectural, thermal, power consumption characteristics, and the like.

[0175] According to one embodiment, processor 1610 executes instructions that control data processing operations of a general type. Coprocessor instructions may be embedded within the instructions. Processor 1610 recognizes these coprocessor instructions as belonging to a type that should be executed by the attached coprocessor 1645. Accordingly, processor 1610 issues these coprocessor instructions (or control signals representing coprocessor instructions) to coprocessor 1645 on a coprocessor bus or other interconnect. The one or more coprocessors 1645 accept and execute the received coprocessor instructions.

[0176] Fig. 17 shows a block diagram of a first, more specific example system 1700 according to an embodiment of the present invention. As shown in Fig.17, multiprocessor system 1700 is a point-to-point interconnect system and includes a first processor 1770 and a second processor 1780 coupled via a point-to-point interconnect 1750. Each of processors 1770 and 1780 may be any version of processor 1500. According to one embodiment of the invention, processors 1770 and 1780 are processors 1610 and 1615, respectively, while coprocessor 1738 is coprocessor 1645. According to another embodiment, processors 1770 and 1780 are processor 1610 and coprocessor 1645, respectively.

[0177] Processors 1770 and 1780 are shown as having integrated memory controller (IMC) units 1772 and 1782, respectively. Processor 1770 also includes point-to-point (PP) interfaces 1776 and 1778 as part of its bus control units; similarly, the second processor 1780 includes PP interfaces 1786 and 1788. Processors 1770, 1780 can exchange information using PP interface circuits 1778, 1788 via a point-to-point (PP) interface 1750. As shown in Fig. 17, the IMCs 1772 and 1782 couple the processors to respective memories, namely a memory 1732 and a memory 1734, which may be portions of main memory locally attached to the respective processors.

[0178] Processors 1770, 1780 may each exchange information with a chipset 1790 via individual PP interfaces 1752, 1754 using point-to-point interface circuits 1776, 1794, 1786, 1798. Chipset 1790 may optionally exchange information with coprocessor 1738 via a high-performance interface 1792. According to one embodiment, coprocessor 1738 is a special-purpose processor, such as a high-throughput MIC processor, a network or communications processor, a compression engine, a graphics processor, a GPGPU, an embedded processor, or the like.

[0179] A shared cache (not shown) may be either included in one of the processors or may be external to the two processors but connected to the processors via a PP interconnect so that local cache information of one or both processors may be stored in the shared cache if one processor is placed in a low-power mode.

[0180] The chipset 1790 may be coupled to a first bus 1716 via an interface 1796. According to one embodiment, the first bus 1716 may be a peripheral component interconnect (PCI) bus or a bus such as a PCI Express bus or other third-generation I / O interconnect bus, although the scope of the present invention is not so limited.

[0181] As in Fig.17, various I / O devices 1714 may be coupled to the first bus 1716 along with a bus bridge 1718 coupling the first bus 1716 to a second bus 1720. According to one embodiment, one or more additional processors 1715, such as coprocessors, high-throughput MIC processors, GPGPUs, accelerators (such as graphics accelerators or digital signal processing (DSP) units, field-programmable gate arrays, or other processors, are coupled to the first bus 1716. According to one embodiment, the second bus 1720 may be a low pin count (LPC) bus.Various devices may be coupled to a second bus 1720, including, for example, a keyboard and / or mouse 1722, communication devices 1727, and a storage unit 1728 such as a disk drive or other mass storage device, which may include instructions / code and data 1730, according to one embodiment. Furthermore, an audio I / O 1724 may be coupled to the second bus 1720. It should be noted that other architectures are also possible. For example, instead of the point-to-point architecture of FIG. Fig. 17 a system may implement a multi-drop bus or other such architecture.

[0182] Fig. 18 shows a block diagram of a second, more specific example system 1800 according to an embodiment of the present invention. Like elements in the Fig. 17 and Fig. 18 have the same reference numbers, and certain aspects of Fig.17 were from Fig. 18 has been omitted to avoid other aspects of Fig. 18 become unclear.

[0183] Fig. 18 shows that processors 1770, 1780 may include integrated memory and I / O control logic ("CL") 1772, 1782, respectively. Accordingly, CL 1772, 1782 include integrated memory controllers and I / O control logic. Fig. Figure 18 shows that not only are the memories 1732, 1734 coupled to the CL 1772, 1782, but also that I / O devices 1814 are coupled to the control logic 1772, 1782. Legacy I / O devices 1815 are coupled to the chipset 1790.

[0184] Fig. 19 shows a block diagram of an SoC 1900 according to an embodiment of the present invention. Similar elements in Fig. 15 have the same reference numbers. Boxes marked with dashed lines are optional features on more advanced SoCs. Fig.19, one or more interconnect units 1902 are coupled to an application processor 1910 having a set of one or more cores 1502A-N with cache units 1504A-N and one or more shared cache units 1506, a system agent unit 1510, one or more bus control units 1516, one or more integrated memory control units 1514, a set of one or more coprocessors 1920, which may include integrated graphics logic, an image processor, an audio processor, and a video processor, a static random access memory (SRAM) unit 1930, a direct memory access (DMA) unit 1932, and a display unit 1940 for coupling the one or more external displays.According to one embodiment, the one or more coprocessors 1920 comprise a special-purpose processor such as, for example, a network or communications processor, a compression engine, a GPGPU, a high-throughput MIC processor, an embedded processor, or the like.

[0185] Embodiments of the mechanisms disclosed herein may be implemented in hardware, software, firmware, or a combination of such implementation approaches. Embodiments of the invention may be implemented as computer programs or program code executing on programmable systems comprising at least one processor, a memory system (including volatile and non-volatile memory and / or storage elements), at least one input device, and at least one output device.

[0186] Program code in the style of Fig.The code 1730 shown in Figure 17 can be used to input instructions for performing the functions described herein and for generating output information. The output information can be applied to one or more output devices in a known manner. For the purposes of this application, a processing system includes any system that includes a processor, such as a digital signal processor (DSP), a microcontroller, an application-specific integrated circuit (ASIC), or a microprocessor.

[0187] The program code can be implemented in a high-level procedural or object-oriented programming language to communicate with a processing system. The program code can also be implemented in assembly or machine language, if desired. Indeed, the mechanisms described here are not limited in scope to any particular programming language. In any case, the language can be a compiled or an interpreted language.

[0188] One or more aspects of at least one embodiment may be implemented by representative instructions stored on a machine-readable medium representing various logic within the processor that, when read by a machine, cause the machine to form logic for performing the techniques described herein. These representations, known as "IP cores," may be stored on a physical machine-readable medium and provided to various customers or manufacturing facilities for loading into the manufacturing machines that actually form the logic or processor.

[0189] Such machine-readable storage media may include, without limitation, non-transitory, physical assemblies of articles manufactured or formed by a machine or device, including storage media such as hard disks, other types of disks including floppy disks, optical disks, compact disk read-only memory (CD-ROM), rewritable compact disks (CD-RW), and magneto-optical disks, semiconductor devices such as read-only memory (ROM), random access memory (RAM) such as dynamic random access memory (DRAM), static random access memory (SRAM), erasable programmable read-only memory (EPROM), flash memory, electrically erasable programmable read-only memory (EEPROM), phase-change memory (PCM), magnetic or optical cards, or any other type of medium suitable for storing electronic instructions.

[0190] Accordingly, embodiments of the invention also include non-transitory, physical machine-readable media containing instructions or design data, such as Hardware Description Language (HDL), that define structures, circuits, devices, processors, and / or system features described herein. These embodiments may also be referred to as program products. EMULATION (INCLUDING BINARY TRANSLATION, CODE MORPHING, ETC.)

[0191] In some cases, an instruction converter may be used to convert an instruction from a source instruction set to a target instruction set. For example, the instruction converter may translate (e.g., using static binary translation, dynamic binary translation, including dynamic compilation), morph, emulate, or otherwise convert an instruction to one or more other instructions to be processed by the core. The instruction converter may be implemented in software, hardware, firmware, or a combination thereof. The instruction converter may be on a processor, off a processor, or partially on and off a processor.

[0192] Fig.Figure 20 is a block diagram illustrating the use of a software instruction converter to convert binary instructions in a source instruction set to binary instructions in a target instruction set, according to some embodiments of the invention. According to the illustrated embodiment, the instruction converter is a software instruction converter, although the instruction converter may alternatively be implemented in software, firmware, hardware, or various combinations thereof. Fig.20 shows a program in a high-level language 2002 that can be compiled using an x86 compiler 2004 to produce an x86 binary code 2006 that can be natively executed by a processor having at least one core 2016 with an x86 instruction set.The at least one-core processor 2016 with an x86 instruction set represents a processor capable of performing substantially the same functions as an Intel processor with at least one core with an x86 instruction set by (1) compatibly executing a substantial portion of the instruction set of the Intel core with an x86 instruction set, or (2) compatibly executing or otherwise processing object code versions of applications or other software intended to run on an Intel processor with at least one core with an x86 instruction set to achieve substantially the same result as an Intel processor with at least one core with an x86 instruction set. The x86 compiler 2004 represents a compiler capable of producing x86 binary code 2006 (e.g., object code) to be executed on the at least one-core processor 2016 with an x86 instruction set, with or without additional link processing.Similarly, . Fig.20, the high-level language 2002 program, which can be compiled using an alternative instruction set compiler 2008 to produce an alternative instruction set binary 2010 that can be natively executed by a processor having at least one core 2014 with an x86 instruction set (e.g., a processor having cores executing the MIPS instruction set from MIPS Technologies of Sunnyvale, CA and / or the ARM instruction set from ARM Holdings of Sunnyvale, CA). The instruction converter 2012 is used to convert the x86 binary 2006 into code that can be natively executed by the processor without a core 2014 with an x86 instruction set.This converted code is likely not the same as the alternative instruction set binary code 2010, because an instruction converter capable of doing so is difficult to create. However, the converted code provides the general operation and may consist of instructions from the alternative instruction set. Accordingly, the instruction converter 2012 represents software, firmware, hardware, or a combination thereof that enables a processor or other electronic device that does not have a processor or core with an x86 instruction set to execute the x86 binary code 2006 through emulation, simulation, or other process. MORE EXAMPLES

[0193] Example 1 provides an exemplary processor for executing a sparse matrix multiplication (SMM) instruction, comprising: fetch circuitry for fetching the SMM instruction having fields for specifying locations of a first, a second, and an output matrix, the specified second matrix being a sparse matrix, the fetch circuitry further configured to retrieve elements of the specified first and second matrix from a memory and store them in a register file; decode circuitry for decoding the fetched SMM instruction; execution circuitry configured, in response to the decoded SMM instruction, to pack the second matrix stored in the register file by replacing one or more zero-valued elements with non-zero elements yet to be processed;wherein each of the replaced elements further comprises a field for identifying its logical position within the second matrix, and the execution circuitry is further configured to generate, for each non-zero element at row M and column K of the specified first matrix, a product of the non-zero element and each corresponding non-zero element at row K and column N of the packed second matrix and to accumulate each generated product with a previous value of a corresponding element at row M and column N of the specified output matrix.

[0194] Example 2 includes the subject matter of the example processor of Example 1, wherein the execution circuitry is configured to, for each row K of the specified second matrix, determine whether the row contains zero-valued elements, and if so, determine whether the row contains non-zero elements that remain to be processed based on the zero-valued element, and if so, for each zero-valued element with a non-zero element still to be processed, fold the non-zero element into the zero-valued element.

[0195] Example 3 includes the subject matter of the example processor of Example 1, wherein the execution circuitry is configured to, for each column N of the specified second matrix, determine whether the column contains zero-valued elements, and if so, determine whether any of the P elements of a subsequent column is a non-zero value, and for each zero-valued element with a non-zero element in the subsequent column, push the non-zero element into the zero-valued element.

[0196] Example 4 includes the subject matter of the example processor of Example 1, wherein each of the elements stored in the register file includes a field for specifying whether it has a zero value, and wherein the execution circuitry is configured to use the field when determining whether the element has a zero value.

[0197] Example 5 includes the subject matter of the example processor of Example 1, wherein the execution circuitry is configured to avoid generating any products of zero-valued elements.

[0198] Example 6 includes the subject matter of the example processor of Example 1, wherein the execution circuitry is configured to have a processing array of (X x Y) processing units, where X is less than M and Y is less than N, and wherein the execution circuitry is configured to use the processing array iteratively over multiple clock cycles to perform the same processing as an actual physical array of (M x N) processing units.

[0199] Example 7 includes the subject matter of the example processor of Example 1, wherein the execution circuitry is configured to have a processing array of (X x Y) processing units, where X is less than M and Y is less than N, wherein the execution circuitry is configured to cascade multiple instances of the processing array to perform the same processing as an actual physical array of (M x N) processing units.

[0200] Example 8 provides an exemplary method for executing a sparse matrix multiplication (SMM) instruction, comprising: fetching the SMM instruction having fields for specifying a first, a second, and an output matrix using fetch circuitry, wherein the specified second matrix is ​​a sparse matrix, the fetch circuitry further configured to retrieve elements of the specified first and second matrix from a memory and store them in a register file, decoding the fetched SMM instruction using the decoding circuitry, responding, by the execution circuitry, to the decoded SMM instruction by packing the specified second matrix stored in the register file by replacing one or more zero-valued elements with non-zero elements yet to be processed,wherein each replaced element in the packed second matrix has a field for identifying its logical position within the second matrix, and further using the execution circuitry: for each non-zero element at row M and column K of the specified first matrix, generating a product of the non-zero element and each corresponding non-zero element at row K and column N of the packed second matrix and accumulating each generated product and a previous value of a corresponding element at row M and column N of the specified output matrix.

[0201] Example 9 includes the subject matter of the exemplary method of Example 8, wherein the execution circuitry is configured to, for each row K of the specified second matrix, determine whether the row contains zero-valued elements, and if so, determine whether the row contains non-zero elements that remain to be processed based on the zero-valued element, and if so, for each zero-valued element with a non-zero element still to be processed, fold the non-zero element into the zero-valued element.

[0202] Example 10 includes the subject matter of the exemplary method of Example 8, wherein the execution circuitry is configured to, for each column N of the specified second matrix, determine whether the column contains zero-valued elements, and if so, determine whether any of the P elements of a subsequent column is a non-zero value, and for each zero-valued element with a non-zero element in the subsequent column, push the non-zero element into the zero-valued element.

[0203] Example 11 includes the subject matter of the exemplary method of Example 8, wherein each of the elements stored in the register file includes a field for specifying whether it has a zero value, and wherein the execution circuitry is configured to use the field when determining whether the element has a zero value.

[0204] Example 12 includes the subject matter of the exemplary method of Example 8, wherein the execution circuitry is configured to avoid generating any products of elements with zero values.

[0205] Example 13 includes the subject matter of the exemplary method of Example 8, wherein the execution circuitry is configured to have a processing array of (X x Y) processing units, where X is less than M and Y is less than N, and wherein the execution circuitry is configured to use the processing array iteratively over multiple clock cycles to perform the same processing as an actual physical array of (M x N) processing units.

[0206] Example 14 includes the subject matter of the exemplary method of Example 8, wherein the execution circuitry is configured to have a processing array of (X x Y) processing units, where X is less than M and Y is less than N, wherein the execution circuitry is configured to cascade multiple instances of the processing array to perform the same processing as an actual physical array of (M x N) processing units.

[0207] Example 15 provides an exemplary system for executing a sparse matrix multiplication (SMM) instruction, comprising: a memory, fetch circuitry for fetching the SMM instruction having fields for specifying locations of the first, second, and output matrices, wherein the specified second matrix is ​​a sparse matrix, the fetch circuitry further configured to retrieve elements of the specified first and second matrices from the memory and store them in a register file, decode circuitry for decoding the fetched SMM instruction, execution circuitry configured, in response to the decoded SMM instruction, to pack the second matrix stored in the register file by replacing one or more zero-valued elements with non-zero elements yet to be processed,wherein each of the replaced elements further comprises a field for identifying its logical position within the second matrix, and the execution circuitry is further configured to generate, for each non-zero element at row M and column K of the specified first matrix, a product of the non-zero element and each corresponding non-zero element at row K and column N of the packed second matrix and to accumulate each generated product with a previous value of a corresponding element at row M and column N of the specified output matrix.

[0208] Example 16 includes the subject matter of the exemplary system of Example 15, wherein the execution circuitry is configured to, for each row K of the specified second matrix, determine whether the row contains zero-valued elements, and if so, determine whether the row contains non-zero elements that remain to be processed based on the zero-valued element, and if so, for each zero-valued element with a non-zero element still to be processed, fold the non-zero element into the zero-valued element.

[0209] Example 17 includes the subject matter of the exemplary system of Example 15, wherein the execution circuitry is configured to, for each column N of the specified second matrix, determine whether the column contains zero-valued elements, and if so, determine whether any of the P elements of a subsequent column is a non-zero value, and for each zero-valued element with a non-zero element in the subsequent column, push the non-zero element into the zero-valued element.

[0210] Example 18 includes the subject matter of the example system of Example 15, wherein each of the elements stored in the register file includes a field for specifying whether it has a zero value, and wherein the execution circuitry is configured to use the field when determining whether the element has a zero value.

[0211] Example 19 includes the subject matter of the exemplary system of Example 15, wherein the execution circuitry is configured to avoid generating any products of elements with zero values.

[0212] Example 20 includes the subject matter of the exemplary method of System 15, wherein the execution circuitry is configured to have a processing array of (X x Y) processing units, where X is less than M and Y is less than N, and wherein the execution circuitry is configured to use the processing array iteratively over multiple clock cycles to perform the same processing as an actual physical array of (M x N) processing units.

Claims

[1] Device comprising: a plurality of registers for storing a first matrix and a second matrix; and a first circuit coupled to the plurality of registers, the first circuit executing a matrix multiplication and accumulation instruction to: receive a packed matrix representing a sparse matrix, where fields identify logical positions of elements in the sparse matrix that are involved in packing the sparse matrix; perform matrix multiplications on corresponding elements in corresponding rows and corresponding columns of the packed matrix and the first matrix to produce corresponding products; and to accumulate the products with corresponding elements in corresponding rows and corresponding columns of the second matrix, wherein the apparatus further comprises a second circuit coupled to the first circuit, the second circuit generating the packed matrix by packing the sparse matrix, the second circuit analyzing a window in a row of the sparse matrix to generate the packed matrix and shifting non-zero elements in the window into positions of zero-valued elements in the window. [2] The apparatus of claim 1, wherein the matrix multiplication and accumulation instruction identifies a set of matrix dimensions from a plurality of possible sets of matrix dimensions. [3] The apparatus of claim 1 or 2, wherein the matrix multiplication and accumulation instruction identifies a data format of the first matrix as one of at least integer, half-precision floating point, and single-precision floating point. [4] The device of any one of claims 1 to 3, wherein the packed matrix has half as many columns as the sparse matrix. [5] Apparatus according to any one of claims 1 to 4, wherein the first circuit serves to decompose the first matrix into submatrices before performing the matrix multiplications. [6] The device according to any one of claims 1 to 5, wherein a size of the window is wider than two columns of the sparse matrix. [7] The apparatus of any one of claims 1 to 6, further comprising an instruction converter for converting the matrix multiplication and accumulation instruction into one or more instructions of another instruction set executable by the first circuit. [8] The device according to any one of claims 1 to 7, wherein in the packed matrix at least some non-zero elements of the sparse matrix are shifted into positions of zero-valued elements of the sparse matrix. [9] Apparatus according to any one of claims 1 to 8, wherein the fields identify logical positions of a subset of all elements of the same type in the sparse matrix that participate in packing the sparse matrix. [10] Apparatus according to any one of claims 1 to 9, wherein the respective elements with which the matrix multiplications are performed correspond to the fields. [11] Computer system comprising: a system memory; and a device coupled to the system memory, the device comprising: a plurality of registers for storing a first matrix and a second matrix; and a first circuit coupled to the plurality of registers, the first circuit executing a matrix multiplication and accumulation instruction to: receive a packed matrix representing a sparse matrix, where fields identify logical positions of elements in the sparse matrix that are involved in packing the sparse matrix; perform matrix multiplications on corresponding elements in corresponding rows and corresponding columns of the packed matrix and the first matrix to produce corresponding products; and to accumulate the products with corresponding elements in corresponding rows and corresponding columns of the second matrix, wherein the apparatus further comprises a second circuit coupled to the first circuit, the second circuit generating the packed matrix by packing the sparse matrix, the second circuit analyzing a window in a row of the sparse matrix to generate the packed matrix and shifting non-zero elements in the window into positions of zero-valued elements in the window. [12] The computer system of claim 11, wherein the matrix multiplication and accumulation instruction identifies a set of matrix dimensions from a plurality of possible sets of matrix dimensions, wherein the matrix multiplication and accumulation instruction identifies a data format of the first matrix as one of at least integer, half-precision floating point, and single-precision floating point, and wherein the computer system further comprises a mass storage device coupled to the processor. [13] The computer system of claim 11 or 12, wherein the packed matrix has half as many columns as the sparse matrix, and wherein the computer system further comprises a communication device coupled to the processor. [14] A computer system according to any one of claims 11 to 13, wherein the first circuit is for decomposing the first matrix into smaller submatrices prior to performing the matrix multiplications, and wherein the computer system further comprises a coprocessor coupled to the processor. [15] The computer system of any one of claims 11 to 14, wherein the packed matrix has at least some non-zero elements of the sparse matrix shifted into positions of zero-valued elements of the sparse matrix, and wherein the computer system further comprises a communication device coupled to the processor. [16] A computer system according to any one of claims 11 to 15, wherein the fields identify logical positions of a subset of all elements of the same type in the sparse matrix that participate in packing the sparse matrix, the corresponding elements on which the matrix multiplications are performed corresponding to the fields, and wherein the computer system further comprises a coprocessor coupled to the processor. [17] Method comprising: Storing a first matrix and a second matrix in a plurality of registers; and Execute a matrix multiplication and accumulation instruction, including: Receiving a packed matrix representing a sparse matrix, wherein fields identify logical positions of elements in the sparse matrix that participate in packing the sparse matrix; performing matrix multiplications on corresponding elements in corresponding rows and corresponding columns of the packed matrix and the first matrix to produce corresponding products; and Accumulating the products with corresponding elements in corresponding rows and corresponding columns of the second matrix, wherein the method further comprises generating the packed matrix by packing the sparse matrix, wherein generating the packed matrix comprises analyzing a window in a row of the sparse matrix and shifting non-zero elements in the window into positions of zero-valued elements in the window. [18] The method of claim 17, wherein the matrix multiplication and accumulation instruction identifies a set of matrix dimensions from a plurality of possible sets of matrix dimensions, and wherein the matrix multiplication and accumulation instruction identifies a data format of the first matrix as one of at least integer, half-precision floating point, and single-precision floating point. [19] A method according to claim 17 or 18, wherein the packed matrix has half as many columns as the sparse matrix. [20] Method according to one of claims 17 to 19, wherein the first matrix is ​​decomposed into smaller submatrices before performing the matrix multiplications. [21] A method according to any one of claims 17 to 20, wherein the fields identify logical positions of a subset of all elements of the same type in the sparse matrix that participate in packing the sparse matrix, the corresponding elements on which the matrix multiplications are performed corresponding to the fields. [22] One or more non-transitory machine-readable storage media storing instructions which, when executed by one or more machines, cause the one or more machines to perform the method of any one of claims 17 to 21.

Images