Accelerator for sparse-dense matrix multiplication

Abstract

The disclosed embodiments relate to an accelerator for sparse-dense matrix instructions. In one example, a processor to execute a sparse-dense matrix multiplication instruction includes fetch circuitry to fetch a sparse-dense matrix multiplication instruction having fields to specify an opcode, a dense output matrix, a dense source matrix, and a sparse source matrix having a sparseness of non-zero elements, the sparseness is less than one; the decoding circuit is used for decoding the extracted sparse-dense matrix multiplication instruction; execution circuitry to execute the decoded sparse-dense matrix multiplication instruction to, for each non-zero element at rows M and columns K of the specified sparse source matrix: generate a product of the non-zero element with each corresponding dense element at rows K and columns N of the specified dense source matrix; and generating a cumulative sum of each generated product and previous values of corresponding output elements at rows M and columns N of the specified dense output matrix.

Description

This application is a divisional application of the invention patent application with application number 201910227563.7, application date March 25, 2019, priority date March 28, 2018, and titled “Accelerator for Sparse-Dense Matrix Multiplication”. Technical Field

[0001] The present disclosure relates to the field of processing logic, microprocessors, and associated instruction set architectures, and more particularly to accelerators for sparse-dense matrix instructions. Background Art

[0002] Deep learning is a class of machine learning algorithms. Deep learning architectures such as deep neural networks have been applied to fields including computer vision, speech recognition, natural language processing, audio recognition, social network filtering, machine translation, bioinformatics, and drug design.

[0003] Both tools used for deep learning, inference and training, tend to favor low-precision arithmetic. Maximizing the throughput of deep learning algorithms and computations can help meet the demands of deep learning processors, such as those that perform deep learning in data centers.

[0004] Sparse-dense matrix multiplication (SDMM) operations are useful in deep learning contexts. However, traditional CPU and GPU instruction set architectures require symmetric inputs with the same density, which limits the ability to gain performance benefits by exploiting the sparsity of sparse input matrices. BRIEF DESCRIPTION OF THE DRAWINGS

[0005] The present invention is illustrated by way of example and not by way of limitation in the accompanying figures, in which like reference numerals indicate similar elements, and in which:

[0006] Figure 1 is a block diagram illustrating processing components for executing a sparse-dense matrix multiplication (SDMM) instruction, such as a virtual neural network instruction (SDMVNNI), according to some embodiments;

[0007] Figure 2A is a block diagram illustrating data flow for processing a sparse-dense matrix multiply (SDMM) instruction according to some embodiments;

[0008] Figure 2B is a block diagram illustrating data flow for processing a sparse-dense matrix multiply (SDMM) instruction according to some embodiments;

[0009] Figure 3 is a block diagram illustrating execution circuitry for processing a sparse-dense matrix multiplication (SDMM) instruction according to some embodiments;

[0010] Figure 4 is a block diagram illustrating execution circuitry for processing a sparse-dense matrix multiplication (SDMM) instruction according to some embodiments;

[0011] Figure 5 is pseudo code illustrating execution circuitry for processing a sparse-dense matrix multiplication (SDMM) instruction according to some embodiments;

[0012] Figure 6 is a process flow diagram illustrating execution of a sparse-dense matrix multiplication (SDMM) instruction by a processor according to some embodiments;

[0013] Figure 7 is a block diagram illustrating the format of a sparse-dense matrix multiply (SDMM) instruction according to some embodiments;

[0014] Figures 8A-8B is a block diagram illustrating a generic vector friendly instruction format and instruction templates thereof according to some embodiments;

[0015] Fig. 8A is a block diagram illustrating a generic vector friendly instruction format and its class A instruction templates according to some embodiments;

[0016] Figure 8B is a block diagram illustrating a generic vector friendly instruction format and its class B instruction templates according to some embodiments;

[0017] Fig. 9A is a block diagram illustrating an exemplary specific vector friendly instruction format according to some embodiments;

[0018] Fig. 9B is a block diagram illustrating the fields that make up the complete opcode field in a specific vector friendly instruction format according to one embodiment of the present invention;

[0019] Fig. 9C is a block diagram illustrating fields constituting a register index field in a specific vector friendly instruction format according to one embodiment of the present invention;

[0020] Fig.9D is a block diagram illustrating fields constituting an extended operation field in a specific vector friendly instruction format according to one embodiment of the present invention;

[0021] Fig.10 is a block diagram of a register architecture according to one embodiment of the present invention;

[0022] Fig.11A is a block diagram illustrating both an exemplary in-order pipeline and an exemplary register renaming out-of-order issue / execution pipeline according to some embodiments;

[0023] Fig. 11Bis a block diagram illustrating both an exemplary embodiment of an in-order architecture core and an exemplary register renaming, out-of-order issue / execution architecture core to be included in a processor in accordance with some embodiments;

[0024] Figures 12A-12B A block diagram illustrating a more specific exemplary in-order core architecture, which would be one of several logic blocks in a chip (including other cores of the same type and / or different types);

[0025] Fig. 12A is a block diagram of a single processor core and its connection to an on-die interconnect network and its local subset of a level 2 (L2) cache in accordance with some embodiments;

[0026] Fig. 12B According to some embodiments Fig. 12A An expanded view of a portion of a processor core in;

[0027] Fig.13 is a block diagram of a processor that may have more than one core, may have an integrated memory controller, and may have integrated graphics according to some embodiments;

[0028] Figure 14-17 is a block diagram of an exemplary computer architecture;

[0029] Fig.14 A block diagram showing a system according to an embodiment of the present invention;

[0030] Fig.15 is a block diagram of a first more specific exemplary system according to an embodiment of the present invention;

[0031] Fig.16 is a block diagram of a second more specific exemplary system according to an embodiment of the present invention;

[0032] Fig.17 is a block diagram of a system on a chip (SoC) according to an embodiment of the present invention; and

[0033] Fig.18 is a block diagram comparing the use of a software instruction converter to convert binary instructions in a source instruction set to binary instructions in a target instruction set, according to some embodiments. DETAILED DESCRIPTION

[0034] In the following description, numerous specific details are set forth. However, it should be understood that some embodiments may be implemented without these specific details. In other instances, well-known circuits, structures, and techniques are not shown in detail to avoid obscuring the understanding of this description.

[0035] References in the specification to "one embodiment," "an embodiment," "an example embodiment," etc. indicate that the described embodiment may include a feature, structure, or characteristic, but every embodiment may not necessarily include the feature, structure, or characteristic. Furthermore, such phrases do not necessarily refer to the same embodiment. Furthermore, when a feature, structure, or characteristic is described with respect to an embodiment, it is considered to be within the knowledge of those skilled in the art to affect such feature, structure, or characteristic with respect to other embodiments if explicitly described.

[0036] The disclosed embodiments maximize the execution throughput of sparse-dense matrix multiplication (SDMM) instructions with variable precision inputs, such as virtual neural network (VNN) matrix multiplication instructions with one sparse matrix input and one dense matrix input. By using the circuits disclosed herein, it is expected that the disclosed SDMM instructions will produce performance gains relative to matrix multiplication instructions with symmetric operands.

[0037] By not using conventional symmetric matrix multiplication circuit, it is expected that disclosed embodiment improves the SDMM instruction throughput of computer with the factor proportional to the sparsity of sparse input matrix.As used herein, sparsity is relevant with the ratio of matrix element with zero value or null value.For example, it is expected that the SDMM instruction that the sparse matrix with 0.125 sparsity that only 1 / 8 element has non-zero value operates has 8 times of throughput increase.For another example, it is expected that the SDMM instruction that the sparse matrix with 0.05 sparsity that only 1 / 20 element has non-zero value operates has 20 times of throughput increase.

[0038] In some embodiments, the SDMM instruction is executed by an execution circuit with a SIMD processing channel using a grid of fused multiply-add (FMA) circuits. SIMD channel width may be different in different embodiments. For example, SIMD channel width may include any one of 16 elements, 32 elements, 64 elements, and 128 elements, and these elements may be any one of 8 bits, 16 bits, 32 bits, 64 bits, and 128 bits, but are not limited thereto. SIMD channels are used to execute instructions to multiple data elements in parallel.

[0039] In some embodiments, a multi-block memory is used for intermediate data and result storage of multiple SIMD channels. For example, a SIMD execution circuit with 8 64-bit SIMD channels may use an 8-block memory.

[0040] Figure 11 is a block diagram illustrating a processing component for executing a sparse-dense matrix multiplication (SDMM) instruction such as a virtual neural network instruction (SDMVNNI) with asymmetric inputs according to some embodiments. Virtual neural network instructions are used in deep learning scenarios and are a type of instruction that can benefit from sparse-dense matrix multiplication. There are other types of applications that can benefit from SDMM instructions, such as Galois field new instructions (GFNI). Therefore, the disclosed SDMM instructions are not meant to be limited to VNNI instructions. As shown, storage 101 stores SDMM instructions 103 to be executed.

[0041] The SDMM instruction 103 is fetched from the storage 101 by the fetch circuit 105. The fetched SDMM instruction 107 is decoded by the decode circuit 109. For example, the decode circuit 109 receives the fetched SDMM instruction 107 from the fetch circuit 105. As further described below and with reference to Figure 7 -9, the SDMM instruction format has a field for specifying an opcode, a dense output matrix, a dense source matrix, and a sparse source matrix, the sparse source matrix having a sparsity of non-zero elements less than 1. Decoding circuit 109 decodes the fetched SDMM instruction 107 into one or more operations. In some embodiments, the decoding includes: generating a plurality of micro-operations to be performed by an execution circuit (such as, execution circuit 117). Decoding circuit 109 also decodes instruction suffixes and prefixes (if used). Execution circuit 117 is further described and illustrated below, including at least reference to Figures 2-6 and Figures 11-12 below.

[0042] In some embodiments, the register renaming, register allocation and / or scheduling circuitry 113 provides functionality for one or more of the following: 1) renaming logical operand values ​​to physical operand values ​​(e.g., a register alias table in some embodiments); 2) assigning status bits and flags to decoded instructions; and 3) scheduling decoded SDMM instructions 111 out of an instruction pool for execution on execution circuitry 117 (e.g., using a reservation station in some embodiments).

[0043] Registers (register files) and / or memory 115 store data as operands of decoded SDMM instructions 111 to be operated on or by execution circuitry 117. Exemplary register types include write mask registers, packed data registers, general purpose registers, and floating point registers, as described below with at least reference to Fig.10 Further described and illustrated.

[0044] In some embodiments, write-back circuitry 119 commits the results of execution of decoded SDMM instructions 111 .

[0045] Figure 2AIt is a block diagram illustrating the data flow for processing sparse-dense matrix multiplication (SDMM) instructions according to some embodiments. As shown, SDMM instruction 202 has fields for specifying opcode 204 (SDMMVNNIW), dense output matrix 206, dense source matrix 210 and sparse source matrix 208. As shown, the specified sparse source matrix 212 is logically a matrix of M=3 rows by K=6 columns with a sparsity of non-zero elements approximately equal to 16.67%. In other words, only 16.67% of the elements of the sparse source matrix have non-zero values. When compared with using conventional symmetric matrix multiplication circuits, processing SDMM instructions 202 according to the embodiments disclosed herein improves the throughput of the processor by up to six times. The disclosed embodiments avoid wasting processing cycles on the zero-value elements of the specified sparse source matrix 212. In some embodiments, the sparsity of the specified sparse source matrix 212 is limited to less than 20%. In the context of a virtual neural network, and as shown, sparse source matrices and dense source matrices can represent activation matrices and weight vectors.

[0046] In some embodiments, the specified sparse source matrix 212 is logically an M by K matrix, but only its non-zero elements are stored in memory in a compressed sparse row (CSR) or compressed sparse column (CSC) format, which is prepared in advance in some embodiments. Figure 4 The CSC and CSR formats are further described.

[0047] In some embodiments, the SDMM instruction 202 further specifies an element size (here, the opcode 204 includes a "W" suffix, which specifies a word ( W ord) size). The following at least refers to Figure 7 -9 further illustrates and describes the format of the SDMM instruction. In some embodiments, one or more of the identified matrices are stored in registers, such as in a register file of a processor, for example, as described below with reference to Fig.10 In some embodiments, one or more of the identified matrices are stored in a memory location.

[0048] As shown, the specified sparse source matrix 212 is a matrix with M rows (equal to 3) and K columns (equal to 6) logically, wherein the non-zero element is at (0,0), (3,3) and (2,5). Therefore, the specified sparse source matrix 212 has a sparsity of about 16.67%, and provides a sixfold improvement in processor throughput according to the disclosed embodiment processing instruction. The specified dense source matrix 214 has K rows (equal to 6) and N columns (equal to 6). The specified dense output matrix 216 is shown as having M rows and N columns. As described herein, capital letters M, N and K are used to refer to the maximum dimension of the matrix, and lowercase letters m, n and k are used as the index of the element position in the matrix.

[0049] In operation, as indicated by data flow indicators 218, 220, and 222, for each non-zero element at row m and column k of the specified sparse source matrix 212, the execution circuit generates the product of the non-zero element and all corresponding dense elements at row k of the specified dense source matrix 214. The execution circuit then accumulates each generated product with the previous value of the corresponding output element at row m of the specified dense output matrix 216. In some embodiments, the execution circuit writes the accumulated sum to the corresponding element of the specified dense output matrix 216. In some embodiments, the execution circuit writes the accumulated sum to a scratchpad memory (not shown) before writing to the dense output matrix.

[0050] Figure 2B It is a block diagram illustrating the data flow for processing sparse-dense matrix multiplication (SDMM) instructions according to some embodiments. As shown, the SDMM instruction 252 has a field for specifying an opcode 254 (SDMMVNNI), a dense output matrix 256, a dense source matrix 260, and a sparse source matrix 258. The specified sparse source matrix 262 is logically a matrix of M=4 times K=4, wherein the non-zero elements are in the first column and the third column. Therefore, the specified sparse source matrix 262 has a sparsity of 0.5. In other words, 50% of the elements of the specified sparse source matrix 262 have non-zero values. When compared with using a conventional symmetric matrix multiplication circuit, the SDMM instruction 252 processed according to the embodiment disclosed herein increases the throughput of the processor by two times. The disclosed embodiment avoids wasting processing cycles on zero-value source elements. In some embodiments, the sparsity of the specified sparse source matrix 262 is limited to less than 10%. In some embodiments, one or more of the identified matrices are stored in registers, such as in a register file of a processor, for example, as described below with reference to Fig.10 In some embodiments, one or more of the specified sparse source, dense source, and dense output matrices are stored in a memory location.

[0051] As shown, the specified sparse source matrix 262 is a matrix logically having M rows (equal to 4) and K columns (equal to 4), wherein eight non-zero elements are at columns 0 and 2. The specified dense source matrix 264 has K rows (equal to 4) and N columns (equal to 4). As described herein, uppercase letters M, N, and K are used to refer to the maximum dimensions of a matrix, while lowercase letters m, n, and k are used to refer to the index of the element position within the matrix.

[0052] In operation, according to some embodiments, for each non-zero element at row m and column k of the specified sparse source matrix 262, the execution circuit generates the product of the non-zero element and each corresponding element at row k and column {0, n-1} of the specified dense source matrix 264. In this embodiment, as shown at step 1 266, each of the non-zero elements of the first column of the specified sparse source matrix 262 is multiplied with each element at the corresponding first row of the specified dense source matrix 264. As shown at step 2 268, each of the non-zero elements of the third column of the specified sparse source matrix 262 is multiplied with each element at the corresponding third row of the specified dense source matrix 264. Then, the execution circuit accumulates the products generated in steps 1 266 and 2 268 with the previous values ​​of the corresponding output elements at row m and column n of the specified dense output matrix 270. For simplicity, the previous values ​​of the output matrix are not shown here, but are assumed to be zero. In some embodiments, the execution circuitry writes the accumulated sums to corresponding elements of the dense output matrix. In some embodiments, the execution circuitry writes the accumulated sums to a scratchpad memory before writing to the dense output matrix.

[0053] Figure 3 is a block diagram illustrating an execution circuit for processing a sparse-dense matrix multiplication (SDMM) instruction according to some embodiments. As shown, the SDMM instruction 302 has fields for specifying an opcode 304 (SDMMVNNI), a dense output matrix 306, a sparse source matrix 308, and a dense source matrix 310. As shown, the specified sparse source matrix 312 has a sparsity of non-zero elements of approximately 0.07 for illustration. In other words, approximately 7% of the elements of the specified sparse source matrix have non-zero values. Processing the SDMM instruction according to the embodiments disclosed herein increases the throughput of the processor by approximately 14 times when compared to using conventional symmetric matrix multiplication circuits. The disclosed embodiments avoid wasting processing cycles on zero-valued source elements. In some embodiments, the sparsity of the sparse source matrix is ​​limited to less than 10%. In some embodiments, one or more of the identified matrices are stored in registers, such as in a register file of a processor, for example, as described below with reference to Fig.10 In some embodiments, one or more of the identified matrices are stored in a memory location.

[0054] As shown, the specified sparse source matrix 312 is a matrix with M rows (equal to 8) and K columns (equal to 2), with one non-zero element at row 6 and column 1. The specified dense source matrix 314 has K rows (equal to 2) and N columns (equal to 6). The specified dense output matrix 316 is shown as having M rows (equal to 8) and N columns (equal to 6).

[0055] In operation, according to some embodiments, for each non-zero element at row m and column k of the specified sparse source matrix 312, the execution circuit generates the product of the non-zero element and each corresponding dense element at row k and column n of the specified dense source matrix 314. As shown, the non-zero element at element (6, 1) of the specified sparse source matrix 312 is multiplied with each element at the corresponding row 1 of the specified dense source matrix 314 using multiplier 318. Then, the execution circuit uses adder / accumulator 320 to generate the cumulative sum of the product generated by multiplier 318 and the previous value of the corresponding element of the specified dense output matrix 316. Here, the previous values ​​of the output matrix are not shown, but for simplicity, they are assumed to be zero. In some embodiments, the execution circuit writes the cumulative sum to the corresponding element of the specified dense output matrix 316. In some embodiments, the execution circuit writes the cumulative sum to the temporary register memory 322 before writing to the specified dense output matrix 316.

[0056] At least for reference Figure 4-6 11-12 further illustrate and discuss execution circuits for executing SDMM instructions according to the disclosed embodiments.

[0057] Figure 4 4 is a block diagram illustrating execution circuitry 411 for processing a sparse-dense matrix multiplication (SDMM) instruction according to some embodiments. As shown, the SDMM instruction 402 has fields for specifying an opcode 404 (SDMMVNNI), a dense output matrix 406, a sparse source matrix 408, and a dense source matrix 410. In the illustrated embodiment, the specified dense output, dense source, and sparse source matrices have dimensions M=N=256 and K=512 with 2-byte precision, and the specified sparse source matrix has a sparsity of 0.125. Compressed Sparse Row / Compressed Sparse Column Format

[0058] In some embodiments, the specified sparse source matrix is ​​stored in a sparse format in memory 412 , such that execution circuitry 411 reads the sparsely formatted matrix but buffers only non-zero values ​​in sparse buffer 414 .

[0059] However, in order to avoid unnecessary memory accesses and save memory space, some embodiments store only the non-zero elements of the sparse source matrix in a compressed sparse row (CSR) or compressed sparse column (CSC) format in the memory 412. With the CSR and CSC formats, only the non-zero elements of the sparse source matrix organized in a row-major format or a column-major format are stored, respectively. In some embodiments, a CSR formatted or CSC formatted sparse source matrix is ​​prepared in the memory 412 prior to the operation performed by dedicated hardware or software. Sparse Buffer 414

[0060] The specified sparse source matrix in the illustrated embodiment is logically a 256X512 matrix with 32 non-zero elements per column, i.e., a sparsity of 0.125. The illustrated embodiment uses the CSC format to store the sparse source matrix in the sparse buffer 414, and each column of the sparse source sparse matrix is ​​divided into 8 blocks based on the row index M. In operation, 32 non-zero elements per column of the sparse source matrix are stored in 8 blocks of the sparse buffer 414, and ideally, the 32 non-zero elements per column are evenly distributed among the 8 blocks of the sparse buffer 414. In some embodiments, each entry in the sparse buffer entry uses up to five bytes, including two bytes for storing data values, and up to 3 bytes for specifying the matrix position of the element in the specified sparse source matrix. In some embodiments, the specified sparse source matrix logically has M=16 rows and K=16 columns, and the matrix position of 1 byte specifies the offset of the element within the 256 elements. In some embodiments, the specified sparse source matrix has M=16 rows and K=16 columns, and a 1-byte matrix position includes a nibble to specify the column in which the element is located and a nibble to specify the row in which the element is located. Multiplier array 416

[0061] In the illustrated embodiment, the multiplier array has a size of 8×32, wherein 8 rows of the multiplier array are connected to 8 blocks of the sparse buffer 414, thereby providing 8 multiplier values ​​per cycle required for multiplication. The 32 elements of the row k of the specified dense source matrix forming the multiplicand are broadcast and multiplied across 8 elements of the sparse buffer 414, so that 256 multiplications are performed per cycle. Then, the partial products of the 32 elements generated for each block are accumulated in the accumulator array 418. In some embodiments, the 32 elements of the specified dense source matrix are buffered in registers (not shown) before multiplication. In some embodiments, as shown, when the 32 elements are loaded from the memory 412, they are fed into the multiplier array 416. In some embodiments, the multiplier array 416 includes a grid of fused multiply-add (FMA) hardware units. Accumulator array 418

[0062] As shown, the accumulator array 418 includes 256 accumulators divided into 8 blocks, each block connected to the sparse buffer 414 and the corresponding block of the multiplier array 416. In operation, the accumulators in the accumulator array 418 (32 per block) accumulate the products generated by the multiplier array 416 with the previous values ​​of the corresponding elements of the dense output matrix specified by the dense output matrix 406 field of the SDMM instruction 402. Rounding and saturation circuit 420

[0063] In some embodiments, the products generated by multiplier array 416 and accumulated by accumulator array 418 are high precision intermediate results represented by at least twice the number of bits used by the data elements of the specified matrix. In some embodiments, rounding and saturation circuit 420 saturates the intermediate results to a predefined maximum value and rounds them to fit the number of bits of the elements of the dense output matrix specified by the dense output matrix 406 field of SDMM instruction 402, which is 16 bits here.

[0064] In the case of floating point arithmetic, the rounding and saturation circuit 420 can round the intermediate results according to the IEEE 754 floating point standard, which was established by the Institute of Electrical and Electronics Engineers in 1985 and updated in 2008. The IEEE 754 floating point standard defines rounding rules to be applied, including round to even, round to zero, round to positive infinity, and round to negative infinity. In some embodiments, the rounding and saturation circuit 420 includes a software accessible rounding control register (not shown) to specify the rounding rule to be applied.

[0065] In some embodiments, each of the FMA hardware units in multiplier array 416 performs rounding by itself. In some embodiments, each of the FMA hardware units in multiplier array 416 checks for saturation by itself and performs saturation. Register 422

[0066] As shown, execution circuit 411 includes a temporary register 422 to store intermediate execution results. In some embodiments, as here, temporary register 422 is a 32kB memory divided into eight (8) blocks, each block connected one-to-one with a corresponding row of the multiplier array and a corresponding block of the accumulator. In some embodiments, all blocks of temporary register 422 communicate in parallel with corresponding blocks of accumulator array 418, thereby generating a high bandwidth connection. The CSC format is used for multiplication, whereby only the non-zero elements of the specified sparse source matrix are stored in sparse buffer 414 and provided to multiplier array 416, which eliminates the need to use expensive gather-scatter circuits to gather non-zero data and provide non-zero data to multiplier array 416 and accumulator array 418.

[0067] In some embodiments, such as here, execution circuit 411 utilizes one or more single instruction multiple data (SIMD) processing channels (e.g., 8 channels) to perform the same operation on multiple data elements simultaneously. In some embodiments, the SIMD processing channel has a channel width of 32 elements, and 8 SIMD processing channels are used to perform 256 operations on 256 elements of data.

[0068] In some embodiments, two or more SIMD processing channels operate concurrently and in parallel. Without limitation, the number of channels in a SIMD processor and the number of elements assigned to each channel may be different. According to some embodiments, without limitation, a SIMD processing channel is implemented with a channel width of 8 elements, 16 elements, or 32 elements, where the element width is 8 bits, 16 bits, 32 bits, or 64 bits.

[0069] In some embodiments, the execution circuit 411 executes the SDMM instruction 402 in the following manner: using a fused multiply-add (FMA) hardware unit to perform a multiply-accumulate operation to generate the product of each non-zero element of the specified sparse source matrix and each element of the elements in the corresponding row of the specified dense source matrix, and accumulating the products with the previous values ​​of the corresponding elements of the dense output matrix specified by the dense output matrix 406 field of the SDMM instruction 402.

[0070] As used herein, the term "corresponding" has different interpretations based on its context. In the context of generating a product, the corresponding element of the dense source matrix specified by the dense source matrix 410 field corresponding to each non-zero element (m, k) of the sparse source matrix specified by the sparse source matrix 408 field of the SDMM instruction 402 is the corresponding element (k, n) in the corresponding row k of the dense source matrix specified by the dense source matrix 410 field of the SDMM instruction 402. In the context of accumulating the product with the previous contents of the dense output matrix specified by the dense output matrix 406 field of the SDMM instruction 402, the corresponding elements of the specified dense output matrix are those elements at the position (m, n).

[0071] Thus, execution circuitry 411 increases the throughput of a processor incorporating the execution circuitry by a factor of 8 by executing SDMM instruction 402 specifying a sparse source matrix 408 having a sparsity of 0.125, when compared to using symmetric matrix multiplication circuitry.

[0072] At least for reference Figure 3-6 11-12 , execution circuitry for executing SDMM instructions according to the disclosed embodiments is further illustrated and discussed.

[0073] Figure 5 502 and 504 illustrate the implementation of the VNNI matrix multiplication instruction for symmetric source operands. As described herein, uppercase letters M, N, and K are used to refer to the maximum dimensions of the matrix, while lowercase letters m, n, and k are used to refer to the index of the element position within the matrix. Both pseudocode 502 and pseudocode 504 illustrate the result of executing the VNNI instruction to generate an output matrix C of M rows by N columns (M x N) on a source matrix A of M rows by K columns (M x K) and a source matrix B of K rows by N columns (K x N). On the other hand, pseudocodes 506 and 508 illustrate the execution of the SDMM instruction according to the disclosed embodiment, wherein only the non-zero elements of the sparse source matrix A are processed, thereby increasing the processor throughput in proportion to the sparsity of the source matrix A.

[0074] At least for reference Figure 3-6 11-12 , execution circuitry for executing SDMM instructions according to the disclosed embodiments is shown and discussed.

[0075] Figure 6It is a process flow diagram illustrating the execution of sparse-dense matrix multiplication (SDMM) instructions by a processor according to some embodiments. At 602, the processor uses a fetch circuit to fetch a sparse-dense matrix multiplication instruction from code storage, the instruction having a field for specifying an opcode, a dense output matrix, a dense source matrix, and a sparse source matrix, the sparse source matrix having a sparsity of non-zero elements, wherein the sparsity is less than one. One source is a sparse matrix and another source is a dense matrix, to this extent, the SDMM instructions fetched at 602 can be referred to as asymmetric SDMM instructions. At 604, the processor is decoded by a decoding circuit to the SDMM instructions fetched. At 606, the processor optionally schedules the execution of the decoded SDMM instructions by a SIMD execution circuit. Scheduling the execution of the decoded instructions may occur at different times or may not occur at all, to this extent, operation 606 is optional, as indicated by its dotted line boundary. At 608, the processor executes the decoded SDMM instruction by the execution circuit to: for each non-zero element at row M and column K of the sparse source matrix, generate the product of the non-zero element and each corresponding dense element at row K and column N of the dense source matrix; and accumulate each generated product with the previous value of the corresponding output element at row M and column N of the dense output matrix. At 610, the processor optionally commits or retires the executed SDMM instruction. Operation 610 may occur at different times or may not occur at all, to the extent that operation 610 is optional, as indicated by its dashed boundary.

[0076] At least for reference Figure 3-6 11-12 , execution circuitry for executing SDMM instructions according to the disclosed embodiments is shown and discussed.

[0077] Figure 7 700 is an exemplary format of a sparse-dense matrix multiplication (SDMM) instruction according to some embodiments. As shown, the SDMM instruction 700 includes an opcode 702 (SDMMVNNI*) and fields for specifying a dense output matrix 704, a dense source matrix 706, and a sparse source matrix 708. The SDMM instruction 700 further includes optional fields for specifying element size 710 and M, N, and K dimensions 712, 714, and 716. In some embodiments, one or more of the optional dimensions 712, 714, and 716 are specified by software programmable model-specific registers (MSRs), which may be predetermined MSRs. Figure 7As shown, the optional element size 710 can be specified as part of the opcode, and the M, N and K dimensions 712, 714 and 716 can be specified by the MSR, as part of the opcode, or a combination thereof. The opcode 702 is shown as including an asterisk to indicate that it can optionally include additional prefixes or suffixes to specify instruction behavior. For example, the opcode 702 can include a suffix such as "B", "W", "D", or "Q" to specify element sizes of eight bits, sixteen bits, thirty-two bits, and sixty-four bits, respectively. If the SDMM instruction 700 does not specify any of the optional parameters, a predetermined default value is used. Reference is made below to Fig. 8A , Figure 8B and Figures 9A-9D The format of the SDMM instruction is further shown and described. Prepare the compressed sparse source matrix in advance

[0078] Reference again Figure 2B In some embodiments, the specified compressed sparse source matrix (CSR or CSC) is prepared in advance. Figure 4 The CSC format and the CSR format are further shown and described.

[0079] In some embodiments, the SDMM instruction 700 is used to cause the processor to generate a compressed format (CSR or CSC, as shown in FIG. Figure 4 ) prepares a sparse source matrix. In some embodiments, the opcode 702 may include a prefix or suffix, such as "PREP", to indicate that the processor is preparing a compressed sparse source matrix. In such embodiments, the dense output matrix 704 field specifies a memory address for storing the compressed sparse source matrix, and the sparse source matrix 708 specifies a memory location where a sparse data set is stored, the sparse data set comprising a large block of memory, a sparsity with valid elements, the validity of which is determined by the data value itself (e.g., invalid is empty, zero or below a threshold), or by a control field attached to each element and indicating the validity of each element. In such embodiments, the size of the sparse data set is specified by the dense source matrix 706 field.

[0080] In one example, the SDMM instruction 700 includes an opcode 702 with a "PREP" suffix, specifies a dense output matrix 704 to which a compact sparse source matrix is ​​stored, uses a sparse source matrix 708 field to specify the location of a sparse data set including tens, hundreds, thousands, millions or billions of data elements, wherein the invalid elements have null values, and uses a dense source matrix 706 field to specify the size of the sparse data set. In response, the processor loads data elements from the specified sparse source matrix 708 position, uses the dense source matrix 706 field to determine the size of the sparse data set, determines whether each data element is valid, and writes valid elements to the specified dense output matrix 704 position in a compressed format (CSR or CSC), which is a cache line aligned position in some embodiments. Therefore, the processor compacts the valid elements of the sparse data set to the specified output matrix in a compressed format (CSR or CSC), and the output matrix can be used as a sparse source matrix for subsequent SDMM instructions.

[0081] In another example, the SDMM instruction 700 includes an opcode 702 with a "PREP" suffix, specifies a dense output matrix 704 for storing a sparse source matrix therein, uses a sparse source matrix 708 field to specify the position of a sparse data set including tens, hundreds, thousands, millions or billions of data elements, wherein each element includes a control field with at least a valid bit, and uses a dense source matrix 706 field to specify the size (i.e., the number of elements) of the sparse data set. In response, the processor loads data elements from the specified sparse source matrix 708 position, uses the dense source matrix 706 field to determine the size of the sparse data set, determines whether each data element is valid, and writes valid elements to the specified dense output matrix 704 position in a compressed format (CSR or CSC), which is a cache line aligned position in some embodiments. Therefore, the processor compacts the valid elements of the sparse data set to the specified output matrix in a compressed format (CSR or CSC), and the output matrix can be used as a sparse source matrix for subsequent SDMM instructions.

[0082] In some embodiments, a processor responding to an instance of an SDMM instruction 700 having an opcode with a "PREP" suffix operates in the background by opportunistically scheduling the execution of its necessary loads and stores only when the processor activity level is below a threshold. In some embodiments, a processor responding to an instance of an SDMM instruction 700 having an opcode with a "PREP" suffix generates an interrupt when the compression of the specified sparse data set is completed. Instruction Set

[0083] An instruction set may include one or more instruction formats. A given instruction format may define various fields (e.g., the number of bits, the position of the bits) to specify the operation to be performed (e.g., an opcode) and the operands and / or other data fields (e.g., a mask) on which the operation will be performed, etc. Some instruction formats are further decomposed by the definition of instruction templates (or sub-formats). For example, an instruction template for a given instruction format may be defined as having different subsets of the fields of the instruction format (the included fields are usually in the same order, but at least some fields have different bit positions because fewer fields are included), and / or defined as having a given field interpreted in a different manner. Thus, each instruction of the ISA is expressed using a given instruction format (and, if defined, according to each of the instruction templates in the instruction format) and includes fields for specifying operations and operands. For example, an exemplary ADD (addition) instruction has a specific opcode and instruction format, the specific instruction format including an opcode field for specifying the opcode and an operand field for selecting operands (source 1 / destination and source 2); and the ADD instruction appears in an instruction stream with specific contents in the operand field for selecting a specific operand. A set of SIMD extensions known as Advanced Vector Extensions (AVX) (AVX1 and AVX2) and utilizing a vector extension (VEX) encoding scheme have been introduced and / or released (see, e.g., September 2014 64 and IA-32 Architectures Software Developer's Manual; and see the October 2014 Advanced Vector Extensions Programming Reference). Example instruction format

[0084] Embodiments of the (multiple) instructions described herein can be embodied in different formats. In addition, exemplary systems, architectures, and pipelines are described in detail below. Embodiments of the (multiple) instructions can be executed on such systems, architectures, and pipelines, but are not limited to those systems, architectures, and pipelines described in detail. Generic vector friendly instruction format

[0085] The vector friendly instruction format is an instruction format suitable for vector instructions (e.g., there are specific fields dedicated to vector operations). Although embodiments are described in which both vector and scalar operations are supported through the vector friendly instruction format, alternative embodiments only use vector operations through the vector friendly instruction format.

[0086] Figure 8A-8B is a block diagram illustrating a generic vector friendly instruction format and instruction templates thereof according to some embodiments. Fig. 8A is a block diagram illustrating a generic vector friendly instruction format and its class A instruction templates according to some embodiments; and Figure 8B800 is a block diagram illustrating a generic vector friendly instruction format and its class B instruction templates according to some embodiments. Specifically, class A and class B instruction templates are defined for the generic vector friendly instruction format 800, both of which include instruction templates with no memory access 805 and instruction templates with memory access 820. The term "generic" in the context of the vector friendly instruction format refers to an instruction format that is not tied to any particular instruction set.

[0087] Although some embodiments will be described in which the vector friendly instruction format supports the following cases: 64-byte vector operand length (or size) with 32-bit (4-byte) or 64-bit (8-byte) data element width (or size) (and thus, a 64-byte vector consists of 16 doubleword-sized elements, or alternatively, 8 quadword-sized elements); 64-byte vector operand length (or size) with 16-bit (2-byte) or 8-bit (1-byte) data element width (or size); 32-byte vector operand length (or size) with 32-bit (4-byte) or 64-bit (8-byte) data element width (or size); byte), 64-bit (8-byte), 16-bit (2-byte), or 8-bit (1-byte) data element width (or size); and 16-byte vector operand length (or size) with 32-bit (4-byte), 64-bit (8-byte), 16-bit (2-byte), or 8-bit (1-byte) data element width (or size); however, alternative embodiments may support larger, smaller, and / or different vector operand sizes (e.g., 256-byte vector operands) with larger, smaller, or different data element widths (e.g., 128-bit (16-byte) data element width).

[0088] Fig. 8A The class A instruction templates include: 1) within the instruction templates without memory access 805, there are shown instruction templates for full rounding control type operations 810 without memory access and instruction templates for data transformation type operations 815 without memory access; and 2) within the instruction templates for memory access 820, there are shown instruction templates for timeliness of memory access 825 and instruction templates for non-timeliness of memory access 830. Figure 8B The Class B instruction templates include: 1) within the instruction template without memory access 805, an instruction template for a partial rounding control type operation 812 with write mask control without memory access and an instruction template for a vsize type operation 817 with write mask control without memory access are shown; and 2) within the instruction template with memory access 820, an instruction template for a write mask control 827 with memory access is shown.

[0089] The generic vector friendly instruction format 800 includes the following listed in accordance with Figures 8A-8B The following fields are in the order shown in the figure.

[0090] Format field 840 - The specific value in this field (the instruction format identifier value) uniquely identifies the vector friendly instruction format, and thereby identifies that the instruction appears in the vector friendly instruction format in the instruction stream. Thus, this field is optional in the sense that it is not required for instruction sets that only have a generic vector friendly instruction format.

[0091] Basic operation field 842 - its content distinguishes different basic operations.

[0092] Register index field 844 - its content specifies the location of the source or destination operand in a register or in memory, either directly or through address generation. These fields include enough bits to select N registers from a PxQ (e.g., 32x512, 16x128, 32x1024, 64x1024) register file. Although in one embodiment N may be up to three source registers and one destination register, alternative embodiments may support source and destination registers (e.g., may support up to two sources, one of which is also used as a destination; may support up to three sources, one of which is also used as a destination; may support up to two sources and one destination).

[0093] Modifier field 846 - its content distinguishes instructions appearing in the generic vector instruction format that specify memory access from instructions appearing in the generic vector instruction format that do not specify memory access; i.e., distinguishes between instruction templates with no memory access 805 and instruction templates with memory access 820. Memory access operations read and / or write to the memory hierarchy (in some cases, using values ​​in registers to specify source and / or destination addresses), while non-memory access operations do not (e.g., the source and destination are registers). Although in one embodiment, this field also selects between three different ways to perform memory address calculations, alternative embodiments may support more, fewer, or different ways to perform memory address calculations.

[0094] Extended Operation Field 850 - Its contents distinguish which of various operations are to be performed in addition to the base operation. This field is context specific. In one embodiment of the invention, this field is divided into a class field 868, an alpha field 852, and a beta field 854. The extended operation field 850 allows multiple common groups of operations to be performed in a single instruction instead of 2, 3, or 4 instructions.

[0095] Scale field 860 - its content allows for memory address generation (e.g., for use with (2 比例 *The address generated by (index + base address) is scaled proportionally to the contents of the index field.

[0096] Displacement field 862A - its contents are used as part of memory address generation (e.g., for use with (2 比例 *address generation of (index+base+displacement)).

[0097] Displacement Factor field 862B (note that the juxtaposition of displacement field 862A directly over displacement factor field 862B indicates that one or the other is used) - its contents are used as part of address generation; it specifies the displacement factor by which the size (N) of the memory access will be scaled - where N is the number of bytes in the memory access (e.g., for use with (2 比例 * index + base address + scaled displacement). Redundant low-order bits are ignored, and therefore the contents of the displacement factor field are multiplied by the total size of the memory operand (N) to generate the final displacement that will be used in calculating the effective address. The value of N is determined by the processor hardware at runtime based on the full opcode field 874 (described later in this document) and the data manipulation field 854C. The displacement field 862A and the displacement factor field 862B are optional in the sense that they are not used for instruction templates without memory access 805 and / or different embodiments may implement only one of the two or neither of the two.

[0098] Data element width field 864 - its contents distinguish which of a number of data element widths will be used (in some embodiments for all instructions; in other embodiments for only some of the instructions). This field is optional in the sense that it is not required if only one data element width is supported and / or some aspect of the opcode is used to support the data element width.

[0099] Write mask field 870 - its content controls, on a data element position by data element position basis, whether the data element position in the destination vector operand reflects the results of the base and augmented operations. Class A instruction templates support merge-write masking, while class B instruction templates support both merge-write masking and zeroing-write masking. When merged, the vector mask allows any set of elements in the destination to be protected from update during the execution of any operation (specified by the base and augmented operations); in another embodiment, the old value of each element of the destination where the corresponding mask bit has a 0 is maintained. Conversely, when zeroed, the vector mask allows any set of elements in the destination to be zeroed during the execution of any operation (specified by the base and augmented operations); in one embodiment, the elements of the destination are set to 0 when the corresponding mask bit has a 0 value. A subset of this functionality is the ability to control the vector length of the operation being performed (i.e., the span from the first to the last element being modified), however, the elements being modified do not have to be consecutive. Thus, the write mask field 870 allows partial vector operations, which include loads, stores, arithmetic, logic, etc. Although some embodiments are described in which the contents of write mask field 870 select one of many write mask registers that contains a write mask to be used (and thereby, the contents of write mask field 870 indirectly identify the mask to be performed), alternative embodiments alternatively or additionally allow the contents of mask write field 870 to directly specify the mask to be performed.

[0100] Immediate field 872 - its contents allow specification of an immediate value. This field is optional in the sense that it is not present in implementations of the generic vector friendly format that do not support immediate values ​​and is not present in instructions that do not use immediate values.

[0101] Class field 868 - its content distinguishes between instructions of different classes. Figure 8A-8B , the contents of this field select between class A and class B instructions. Figure 8A-8B In , rounded squares are used to indicate that a specific value exists in a field (for example, in Figure 8A-8B 868A and B respectively for class field 868). Class A instruction template

[0102] In the case of the class A non-memory access 805 instruction templates, the alpha field 852 is interpreted as an RS field 852A whose content distinguishes which of the different extended operation types is to be performed (e.g., the instruction templates for the no-memory access rounding type operation 810 and the no-memory access data transformation type operation 815 specify rounding 852A.1 and data transformation 852A.2, respectively), while the beta field 854 distinguishes which of the specified types of operations is to be performed. In the no-memory access 805 instruction templates, the scale field 860, the displacement field 862A, and the displacement scale field 862B are not present. Instruction templates without memory access - full rounding control type operations

[0103] In the instruction templates of the full rounding control type operation 810 without memory access, the beta field 854 is interpreted as a rounding control field 854A that provides static rounding for its contents (multiple). Although in the described embodiment the rounding control field 854A includes a suppress all floating point exceptions (SAE) field 856 and a rounding operation control field 858, alternative embodiments may support both concepts, may encode both concepts into the same field, or may have only one or the other of these concepts / fields (e.g., may have only the rounding operation control field 858).

[0104] SAE field 856 - its content distinguishes whether exception event reporting is disabled; when the content of SAE field 856 indicates that suppression is enabled, the given instruction does not report any kind of floating point exception flags and does not invoke any floating point exception handler.

[0105] Round operation control field 858 - its content distinguishes which of a set of rounding operations is to be performed (e.g., round up, round down, round toward zero, and round to nearest). Thus, the round operation control field 858 allows the rounding mode to be changed on an instruction-by-instruction basis. In one embodiment of the invention in which the processor includes a control register for specifying the rounding mode, the content of the round operation control field 850 overrides the register value. Instruction templates without memory access - data transformation operations

[0106] In instruction templates for data transform type operations 815 without memory access, the beta field 854 is interpreted as a data transform field 854B, the contents of which distinguish which of a number of data transforms is to be performed (eg, no data transform, blend, broadcast).

[0107] In the case of an instruction template of class A memory access 820, the alpha field 852 is interpreted as an eviction hint field 852B, the content of which distinguishes which of the eviction hints is to be used (in Fig. 8A852B.1 and 852B.2 are specified for the instruction templates of memory access temporal 825 and memory access non-temporal 830, respectively, and the β field 854 is interpreted as a data manipulation field 854C, the content of which distinguishes which of a plurality of data manipulation operations (also referred to as primitives) is to be performed (e.g., no manipulation, broadcast, up-conversion of the source, and down-conversion of the destination). The instruction template of memory access 820 includes a scale field 860 and optionally includes a displacement field 862A or a displacement scale field 862B.

[0108] Vector memory instructions use the conversion support to perform vector loads from memory and vector stores to memory. As usual vector instructions, vector memory instructions transfer data to / from memory in a data element-wise manner, where the elements transferred are specified by the contents of the vector mask selected as the writemask. Instruction templates for memory accesses - time-sensitive

[0109] Temporal data is data that is likely to be reused quickly enough to benefit from cache operations. However, this is a hint, and different processors can implement it in different ways, including ignoring the hint completely. Instruction templates for memory accesses - non-temporal

[0110] Non-temporal data is data that is unlikely to be reused quickly enough to benefit from cache operations in the first level cache and should be given eviction priority. However, this is a hint, and different processors can implement it in different ways, including ignoring the hint completely. Class B instruction template

[0111] In the case of class B instruction templates, the alpha field 852 is interpreted as a write mask control (Z) field 852C, the content of which distinguishes whether the write mask controlled by the write mask field 870 should be merge or zero.

[0112] In the case of a class B non-memory access 805 instruction template, a portion of the beta field 854 is interpreted as an RL field 857A, the contents of which distinguish which of the different extended operation types are to be performed (e.g., the instruction template for the no memory access write mask control partial rounding control type operation 812 and the instruction template for the no memory access write mask control VSIZE type operation 817 specify rounding 857A.1 and vector length (VSIZE) 857A.2, respectively), while the remainder of the beta field 854 distinguishes which of the specified types of operations are to be performed. In the no memory access 805 instruction template, the scale field 860, the displacement field 862A, and the displacement scale field 862B are not present.

[0113] In the instruction template of the write mask control partial round control type operation 810 without memory access, the rest of the beta field 854 is interpreted as the round operation field 859A, and exception event reporting is disabled (the given instruction does not report any kind of floating point exception flag and does not invoke any floating point exception handler).

[0114] Round operation control field 859A - As with round operation control field 858, its contents distinguish which of a set of rounding operations (e.g., round up, round down, round toward zero, and round to nearest) is to be performed. Thus, round operation control field 859A allows the rounding mode to be changed on an instruction-by-instruction basis. In one embodiment of the invention in which the processor includes a control register for specifying the rounding mode, the contents of round operation control field 850 override that register value.

[0115] In the instruction template for a write mask control VSIZE type operation 817 without memory access, the remainder of the β field 854 is interpreted as a vector length field 859B, the contents of which distinguish which of a number of data vector lengths (e.g., 128 bytes, 256 bytes, or 512 bytes) is to be performed.

[0116] In the case of a class B memory access 820 instruction template, a portion of the beta field 854 is interpreted as a broadcast field 857B, the content of which distinguishes whether a broadcast-type data manipulation operation is to be performed, and the remainder of the beta field 854 is interpreted as a vector length field 859B. The memory access 820 instruction template includes a scale field 860 and optionally includes a displacement field 862A or a displacement scale field 862B.

[0117] With respect to the generic vector friendly instruction format 800, the full opcode field 874 is shown to include the format field 840, the base operation field 842, and the data element width field 864. Although one embodiment is shown in which the full opcode field 874 includes these fields, in embodiments that do not support all of these fields, the full opcode field 874 includes less than these fields. The full opcode field 874 provides an operation code (opcode).

[0118] The augment operation field 850, the data element width field 864, and the write mask field 870 allow these features to be specified on an instruction-by-instruction basis in the generic vector friendly instruction format.

[0119] The combination of the write mask field and the data element width field creates various types of instructions because these instructions allow the mask to be applied based on different data element widths.

[0120] The various instruction templates that appear within class A and class B are beneficial in different situations. In some embodiments, different processors or different cores within a processor may support only class A, only class B, or both classes. For example, a high-performance general purpose out-of-order core intended for general purpose computing may support only class B, a core intended to be used primarily for graphics and / or scientific (throughput) computing may support only class A, and a core intended for both general purpose computing and graphics and / or scientific (throughput) computing may support both class A and class B (of course, a core with some mix of templates and instructions from these two classes, but not all templates and instructions from these two classes, is within the scope of the present invention). Likewise, a single processor may include multiple cores, all of which support the same class, or different cores in which support different classes. For example, in a processor with separate graphics cores and general purpose cores, one of the graphics cores intended to be used primarily for graphics and / or scientific computing may support only class A, while one or more of the general purpose cores may be a high-performance general purpose core with out-of-order execution and register renaming intended for general purpose computing that supports only class B. Another processor that does not have a separate graphics core may include one or more general-purpose in-order or out-of-order cores that support both class A and class B. Of course, in different embodiments, features from one class may also be implemented in the other class. A program written in a high-level language will be made (e.g., just-in-time or statically compiled) into a variety of different executable forms, including: 1) a form with only instructions of the class(es) supported by the target processor for execution; or 2) a form with alternative routines written using different combinations of instructions of all classes and with control flow code that selects these routines to execute based on instructions supported by the processor currently executing the code. Exemplary Specialized Vector Friendly Instruction Format

[0121] Fig. 9A is a block diagram illustrating an exemplary specific vector friendly instruction format in accordance with some embodiments. Fig. 9A A specific vector friendly instruction format 900 is shown that specifies the location, size, interpretation, and order of various fields, as well as the values ​​of some of those fields, and in this sense, the specific vector friendly instruction format 900 is specific. The specific vector friendly instruction format 900 can be used to extend the x86 instruction set, and thus some of the fields are similar or identical to those used in the existing x86 instruction set and its extensions (e.g., AVX). The format remains consistent with the prefix encoding field, real opcode byte field, MOD R / M field, SIB field, displacement field, and immediate field of the existing x86 instruction set with extensions. The fields from FIG. 8 are illustrated, and the fields from Fig. 9A The fields of are mapped to the fields from Figure 8.

[0122] It should be understood that although some embodiments are described with reference to the specific vector friendly instruction format 900 in the context of the general vector friendly instruction format 800 for illustrative purposes, the present invention is not limited to the specific vector friendly instruction format 900 unless otherwise stated. For example, the general vector friendly instruction format 800 contemplates various possible sizes of various fields, while the specific vector friendly instruction format 900 is shown as having fields of specific sizes. As a specific example, although the data element width field 864 is illustrated as a one-bit field in the specific vector friendly instruction format 900, the present invention is not limited thereto (i.e., the general vector friendly instruction format 800 contemplates other sizes of the data element width field 864).

[0123] The generic vector friendly instruction format 800 includes the following listed in accordance with Fig. 9A The following fields are in the order shown in the figure.

[0124] EVEX prefix (bytes 0-3) 902 - encoded in four-byte form.

[0125] Format field 840 (EVEX byte 0, bits [7:0]) - The first byte (EVEX byte 0) is the format field 840, and it contains 0x62 (a unique value used to distinguish the vector friendly instruction format in one embodiment of the present invention).

[0126] The second through fourth bytes (EVEX bytes 1-3) include a number of bit fields that provide specific capabilities.

[0127] REX field 905 (EVEX byte 1, bits [7-5]) - consists of the EVEX.R bit field (EVEX byte 1, bits [7]-R), the EVEX.X bit field (EVEX byte 1, bits [6]-X), and (857BEX byte 1, bits [5]-B). The EVEX.R, EVEX.X, and EVEX.B bit fields provide the same functionality as the corresponding VEX bit fields and are encoded using 1's complement form, i.e., ZMM0 is encoded as 1111B and ZMM15 is encoded as 0000B. The other fields of these instructions encode the lower three bits of the register index (rrr, xxx, and bbb) as is known in the art, whereby Rrrr, Xxxx, and Bbbb may be formed by adding EVEX.R, EVEX.X, and EVEX.B.

[0128] REX' field 810 - This is the first portion of the REX' field 810 and is the EVEX.R' bit field (EVEX byte 1, bit [4] - R') used to encode the upper 16 or lower 16 registers of the extended 32 register set. In one embodiment of the invention, this bit is stored in a bit-reversed format along with the other bits indicated below to distinguish (in 32-bit mode of the known x86) from a BOUND instruction whose real opcode byte is 62, but does not accept a value of 11 in the MOD field in the MOD R / M field (described below); some alternative embodiments do not store this indicated bit and the other indicated bits below in an inverted format. A value of 1 is used to encode the lower 16 registers. In other words, R'Rrrr is formed by combining EVEX.R', EVEX.R, and the other RRRs from the other fields.

[0129] Opcode map field 915 (EVEX byte 1, bits [3:0] – mmmm) - its contents encode the implied leading opcode byte (0F, 0F 38, or 0F 3).

[0130] Data element width field 864 (EVEX byte 2, bit [7] – W) – represented by the notation EVEX.W. EVEX.W is used to define the granularity (size) of the data type (32-bit data element or 64-bit data element).

[0131] EVEX.vvvv 920 (EVEX byte 2, bits [6:3] - vvvv) - The role of EVEX.vvvv may include the following: 1) EVEX.vvvv encodes the first source register operand specified in inverted (1's complement) form and is valid for instructions with two or more source operands; 2) EVEX.vvvv encodes the destination register operand specified in 1's complement form for a particular vector displacement; or 3) EVEX.vvvv does not encode any operand, this field is reserved, and should contain 1111b. Thus, the EVEX.vvvv field 920 encodes the 4 low-order bits of the first source register designator stored in inverted (1's complement) form. Depending on the instruction, additional different EVEX bit fields are used to extend the designator size to 32 registers.

[0132] EVEX.U 868 Class field (EVEX byte 2, bit [2] - U) - if EVEX.U = 0, it indicates Class A or EVEX.U0; if EVEX.U = 1, it indicates Class B or EVEX.U1.

[0133] Prefix encoding field 925 (EVEX byte 2, bits [1:0]-pp) - provides additional bits for the base operation field. In addition to providing support for legacy SSE instructions in EVEX prefix format, this also has the benefit of compressing the SIMD prefix (EVEX prefix only requires 2 bits, rather than requiring bytes to express the SIMD prefix). In one embodiment, to support legacy SSE instructions that use SIMD prefixes (66H, F2H, F3H) in both legacy format and in EVEX prefix format, these legacy SIMD prefixes are encoded into the SIMD prefix encoding field; and at runtime are expanded into the legacy SIMD prefix before being provided to the decoder's PLA (so that the PLA can execute both these legacy instructions in legacy format and these legacy instructions in EVEX format without modification). Although newer instructions can use the contents of the EVEX prefix encoding field directly as an opcode extension, for consistency, specific embodiments expand in a similar manner, but allow for different meanings specified by these legacy SIMD prefixes. Alternative embodiments may redesign the PLA to support 2-bit SIMD prefix encodings, and thus do not require expansion.

[0134] alpha field 852 (EVEX byte 3, bit [7] - EH, also known as EVEX.EH, EVEX.rs, EVEX.RL, EVEX.WriteMaskControl, and EVEX.N; also illustrated as alpha) - As previously described, this field is context specific.

[0135] Beta field 854 (EVEX byte 3, bits [6:4] - SSS, also called EVEX.s 2-0 ,EVEX.r 2-0 , EVEX.rr1, EVEX.LL0, EVEX.LLB, also illustrated as βββ) - As mentioned earlier, this field is context-specific.

[0136] REX' field 810 - This is the remainder of the REX' field and is the EVEX.V' bit field (EVEX byte 3, bit [3] - V') that can be used to encode the upper 16 or lower 16 registers of the extended 32 register set. This bit is stored in a bit-reversed format. A value of 1 is used to encode the lower 16 registers. In other words, V'VVVV is formed by combining EVEX.V', EVEX.vvvv.

[0137] Write mask field 870 (EVEX byte 3, bits [2:0] - kkk) - its contents specify the index of a register in the write mask register, as previously described. In one embodiment of the invention, a special value EVEX.kkk = 000 has special behavior that implies no write mask is used for a particular instruction (this can be implemented in a variety of ways, including using a write mask that is hardwired to all objects or hardware that bypasses the mask hardware).

[0138] The real opcode field 930 (byte 4) is also called the opcode byte. A portion of the opcode is specified in this field.

[0139] The MOD R / M field 940 (byte 5) includes a MOD field 942, a Reg field 944, and an R / M field 946. As previously described, the content of the MOD field 942 distinguishes memory access operations from non-memory access operations. The role of the Reg field 944 can be attributed to two situations: encoding a destination register operand or a source register operand; or being treated as an opcode extension and not used to encode any instruction operand. The role of the R / M field 946 may include the following: encoding an instruction operand that references a memory address; or encoding a destination register operand or a source register operand.

[0140] Scale, Index, Base (SIB) Byte (Byte 6) - As previously described, the contents of the scale field 850 are used for memory address generation. SIB.xxx 954 and SIB.bbb 956 - The contents of these fields have been mentioned previously for register indexes Xxxx and Bbbb.

[0141] Displacement field 862A (bytes 7-10) - When the MOD field 942 contains 10, bytes 7-10 are the displacement field 862A and it works the same as a traditional 32-bit displacement (disp32) and works at byte granularity.

[0142] Displacement Factor Field 862B (Byte 7) - When the MOD field 942 contains 01, byte 7 is the displacement factor field 862B. The location of this field is the same as that of the traditional x86 instruction set 8-bit displacement (disp8) that works at byte granularity. Since disp8 is sign-extended, it can only address between -128 and 127 byte offsets; in terms of a 64-byte cache line, disp8 uses 8 bits that can be set to only four really useful values ​​-128, -64, 0, and 64; since a larger range is often needed, disp32 is used; however, disp32 requires 4 bytes. In contrast to disp8 and disp32, displacement factor field 862B is a reinterpretation of disp8; when displacement factor field 862B is used, the actual displacement is determined by multiplying the contents of the displacement factor field by the size (N) of the memory operand access. This type of displacement is referred to as disp8*N. This reduces the average instruction length (a single byte for the displacement, but with a much larger range). Such compressed displacements are based on the assumption that the effective displacement is a multiple of the granularity of the memory access, and thus the redundant low-order bits of the address offset do not need to be encoded. In other words, the displacement factor field 862B replaces the traditional x86 instruction set 8-bit displacement. Thus, the displacement factor field 862B is encoded in the same manner as the x86 instruction set 8-bit displacement (therefore, there is no change in the ModRM / SIB encoding rules), the only difference being that disp8 is overloaded to disp8*N. In other words, there is no change in the encoding rules or encoding length, and only a change in the hardware interpretation of the displacement value (which requires scaling the displacement by the size of the memory operand to obtain a byte-based address offset). The immediate field 872 operates as previously described. Full opcode field

[0143] Fig. 9B 874, and a data element width (W) field 864. The base operation field 842 includes a prefix encoding field 925, an opcode map field 915, and a real opcode field 930. Register Index Field

[0144] Fig. 9C844, a block diagram illustrating fields of a specific vector friendly instruction format 900 that make up a register index field 844 according to one embodiment of the present invention. Specifically, the register index field 844 includes a REX field 905, a REX' field 910, a MODR / M.reg field 944, a MODR / Mr / m field 946, a VVVV field 920, a xxx field 954, and a bbb field 956. Expand the operation field

[0145] Fig.9D 850 is a block diagram illustrating the fields of the specific vector friendly instruction format that make up the extended operation field 850 according to one embodiment of the present invention. When the class (U) field 868 contains 0, it indicates EVEX.U0 (class A 868A); when it contains 1, it indicates EVEX.U1 (class B 868B). When U=0 and the MOD field 942 contains 11 (indicating a no memory access operation), the alpha field 852 (EVEX byte 3, bit [7] - EH) is interpreted as the rs field 852A. When the rs field 852A contains 1 (round 852A.1), the beta field 854 (EVEX byte 3, bits [6:4] - SSS) is interpreted as the round control field 854A. The round control field 854A includes a one-bit SAE field 856 and a two-bit round operation field 858. When the rs field 852A contains 0 (data transformation 852A.2), the beta field 854 (EVEX byte 3, bits [6:4]-SSS) is interpreted as a three-bit data transformation field 854B. When U=0 and the MOD field 942 contains 00, 01, or 10 (indicating a memory access operation), the alpha field 852 (EVEX byte 3, bits [7]-EH) is interpreted as an eviction hint (EH) field 852B, and the beta field 854 (EVEX byte 3, bits [6:4]-SSS) is interpreted as a three-bit data manipulation field 854C.

[0146] When U=1, the alpha field 852 (EVEX byte 3, bit [7]-EH) is interpreted as the write mask control (Z) field 852C. When U=1 and the MOD field 942 contains 11 (indicating a no memory access operation), a portion of the beta field 854 (EVEX byte 3, bit [4]-S0) is interpreted as the RL field 857A; when it contains 1 (rounded 857A.1), the remainder of the beta field 854 (EVEX byte 3, bits [6-5]-S0) is interpreted as the RL field 857A. 2-1 ) is interpreted as the rounding operation field 859A, and when the RL field 857A contains 0 (VSIZE 857.A2), the remainder of the beta field 854 (EVEX byte 3, bits [6-5] -S 2-1 ) is interpreted as the vector length field 859B (EVEX byte 3, bits [6-5] – L 1-0). When U=1 and the MOD field 942 contains 00, 01, or 10 (indicating a memory access operation), the beta field 854 (EVEX byte 3, bits [6:4] - SSS) is interpreted as the vector length field 859B (EVEX byte 3, bits [6-5] - L 1-0 ) and the broadcast field 857B (EVEX byte 3, bit [4]–B). Exemplary Register Architecture

[0147] Fig.10 1 is a block diagram of a register architecture 1000 according to one embodiment of the present invention. In the illustrated embodiment, there are 32 512-bit wide vector registers 1010; these registers are referenced as zmm0 through zmm31. The lower order 256 bits of the lower 16 zmm registers are overlaid on registers ymm0-16. The lower order 128 bits of the lower 16 zmm registers (the lower order 128 bits of the ymm registers) are overlaid on registers xmm0-15. The specific vector friendly instruction format 900 operates on these overlaid register files, as illustrated in the following table.

[0148] In other words, the vector length field 859B selects between a maximum length and one or more other shorter lengths, where each such shorter length is half the previous length, and instruction templates without a vector length field 859B operate on the maximum vector length. In addition, in one embodiment, the class B instruction templates of the dedicated vector friendly instruction format 900 operate on packed or scalar single / double precision floating point data and packed or scalar integer data. Scalar operations are operations performed on the lowest order data element position in the zmm / ymm / xmm register; depending on the embodiment, higher order data element positions either remain the same as before the instruction or are reset to zero.

[0149] Write mask registers 1015 - In the illustrated embodiment, there are 8 write mask registers (k0 to k7), each of which is 64 bits in size. In an alternative embodiment, the size of the write mask registers 1015 is 16 bits. As previously described, in one embodiment of the present invention, the vector mask register k0 cannot be used as a write mask; when the encoding that normally indicates k0 is used as a write mask, it selects a hardwired write mask of 0xFFFF, effectively disabling the use of a write mask for that instruction.

[0150] General purpose registers 1025 - In the embodiment shown, there are sixteen 64-bit general purpose registers that are used with existing x86 addressing modes to address memory operands. These registers are referenced by the names RAX, RBX, RCX, RDX, RBP, RSI, RDI, RSP, and R8 through R15.

[0151] A scalar floating point stack register file (x87 stack) 1045 on which an MMX packed integer flat register file 1050 is overlaid - in the illustrated embodiment, the x87 stack is an eight-element stack used to perform scalar floating point operations on 32 / 64 / 80-bit floating point data using the x87 instruction set extension; while MMX registers are used to perform operations on 64-bit packed integer data, as well as to hold operands for some operations performed between MMX and XMM registers.

[0152] Alternative embodiments may use wider or narrower registers. In addition, alternative embodiments may use more, fewer or different register files and registers. Exemplary Core Architectures, Processors, and Computer Architectures

[0153] Processor cores can be implemented in different ways, for different purposes, and in different processors. For example, implementations of such cores may include: 1) a general-purpose in-order core intended for general-purpose computing; 2) a high-performance general-purpose out-of-order core intended for general-purpose computing; 3) a special-purpose core intended primarily for graphics and / or scientific (throughput) computing. Implementations of different processors may include: 1) a CPU that includes one or more general-purpose in-order cores intended for general-purpose computing and / or one or more general-purpose out-of-order cores intended for general-purpose computing; and 2) a coprocessor that includes one or more special-purpose cores intended primarily for graphics and / or scientific (throughput). Such different processors lead to different computer system architectures, which may include: 1) a coprocessor on a separate chip from the CPU; 2) a coprocessor in the same package as the CPU but on a separate die; 3) a coprocessor on the same die as the CPU (in which case such a coprocessor is sometimes referred to as dedicated logic or as a dedicated core, such as integrated graphics and / or scientific (throughput) logic); and 4) a system on a chip, which may include the described CPU (sometimes referred to as application core(s) or application processor(s), the coprocessor described above, and additional functionality on the same die. An exemplary core architecture is described next, followed by a description of an exemplary processor and computer architecture. Exemplary Core Architecture In-order and out-of-order core block diagram

[0154] Fig.11Ais a block diagram illustrating an exemplary in-order pipeline and an exemplary register-renaming out-of-order issue / execution pipeline in accordance with some embodiments. Fig. 11B is a block diagram illustrating an exemplary embodiment of an in-order architecture core and an exemplary register renaming, out-of-order issue / execution architecture core to be included in a processor in accordance with some embodiments. Figure 11A-11B The solid line boxes in illustrate the in-order pipeline and in-order core, while the optional addition of dashed line boxes illustrates the register renaming, out-of-order issue / execution pipeline and core. Considering that the in-order aspect is a subset of the out-of-order aspect, the out-of-order aspect will be described.

[0155] exist Fig.11A , the processor pipeline 1100 includes a fetch stage 1102, a length decode stage 1104, a decode stage 1106, an allocation stage 1108, a rename stage 1110, a scheduling (also known as dispatch or issue) stage 1112, a register read / memory read stage 1114, an execute stage 1116, a write back / memory write stage 1118, an exception handling stage 1122, and a commit stage 1124.

[0156] Fig. 11B A processor core 1190 is shown, which includes a front end unit 1130, which is coupled to an execution engine unit 1150, and both the front end unit 1130 and the execution engine unit 1150 are coupled to a memory unit 1170. The core 1190 can be a reduced instruction set computing (RISC) core, a complex instruction set computing (CISC) core, a very long instruction word (VLIW) core, or a hybrid or alternative core type. As another option, the core 1190 can be a special-purpose core, such as, for example, a network or communication core, a compression engine, a coprocessor core, a general-purpose computing graphics processing unit (GPGPU) core, a graphics core, and the like.

[0157] The front end unit 1130 includes a branch prediction unit 1132, which is coupled to an instruction cache unit 1134, which is coupled to an instruction translation lookaside buffer (TLB) 1136, which is coupled to an instruction fetch unit 1138, which is coupled to a decode unit 1140. The decode unit 1140 (or decoder) can decode the instruction and generate one or more micro-operations, microcode entry points, microinstructions, other instructions, or other control signals decoded from the original instruction, or otherwise reflecting the original instruction, or derived from the original instruction as output. The decode unit 1140 can be implemented using a variety of different mechanisms. Examples of suitable mechanisms include, but are not limited to, lookup tables, hardware implementations, programmable logic arrays (PLA), microcode read-only memories (ROMs), etc. In one embodiment, the core 1190 includes a microcode ROM or other medium (e.g., in the decode unit 1140, or otherwise within the front end unit 1130) that stores microcode for certain macroinstructions. Decode unit 1140 is coupled to rename / allocator unit 1152 in execution engine unit 1150 .

[0158] The execution engine unit 1150 includes a rename / allocator unit 1152 coupled to a retirement unit 1154 and a set of one or more scheduler units 1156. The scheduler unit(s) 1156 represent any number of different schedulers, including reservation stations, central instruction windows, etc. The scheduler unit(s) 1156 are coupled to physical register file(s) units 1158. Each of the physical register file(s) units 1158 represents one or more physical register files, where different physical register files store one or more different data types, such as scalar integers, scalar floating points, packed integers, packed floating points, vector integers, vector floating points, status (e.g., an instruction pointer that is the address of the next instruction to be executed), etc. In one embodiment, the physical register file(s) units 1158 include a vector register unit, a write mask register unit, and a scalar register unit. These register units may provide architectural vector registers, vector mask registers, and general purpose registers. The physical register file(s) units 1158 are overlapped by the retirement unit 1154 to illustrate various ways in which register renaming and out-of-order execution may be implemented (e.g., using reorder buffer(s) and retirement register file(s); using future files(s), history buffer(s), retirement register file(s); using register maps and register pools, etc.). The retirement unit 1154 and the physical register file(s) units 1158 are coupled to the execution cluster(s) 1160. The execution cluster(s) 1160 include a set 1162 of one or more execution units and a set 1164 of one or more memory access units. The execution unit 1162 may perform various operations (e.g., shifts, additions, subtractions, multiplications) and may perform on various data types (e.g., scalar floating point, packed integer, packed floating point, vector integer, vector floating point). While some embodiments may include multiple execution units dedicated to a particular function or set of functions, other embodiments may include only one execution unit or multiple execution units that all perform all functions. Scheduler unit(s) 1156, physical register file(s) units 1158, and execution cluster(s) 1160 are shown as potentially multiple, as some embodiments create separate pipelines for certain types of data / operations (e.g., scalar integer pipelines, scalar floating point / packed integer / packed floating point / vector integer / vector floating point pipelines, and / or memory access pipelines each with its own scheduler unit, physical register file(s) units, and / or execution cluster—and in the case of separate memory access pipelines, certain embodiments are implemented where only the execution cluster of that pipeline has memory access unit(s) 1164). It should also be understood that where separate pipelines are used, one or more of these pipelines may be out-of-order issue / execution, and the rest may be in-order.

[0159] A set of memory access units 1164 is coupled to a memory unit 1170, which includes a data TLB unit 1172, which is coupled to a data cache unit 1174, which is coupled to a second level (L2) cache unit 1176. In an exemplary embodiment, the memory access unit 1164 may include a load unit, a store address unit, and a store data unit, each of which is coupled to the data TLB unit 1172 in the memory unit 1170. The instruction cache unit 1134 is also coupled to a second level (L2) cache unit 1176 in the memory unit 1170. The L2 cache unit 1176 is coupled to one or more other levels of cache and ultimately to the main memory.

[0160] As an example, an exemplary register renaming out-of-order issue / execution core architecture may implement pipeline 1100 as follows: 1) instruction fetch 1138 performs fetch stage 1102 and length decode stage 1104; 2) decode unit 1140 performs decode stage 1106; 3) rename / allocator unit 1152 performs allocation stage 1108 and rename stage 1110; 4) (multiple) scheduler units 1156 perform scheduling stage 1112; 5) (multiple) physical register file units 1158 and memory units 1170 perform register read / memory read stage 1114; execution cluster 1160 performs execution stage 1116; 6) memory unit 1170 and (multiple) physical register file units 1158 perform write back / memory write stage 1118; 7) each unit may be involved in an exception handling stage 1122; and 8) retirement unit 1154 and (multiple) physical register file units 1158 perform commit stage 1124.

[0161] Core 1190 may support one or more instruction sets (e.g., the x86 instruction set (with some extensions that have been added with newer versions); the MIPS instruction set of MIPS Technologies, Inc. of Sunnyvale, California; the ARM instruction set of ARM Holdings, Inc. of Sunnyvale, California (with optional additional extensions such as NEON)), including the instruction(s) described herein. In one embodiment, core 1190 includes logic to support packed data instruction set extensions (e.g., AVX1, AVX2), thereby allowing operations used by many multimedia applications to be performed using packed data.

[0162] It should be understood that a core may support multithreading (executing a set of two or more operations or threads in parallel), and that this multithreading may be accomplished in a variety of ways, including time-division multithreading, simultaneous multithreading (where a single physical core provides a logical core for each of the threads that the physical core is simultaneously multithreading), or a combination thereof (e.g., time-division fetching and decoding and subsequent operations such as Hyper-Threading (Simultaneous Multithreading).

[0163] Although register renaming is described in the context of out-of-order execution, it should be understood that register renaming can be used in an in-order architecture. Although the illustrated embodiment of the processor also includes separate instruction and data cache units 1134 / 1174 and a shared L2 cache unit 1176, alternative embodiments may have a single internal cache for both instructions and data, such as, for example, a first level (L1) internal cache or multiple levels of internal cache. In some embodiments, the system may include a combination of internal caches and external caches external to the core and / or processor. Alternatively, all caches may be external to the core and / or processor. Specific Exemplary In-Order Core Architecture

[0164] Figure 12A-12B A block diagram illustrating a more specific exemplary in-order core architecture, which would be one logic block among several logic blocks in a chip (including other cores of the same type and / or different types). Depending on the application, the logic block communicates with some fixed function logic, memory I / O interfaces, and other necessary I / O logic via a high bandwidth interconnect network (e.g., a ring network).

[0165] At least for reference Figure 3-6 11-12 illustrate and discuss execution circuits for executing SDMM instructions according to the disclosed embodiments.

[0166] Fig. 12A1 is a block diagram of a single processor core and its connection to an on-die interconnect network 1202 and its local subset 1204 of a second level (L2) cache according to some embodiments. In one embodiment, the instruction decoder 1200 supports the x86 instruction set with a packed data instruction set extension. The L1 cache 1206 allows low latency access to cache memory into the scalar and vector units. Although in one embodiment (to simplify the design), the scalar unit 1208 and the vector unit 1210 use separate register sets (scalar registers 1212 and vector registers 1214, respectively), and data transferred between these registers is written to memory and then read back from the first level (L1) cache 1206, alternative embodiments may use different approaches (e.g., using a single register set or including a communication path that allows data to be transferred between the two register files without being written and read back).

[0167] The local subset 1204 of the L2 cache is part of the global L2 cache, which is divided into multiple separate local subsets, one local subset for each processor core. Each processor core has a direct access path to its own local subset 1204 of the L2 cache. The data read by the processor core is stored in its L2 cache subset 1204 and can be quickly accessed in parallel with other processor cores accessing their own local L2 cache subsets. The data written by the processor core is stored in its own L2 cache subset 1204 and is cleared from other subsets when necessary. The ring network ensures the consistency of shared data. The ring network is bidirectional to allow agents such as processor cores, L2 caches and other logic blocks to communicate with each other within the chip. Each ring data path is 1012 bits wide in each direction.

[0168] Fig. 12B According to some embodiments Fig. 12A An expanded view of a portion of a processor core in FIG. Fig. 12B Includes the L1 data cache 1206A portion of the L1 cache 1204, as well as more details about the vector unit 1210 and vector registers 1214. Specifically, the vector unit 1210 is a 16-wide vector processing unit (VPU) (see 16-wide ALU 1228) that executes one or more of integer, single-precision floating point, and double-precision floating point instructions. The VPU supports blending of register inputs through blending unit 1220, numerical conversion through numerical conversion units 1222A-B, and copying of memory inputs through copy unit 1224. Write mask register 1226 allows asserted vector writes.

[0169] At least for reference Figure 3-611-12 illustrate and discuss execution circuits for executing SDMM instructions according to the disclosed embodiments.

[0170] Fig.13 is a block diagram of a processor 1300 that may have more than one core, may have an integrated memory controller, and may have integrated graphics, according to some embodiments. Fig.13 The solid line box in the figure illustrates a processor 1300 having a single core 1302A, a system agent 1310, a set 1316 of one or more bus controller units, while the optional addition of the dashed line box illustrates an alternative processor 1300 having multiple cores 1302A-N, a set 1314 of one or more integrated memory controller units in the system agent unit 1310, and dedicated logic 1308.

[0171] Thus, different implementations of processor 1300 may include: 1) a CPU, where the dedicated logic 1308 is integrated graphics and / or scientific (throughput) logic (which may include one or more cores), and cores 1302A-N are one or more general purpose cores (e.g., general purpose in-order cores, general purpose out-of-order cores, combinations of the two); 2) a coprocessor, where cores 1302A-N are a large number of dedicated cores intended primarily for graphics and / or scientific (throughput); and 3) a coprocessor, where cores 1302A-N are a large number of general purpose in-order cores. Thus, processor 1300 may be a general purpose processor, a coprocessor, or a special purpose processor, such as, for example, a network or communication processor, a compression engine, a graphics processor, a GPGPU (general purpose graphics processing unit), a high throughput integrated many-core (MIC) coprocessor (including 30 or more cores), an embedded processor, and the like. The processor may be implemented on one or more chips. Processor 1300 may be part of one or more substrates and / or may be implemented on one or more substrates using any of a number of process technologies (such as, for example, BiCMOS, CMOS, or NMOS).

[0172] The memory hierarchy includes one or more levels of cache within the core, a set of one or more shared cache units 1306, and external memory (not shown) coupled to a set of integrated memory controller units 1314. The set of shared cache units 1306 may include one or more intermediate levels of cache, such as a second level (L2), a third level (L3), a fourth level (L4), or other levels of cache, a last level cache (LLC), and / or combinations thereof. While in one embodiment, a ring-based interconnect unit 1312 interconnects the integrated graphics logic 1308 (the integrated graphics logic 1308 is an example of dedicated logic and is also referred to herein as dedicated logic), the set of shared cache units 1306, and the system agent unit 1310 / (multiple) integrated memory controller units 1314, alternative embodiments may interconnect such units using any number of well-known techniques. In one embodiment, coherency is maintained between the one or more cache units 1306 and the cores 1302A-N.

[0173] In some embodiments, one or more of the cores 1302A-N may be multithreaded. The system agent 1310 includes those components that coordinate and operate the cores 1302A-N. The system agent unit 1310 may include, for example, a power control unit (PCU) and a display unit. The PCU may be, or may include, the logic and components required to regulate the power state of the cores 1302A-N and the integrated graphics logic 1308. The display unit is used to drive one or more externally connected displays.

[0174] The cores 1302A-N may be homogeneous or heterogeneous in terms of the architectural instruction set; that is, two or more of the cores 1302A-N may be capable of executing the same instruction set, while other cores may be capable of executing only a subset of the instruction set or a different instruction set. Exemplary Computer Architecture

[0175] Figure 14-17 is a block diagram of an exemplary computer architecture. Other system designs and configurations known in the art for laptops, desktops, handheld PCs, personal digital assistants, engineering workstations, servers, network devices, network hubs, switches, embedded processors, digital signal processors (DSPs), graphics devices, video game devices, set-top boxes, microcontrollers, cellular phones, portable media players, handheld devices, and various other electronic devices are also suitable. In general, a wide variety of systems or electronic devices that can contain processors and / or other execution logic as disclosed herein are generally suitable.

[0176] Reference now Fig.14, shown is a block diagram of a system 1400 according to one embodiment of the present invention. The system 1400 may include one or more processors 1410, 1415 coupled to a controller hub 1420. In one embodiment, the controller hub 1420 includes a graphics memory controller hub (GMCH) 1490 and an input / output hub (IOH) 1450 (which may be on separate chips); the GMCH 1490 includes memory and a graphics controller to which a memory 1440 and a coprocessor 1445 are coupled; and the IOH 1450 couples an input / output (I / O) device 1460 to the GMCH 1490. Alternatively, one or both of the memory and the graphics controller are integrated within a processor (as described herein), the memory 1440 and the coprocessor 1445 are directly coupled to the processor 1410, and the controller hub 1420 and the IOH 1450 are in a single chip.

[0177] The optional nature of the additional processor 1415 is Fig.14 Each processor 1410 , 1415 may include one or more of the processing cores described herein, and may be a version of processor 1300 .

[0178] The memory 1440 may be, for example, a dynamic random access memory (DRAM), a phase change memory (PCM), or a combination of the two. For at least one embodiment, the controller hub 1420 communicates with the processor(s) 1410, 1415 via a multi-drop bus such as a front-side bus (FSB), a point-to-point interface such as a Quick Path Interconnect (QPI), or a similar connection 1495.

[0179] In one embodiment, coprocessor 1445 is a special purpose processor such as, for example, a high throughput MIC processor, a network or communication processor, a compression engine, a graphics processor, a GPGPU, an embedded processor, etc. In one embodiment, controller hub 1420 may include an integrated graphics accelerator.

[0180] There may be various differences between the physical resources 1410 , 1415 in terms of a range of quality metrics including architectural, microarchitectural, thermal, power consumption characteristics, etc.

[0181] In one embodiment, processor 1410 executes instructions that control general types of data processing operations. Embedded within these instructions may be coprocessor instructions. Processor 1410 recognizes these coprocessor instructions as being of a type that should be executed by an attached coprocessor 1445. Accordingly, processor 1410 issues these coprocessor instructions (or control signals representing coprocessor instructions) to coprocessor 1445 on a coprocessor bus or other interconnect. Coprocessor(s) 1445 accept and execute the received coprocessor instructions.

[0182] See now Fig.15 , shown is a block diagram of a first more specific exemplary system 1500 according to an embodiment of the present invention. Fig.15 As shown in FIG. 1 , multiprocessor system 1500 is a point-to-point interconnect system and includes a first processor 1570 and a second processor 1580 coupled via a point-to-point interconnect 1550. Each of processors 1570 and 1580 may be a version of processor 1300. In one embodiment of the present invention, processors 1570 and 1580 are processors 1410 and 1415, respectively, and coprocessor 1538 is coprocessor 1445. In another embodiment, processors 1570 and 1580 are processor 1410 and coprocessor 1445, respectively.

[0183] Processors 1570 and 1580 are shown as including integrated memory controller (IMC) units 1572 and 1582, respectively. Processor 1570 also includes point-to-point (PP) interfaces 1576 and 1578 as part of its bus controller unit; similarly, second processor 1580 includes PP interfaces 1586 and 1588. Processors 1570, 1580 can exchange information via PP interface 1550 using point-to-point (PP) interface circuits 1578, 1588. Fig.15 As shown in FIG. 1 , IMCs 1572 and 1582 couple the processors to respective memories, namely, memory 1532 and memory 1534, which may be portions of main memory locally attached to the respective processors.

[0184] Processors 1570, 1580 may each exchange information with chipset 1590 via respective PP interfaces 1552, 1554 using point-to-point interface circuits 1576, 1594, 1586, 1598. Chipset 1590 may optionally exchange information with coprocessor 1538 via high performance interface 1592. In one embodiment, coprocessor 1538 is a special purpose processor such as, for example, a high throughput MIC processor, a network or communication processor, a compression engine, a graphics processor, a GPGPU, an embedded processor, or the like.

[0185] A shared cache (not shown) may be included in either processor, or external to both processors but connected to the processors via the PP interconnect, so that if the processors are placed in a low power mode, local cache information of either or both processors may be stored in the shared cache.

[0186] Chipset 1590 may be coupled to first bus 1516 via interface 1596. In one embodiment, first bus 1516 may be a peripheral component interconnect (PCI) bus or a bus such as a PCI Express bus or another third generation I / O interconnect bus, but the scope of the invention is not limited in this regard.

[0187] like Fig.15 As shown in , various I / O devices 1514 may be coupled to the first bus 1516 along with a bus bridge 1518 that couples the first bus 1516 to a second bus 1520. In one embodiment, one or more additional processors 1515 such as a coprocessor, a high throughput MIC processor, a GPGPU, an accelerator (such as, for example, a graphics accelerator or a digital signal processing (DSP) unit), a field programmable gate array, or any other processor are coupled to the first bus 1516. In one embodiment, the second bus 1520 may be a low pin count (LPC) bus. In one embodiment, various devices may be coupled to the second bus 1520 including, for example, a keyboard and / or mouse 1522, communication devices 1527, and a storage unit 1528 such as a disk drive or other mass storage device which may include instructions / code and data 1530. Additionally, an audio I / O 1524 may be coupled to the second bus 1520. Note that other architectures are possible. For example, instead of Fig.15 Instead of a point-to-point architecture, the system may implement a multi-drop bus or other such architecture.

[0188] Reference now Fig.16 , shown is a block diagram of a second more specific exemplary system 1600 according to an embodiment of the present invention. Fig.15 and 16 Similar elements in the same reference numerals are used, and Fig.16 Omitted Fig.15 to avoid confusion Fig.16 other aspects.

[0189] Fig.16 The illustrated processors 1570, 1580 may include integrated memory and I / O control logic ("CL") 1572 and 1582, respectively. Thus, the CL 1572, 1582 includes an integrated memory controller unit and includes I / O control logic. Fig.16The illustration shows not only that memories 1532, 1534 are coupled to CL 1572, 1582, but also that I / O devices 1614 are coupled to control logic 1572, 1582. Legacy I / O devices 1615 are coupled to chipset 1590.

[0190] Reference now Fig.17 , shown is a block diagram of a SoC 1700 according to an embodiment of the present invention. Fig.13 Similar elements in the same figure are marked with similar reference numerals. In addition, the dashed boxes are optional features on more advanced SoCs. Fig.17 , interconnect unit(s) 1702 are coupled to: application processor 1710, which includes a set of one or more cores 1302A-N and a shared cache unit(s) 1306, the set of one or more cores 1302A-N including cache units 1304A-N; system agent unit 1310; bus controller unit(s) 1316; integrated memory controller unit(s) 1314; a set of one or more coprocessors 1720, which may include integrated graphics logic, image processors, audio processors, and video processors; static random access memory (SRAM) unit 1730; direct memory access (DMA) unit 1732; and a display unit 1740 for coupling to one or more external displays. In one embodiment, coprocessor(s) 1720 include special purpose processors such as, for example, network or communication processors, compression engines, GPGPUs, high throughput MIC processors, or embedded processors, among others.

[0191] The various embodiments of the mechanisms disclosed herein may be implemented in hardware, software, firmware, or a combination of such implementations. Some embodiments may be implemented as a computer program or program code executed on a programmable system comprising at least one processor, a storage system (including volatile and non-volatile memory and / or storage elements), at least one input device, and at least one output device.

[0192] Program code (such as Fig.15 The code 1530 illustrated in FIG. 15 is applied to input instructions to perform the functions described herein and generate output information. The output information can be applied to one or more output devices in a known manner. For the purposes of this application, a processing system includes any system having a processor, such as, for example, a digital signal processor (DSP), a microcontroller, an application specific integrated circuit (ASIC), or a microprocessor.

[0193] The program code can be implemented with a high-level process-oriented programming language or an object-oriented programming language to communicate with the processing system. If necessary, the program code can also be implemented with assembly language or machine language. In fact, the mechanism described herein is not limited to the scope of any programming language. In any case, the language can be a compiled language or an interpreted language.

[0194] One or more aspects of at least one embodiment may be implemented by representative instructions stored on a machine-readable medium, which represent various logic in a processor, which when read by a machine causes the machine to fabricate logic for performing the techniques described herein. Such representations, known as "IP cores," may be stored on a tangible, machine-readable medium and supplied to various customers or manufacturing facilities to load into a manufacturing machine that manufactures the logic or processor.

[0195] Such machine-readable storage media may include, but are not limited to, a non-transitory, tangible arrangement of an article manufactured or formed by a machine or apparatus, including storage media such as a hard disk; any other type of disk, including a floppy disk, an optical disk, a compact disk-read only memory (CD-ROM), a compact disk-rewritable (CD-RW), and a magneto-optical disk; semiconductor devices, such as read-only memory (ROM), random access memory (RAM) such as dynamic random access memory (DRAM) and static random access memory (SRAM), erasable programmable read-only memory (EPROM), flash memory, electrically erasable programmable read-only memory (EEPROM); phase change memory (PCM); a magnetic card or an optical card; or any other type of medium suitable for storing electronic instructions.

[0196] Therefore, some embodiments also include non-transitory tangible machine-readable media containing instructions or containing design data, such as hardware description language (HDL), which defines the structures, circuits, devices, processors and / or system features described herein. These embodiments are also referred to as program products. Simulation (including binary conversion, code deformation, etc.)

[0197] In some cases, the instruction converter can be used to convert instructions from a source instruction set to a target instruction set. For example, the instruction converter can convert (e.g., using static binary conversion, dynamic binary conversion including dynamic compilation), deform, simulate or otherwise convert instructions into one or more other instructions to be processed by the core. The instruction converter can be implemented in software, hardware, firmware, or a combination thereof. The instruction converter can be on the processor, outside the processor, or partially on the processor and partially outside the processor.

[0198] Fig.181 is a block diagram of a method for converting binary instructions in a source instruction set into binary instructions in a target instruction set using a software instruction converter according to some embodiments. In the illustrated embodiment, the instruction converter is a software instruction converter, but alternatively, the instruction converter can be implemented in software, firmware, hardware, or various combinations thereof. Fig.18 It is shown that a program in a high-level language 1802 can be compiled using an x86 compiler 1804 to generate x86 binary code 1806 that can be natively executed by a processor 1816 having at least one x86 instruction set core. A processor 1816 having at least one x86 instruction set core represents any processor that performs substantially the same functions as an Intel processor having at least one x86 instruction set core by compatibly executing or otherwise processing: 1) a substantial portion of the instruction set of an Intel x86 instruction set core, or 2) an object code version of an application or other software that is targeted to run on an Intel processor having at least one x86 instruction set core to achieve substantially the same results as an Intel processor having at least one x86 instruction set core. The x86 compiler 1804 represents a compiler that can be operated to generate x86 binary code 1806 (e.g., object code) that can be executed on a processor 1816 having at least one x86 instruction set core with or without additional linking processing. Similarly, Fig.18 It is shown that a program in a high-level language 1802 can be compiled using an alternative instruction set compiler 1808 to generate an alternative instruction set binary code 1810 that can be natively executed by a processor 1814 that does not have at least one x86 instruction set core (e.g., a processor that has a core that executes the MIPS instruction set of MIPS Technologies, Inc. of Sunnyvale, California, and / or executes the ARM instruction set of ARM Holdings, Inc. of Sunnyvale, California). The instruction converter 1812 is used to convert the x86 binary code 1806 into code that can be natively executed by the processor 1814 that does not have an x86 instruction set core. The converted code is unlikely to be the same as the alternative instruction set binary code 1810 because an instruction converter that can do so is difficult to manufacture; however, the converted code will complete the general operation and be composed of instructions from the alternative instruction set. Therefore, the instruction converter 1812 represents software, firmware, hardware, or a combination thereof that allows a processor or other electronic device that does not have an x86 instruction set processor or core to execute the x86 binary code 1806 through simulation, emulation, or any other process. Further examples

[0199] Example 1 provides an exemplary processor for executing a sparse-dense matrix multiplication instruction, comprising: a fetch circuit for fetching a sparse-dense matrix multiplication instruction from a code storage, the sparse-dense matrix multiplication instruction having fields for specifying an opcode, a dense output matrix, a dense source matrix, and a sparse source matrix, the sparse source matrix having a sparsity of non-zero elements, the sparsity being less than one; a decoding circuit for decoding the fetched sparse-dense matrix multiplication instruction; an execution circuit for executing the decoded sparse-dense matrix multiplication instruction to perform the following operations for each non-zero element at row M and column K of the sparse source matrix: generating the product of the non-zero element and each corresponding dense element at row K and column N of the dense source matrix; and generating a cumulative sum of each generated product and the previous value of the corresponding output element at row M and column N of the dense output matrix.

[0200] Example 2 includes the substance of the example processor of Example 1, wherein the sparsity is 0.125 or less.

[0201] Example 3 includes the substance of the exemplary processor of Example 1, wherein the sparse-dense matrix multiplication instruction further specifies a size of data elements of the specified matrix, which is one of 8 bits, 16 bits, 32 bits, 64 bits, and 128 bits, and the size is specified as part of an operand or opcode of the instruction.

[0202] Example 4 includes the substance of the exemplary processor of Example 1, wherein the sparse-dense matrix multiplication instruction further specifies a format of data elements of the specified matrix, which is one of fixed point, single precision floating point, double precision floating point, extended precision floating point, and extended double precision floating point.

[0203] Example 5 includes the substance of the exemplary processor of any one of Examples 1-4, wherein the execution circuit generates products having the same size as the data elements of the specified matrix using the multipliers, generates accumulated sums having a size twice the size of the data elements of the specified matrix using the accumulators, and performs saturation and rounding on each of the accumulated sums before writing the saturated and rounded accumulated sums to the specified dense output matrix.

[0204] Example 6 includes the substance of the exemplary processor of any of Examples 1-4, wherein the execution circuit includes a single instruction multiple data (SIMD) execution circuit, the SIMD execution circuit including multiple multipliers and multiple accumulators to perform decoded sparse-dense matrix multiplication on multiple non-zero elements of a sparse source matrix in parallel, the multiple non-zero elements including one of 64 elements, 128 elements, 256 elements, and 512 elements.

[0205] Example 7 includes the substance of the exemplary processor of any one of Examples 1-4, further comprising: a memory read circuit for reading the specified matrix and writing the result to the memory, and wherein the sparse source matrix is ​​stored in the memory in a compressed format, the compressed format includes only the non-zero elements of the sparse source matrix, each non-zero element is represented by a data value and a matrix position, and wherein the memory read circuit reads the sparse source matrix in a compressed format.

[0206] Example 8 includes the substance of the exemplary processor of Example 7, wherein the compressed format is a compressed sparse row (CSR) format that stores non-zero elements in row-major order or a compressed sparse column (CSC) format that stores non-zero elements in column-major order.

[0207] Example 9 includes the substance of the exemplary processor of Example 8, wherein the sparse source matrix includes a predetermined number of rows and a predetermined number of columns, and wherein the matrix position specifies an index of a non-zero element within the sparse source matrix.

[0208] Example 10 includes the substance of the exemplary processor of Example 8, wherein the sparse source matrix includes a predetermined number of rows and a predetermined number of columns, wherein when the CSR format is used, the compressed format further specifies the number of non-zero elements in each row and the matrix positions are specified as the indices of the non-zero elements within the row, and wherein when the CSC format is used, the compressed format further specifies the number of non-zero elements in each column and the matrix positions are specified as the indices of the non-zero elements within the column.

[0209] Example 11 provides an exemplary method for executing a sparse-dense matrix multiplication instruction, comprising: using a fetch circuit to fetch a sparse-dense matrix multiplication instruction from a code storage, the sparse-dense matrix multiplication instruction having fields for specifying an opcode, a dense output matrix, a dense source matrix, and a sparse source matrix, the sparse source matrix having a sparsity of non-zero elements, the sparsity being less than one; decoding the fetched sparse-dense matrix multiplication instruction by a decoding circuit; executing the decoded sparse-dense matrix multiplication instruction by an execution circuit to perform the following operations for each non-zero element at row M and column K of the sparse source matrix: generating the product of the non-zero element and each corresponding dense element at row K and column N of the dense source matrix; and generating the cumulative sum of each generated product and the previous value of the corresponding output element at row M and column N of the dense output matrix.

[0210] Example 12 includes the substance of the exemplary method of Example 11, wherein the sparsity is 0.125 or less.

[0211] Example 13 includes the substance of the exemplary method of Example 11, wherein the sparse-dense matrix multiplication instruction further specifies a size of data elements of the specified matrix, which size is one of 8 bits, 16 bits, 32 bits, 64 bits, and 128 bits, and the size is specified as part of an operand or opcode of the instruction.

[0212] Example 14 includes the substance of the exemplary method of Example 11, wherein the sparse-dense matrix multiplication instruction further specifies a format of data elements of the specified matrix, which is one of fixed point, single precision floating point, double precision floating point, extended precision floating point, and extended double precision floating point.

[0213] Example 15 includes the substance of the exemplary method of any of Examples 11-14, wherein the execution circuit uses a multiplier to generate products having the same size as the data elements of a specified matrix, uses an accumulator to generate accumulated sums having a size twice the size of the data elements of the specified matrix, and performs saturation and rounding on each of the accumulated sums before writing the saturated and rounded accumulated sums to the specified dense output matrix.

[0214] Example 16 includes the substance of the exemplary method of any one of Examples 11-14, wherein the execution circuit includes a single instruction multiple data (SIMD) execution circuit, the SIMD execution circuit including multiple multipliers and multiple accumulators to perform decoded sparse-dense matrix multiplication on multiple non-zero elements of a sparse source matrix in parallel, the multiple non-zero elements including one of 64 elements, 128 elements, 256 elements, and 512 elements.

[0215] Example 17 includes the substance of the exemplary method of any one of Examples 11-14, further comprising: a memory read circuit for reading the specified matrix and writing the result to the memory, and wherein the sparse source matrix is ​​stored in the memory in a compressed format, the compressed format includes only the non-zero elements of the sparse source matrix, each non-zero element is represented by a data value and a matrix position, and wherein the memory read circuit reads the sparse source matrix in a compressed format.

[0216] Example 18 includes the substance of the exemplary method of Example 17, wherein the compressed format is a compressed sparse row (CSR) format that stores non-zero elements in row-major order or a compressed sparse column (CSC) format that stores non-zero elements in column-major order.

[0217] Example 19 includes the substance of the exemplary method of Example 18, wherein the sparse source matrix comprises a predetermined number of rows and a predetermined number of columns, and wherein the matrix position specifies the index of the non-zero element within the sparse source matrix.

[0218] Example 20 includes the substance of the exemplary method of Example 18, wherein the sparse source matrix comprises a predetermined number of rows and a predetermined number of columns, wherein when the CSR format is used, the compressed format further specifies the number of non-zero elements in each row and the matrix positions are specified as the indices of the non-zero elements within the row, and wherein when the CSC format is used, the compressed format further specifies the number of non-zero elements in each column and the matrix positions are specified as the indices of the non-zero elements within the column.

[0219] Example 21 provides an exemplary system for executing a sparse-dense matrix multiplication (SDMM) instruction, comprising: a memory; and a processor, the processor comprising: a device for retrieving an SDMM instruction from a code storage, the SDMM instruction having fields for specifying an opcode, a dense output matrix, a dense source matrix, and a sparse source matrix, the sparse source matrix having a sparsity of non-zero elements, the sparsity being less than one; a device for decoding the retrieved SDMM instruction; and a device for executing the decoded SDMM instruction to perform the following operations for each non-zero element at row M and column K of the sparse source matrix: generate the product of the non-zero element and each corresponding dense element at row K and column N of the dense source matrix; and generate the cumulative sum of each generated product and the previous value of the corresponding output element at row M and column N of the dense output matrix.

[0220] Example 22 includes the substance of the exemplary system of Example 21, wherein the SDMM instruction further specifies a size of data elements of the specified matrix, the size being one of 8 bits, 16 bits, 32 bits, 64 bits, and 128 bits, the size being specified as part of an operand or opcode of the instruction.

[0221] Example 23 includes the substance of the exemplary system of Example 21, wherein the apparatus for performing generates products having the same size as the data elements of a specified matrix using multipliers, generates accumulated sums having a size twice the size of the data elements of the specified matrix using accumulators, and performs saturation and rounding on each of the accumulated sums before writing the saturated and rounded accumulated sums to the specified dense output matrix.

[0222] Example 24 includes the substance of the exemplary system of Example 21, wherein the sparse source matrix is ​​stored in a memory in a compressed format, the compressed format includes only the non-zero elements of the sparse source matrix, each non-zero element is represented by a data value and a matrix position, and wherein the compressed format is a compressed sparse row (CSR) format that stores the non-zero elements in row-major order or a compressed sparse column (CSC) format that stores the non-zero elements in column-major order.

[0223] Example 25 provides an exemplary non-transitory machine-readable medium, comprising instructions that, when executed by a processor, cause the processor to perform a sparse-dense matrix multiplication (SDMM) instruction by: using a fetch circuit to fetch the SDMM instruction from code storage, the SDMM instruction having fields for specifying an opcode, a dense output matrix, a dense source matrix, and a sparse source matrix, the sparse source matrix having a sparsity of non-zero elements that is less than one; decoding the fetched SDMM instruction by a decoding circuit; and executing the decoded SDMM instruction by an execution circuit to perform the following operations for each non-zero element at row M and column K of the sparse source matrix: generating the product of the non-zero element with each corresponding dense element at row K and column N of the dense source matrix; and generating a cumulative sum of each generated product with the previous value of the corresponding output element at row M and column N of the dense output matrix.

[0224] Example 26 includes the substance of the exemplary non-transitory machine-readable medium of Example 25, wherein the execution circuit uses a multiplier to generate products having the same size as the data elements of a specified matrix, uses an accumulator to generate accumulated sums having a size twice the size of the data elements of the specified matrix, and performs saturation and rounding on each of the accumulated sums before writing the saturated and rounded accumulated sums to the specified dense output matrix.

[0225] Example 27 includes the substantial contents of the exemplary non-transitory machine-readable medium of Example 25, wherein the sparse source matrix is ​​stored in a memory in a compressed format, the compressed format comprising only the non-zero elements of the sparse source matrix, each non-zero element being represented by a data value and a matrix position, and wherein the compressed format is a compressed sparse row (CSR) format storing the non-zero elements in row-major order or a compressed sparse column (CSC) format storing the non-zero elements in column-major order.

[0226] Example 28 includes the substance of the exemplary non-transitory machine-readable medium of Example 25, wherein the execution circuit includes a single instruction multiple data (SIMD) execution circuit, the SIMD execution circuit including multiple multipliers and multiple accumulators to perform the decoded SDMM on multiple non-zero elements of the sparse source matrix in parallel, the multiple non-zero elements including one of 16 elements, 32 elements and 64 elements.

[0227] Example 29 includes the substance of the exemplary non-transitory machine-readable medium of Example 25, wherein the sparse source matrix is ​​stored in a memory in a compressed format, the compressed format includes only non-zero elements of the sparse source matrix, each non-zero element is represented by a data value and a matrix position, and wherein a memory read circuit reads the sparse source matrix in a compressed format, and wherein the compressed format is a compressed sparse row (CSR) format that stores non-zero elements in row-major order or a compressed sparse column (CSC) format that stores non-zero elements in column-major order.

Claims

1. A processor, comprising: fetching circuitry, configured to fetch a sparse-dense matrix multiplication SDMM instruction from a memory, the SDMM instruction having fields for specifying an operation code, a result matrix, a first source matrix, and a second source matrix, the operation code being configured to indicate a sparse-dense matrix multiplication operation, the result matrix having a matrix data element dimension of M x N, the first source matrix having a source matrix data element dimension of K x N, the second source matrix representing a sparse source matrix having a source matrix data element dimension of M x K, wherein the matrix data elements in the K dimension of the sparse source matrix include zero-valued data elements and remaining data elements, wherein in dimension K, the second source matrix is ​​used to include the remaining data elements, the remaining data elements having position values ​​associated therewith; a plurality of registers for storing a plurality of source data elements of the first source matrix, for storing the remaining data elements of the second source matrix, and for storing the position values ​​associated with the remaining data elements, each position value being associated with a position of a corresponding one of the remaining data elements within the sparse source matrix; as well as The execution circuit is configured to perform the following operations in response to the SDMM instruction: multiply the remaining data elements with corresponding data elements of the first source matrix according to the position value to generate a plurality of products, and accumulate subsets of the plurality of products to generate corresponding data elements of the result matrix.

2. The processor of claim 1, wherein: Each position value indicates the position of a corresponding remaining data element within the sparse source matrix.

3. The processor of claim 2, wherein: The execution circuit is used to use each position value to identify the position of the corresponding remaining data element within the sparse source matrix in response to the SDMM instruction, and the position is used to indicate to the execution circuit one or more source data elements of the first source matrix to be multiplied with the corresponding remaining data element.

4. The processor of claim 1, wherein: The SDMM instruction further specifies the size and data format of the data elements of: the sparse source matrix, the first source matrix, and the result matrix.

5. The processor of claim 1, wherein: The SDMM instruction specifies a data element size and format of 16-bit floating point FP16 for the second source matrix and the first source matrix, and specifies a data element size and format of 32-bit floating point FP32 for the result matrix.

6. The processor of claim 1, wherein: The SDMM instruction specifies a data element size and format of 8-bit integers for the second source matrix and the first source matrix, and specifies a data element size and format of 32-bit integers for the result matrix.

7. The processor of claim 1, wherein: The execution circuit comprises: A plurality of multipliers are configured to operate in parallel, each multiplier being configured to multiply a data element of the first source matrix with corresponding remaining data elements of the second source matrix according to a corresponding position value to generate a corresponding product of the plurality of products.

8. The processor of claim 1, wherein: The execution circuit comprises a single instruction multiple data (SIMD) execution circuit, and the SIMD execution circuit comprises a plurality of channels, and is used for executing the SDMM instructions in parallel in the plurality of channels.

9. The processor of claim 1, further comprising: A memory access circuit is configured to read the first source matrix and read one of the sparse source matrix and the second source matrix, and write the result to a memory, wherein the second source matrix is ​​stored in the memory in a compressed format, the compressed format comprising only the remaining data elements of the sparse source matrix, each remaining data element being used to be represented by a data value and a position value, and wherein the execution circuit reads the second source matrix in the compressed format.

10. The processor of claim 1, wherein: Each group of four matrix data elements in the K dimension comprises two zero-valued data elements and two remaining data elements, and wherein in the K dimension, the second source matrix is ​​used to include the two remaining data elements for each group of four matrix data elements, the two remaining data elements having position values ​​associated therewith.

11. A method comprising: fetching a sparse-dense matrix multiplication SDMM instruction from a memory, the SDMM instruction having fields for specifying an operation code, a result matrix, a first source matrix, and a second source matrix, the operation code being used to indicate a sparse-dense matrix multiplication operation, the result matrix having a matrix data element dimension of M x N, the first source matrix having a source matrix data element dimension of K x N, the second source matrix representing a sparse source matrix having a source matrix data element dimension of M x K, wherein the matrix data elements in the K dimension of the sparse source matrix include zero-valued data elements and remaining data elements, wherein in dimension K, the second source matrix is ​​used to include the remaining data elements, the remaining data elements having position values ​​associated therewith; storing a plurality of source data elements of the first source matrix, the remaining data elements of the second source matrix, and the position values ​​associated with the remaining data elements, each position value being associated with a position of a corresponding one of the remaining data elements within the sparse source matrix; and In response to the SDMM instruction, the remaining data elements are multiplied with corresponding data elements of the first source matrix according to the position values ​​to generate a plurality of products, and subsets of the plurality of products are accumulated to generate corresponding data elements of the result matrix.

12. A computer-readable medium comprising instructions which, when executed by a computer, cause the computer to perform the method of claim 11.

13. A processor core, comprising: Instruction decoder; a scalar unit coupled to the instruction decoder, the scalar unit using a scalar register; a vector unit coupled to the instruction decoder, the vector unit using vector registers; and an L1 cache that allows low latency access to the scalar registers and the vector registers, The processor core uses a local subset of the global L2 cache and has a direct access path to the local subset.

14. The processor core according to claim 13, wherein: The vector unit is a 16-wide vector processing unit VPU that executes one or more of integer, single-precision floating point, and double-precision floating point instructions.

15. A method comprising: Compiling a program in a high-level programming language using an x86 compiler to generate x86 binary code that is natively executed by a first processor having at least one x86 instruction set core; Using an instruction converter, the x86 binary code is converted into an alternative binary code that can be natively executed by a second processor that does not have an x86 instruction set core.

Images