System and method for executing instructions to transform a matrix into a row-interleaved format

Abstract

Systems and methods for executing an instruction to transform a matrix into a row interleaved format are disclosed. The disclosed embodiments relate to systems and methods for executing an instruction to transform a matrix into a row interleaved format. In one example, a processor includes fetch and decode circuitry to fetch and decode an instruction having fields to specify an opcode and locations of a source and destination matrix, where the opcode indicates the processor to transform the specified source matrix into the specified destination matrix having a row interleaved format, and execution circuitry to respond to the decoded instruction by transforming the specified source matrix into the specified RowInt formatted destination matrix by interleaving J elements of each J element sub-column of the specified source matrix into K width sub-matrices of the specified destination matrix in a row major or column major order, the K width sub-matrices having K columns and enough rows to hold the J elements.

Description

[0001] This invention patent application is a divisional application of the invention patent application filed on August 27, 2019, with application number 201910795336.4 and title "System and method for executing instructions to transform a matrix into a row interleaved format". Technical Field

[0002] The field of this invention relates generally to computer processor architecture, and more specifically to systems and methods for executing instructions for transforming a matrix into a row-interlaced format. Background Technology

[0003] Matrices are becoming increasingly important in many computational tasks, such as machine learning and other batch data processing. Deep learning is a class of machine learning algorithms. Deep learning architectures, such as deep neural networks, have been applied in fields including computer vision, speech recognition, natural language processing, audio recognition, social network filtering, machine translation, bioinformatics, and drug design.

[0004] The two tools used for deep learning—inference and training—tend to rely on low-precision arithmetic. Maximizing the throughput of deep learning algorithms and computations can help meet the demands of deep learning processors, such as those executing deep learning in data centers.

[0005] Matrix-matrix multiplication (also known as GEMM or generalized matrix multiplication) is a common computationally intensive operation on today's processors. Specialized hardware for matrix multiplication (e.g., GEMM) is a good choice for improving peak computation (and energy efficiency) in certain applications such as deep learning. Some of these applications, including deep learning, can operate on input data elements with relatively few bits without sacrificing accuracy, provided that the output element has enough bits (i.e., more than the input). Attached Figure Description

[0006] The invention is illustrated by way of example and not limitation in the accompanying drawings, in which similar reference numerals indicate similar elements, wherein:

[0007] Figure 1A The illustration shows an embodiment of the configured chips;

[0008] Figure 1B The illustration shows an embodiment of the configured chips;

[0009] Figure 2 Several examples of matrix storage are illustrated;

[0010] Figure 3 The illustration shows an embodiment of a system that utilizes a matrix (slice) to operate an accelerator;

[0011] Figure 4and Figure 5 Different embodiments of how to use a matrix operation accelerator to share memory are shown;

[0012] Figure 6 The illustration shows an example of using slice matrix multiplication accumulation operation (“TMMA”);

[0013] Figure 7 This illustration shows an example of an iterative execution subset of chained fused multiplication-accumulation instructions;

[0014] Figure 8 This illustration shows an example of an iterative execution subset of chained fused multiplication-accumulation instructions;

[0015] Figure 9 This illustration shows an example of an iterative execution subset of chained fused multiplication-accumulation instructions;

[0016] Figure 10 This illustration shows an example of an iterative execution subset of chained fused multiplication-accumulation instructions;

[0017] Figure 11 The illustration shows a SIMD implementation of a power of 2 according to an embodiment, wherein the accumulator uses an input size larger than the size of the input to the multiplier;

[0018] Figure 12 The diagram illustrates an embodiment of a system utilizing a matrix manipulation circuit;

[0019] Figure 13 The illustration shows an embodiment of a processor core pipeline that supports matrix operations using chips;

[0020] Figure 14 The illustration shows an embodiment of a processor core pipeline that supports matrix operations using chips;

[0021] Figure 15 The diagram shows examples of matrices expressed in row-major and column-major formats.

[0022] Figure 16 Illustrated examples of the use of matrices (slices);

[0023] Figure 17 An example of how to use a matrix (piece) is illustrated;

[0024] Figure 18 The illustration shows the configuration support for the use of chips according to an embodiment;

[0025] Figure 19 The illustration shows an embodiment describing the supported matrices (slices);

[0026] Figures 20(A)-20(D) An example of (multiple) registers;

[0027] Figure 21 This is a block diagram illustrating the use of the TILETFM2RI instruction to accelerate matrix multiplication according to some embodiments;

[0028] Figure 22A The illustration shows an exemplary execution of the TILETFM2RI instruction according to some embodiments;

[0029] Figure 22B The illustration shows an exemplary execution of the TILETFM2RI instruction according to some embodiments;

[0030] Figure 23 The illustration shows an embodiment of a processor executing a flow for processing TILETFM2RI instructions;

[0031] Figure 24 This is a block diagram illustrating the format of the TILETFM2RI instruction according to some embodiments;

[0032] Figures 25A-25B This is a block diagram illustrating a general vector-friendly instruction format and its instruction template according to an embodiment;

[0033] Figure 25A This is a block diagram illustrating a general vector-friendly instruction format and its Class A instruction template according to an embodiment;

[0034] Figure 25B This is a block diagram illustrating the general vector-friendly instruction format and its Class B instruction template according to an embodiment;

[0035] Figure 26A This is a block diagram illustrating an exemplary dedicated vector-friendly instruction format according to an embodiment;

[0036] Figure 26B This is a block diagram illustrating a field with a dedicated vector-friendly instruction format that constitutes a complete opcode field according to one embodiment;

[0037] Figure 26C This is a block diagram illustrating a field with a dedicated vector-friendly instruction format that constitutes a register index field according to one embodiment;

[0038] Figure 26D This is a block diagram illustrating a field with a dedicated vector-friendly instruction format that constitutes an extended operation field according to one embodiment;

[0039] Figure 27 This is a block diagram of a register architecture according to one embodiment;

[0040] Figure 28A This is a block diagram illustrating both an exemplary ordered pipeline and an exemplary out-of-order release / execution pipeline for register renaming, according to an embodiment.

[0041] Figure 28B This is a block diagram illustrating both an exemplary embodiment of an ordered architecture core to be included in a processor according to an embodiment and an exemplary out-of-order release / execution architecture core with register renaming.

[0042] Figures 29A-29B The diagram illustrates a more specific example of an ordered core architecture, which will be one of several logical blocks in a chip (including other cores of the same type and / or different types);

[0043] Figure 29A It is a block diagram of a single processor core according to an embodiment, its connection to the on-die interconnect network, and a local subset of its second-level (L2) cache;

[0044] Figure 29B According to the embodiments Figure 29A An expanded diagram of a portion of the processor core;

[0045] Figure 30 This is a block diagram of a processor according to an embodiment, which may have more than one core, may have an integrated memory controller, and may have an integrated graphics device;

[0046] Figures 31-34 This is a block diagram of an exemplary computer architecture;

[0047] Figure 31 A block diagram of a system according to an embodiment of the present invention is shown;

[0048] Figure 32 This is a block diagram of a first, more specific, exemplary system according to an embodiment of the present invention;

[0049] Figure 33 This is a block diagram of a second, more specific, exemplary system according to an embodiment of the present invention;

[0050] Figure 34 This is a block diagram of a System-on-a-Chip (SoC) according to an embodiment of the present invention; and

[0051] Figure 35 This is a block diagram illustrating the use of a software instruction converter to convert binary instructions in a source instruction set into binary instructions in a target instruction set, according to an embodiment. Detailed Implementation

[0052] Numerous specific details are set forth in the following description. However, it should be understood that embodiments may be practiced without these specific details. In other instances, well-known circuits, structures, and techniques have not been shown in detail so as not to obscure the understanding of this description.

[0053] References to "an embodiment," "embodiment," "example embodiment," etc., in the specification indicate that the described embodiment may include a specific feature, structure, or characteristic; however, each embodiment may not necessarily include that specific feature, structure, or characteristic. Furthermore, such phrases do not necessarily refer to the same embodiment. Additionally, when a specific feature, structure, or characteristic is described in connection with an embodiment, it is believed that the influence of such feature, structure, or characteristic on such feature, structure, or characteristic in conjunction with other embodiments, whether explicitly described or not, is within the knowledge of those skilled in the art.

[0054] In many mainstream processors, handling matrices is a difficult and / or instruction-intensive task. For example, multiple rows of a matrix may be placed into multiple compact data (e.g., SIMD or vector) registers, and then operations can be performed on those rows individually. For instance, depending on the data size, adding two 8x2 matrices might require loading or pooling into four compact data registers. A first addition is then performed on the compact data registers corresponding to the first row of each matrix, and a second addition is performed on the compact data registers corresponding to the second row of each matrix. The resulting compact data registers are then distributed back into memory. While this scenario may be acceptable for small matrices, it is generally unacceptable for larger matrices.

[0055] discuss

[0056] This document describes mechanisms for supporting matrix operations in computer hardware such as central processing units (CPUs), graphics processing units (GPUs), and accelerators. Matrix operations utilize 2-dimensional (2-D) data structures representing one or more compact regions of memory (such as registers). Throughout this specification, these 2-D data structures are referred to as slices. Note that a matrix can be smaller than a slice (using less than the entirety of a slice), or multiple slices can be utilized (the matrix is ​​larger than any single slice). Throughout this specification, the matrix (slice) language is used to indicate operations performed using slices that affect the matrix; whether the matrix is ​​larger than any single slice is generally irrelevant.

[0057] Each slice can be acted upon by different operations, such as those detailed herein, including but not limited to: matrix (slice) multiplication, slice addition, slice subtraction, slice diagonal, slice zeroing, slice transformation, slice dot product, slice broadcast, slice row broadcast, slice column broadcast, slice multiplication, slice multiplication and accumulation, slice movement, etc. Furthermore, in the future, support for operators using scaling and / or biasing can be used in conjunction with these operations or to support non-numerical applications such as OpenCL "local memory," data compression / decompression, etc. Instructions for performing ternary slice operations (TILETFM2RI) are also described herein.

[0058] Multiple portions of storage (such as (non-volatile and volatile) memory, registers, caches, etc.) are arranged as slices with different horizontal and vertical scales. For example, a slice may have a horizontal scale of 4 (e.g., four rows of a matrix) and a vertical scale of 8 (e.g., eight columns of a matrix). Typically, the horizontal scale is related to the element size (e.g., 2-bit, 4-bit, 8-bit, 16-bit, 32-bit, 64-bit, 128-bit, etc.). Multiple data types (single-precision floating-point, double-precision floating-point, integers, etc.) can be supported.

[0059] Exemplary use of a configured chip

[0060] In some embodiments, slice parameters can be configured. For example, a given slice can be configured to provide slice options. Exemplary slice options include, but are not limited to, the number of rows in the slice, the number of columns in the slice, whether the slice is valid, and whether the slice consists of pairs of slices of equal size.

[0061] Figure 1A The illustration shows an embodiment of the configured chips. As shown, the 4kB of application memory 102 has four 1kB chips stored thereon—chip t0 104, chip t1 106, chip t2 108, and chip t3 110. In this example, these four chips are not composed of pairs, and each chip has elements arranged in rows and columns. Chips t0 104 and t1 106 have 4-byte elements (e.g., single-precision data) with K rows and N columns, where K = 8 and N = 32. Chips t2 108 and t3 110 have 8-byte elements (e.g., double-precision data) with K rows and N / 2 columns. Since the width of a double-precision operand is twice that of a single-precision operand, this configuration aligns with the palette used to provide chip options, providing at least 4kB of total storage for at least four names. In operation, chips can be loaded from and stored in memory using load and store operations. Depending on the instruction encoding scheme used, the amount of application memory available, as well as the size, number, and configuration of available chips, vary.

[0062] Figure 1BThe illustration shows an embodiment of the configured chips. As shown, 4kB of application memory 122 has two pairs of 1kB chips stored thereon. The first pair is chip t4L 124 and chip t4R 126, and the second pair is chip t5L 128 and chip t5R 130. As shown, the chip pairs are divided into left and right chips. In other embodiments, the chip pairs are divided into even-numbered and odd-numbered chips. In this example, each of the four chips has elements arranged in rows and columns. Chips t4L 124 and t4R 126 have 4-byte elements (e.g., single-precision floating-point data) with K rows and N columns, where K = 8 and N = 32. Chips t5L 128 and t5R 130 have 8-byte elements (e.g., double-precision floating-point data) with K rows and N / 2 columns. Since the width of a double-precision operand is twice that of a single-precision operand, this configuration aligns with the palette used to provide chip options, providing at least 4kB of total storage for at least two names. Figure 1A The four chips use four names, each name representing a 1kB chip, while Figure 1B Two chip pairs can be specified using two names. In some embodiments, chip instructions accept the names of the paired chips as operands. During operation, load and store operations can be used to load chips from memory and store chips into memory. The amount of available application memory, as well as the size, number, and configuration of available chips, vary depending on the instruction encoding scheme used.

[0063] In some embodiments, slice parameters are definable. For example, a "palette" is used to provide slice options. Exemplary options include, but are not limited to, the number of slice names, the number of bytes in the stored rows, the number of rows and columns in the slice, and so on. For example, the maximum "height" (number of rows) of a slice can be defined as:

[0064] Maximum number of rows per slice = number of constructed storage / (number of palette names * number of bytes per row).

[0065] This allows applications to write to names so that fixed usage can take advantage of different storage sizes across implementations.

[0066] The slice configuration (“TILECONFIG”) directive is used to configure slices, where specific slice usage is defined within the selected palette. This declaration includes the number of slice names to be used, the requested number of rows and columns for each name (slice), and in some embodiments, the requested data type for each slice. In some embodiments, a consistency check is performed during the execution of the TILECONFIG directive to determine its compliance with palette entries.

[0067] Exemplary slice storage type

[0068] Figure 2Several examples of matrix storage are illustrated. In (A), slices are stored in memory. As shown, each "row" consists of four compact data elements. To reach the next "row," a stride value is used. Note that rows can be stored contiguously in memory. When slice storage does not map the row width of the underlying memory array, stride memory access allows access to a row and subsequently to the next row.

[0069] Loading a slice from memory and accessing a slice of memory typically involves step-by-step access from application memory to compressed data lines. Exemplary TILELOAD and TILESTORE instructions, or other instructions for application memory that serve as the TILE(slice) operand in load operation instructions, are in some embodiments reversible to handle page faults, unmasked floating-point exceptions, and / or interrupts for up to 2* lines per instruction.

[0070] In (B), the matrix is ​​stored in a slice of registers, such as compact data registers (Single Instruction Multiple Data (SIMD) or vector registers). In this example, the slice is stacked on three physical registers. Typically, contiguous registers are used; however, this is not always necessary.

[0071] In (C), the matrix is ​​stored in on-chip non-register memory accessible by fused multiply-accumulate (FMA) circuitry used in on-chip operations. This memory may be within or adjacent to the FMA. Furthermore, in some embodiments, as discussed below, this memory may be used for data elements rather than for entire rows or entire slices.

[0072] The CPUID reports the supported parameters of the TMMA architecture. In some embodiments, the information list includes the maximum height and the maximum SIMD scale. Configuring the TMMA architecture requires specifying the scale of each slice, the element size of each slice, and the palette identifier. This configuration is accomplished by executing the TILECONFIG directive.

[0073] Successful execution of the TILECONFIG instruction enables subsequent TILE operations. The TILERELEASEALL instruction clears the chip configuration and disables TILE operations (until the next TILECONFIG instruction is executed). In some embodiments, XSAVE, XSTORE, etc., are used during chip context switching. In some embodiments, two XCR0 bits are used in XSAVE: one for TILECONFIG metadata and one bit corresponding to the actual chip payload data.

[0074] TILECONFIG not only configures slice usage but also sets status variables that indicate whether the program is in the code region when the slice is configured. The implementation can enumerate restrictions on other instructions that can be used with the slice region, such as no use of existing register sets, etc.

[0075] Exiting a slice region is typically accomplished using the TILERELEASEALL command. This command takes no parameters and quickly invalidates all slices (indicating that no more data needs to be saved or recovered) and clears the internal state corresponding to the slice region.

[0076] In some embodiments, a slice operation will zero out any rows and columns that exceed the scale specified by the slice configuration. For example, as each row is written, the slice operation will zero out data exceeding the configured number of columns (taking into account the element size). For instance, for a 64-byte row and a slice configured with 10 rows and 12 columns, a write operation to an FP32 element will write output / result data in 12*4 bytes per row for the first 10 rows and zero out the remaining 4*4 bytes in each row. The slice operation also completely zeroes out any rows after the first 10 configured rows. When using a 1K slice with 64-byte rows, there will be 16 rows, so in this example, the last 6 rows will also be zeroed out.

[0077] In some embodiments, when loading data, a context recovery instruction (e.g., XRSTOR) forces that data exceeding the configured number of rows in the slice be maintained at zero. If no valid configuration exists, all rows are zeroed out. XRSTOR on slice data can load useless information from columns exceeding those configured columns. It should not be possible for XRSTOR to clear data exceeding the configured number of columns because there is no element width associated with the slice configuration.

[0078] When writing an entire TILE memory area to memory, a context save (e.g., XSAVE) exposes the entire TILE memory area. If XRSTOR loads useless data into the rightmost part of the slice, that data will be saved by XSAVE. For lines exceeding the number specified for each slice, XSAVE will write zeros.

[0079] In some embodiments, slice instructions are resumable. Memory access operations can be restarted after a page fault. Calculation instructions handling floating-point operations also allow unmasked floating-point exceptions, thanks to exception masks controlled by control and / or status registers.

[0080] To support the restart of instructions after these events, these instructions store information in the start register, which is detailed below.

[0081] Matrix (Slice) Operating System

[0082] Exemplary hardware support

[0083] Figure 3 The illustration depicts an embodiment of a system utilizing a matrix (slice) manipulation accelerator. In this illustration, a host processor / processing system 301 passes commands 311 (e.g., matrix manipulation operations, such as arithmetic or matrix manipulation operations, or load and store operations) to a matrix manipulation accelerator 307. However, this is shown in this manner for discussion purposes only. As detailed later, the accelerator 307 may be part of a processing core. Typically, commands 311, as slice manipulation operator instructions, refer to the slice in register-register (“reg-reg”) or register-memory (“reg-mem”) format. Other commands, such as TILESTORE, TILELOAD, TILECONFIG, etc., do not perform data manipulation on the slice. Commands may be decoded instructions (e.g., micro-operations) or macro instructions to be processed by the accelerator 307.

[0084] In this example, the coherent memory interface 303 is coupled to the host processor / processing system 301 and the matrix operation accelerator 307, enabling them to share memory. Figure 4 and Figure 5 Different embodiments of how to use a matrix operation accelerator to share memory are shown. For example... Figure 4 As shown, the host processor 401 and the matrix operation accelerator circuit 405 share the same memory 403. Figure 5 The illustration shows an embodiment where the host processor 501 and matrix operation accelerator 505 do not share memory but can access each other's memory. For example, processor 501 can access on-chip memory 507 and utilizes its host memory 503 as usual. Similarly, matrix operation accelerator 505 can access host memory 503, but more typically uses its own memory 507. Note that these memories can be of different types.

[0085] In some embodiments, a stacked structure on top of physical registers is used to support slices. For example, depending on the implementation, a slice may utilize 16 1024-bit registers, 32 512-bit registers, and so on. In some embodiments, matrix operations utilize 2-dimensional (2-D) data structures representing one or more compact regions of memory (such as registers). Throughout this specification, these 2-D data structures are referred to as slices or slice registers.

[0086] In some embodiments, the matrix operation accelerator 307 includes a plurality of FMAs 309 coupled to a data buffer 305 (in some implementations, one or more of these buffers 305 are stored in an FMA in a grid as shown). The data buffers 305 buffer slices loaded from memory and / or stored to memory (e.g., using slice load or slice store instructions). The data buffers may be, for example, a plurality of registers. Typically, these FMAs are arranged as a grid of linked FMAs 309s capable of reading and writing slices. In this example, the matrix operation accelerator 307 is used to perform matrix multiplication operations using slices T0, T1, and T2. At least one slice is accommodated in an FMA grid 309. In some embodiments, all slices in the operation are stored in an FMA grid 309. In other embodiments, only a subset is stored in an FMA grid 309. As shown, T1 is accommodated, while T0 and T2 are not. Note that A, B, and C refer to the matrices of these slices, which may or may not occupy the entire space of the slice.

[0087] Figure 6 The illustration shows an example of using slice matrix multiplication accumulation operation (“TMMA”).

[0088] The number of rows in the matrix (slice A 601) matches the number of cascaded (chained) FMAs, which include computational latency. The implementation is free to recycle on grids of smaller heights, but the computation remains the same.

[0089] The source / destination vectors come from a slice of N rows (slice C 605), and the grid 611 of the FMA performs N vector-matrix operations, resulting in a complete instruction to perform matrix multiplication of the slice. Slice B 603 is another vector source and provides "broadcast" items to the FMA at each level.

[0090] In operation, in some embodiments, the elements of matrix B (stored in slice B 603) are scattered across a rectangular grid of the FMA. Matrix B (stored in slice A 601) has its row elements transformed to match the column scale of the rectangular grid of the FMA. At each FMA in the grid, the elements of A and B are multiplied and added to the incoming addend (from the diagram above), and the outgoing sum is passed to the next row of the FMA (or the final output).

[0091] The latency of a single step is proportional to K (the row height of matrix B), and the dependent TMMA typically (in a single slice or across slices) has enough source-destination rows to hide this latency. Implementations can also segment the SIMD (compact data element) scale M (the row height of matrix A) across time steps, but this only changes the constant multiplied by K. When the program specifies K smaller than the maximum value enumerated by TMACC, the implementation is free to use either a "mask" or "early out".

[0092] The latency of the entire TMMA is proportional to N*K. The repetition rate is proportional to N. The number of MACs per TMMA instruction is N*K*M.

[0093] Figure 7 This illustration depicts an embodiment of a subset of the iterative execution of a chained fused multiply-accumulate instruction. Specifically, it illustrates the execution circuitry for iteratively executing a compressed data element position at a destination. In this embodiment, the chained fused multiply-accumulate operates against a signed source, where the accumulator is twice the size of the input data.

[0094] The first signed source (source 1 701) and the second signed source (source 2 703) each have four compact data elements. Each of these compact data elements stores signed data such as floating-point data. The third signed source (source 3 709) has two compact data elements, each of which stores signed data. The size of the first signed source 701 and the size of the second signed source 703 are half the size of the third signed source (initial value or previous result) 709. For example, the first signed source 701 and the second signed source 703 may have 32-bit compact data elements (e.g., single-precision floating-point), while the third signed source 709 may have 64-bit compact data elements (e.g., double-precision floating-point).

[0095] This diagram only shows the most valid two compressed data element positions of the first signed source 701 and the second signed source 703, and the most valid compressed data element position of the third signed source 709. Of course, other compressed data element positions will also be processed.

[0096] As shown, compacted data elements are processed in pairs. For example, multiplier circuit 705 multiplies the data from the most valid compacted data element positions of the first signed source 701 and the second signed source 703, and multiplier circuit 707 multiplies the data from the second most valid compacted data element positions of the first signed source 701 and the second signed source 703. In some embodiments, these multiplier circuits 705 and 707 are reused for other compacted data element positions. In other embodiments, additional multiplier circuitry is used to enable parallel processing of compacted data elements. In some contexts, parallel execution is accomplished using channels of the same size as the signed third source 709. The results of each of these multiplications are added using adder circuit 711.

[0097] (Using different adders 713 or the same adder 711) add the result of the addition of these multiplications to the data at the position of the most effective compacted data element from the signed source 3709.

[0098] Finally, the result of the second addition is stored in the signed destination 715 in the compacted data element position corresponding to the used compacted data element position from the signed third source 709, or if there is a next iteration, the result of the second addition is passed to that next iteration. In some embodiments, a write mask is applied to this storage such that storage occurs if the corresponding write mask (bit) is set, and storage does not occur if the corresponding write mask (bit) is not set.

[0099] Figure 8 This illustration depicts an embodiment of a subset of the iterative execution of a chained fused multiply-accumulate instruction. Specifically, it illustrates the execution circuitry for iteratively executing a compressed data element position at a destination. In this embodiment, the chained fused multiply-accumulate operates against a signed source, where the accumulator is twice the size of the input data.

[0100] The first signed source (source 1 801) and the second signed source (source 2 803) each have four compact data elements. Each of these compact data elements stores signed data such as integer data. The third signed source (source 3 809) has two compact data elements, each of which stores signed data. The size of the first signed source 801 and the size of the second signed source 803 are half the size of the third signed source 809. For example, the first signed source 801 and the second signed source 803 may have 32-bit compact data elements (e.g., single-precision floating-point), while the third signed source 809 may have 64-bit compact data elements (e.g., double-precision floating-point).

[0101] This diagram only shows the most valid two compressed data element positions of the first signed source 801 and the second signed source 803, as well as the most valid compressed data element position of the third signed source 809. Other compressed data element positions will, of course, be processed as well.

[0102] As shown, compacted data elements are processed in pairs. For example, multiplier circuit 805 multiplies the data from the most valid compacted data element positions of the first signed source 801 and the second signed source 803, and multiplier circuit 807 multiplies the data from the second most valid compacted data element positions of the first signed source 801 and the second signed source 803. In some embodiments, these multiplier circuits 805 and 807 are reused for other compacted data element positions. In other embodiments, additional multiplier circuitry is used to enable parallel processing of compacted data elements. In some contexts, parallel execution is accomplished using a channel of size 809, which is the size of the signed third source (initial value or result of previous iterations). Addition / saturation circuit 813 adds the result of each of the multiple multiplications to the signed third source 809.

[0103] When addition results in an excessively large value, the adder / saturation (accumulator) circuit 813 preserves the sign of the operand. Specifically, saturation evaluation occurs for infinite-precision results between multiplexed additions and writes to the destination or the next iteration. When the accumulator 813 is floating-point and the input is an integer, the sum of the product and the floating-point accumulator input value are converted to an infinite-precision value (a fixed-point number with hundreds of digits), the multiplication result is added to the third input, and a single rounding to the actual accumulator type is performed.

[0104] Unsigned saturation means that the output value is limited to the maximum unsigned number (all 1s) of that element width. Signed saturation means that the value is limited to the range between the minimum negative number and the maximum positive number of that element width (e.g., for a byte, the range is from -128 (=-2^7) to 127 (=2^7-1)).

[0105] The result of addition and saturation check is stored in the signed result 815 in the corresponding compacted data element position from the signed third source 809, or the result is passed to the next iteration if there is one. In some embodiments, a write mask is applied to this storage such that storage occurs if the corresponding write mask (bit) is set, and storage does not occur if the corresponding write mask (bit) is not set.

[0106] Figure 9This illustration depicts an embodiment of a subset of the iterative execution of a chained fused multiply-accumulate instruction. Specifically, it illustrates the execution circuitry for iteratively executing a compressed data element position at the destination. In this embodiment, the chained fused multiply-accumulate operates on both signed and unsigned sources, where the accumulator is four times the size of the input data.

[0107] The first signed source (source 1 901) and the second unsigned source (source 2 903) each have four compacted data elements. Each of these compacted data elements contains data such as floating-point or integer data. The third signed source (initial value or result 915) has compacted data elements that store signed data. The size of the first source 901 and the second source 903 is one-quarter the size of the third signed source 915. For example, the first source 901 and the second source 903 may have 16-bit compacted data elements (e.g., words), while the third signed source 915 may have 64-bit compacted data elements (e.g., double-precision floating-point or 64-bit integer).

[0108] This diagram only shows the four most valid compressed data element positions of the first source 901 and the second source 903, as well as the most valid compressed data element position of the third signed source 915. Of course, if there are any other compressed data element positions, those will also be processed.

[0109] As shown in the figure, compacted data elements are processed as quadruples. For example, multiplier circuit 905 multiplies the data from the most significant compacted data element positions of the first source 901 and the second source 903, multiplier circuit 907 multiplies the data from the second most significant compacted data element positions of the first source 901 and the second source 903, multiplier circuit 909 multiplies the data from the third most significant compacted data element positions of the first source 901 and the second source 903, and multiplier circuit 911 multiplies the data from the least significant compacted data element positions of the first source 901 and the second source 903. In some embodiments, before multiplication, the signed compacted data elements of the first source 901 are sign-extended, and the unsigned compacted data elements of the second source 903 are zero-extended.

[0110] In some embodiments, these multiplier circuits 905-911 are reused for other compact data element locations. In other embodiments, additional multiplier circuitry is used to enable parallel processing of compact data elements. In some contexts, parallel execution is accomplished using channels of size 915 of the signed third source. The results of each of these multiplications are added using adder circuitry 913.

[0111] (Using different adders 917 or the same adder 913) add the result of the addition of these multiplications to the data at the position of the most effective compacted data element from the signed source 3 915.

[0112] Finally, the result 919 of the second addition is either stored in the signed destination at the compacted data element position corresponding to the position of the compacted data element used from the signed third source 915, or passed to the next iteration. In some embodiments, a write mask is applied to this storage such that storage occurs if the corresponding write mask (bit) is set, and storage does not occur if the corresponding write mask (bit) is not set.

[0113] Figure 10 This illustration depicts an embodiment of a subset of the iterative execution of a chained fused multiply-accumulate instruction. Specifically, it illustrates the execution circuitry for iteratively executing a compressed data element position at the destination. In this embodiment, the chained fused multiply-accumulate operates on both signed and unsigned sources, where the accumulator is four times the size of the input data.

[0114] The first signed source 1001 and the second unsigned source 1003 each have four compact data elements. Each of these compact data elements stores data such as floating-point data or integer data. The third signed source 1015 (initial or previous result) has compact data elements that store signed data. The size of the first source and the size of the second source are one-quarter the size of the third signed source 1015 (initial or previous result). For example, the first and second sources may have 16-bit compact data elements (e.g., words), while the third signed source 1015 (initial or previous result) may have 64-bit compact data elements (e.g., double-precision floating-point or 64-bit integer).

[0115] This diagram shows the four most valid compressed data element positions of the first signed source 1001 and the second unsigned source 1003, as well as the most valid compressed data element position of the third signed source 1015. Of course, if there are any other compressed data element positions, those will also be processed.

[0116] As shown in the figure, compacted data elements are processed as quadruples. For example, multiplier circuit 1005 multiplies the data from the most significant compacted data element positions of the first signed source 1001 and the second unsigned source 1003; multiplier circuit 1007 multiplies the data from the second most significant compacted data element positions of the first signed source 1001 and the second unsigned source 1003; multiplier circuit 1009 multiplies the data from the third most significant compacted data element positions of the first signed source 1001 and the second unsigned source 1003; and multiplier circuit 1011 multiplies the data from the least significant compacted data element positions of the first signed source 1001 and the second unsigned source 1003. In some embodiments, before multiplication, the signed compacted data elements of the first signed source 1001 are sign-extended, and the unsigned compacted data elements of the second unsigned source 1003 are zero-extended.

[0117] In some embodiments, these multiplier circuits 1005-1011 are reused for other compacted data element locations. In other embodiments, additional multiplier circuitry is used to enable parallel processing of compacted data elements. In some contexts, parallel execution is accomplished using a channel of size 1015 (the initial or previous result). The result of adding these multiplication results is added to the data from the highest valid compacted data element location of the third signed source 1015 (the initial or previous result) using the adder / saturator 1013 circuitry.

[0118] When addition results in a value that is too large or too small for signed saturation, the adder / saturation (accumulator) circuit 1013 preserves the sign of the operand. Specifically, saturation evaluation occurs for infinite-precision results between multiplexed additions and writes to the destination. When the accumulator 1013 is floating-point and the input is an integer, the sum of the product and the floating-point accumulator input value are converted to an infinite-precision value (a fixed-point number of several hundred digits), the multiplication result is added to the third input, and a single rounding to the actual accumulator type is performed.

[0119] The result 1019 of the addition and saturation check is either stored in the signed destination at the compacted data element position corresponding to the position of the compacted data element used from the third signed source 1015 (initial or previous result) or passed to the next iteration. In some embodiments, a write mask is applied to this storage such that storage occurs if the corresponding write mask (bit) is set, and storage does not occur if the corresponding write mask (bit) is not set.

[0120] Figure 11The illustration shows a SIMD implementation of a power of 2 according to an embodiment, where the accumulator uses an input size larger than the size of the input to the multiplier. Note that the source and accumulator values ​​(to the multiplier) can be signed or unsigned. Table 1101 illustrates different configurations for an accumulator with an input size of 2X (in other words, the size of the accumulator input value is twice the size of the compacted data element of the source). For a byte-sized source, the accumulator uses a 16-bit word or half-precision floating-point (HPFP) value. For a word-sized source, the accumulator uses a 32-bit integer or single-precision floating-point (SPFP) value. For an SPFP or 32-bit integer-sized source, the accumulator uses a 64-bit integer or double-precision floating-point (DPFP) value.

[0121] For an accumulator with a 4X input size (in other words, the size of the accumulator input value is four times the size of the compacted data element of the source), Table 1103 illustrates different configurations. For byte-sized sources, the accumulator uses 32-bit integer or single-precision floating-point (SPFP) values. In some embodiments, for word-sized sources, the accumulator uses 64-bit integer or double-precision floating-point (DPFP) values.

[0122] For an accumulator with an input size of 8X (in other words, the size of the accumulator input value is 8 times the size of the compressed data element of the source), Table 1105 illustrates the configuration. For a byte-sized source, the accumulator uses a 64-bit integer.

[0123] As previously mentioned, the matrix operation circuitry can be included in the core or used as an external accelerator. Figure 12 The illustration shows an embodiment of a system utilizing matrix manipulation circuitry. In this illustration, multiple entities are coupled to a ring interconnect 1245.

[0124] Multiple cores, core 0 1201, core 1 1203, core 2 1205, and core N 1207, provide non-chip-based instruction support. In some embodiments, matrix operation circuitry 1251 is located in core 1203, while in other embodiments, matrix operation circuitry 1211 and 1213 are accessible on ring interconnect 1245.

[0125] In addition, one or more memory controllers 1223-1225 are provided to communicate with the memories 1233 and 1231 on behalf of the core and / or matrix operation circuitry.

[0126] Figure 13The illustration depicts an embodiment of a processor core pipeline that supports matrix operations using chips. Branch prediction and decoding circuitry 1303 performs branch prediction, decoding of instructions stored in instruction store 1301, and / or both branch prediction and decoding. For example, the instructions detailed herein may be stored in the instruction store. In some implementations, separate circuitry is used for branch prediction, and in some embodiments, at least some instructions are decoded into one or more micro-operations, microcode entry points, microinstructions, other instructions, or other control signals using microcode 1305. Branch prediction and decoding circuitry 1303 can be implemented using a variety of different mechanisms. Examples of suitable mechanisms include, but are not limited to, lookup tables, hardware implementations, programmable logic arrays (PLAs), microcode read-only memory (ROM), etc.

[0127] Branch prediction and decoding circuitry 1303 is coupled to allocation / renaming circuitry 1307, which in some embodiments is coupled to scheduler circuitry 1309. In some embodiments, these circuits provide register renaming, register allocation, and / or scheduling functionality by performing one or more of the following steps: 1) renaming logical operand values ​​to physical operand values ​​(e.g., a register alias table in some embodiments); 2) assigning status bits and flags to decoded instructions; and 3) (e.g., using a reserved station in some embodiments) scheduling the decoded instructions for execution on execution circuitry outside the instruction pool.

[0128] Scheduler circuitry 1309 represents any number of different schedulers, including reserved stations, central instruction windows, etc. Scheduler circuitry 1309 is coupled to or includes (multiple) physical register files 1315. Each of the (multiple) physical register files 1315 represents one or more physical register files, wherein different physical register files store one or more different data types, such as scalar integers, scalar floating-point numbers, compressed integers, compressed floating-point numbers, vector integers, vector floating-point numbers, status (e.g., an instruction pointer as the address of the next instruction to be executed), slices, etc. In one embodiment, (multiple) physical register files 1315 include vector register circuitry, write mask register circuitry, and scalar register circuitry. These register circuitry can provide architectural vector registers, vector mask registers, and general-purpose registers. Multiple physical register files 1315 are covered by retirement circuitry 1317 to illustrate various ways in which register renaming and out-of-order execution can be implemented (e.g., using multiple reordering buffers and multiple retirement register files, using multiple future files, multiple history buffers, multiple retirement register files, using register mappings and register pools, etc.). Retirement circuitry 1317 and multiple physical register files 1315 are coupled to execution circuitry 1311.

[0129] Although register renaming is described in the context of out-of-order execution, it should be understood that register renaming can be used in ordered architectures. While the illustrated embodiments of the processor may also include separate instruction and data cache units and a shared L2 cache unit, alternative embodiments may also have a single internal cache for both instructions and data, such as, for example, a Level 1 (L1) internal cache, or multiple levels of internal caches. In some embodiments, the system may include a combination of internal caches and external caches located outside the core and / or processor. Alternatively, all caches may be located outside the core and / or processor.

[0130] Execution circuitry 1311 is a collection of one or more execution units, including scalar circuitry 1321, vector / SIMD circuitry 1323, matrix operation circuitry 1327, and memory access circuitry 1325 for accessing cache 1313. The execution circuitry performs various operations (e.g., shift, addition, subtraction, multiplication) and on various data types (e.g., scalar floating-point, compressed integer, compressed floating-point, vector integer, vector floating-point). While some embodiments may include several execution units dedicated to a particular function or set of functions, other embodiments may include only one execution unit or multiple execution units that all perform all functions. Scalar circuitry 1321 performs scalar operations, vector / SIMD circuitry 1323 performs vector / SIMD operations, and matrix operation circuitry 1327 performs the matrix (slice) operations detailed herein.

[0131] As an example, an exemplary register-renaming, out-of-order release / execution core architecture can implement a pipeline as follows: 1) an instruction fetch circuit performs the fetch and length decoding stage; 2) a branch and decode circuit 1303 performs the decoding stage; 3) an allocation / rename circuit 1307 performs the allocation and rename stages; 4) a scheduler circuit 1309 performs the scheduling stage; 5) (multiple) physical register files coupled to or included in the scheduler circuit 1309 and the allocation / rename circuit 1307 and memory units) perform the register read / memory read stage; execution circuit 1311 performs the execution stage; 6) memory units and (multiple) physical register file units perform the write-back / memory write stage; 7) each unit may involve an exception handling stage; and 8) a retirement unit and (multiple) physical register file units perform the commit stage.

[0132] The core may support one or more instruction sets (e.g., the x86 instruction set (with some extensions added with newer versions); the MIPS instruction set of MIPS Technologies, Inc., Sunnyvale, California; the ARM instruction set of ARM Holdings, Inc., Sunnyvale, California (with optional additional extensions such as NEON)), including the instructions described herein. In one embodiment, core 1390 includes logic for supporting compressed data instruction set extensions (e.g., AVX1, AVX2), thereby allowing the use of compressed data to perform operations used by many multimedia applications.

[0133] It should be understood that a core can support multithreading (execution of two or more parallel operations or a set of threads), and this multithreading can be accomplished in various ways, including time-division multithreading, simultaneous multithreading (where a single physical core provides a logical core for each thread in the simultaneous multithreading of the physical core), or combinations thereof (e.g., time-division fetching and decoding, and subsequent operations such as...). Simultaneous multithreading in hyper-threading technology.

[0134] Figure 14 The illustration depicts an embodiment of a processor core pipeline that supports matrix operations using chips. Branch prediction and decoding circuitry 1403 performs branch prediction, decoding of instructions stored in instruction store 1401, and / or both branch prediction and decoding. For example, the instructions detailed herein may be stored in the instruction store. In some implementations, separate circuitry is used for branch prediction, and in some embodiments, at least some instructions are decoded into one or more micro-operations, microcode entry points, microinstructions, other instructions, or other control signals using microcode 1405. Branch prediction and decoding circuitry 1403 can be implemented using a variety of different mechanisms. Examples of suitable mechanisms include, but are not limited to, lookup tables, hardware implementations, programmable logic arrays (PLAs), microcode read-only memory (ROM), etc.

[0135] Branch prediction and decoding circuitry 1403 is coupled to allocation / renaming circuitry 1407, which in some embodiments is coupled to scheduler circuitry 1409. In some embodiments, these circuits provide register renaming, register allocation, and / or scheduling functionality by performing one or more of the following steps: 1) renaming logical operand values ​​to physical operand values ​​(e.g., a register alias table in some embodiments); 2) assigning status bits and flags to decoded instructions; and 3) (e.g., using a reserved station in some embodiments) scheduling the decoded instructions for execution on execution circuitry outside the instruction pool.

[0136] Scheduler circuitry 1409 represents any number of different schedulers, including reserved stations, central instruction windows, etc. Scheduler circuitry 1409 is coupled to or includes physical register files 1415. Each of the physical register files 1415 represents one or more physical register files, where different physical register files store one or more different data types, such as scalar integers, scalar floating-point numbers, compressed integers, compressed floating-point numbers, vector integers, vector floating-point numbers, status (e.g., an instruction pointer as the address of the next instruction to be executed), slices, etc. In one embodiment, physical register files 1415 include vector register circuitry, write mask register circuitry, and scalar register circuitry. These register circuitry can provide architectural vector registers, vector mask registers, and general-purpose registers. Multiple physical register files 1415 are covered by retirement circuitry 1417 to illustrate various ways in which register renaming and out-of-order execution can be implemented (such as using multiple reordering buffers and multiple retirement register files, using multiple future files, multiple history buffers, multiple retirement register files, using register mappings and register pools, etc.). Retirement circuitry 1417 and multiple physical register files 1415 are coupled to execution circuitry 1411.

[0137] Although register renaming is described in the context of out-of-order execution, it should be understood that register renaming can be used in ordered architectures. While the illustrated embodiments of the processor may also include separate instruction and data cache units and a shared L2 cache unit, alternative embodiments may also have a single internal cache for both instructions and data, such as, for example, a Level 1 (L1) internal cache, or multiple levels of internal caches. In some embodiments, the system may include a combination of internal caches and external caches located outside the core and / or processor. Alternatively, all caches may be located outside the core and / or processor.

[0138] The execution circuitry 1411 includes a set of one or more execution circuits 1427 and a set of one or more memory access circuits 1425 for accessing the cache 1413. The execution circuits 1427 perform the matrix (slice) operations detailed herein.

[0139] As an example, an exemplary register-renaming, out-of-order release / execution core architecture can implement a pipeline as follows: 1) an instruction fetch circuit performs the fetch and length decoding stage; 2) a branch and decode circuit 1403 performs the decoding stage; 3) an allocation / rename circuit 1407 performs the allocation and rename stages; 4) a scheduler circuit 1409 performs the scheduling stage; 5) (multiple) physical register files coupled to or included in the scheduler circuit 1409 and the allocation / rename circuit 1407 and memory units) perform the register read / memory read stage; execution circuit 1411 performs the execution stage; 6) memory units and (multiple) physical register file units perform the write-back / memory write stage; 7) each unit may involve an exception handling stage; and 8) a retirement unit and (multiple) physical register file units perform the commit stage.

[0140] The core may support one or more instruction sets (e.g., the x86 instruction set (with some extensions added with newer versions); the MIPS instruction set of MIPS Technologies, Inc., Sunnyvale, California; the ARM instruction set of ARM Holdings, Inc., Sunnyvale, California (with optional additional extensions such as NEON)), including the instructions described herein. In one embodiment, core 1490 includes logic for supporting compressed data instruction set extensions (e.g., AVX1, AVX2), thereby allowing the use of compressed data to perform operations used by many multimedia applications.

[0141] It should be understood that a core can support multithreading (execution of two or more parallel operations or a set of threads), and this multithreading can be accomplished in various ways, including time-division multithreading, simultaneous multithreading (where a single physical core provides a logical core for each thread in the simultaneous multithreading of the physical core), or combinations thereof (e.g., time-division fetching and decoding, and subsequent operations such as...). Simultaneous multithreading in hyper-threading technology.

[0142] layout

[0143] Throughout this specification, a row-based data layout is used to represent data. Column-based users should modify these items according to their orientation. Figure 15 The diagram illustrates examples of matrices expressed in row-major and column-major formats. As shown, matrix A is a 2x3 matrix. When this matrix is ​​stored in row-major format, the data elements in the rows are consecutive. When this matrix is ​​stored in column-major format, the data elements in the columns are consecutive. T *B T =(BA) T This is a well-known property of matrices, where the superscript T indicates a transformation. Reading data as row-major data results in a matrix that looks like a transformation matrix.

[0144] In some embodiments, the semantics of behavior-major are utilized in the hardware, and column-major data will swap the operand order and result in a matrix transformation, but for subsequent column-major reads from memory, it is a correct non-transformation matrix.

[0145] For example, if you have two columns-matrixes to be multiplied:

[0146] abgik ag+bh ai+bj ak+bl

[0147] cd* hjl= cg+dh ci+dj ck+dl

[0148] ef eg+fh ei+fj ek+fl

[0149] (3x2) (2x3) (3x3)

[0150] The input matrix will be stored in linear memory (column-major) as follows:

[0151] acebdf

[0152] as well as

[0153] ghijk l.

[0154] If those matrices are read as row-major matrices at scales of 2x3 and 3x2, they will be represented as follows:

[0155] ace and gh

[0156] bdfij

[0157] kl

[0158] Commutation and matrix multiplication:

[0159] ghace ag+bh cg+dh eg+fh

[0160] ij *bdf= ai+bj ci+dj ei+fj

[0161] kl ak+bl ck+dl ek+fl

[0162] The transformation matrix is ​​shifted out and can then be stored in row-major order:

[0163] ag+bh cg+dh eg+fh ai+bj ci+dj ei+fj ak+bl ck+dl ek+fl

[0164] And it is used in subsequent column-major calculations, and it is the correct untransformed matrix:

[0165] ag+bh ai+bj ak+bl

[0166] cg+dh ci+dj ck+dl

[0167] eg+fh ei+fj ek+fl

[0168] Exemplary use

[0169] Figure 16 This diagram illustrates an example of matrix (slice) usage. In this example, matrix C 1601 comprises two slices, matrix A 1603 comprises one slice, and matrix B 1605 comprises two slices. The diagram shows an example of the inner loop of an algorithm used to compute matrix multiplication. In this example, the two resulting slices tmm0 and tmm1 from matrix C 1601 are used to accumulate intermediate results. When a slice (tmm2) from matrix A 1603 is multiplied by the two slices from matrix B 1605, this slice is reused twice. Pointers are used to load a new A matrix (slice) and two new B matrices (slices) from the direction indicated by the arrow. The outer loop, not shown, adjusts the pointer for slice C.

[0170] The exemplary code shown in the figure includes the use of slice configuration instructions, which are executed to configure slice usage, load the slice, perform a slice loop, store the slice in memory, and release the slice usage.

[0171] Figure 17 The illustration shows an example of the use of a matrix (slice). At 1701, slice usage is configured. For example, the TILECONFIG instruction is executed to configure slice usage, including setting the number of rows and columns for each slice. Typically, at 1703, at least one matrix (slice) is loaded from memory. At 1705, the matrix (slice) is used to perform at least one matrix (slice) operation. At 1707, at least one matrix (slice) is stored out of memory, and at 1709, a context switch may occur.

[0172] Exemplary configuration

[0173] Hardware support for chip configuration

[0174] As discussed above, slice usage typically requires configuration before use. For example, it may not be necessary to use all rows and columns. In some embodiments, not configuring these rows and columns not only saves power but also allows the configuration to determine whether an operation will generate an error. For example, if M and L are not the same, matrix multiplication of the form (N x M)*(L x N) will generally not work.

[0175] In some embodiments, slice support is configured before using the slice matrix. For example, the number of rows and columns per slice, the slices to be used, and so on are configured. The TILECONFIG instruction is an improvement to the computer itself because it provides support for configuring the computer to use (as part of the processor core or as an external device) a matrix accelerator. Specifically, execution of the TILECONFIG instruction causes the configuration to be retrieved from memory and applied to the matrix (slice) settings within the matrix accelerator.

[0176] Chip usage configuration

[0177] Figure 18 The illustration shows the configuration support for the use of chips according to an embodiment. Memory 1801 contains chip descriptions 1803 of the matrices (chips) to be supported.

[0178] The instruction execution resource 1811 of the processor / core 1805 stores various aspects of the chip description 1803 into the chip configuration 1817. The chip configuration 1817 includes a palette table 1813 detailing which chips (number of rows and columns in each chip) are configured for the palette, and matrix support tags for use. Specifically, the instruction execution resource 1811 is configured to use the chip as specified in the chip configuration 1817. The instruction execution resource 1811 may also include machine-specific registers or configuration registers for indicating chip use. Additional values, such as usage midpoint and start values, are also set. The chip configuration 1817 utilizes register(s) 1819 to store chip usage and configuration information.

[0179] Figure 19 The illustration shows an embodiment of the description of the supported matrix (slice). This is the description that will be stored in response to the execution of the STTILECFG instruction. In this example, each field is a byte. In byte [0], the palette ID 1901 is stored. The palette ID is used to index the palette table 1813, which, as defined by the configuration, stores the number of bytes in the slice according to the palette ID and the bytes of each row of the slice associated with that ID.

[0180] Byte 1 storage will be the value stored in the "startRow" register 1903, and byte 2 storage will be the value stored in the startP register 1905. These registers are used to support restart instructions following these events. The startRow value indicates the row that should be used for restarting. The startP value indicates the position within the row used for storage operations when the pair is used, and in some embodiments, the startP value indicates the lower half of the row (in the lower slice of the pair) or the upper half of the row (in the higher slice of the pair). Generally, this position within the row (column) is not required.

[0181] Successful execution of matrix (slice) instructions will set both startRow and startP to zero, with the exception of TILECONFIG and STTILECFG.

[0182] At any point without restarting the interrupted matrix (slice) instructions, it is the software's responsibility to zero out the values ​​of startRow and startP. For example, an unmasked floating-point exception handler might decide to complete the operation in software and change the program counter value for another instruction, typically the next one. In this case, the software exception handler must zero out the startRow and startP values ​​from the exception presented to it by the operating system before resuming the process. The operating system will then use recovery instructions to reload those values.

[0183] Indication of pairs of byte 3 memory slices (1 byte per slice) 1907.

[0184] Bytes 16-17 store the number of rows (1913) and columns (1915) of slice 0, bytes 18-19 store the number of rows and columns of slice 1, and so on. In other words, each 2-byte group specifies the number of rows and columns of the slice. If 2-byte groups are not used to specify slice parameters, they should have a value of zero. Specifying slice parameters for more slices than the implementation limits or palette limits will result in an error. Unconfigured slices are set to an initial state with 0 rows and 0 columns.

[0185] Ultimately, the configuration in memory usually ends with a description such as all zeros for several consecutive bytes.

[0186] Exemplary chips and chip configuration storage

[0187] Figures 20(A)-20(D)An example of multiple registers 1819 is illustrated in Figure 20(A). As shown, each slice (TMM0 2001...TMMN 2003) has separate registers, each storing the row and column dimensions for that particular slice. StartP 2011 and StartRow 2013 are stored in separate registers. One or more status registers 2015 are set (e.g., TILES_CONFIGURED = 1) to indicate that the slice is configured for use.

[0188] Figure 20(B) illustrates multiple registers 1819. As shown, each slice has separate registers for its rows and columns. For example, TMM0 row configuration 2021, TMM0 column configuration 2023, StartP 2011, and StartRow 2013 are stored in separate registers. One or more status registers 2015 are set (e.g., TILES_CONFIGURED = 1) to indicate that the slice is configured for use.

[0189] Figure 20(C) illustrates a single register 1819. As shown, this register stores the slice configuration (rows and columns per slice) 2031, StartP 2011, and StartRow 2013 in a single register as a compact data register. One or more status registers 2015 are set (e.g., TILES_CONFIGURED = 1) to indicate that the slice is configured for use.

[0190] Figure 20(D) illustrates multiple registers 1819. As shown, a single register stores slice configurations (rows and columns per slice) 2031. StartP and StartRow are stored in separate registers 2011 and 2013. One or more status registers 2015 are set (e.g., TILES_CONFIGURED = 1) to indicate that the slice is configured for use.

[0191] Other combinations were conceived, such as combining the start registers into a single register in which they are displayed separately, and so on.

[0192] TILETFM2RI

[0193] As mentioned above, specialized hardware for general matrix multiplication (known as GEMM) is a good choice for improving peak computation (and energy efficiency) in certain applications such as deep learning. Some of these applications, including deep learning, can operate on input data elements with relatively few bits without sacrificing accuracy, provided that the output element has enough bits (i.e., more than the input).

[0194] Therefore, the disclosed methods and systems execute the transformation instruction TILETFM2RI, which transforms a matrix (slice)—that is, a two-dimensional (2D) data block—and writes the result to a row-interleaved (RowInt) formatted matrix / slice / 2D register. Specifically, a RowInt formatted matrix can be used when performing a fused multiply-add (FMA) operation (or other arithmetic) to perform an FMA and then combining (adding together) the results of adjacent operations. For example, all element-wise multiplications specified by a vector or slice FMA can be performed, and all these element-wise multiplications can then be added together with adjacent element pairs to an accumulator that is twice the size of the input value, producing the same total number of bits in the output as in the input. The TILETFM2RI instruction implements this optimization within a GEMM context.

[0195] Deep learning training (and potentially other applications) uses slices in both RowInt and regular formats. Therefore, applications can be accelerated by transforming slices from a traditional 2D block format (i.e., with normal rows and columns) to RowInt format. RowInt format takes a set of consecutive rows from a matrix in "normal" format and merges / interleaves those rows. In the context of GEMM operations, what is ultimately needed is to multiply each row of matrix A by each column of matrix B. If B is in this RowInt format, then each column of matrix B is effectively "shorter".

[0196] Figure 21 This is a block diagram illustrating the use of the TILETFM2RI instruction to accelerate matrix multiplication according to some embodiments. As shown, instruction 2101 includes opcode 2102 (e.g., TILETFM2RI) that instructs the processor to transform a specified source matrix into a specified destination matrix having a row-interleaved (RowInt) format. Specifically, in response to the opcode, the processor interleaves the J elements of each J-element subcolumn of the specified source matrix in row-major order to form a K-width submatrix of the specified destination matrix, the K-width submatrix having K columns and enough rows to hold the J elements.

[0197] Here, J 2108 and K 2110, which are equal to 4 and 2 respectively, will be specified in one or more of several ways: as operands for the TILETFM2RI instruction (as here), as a suffix or prefix to the specified opcode, as part of the immediate value provided by the instruction (e.g., J will be specified by the lower 8 bits of the 16-bit immediate value, while K will be specified by the higher 8 bits of the 16-bit immediate value), as part of a control register programmed by software before the instruction is issued (e.g., XTILECONFIG), or even as an architectural default. J and K may each represent an integer chosen from an unrestricted range of integer values.

[0198] Instruction 2101 further specifies the destination matrix (slice) location 2104 and the source matrix (slice) location 2106. Each specified matrix location can be in any of the following: a memory location, a set of vector registers, and a set of slice registers. Here, the specified source matrix 2112 and destination matrix 2116 each comprise thirty-two (32) word-sized elements. The specified source matrix 2112 comprises four rows and eight columns, while the specified destination matrix 2116 comprises two rows and sixteen columns. As shown, the specified destination matrix 2116 is a transformation of the specified source matrix 2112 in row-interleaved (RowInt) format.

[0199] A system 2100 for executing TILETFM2RI instructions is also shown. The system includes a specified source matrix (chip) 2112, execution circuitry 2114, and a specified destination matrix (chip) 2116. Both the specified destination matrix (chip) 2116 and the specified source matrix 2112 in "normal" form are routed to a matrix multiplication circuit (GEMM) 2118, thereby improving GEMM performance and efficiency.

[0200] Alternatively, inferior methods for transforming matrix data may exist, but they do not achieve the power and performance gains of the disclosed embodiments that execute the TILETFM2RI instructions. In some other methods, software can load data into vector / SIMD registers, perform the transformation using vector instructions, write the reformulated data to memory, and then load the reformulated data into 2D / vector / slice registers. However, format conversion in vector instructions is slow, requires complex software tweaking, and may require more space in the cache.

[0201] The disclosed embodiments improve upon alternative methods by allowing software to execute the TILETFM2RI instruction to transform two-dimensional (2D) data matrices (slices) and place them into RowInt-formatted 2D matrix (slice) registers. Advantageously, using this instruction is intended to be simpler for software compared to methods limited to the use of vector registers.

[0202] In one embodiment, as referenced below Figure 22BFurther shown and described, the processor executes TILETFM2RI instructions to transform words from a pair of source matrices (slices) and store the results in a pair of destination matrices (slices). TILETFM2RI transforms 2D data slices into a so-called RowInt format and places them into one or more 2D (i.e., slice) registers. In one embodiment, the lower-numbered slice of the source register pair represents the first 16 rows of the slice pair, while the higher-numbered slice represents the next 16 rows. The first destination slice will contain data from the first 16 columns of the source register pair, while the second destination slice will contain data from a second set of 16 columns of the source register pair. In this embodiment, the source slice (register block) pair is top / bottom (meaning the first slice holds rows 1-16, while the second slice holds the remaining rows 17-32). The destination slice (register block) pair is left / right (meaning the first slice holds columns 1-16, while the second slice holds the remaining columns 17-32).

[0203] Advantageously, the disclosed embodiments can handle data elements of any size (e.g., 1 byte, 2 bytes, 4 bytes). This includes elements of subbyte size, such as 2-bit or 4-bit data elements.

[0204] As a further advantage, the disclosed embodiments can also handle different numbers of input and output chips, including a single chip, a pair (such as...). Figure 21 and Figure 22A -B (as shown), or more than one pair.

[0205] The input data for this instruction can come from one or more slice registers, from a set of vector registers, or from memory. The instruction can specify multiple slice or vector registers as source and destination, and each can be explicitly encoded. In some embodiments, on the other hand, the instruction specifies a first register (e.g., X), while the remaining registers are fixed functions of the first register (e.g., X+1, X+2, etc., or X+2, X+4, etc.). In some embodiments, the TILETFM2RI instruction specifies a memory location as the location of one or more source matrix (slices) or destination matrix (slices).

[0206] When transforming rectangular slices (with more rows than columns, or vice versa), slice architectures can have asymmetric limitations on the number of rows and columns. Therefore, some embodiments require more or fewer registers than outputs to hold the inputs.

[0207] In some embodiments, the software is used to add padding to an output slice (e.g., to make the slice square) or remove padding from an input slice. The disclosed embodiments allow this to be done. In some embodiments, the TILETFM2RI instruction uses a parameter (e.g., in a register, as an immediate value, or as part of an opcode) that indicates the number of elements to be added as padding to each output row (or column). The value of the padding element can be zero, or some other pre-selected value, or this can be input to the instruction (as a register, an immediate value, or as part of an opcode). Alternatively or additionally, the instruction can take a parameter that indicates the elements to be removed from each input row or column and indicates whether these elements are at the beginning, end, or some of each row / column.

[0208] refer to Figure 21-23 The system and method for executing the TILETFM2RI instructions are further shown and described. References Figure 24-2 6. The format of the TILETFM2RI instruction is further shown and described.

[0209] Exemplary execution

[0210] Figure 22A The illustration shows an exemplary execution of the TILETFM2RI instruction according to some embodiments. As shown, instruction 2201 includes opcode 2202 (e.g., TILETFM2RI) that instructs the processor to transform a specified source matrix into a specified destination matrix having a row-interleaved (RowInt) format. Specifically, in response to the opcode, the processor is configured to interleave J elements of each J-element subcolumn of the specified source matrix in row-major order to form a K-width submatrix of the specified destination matrix, the K-width submatrix having K columns and enough rows to hold J elements.

[0211] Here, J 2208 and K 2210, which are equal to 4 and 2 respectively, will be specified in one or more of several ways: as operands for the TILETFM2RI instruction (as here), as a suffix or prefix to the specified opcode, as part of the immediate value provided by the instruction (e.g., J will be specified by the lower 8 bits of the 16-bit immediate value, and K will be specified by the higher 8 bits of the 16-bit immediate value), as part of a control register programmed by software before the instruction is issued (e.g., XTILECONFIG), or even as an architectural default. J and K may each represent an integer chosen from an unrestricted range of integer values.

[0212] Instruction 2201 further specifies the destination matrix (slice) location 2204 and the source matrix (slice) location 2206. Each specified matrix location can be in any of the following: a memory location, a set of vector registers, and a set of slice registers. Here, the specified source matrix 2212 and destination matrix 2216 each comprise thirty-two (32) word-sized elements. The specified source matrix 2212 comprises four rows and eight columns, while the specified destination matrix 2216 comprises two rows and sixteen columns. As shown, the specified destination matrix 2216 is a row-interleaved (RowInt) transformation of the specified source matrix 2212.

[0213] A system 2200 for executing TILETFM2RI instructions is also shown. The system includes a designated source matrix (chip) 2212, execution circuitry 2214, and a designated destination matrix (chip) 2216.

[0214] In operation, a fetch and decode circuit (not shown) is used to fetch and decode the TILETFM2RI instruction 2201. An execution circuit 2216 is used to respond to the decoded transformation instruction by transforming the specified source matrix into a specified RowInt-formatted destination matrix via the following operation: interleaving the J elements of each J-element subcolumn of the specified source matrix in row-major order to form a K-width submatrix of the specified destination matrix, the K-width submatrix having K columns and enough rows to hold J elements. Reference Figure 22B-23 , Figure 28A -B and Figure 29A -B further illustrates and describes the execution of the TILETFM2RI instruction. (See reference) Figure 24-2 6. The format of the TILETFM2RI instruction is further shown and described.

[0215] Figure 22B The illustration shows an exemplary execution of the TILETFM2RI instruction according to some embodiments. As shown, the format of instruction 2252 includes fields for specifying opcode 2254 (e.g., TILETFM2RI), and destination matrix (slice) position 2256, second destination matrix (slice) position 2258, source matrix (slice) position 2260, and second source matrix (slice) position 2262.

[0216] Instruction 2252 further specifies J 2264 and K 2266, which are equal to 4 and 2 here, respectively. However, it should be noted that J and K can be specified in one or more of several other ways: as operands of the TILETFM2RI instruction, as a suffix or prefix of the specified opcode, as part of the immediate value provided by the instruction (e.g., J would be specified by the lower 8 bits of the 16-bit immediate value, while K would be specified by the higher 8 bits of the 16-bit immediate value), as part of a control register programmed by software before the instruction is issued (e.g., XTILECONFIG), or even as an architectural default. J and K can each represent an integer selected from an unrestricted range of integer values.

[0217] A system 2250 for executing TILETFM2RI instructions 2252 is also shown. The system includes a designated first source matrix (chip) 2268 and a designated second source matrix (chip) 2275. System 2250 further includes execution circuitry 2272 and designated first and second destination matrices (chips) 2274 and 2276.

[0218] In operation, the processor executes the TILETFM2RI instruction 2252 to transform words from a pair of source matrices (slices) and store the results in a pair of destination matrices (slices). TILETFM2RI transforms the specified source 1 2268 matrix (slice) and the specified source 2 2275 matrix (slice) into row-interleaved (RowInt) format and places them in the specified destinations 1 2274 and 2 2276. In one embodiment, as shown, the lower-numbered slice of the source slice pair represents the first 16 rows of the slice pair, while the higher-numbered slice represents the next 16 rows. The first destination slice is used to contain the data from the first 16 columns of the source slice pair, while the second destination slice is used to contain the data from the second set of 16 columns of the source slice pair. In this embodiment, the source slice (register block) pair is presented as top / bottom (meaning the first slice holds rows 1-16, while the second slice holds the remaining rows 17-32). The destination slice (register block) pairs are represented as left / right (meaning the first slice stores columns 1-16, while the second slice stores the remaining columns 17-32). In other embodiments, the destination slice (register block) pairs are represented as top / bottom.

[0219] In operation, a fetch and decode circuit (not shown) is used to fetch and decode the TILETFM2RI instruction 2252. Execution circuitry 2272 is used to respond to the decoded matrix transformation instruction by transforming the specified source matrix into a specified RowInt-formatted destination matrix via: interleaving J elements of each J-element subcolumn of the specified source matrix in row-major order into a K-width submatrix of the specified destination matrix, the K-width submatrix having K columns and enough rows to hold the J elements. In other embodiments, the J elements are interleaved into the K-width submatrix in column-major order.

[0220] refer to Figure 21 , Figure 23 , Figure 28A -B and Figure 29A -B further illustrates and describes the execution of the TILETFM2RI instruction. (See reference) Figure 24-2 6. The format of the TILETFM2RI instruction is further shown and described.

[0221] Execution of (multiple) exemplary methods

[0222] Figure 23 The illustration depicts an embodiment of a processor executing flow 2300 for processing TILETFM2RI instructions. As shown, at 2301, the processor uses fetch circuitry to fetch instructions with a format having fields specifying the opcode and the positions of the source and destination matrices, wherein the opcode instructs the processor to transform the specified source matrix into a specified destination matrix having a row-interleaved (RowInt) format. Specifically, in response to the opcode, the processor interleaves J elements of each J-element subcolumn of the specified source matrix in row-major order into a K-width submatrix of the specified destination matrix, the K-width submatrix having K columns and enough rows to hold J elements.

[0223] J and K will be specified in one or more of several ways: as operands of the TILETFM2RI instruction, as a suffix or prefix of the specified opcode, as part of the immediate value provided by the instruction (e.g., J will be specified by the lower 8 bits of the 16-bit immediate value, and K will be specified by the higher 8 bits of the 16-bit immediate value), as part of a control register programmed by software before the instruction is issued (e.g., XTILECONFIG), or even as an architectural default. J and K may each represent an integer selected from an unrestricted range of integer values.

[0224] At 2303, the processor uses decoding circuitry to decode the fetched instruction. For example, the fetched TILETFM2RI instruction is decoded by decoding circuitry such as that detailed herein. In the context of the system shown, the decoding circuitry is similar to at least the one described in reference [reference needed]. Figure 13 , Figure 14 and Figure 28A -B shows and describes the decoding circuit.

[0225] At 2305, the execution of the decoded instructions is scheduled (as needed), which is optional (as indicated by its dashed border), meaning it can occur at different times or not at all. At 2307, the processor responds to the decoded instructions by using execution circuitry to transform the specified source matrix into a specified RowInt-formatted destination matrix via: interleaving J elements of each J-element subcolumn of the specified source matrix in either row-major or column-major order to form a K-width submatrix of the specified destination matrix, which has K columns and enough rows to hold the J elements. In other embodiments, the J elements are interleaved into the K-width submatrix in column-major order.

[0226] refer to Figure 3-14 The execution circuitry is further shown and described. In some embodiments, the execution circuitry is a matrix operation accelerator, such as accelerator 307 shown and described. Figure 3 An accelerator. In some embodiments, the execution circuit is a matrix operation circuit, such as matrix operation circuit 405. Figure 4 ), 505 Figure 5 ), 1213 Figure 12 ) and 1327 ( Figure 13 ).

[0227] In some embodiments, at 2309, the instruction is submitted or withdrawn, which is optional (as indicated by its dashed border), and is reflected in the fact that it may occur at different times or not at all.

[0228] (Multiple) Exemplary Instruction Formats

[0229] Figure 24 This is a block diagram illustrating the format of the TILETFM2RI instruction according to some embodiments. As shown, the TILETFM2RI instruction 2400 includes a field for specifying an opcode 2402 that instructs the processor to transform a specified source matrix into a specified destination matrix having a row-interleaved (RowInt) format. An example of opcode 2402 is TILETFM2RI*.

[0230] Instruction 2400 further includes a destination matrix (slice) location 2404 and a source matrix (slice) location 2406, and an optional second destination matrix (slice) location 2408, and an optional second source matrix (slice) location 2410. Each of the specified source and destination matrix locations can be in any of a memory location, a set of vector registers, and a set of slice registers. The second source matrix (slice) location 2410 and the second destination matrix (slice) location 2408 are optional, as indicated by their dashed borders, which is reflected in the fact that the TILETFM2RI instruction can specify only a single source and destination, as in Figure 21 In the middle, or you can specify two sources and two destinations, such as in Figure 22B In other embodiments (not shown), TILETFM2RI may specify three or more sources and three or more destinations.

[0231] The TILETFM2RI instruction 2400 further includes several optional parameters to control the processor's behavior, including J 2412, K 2414, element size 2416 (binary digit, nibble, byte, word, double word, or quadword), element format 2418 (compact or scalar single-precision or double-precision floating-point data and compact or scalar integer data), M 2420 (source row), N 2422 (source column), and mask 2424 (a multi-bit value with one bit for each destination element, used to control whether the destination element will be updated, or used to control whether the destination element will be zeroed or merged).

[0232] Opcode 2402 is shown as including an asterisk, which is used to convey that additional prefixes and / or suffixes can be added to specify instruction behavior. One or more of the instruction modifiers 2412, 2414, 2416, 2418, 2420, 2422, and 2424 can be specified by using a prefix or suffix of opcode 2402.

[0233] In some embodiments, one or more of the optional instruction modifiers 2412, 2414, 2416, 2418, 2420, 2422, and 2424 are encoded in an immediate numeric field (not shown) optionally included in instruction 2400. In some embodiments, one or more of the optional instruction modifiers 2412, 2414, 2416, 2418, 2420, 2422, and 2424 are specified via a configuration / status register (e.g., XTILECONFIG). In other words, when any one or more of the optional modifiers 2412, 2414, 2416, 2418, 2420, 2422, and 2424 are not specified by an instruction, they sometimes use implicit parameters inherited from other parts of the chip architecture.

[0234] Detailed exemplary systems, processors, and simulations

[0235] This article details examples of hardware, software, and other components used to execute the instructions described above. For instance, the following description details various aspects of instruction execution, including various pipeline stages such as fetching, decoding, scheduling, execution, and termination.

[0236] Instruction set

[0237] An instruction set may include one or more instruction formats. A given instruction format may define various fields (e.g., number of bits, bit positions) to specify the operation to be performed (e.g., opcode) and operand(s) and / or other data fields (e.g., mask) to which the operation will be performed, and so on. Some instruction formats are further decomposed by defining instruction templates (or subformats). For example, an instruction template for a given instruction format may be defined as a different subset of the fields of that instruction format (the included fields are generally in the same order, but at least some fields have different bit positions because fewer fields are included), and / or defined as a given field that is interpreted in a different way. Thus, each instruction of the ISA is expressed using a given instruction format (and, if defined, according to a given instruction template in the instruction template of that instruction format) and includes fields for specifying the operation and operands. For example, the exemplary ADD (addition) instruction has a specific opcode and instruction format, which includes an opcode field for specifying the opcode and an operand field for selecting operands (source 1 / destination and source 2); and the appearance of the ADD instruction in the instruction stream will cause the operand field to contain specific content for selecting the specific operand. Sets of SIMD extensions known as Advanced Vector Extensions (AVX) (AVX1 and AVX2) and utilizing Vector Extensions (VEX) encoding schemes have been introduced and / or released (see, for example, September 2014). See the IA-64 and IA-32 architecture software developer manual; and also see the October 2014 edition. (Advanced Vector Extension Programming Reference).

[0238] Exemplary instruction format

[0239] Examples of the instructions described herein can be embodied in different formats. Furthermore, exemplary systems, architectures, and pipelines are detailed below. Examples of the instructions can be executed on such systems, architectures, and pipelines, but are not limited to those detailed herein.

[0240] General Vector-Friendly Instruction Format

[0241] A vector-friendly instruction format is an instruction format suitable for vector instructions (e.g., the existence of specific fields dedicated to vector operations). Although an embodiment in which both vector and scalar operations are supported via a vector-friendly instruction format has been described, alternative embodiments use only vector operations via the vector-friendly instruction format.

[0242] Figures 25A-25B This is a block diagram illustrating a general vector-friendly instruction format and its instruction template according to an embodiment. Figure 25A This is a block diagram illustrating a general vector-friendly instruction format and its Class A instruction template according to an embodiment; while Figure 25B This is a block diagram illustrating a general vector-friendly instruction format and its Class B instruction template according to an embodiment. Specifically, Class A and Class B instruction templates are defined for the general vector-friendly instruction format 2500, both of which include instruction templates for no memory access 2505 and instruction templates for memory access 2520. In the context of the vector-friendly instruction format, the term "general" means an instruction format that is not bound to any particular instruction set.

[0243] Although embodiments in which the vector-friendly instruction format supports the following will be described: 64-byte vector operand length (or size) with a 32-bit (4-byte) or 64-bit (8-byte) data element width (or size) (and thus, the 64-byte vector consists of 16 double-word-sized elements, or alternatively 8 quad-word-sized elements); 64-byte vector operand length (or size) with a 16-bit (2-byte) or 8-bit (1-byte) data element width (or size); 32-byte vector operand length (or size) with a 32-bit (4-byte) data element width (or size). ), 64-bit (8-byte), 16-bit (2-byte), or 8-bit (1-byte) data element widths (or dimensions); and 16-byte vector operand lengths (or dimensions) with 32-bit (4-byte), 64-bit (8-byte), 16-bit (2-byte), or 8-bit (1-byte) data element widths (or dimensions); however, alternative embodiments may support larger, smaller, and / or different vector operand sizes (e.g., 256-byte vector operands) with larger, smaller, or different data element widths (e.g., 128-bit (16-byte) data element widths).

[0244] Figure 25A The Class A instruction templates include: 1) within the instruction template for no memory access 2505, an instruction template for a fully round-control operation 2510 without memory access and an instruction template for a data transformation operation 2515 without memory access are shown; and 2) within the instruction template for memory access 2520, an instruction template for the timeliness of memory access 2525 and an instruction template for the non-timeliness of memory access 2530 are shown. Figure 25BThe B-type instruction templates include: 1) within the instruction template for no memory access 2505, an instruction template showing a partial rounding control type operation 2512 for write mask control without memory access and an instruction template showing a vsize type operation 2517 for write mask control without memory access; and 2) within the instruction template for memory access 2520, an instruction template showing a write mask control 2527 for memory access.

[0245] The General Vector Friendly Instruction Format 2500 includes the following as listed in Figures 25A-25B The fields in the order shown in the diagram are as follows.

[0246] Format field 2540 – A specific value in this field (instruction format identifier value) uniquely identifies the vector-friendly instruction format, and thus indicates that the instruction appears in the instruction stream in the vector-friendly instruction format. Therefore, this field is not required for instruction sets that only have a general vector-friendly instruction format, and in this sense, it is optional.

[0247] Basic operation field 2542 – its content distinguishes different basic operations.

[0248] Register index field 2544—its contents, either directly or generated from addresses, specify the location of the source or destination operand in a register or in memory. These fields include a sufficient number of bits to select N registers from a PxQ (e.g., 32x512, 16x128, 32x1024, 64x1024) register file. Although in one embodiment N can be up to three source registers and one destination register, alternative embodiments may support more or fewer source and destination registers (e.g., up to two sources, where one of these sources also serves as the destination; up to three sources, where one of these sources also serves as the destination; up to two sources and one destination).

[0249] The modifier field 2546 distinguishes between instructions in the generic vector instruction format that specify memory access and those that do not; that is, it differentiates between instruction templates with no memory access 2505 and instruction templates with memory access 2520. Memory access operations read and / or write to the memory hierarchy (in some cases, using values ​​in registers to specify the source and / or destination addresses), while non-memory access operations do not (e.g., the source and destination are registers). Although in one embodiment, this field also selects between three different ways to perform memory address calculations, alternative embodiments may support more, fewer, or different ways to perform memory address calculations.

[0250] Extended operation field 2550 – its content distinguishes which of the various operations to be performed in addition to the basic operation. This field is context-dependent. In one embodiment, this field is divided into class field 2568, α field 2552, and β field 2554. Extended operation field 2550 allows multiple sets of common operations to be performed in a single instruction instead of two, three, or four instructions.

[0251] The ratio field 2560—its contents allow for use in memory address generation (e.g., for use with (2...) 比例 The content of the index field is scaled proportionally by the address generated by the index + base address.

[0252] Displacement field 2562A—its contents are used as part of memory address generation (e.g., for use with (2 比例 Address generation (index + base address + offset).

[0253] The displacement factor field 2562B (note that the offset field 2562A directly indicates the use of one or the other on the offset factor field 2562B) – its contents are used as part of address generation; it specifies the displacement factor that will scale the size (N) of the memory access – where N is the number of bytes in the memory access (e.g., for use with (2... 比例 *Address generation (index + base address + scaled displacement). Redundant low-order bits are ignored, and therefore the contents of the displacement factor field are multiplied by the total memory operand size (N) to generate the final displacement that will be used in calculating the effective address. The value of N is determined by the processor hardware at runtime based on the full opcode field 2574 (described later herein) and the data manipulation field 2554C. Displacement field 2562A and displacement factor field 2562B are not used in the instruction template of no memory access 2505 and / or different embodiments may implement only one of them or neither of them; in this sense, displacement field 2562A and displacement factor field 2562B are optional.

[0254] The data element width field 2564—its content distinguishes which of the multiple data element widths will be used (in some embodiments for all instructions; in other embodiments for only some instructions). This field is unnecessary if only one data element width is supported and / or if one aspect of the opcode is used to support the data element width; in this sense, the field is optional.

[0255] The write mask field 2570—its contents control, element-by-element position, whether the position of data elements in the destination vector operand reflects the results of the base and augmentation operations. Type A instruction templates support merge-write masks, while Type B instruction templates support both merge-write masks and zero-write masks. When merging, the vector mask allows protecting any set of elements in the destination from updates during the execution of any operation (specified by the base and augmentation operations); in another embodiment, it preserves the old value of each element in the destination where the corresponding mask bit has a 0. Conversely, when zeroing, the vector mask allows zeroing any set of elements in the destination during the execution of any operation (specified by the base and augmentation operations); in one embodiment, the elements in the destination are set to 0 when the corresponding mask bit has a 0 value. A subset of this functionality is the ability to control the vector length of the operation being performed (i.e., the span from the first to the last element being modified), however, the modified elements do not necessarily have to be consecutive. Thus, the write mask field 2570 allows for partial vector operations, including load, store, arithmetic, logical, etc. Although an embodiment is described in which the content of the write mask field 2570 selects one of a plurality of write mask registers containing the write mask to be used (and thus the content of the write mask field 2570 indirectly identifies the mask to be performed), alternative embodiments alternatively or additionally allow the content of the mask write field 2570 to directly specify the mask to be performed.

[0256] The immediate number field 2572—its contents allow for the specification of an immediate number. This field is optional in implementations of generic vector-friendly formats that do not support immediate numbers and in instructions that do not use immediate numbers.

[0257] Class field 2568 – its content distinguishes instructions from different classes. (See reference) Figures 25A-25B The content of this field allows selection between Class A and Class B instructions. Figures 25A-25B In this context, rounded rectangles are used to indicate that a specific value exists in a field (e.g., in...). Figures 25A-25B The two fields are used for class field 2568, namely class A 2568A and class B 2568B.

[0258] Type A Instruction Template

[0259] In the case of instruction template 2505 for type A non-memory access, the α field 2552 is interpreted as the RS field 2552A, whose content distinguishes which of the different extended operation types to be performed (e.g., for instruction templates specifying rounding 2510 and data transformation 2515 for non-memory access, respectively, rounding 2552A.1 and data transformation 2552A.2), while the β field 2554 distinguishes which of the specified types of operations to be performed. In instruction template 2505 for non-memory access, the scaling field 2560, the displacement field 2562A, and the displacement-scale field 2562B are absent.

[0260] Instruction template without memory access – Full round control operation

[0261] In the instruction template of a fully round-controlled operation 2510 without memory access, the β field 2554 is interpreted as a round control field 2554A that provides static rounding for its(plural) contents. Although the round control field 2554A in the described embodiment includes a suppress all floating-point exception (SAE) field 2556 and a round operation control field 2558, alternative embodiments may support both concepts, which may be encoded as the same field, or only one or the other of these concepts / fields may be present (e.g., only the round operation control field 2558 may be present).

[0262] SAE field 2556 – its content distinguishes whether exception event reporting is disabled; when the content of SAE field 2556 indicates that suppression is enabled, the given instruction does not report any kind of floating-point exception flag and does not invoke any floating-point exception handler.

[0263] The rounding operation control field 2558 distinguishes which of a set of rounding operations to perform (e.g., round up, round down, round to zero, and round to nearest). Thus, the rounding operation control field 2558 allows the rounding mode to be changed instruction-by-instruction. In one embodiment where the processor includes a control register for specifying the rounding mode, the content of the rounding operation control field 2550 overrides that register value.

[0264] Instruction template without memory access – data transformation operations

[0265] In the instruction template of the data transformation operation 2515 without memory access, the β field 2554 is interpreted as the data transformation field 2554B, the contents of which distinguish which of the multiple data transformations to be performed (e.g., no data transformation, mixing, broadcasting).

[0266] In the case of instruction template 2520 for Class A memory access, α field 2552 is interpreted as eviction hint field 2552B, the content of which distinguishes which eviction hint to use (in... Figure 25A In the instruction template for memory access timing 2525 and memory access non-timing 2530, timing 2552B.1 and non-timing 2552B.2 are specified respectively, while the β field 2554 is interpreted as a data manipulation field 2554C, the content of which distinguishes which of the multiple data manipulation operations (also called primitives) to be performed (e.g., no manipulation, broadcast, source upcast, and destination downcast). The instruction template for memory access 2520 includes a scaling field 2560 and optionally includes a displacement field 2562A or a displacement scaling field 2562B.

[0267] Vector memory instructions use translation support to perform vector loads from and vector stores to memory. Like regular vector instructions, vector memory instructions transfer data to / from memory in an element-by-element manner, where the actual elements transferred are specified by the contents of a vector mask selected as the write mask.

[0268] Instruction templates for memory access – time-sensitive

[0269] Time-sensitive data is data that can be reused quickly enough to benefit from cache operations. However, this is a hint, and different processors can implement it in different ways, including ignoring the hint entirely.

[0270] Instruction templates for memory access – non-time-sensitive

[0271] Non-time-sensitive data is data that is unlikely to be reused quickly enough to benefit from cache operations in the L1 cache and should be given eviction priority. However, this is a hint, and different processors can implement it in different ways, including ignoring the hint entirely.

[0272] Type B Instruction Template

[0273] In the case of the B-type instruction template, α field 2552 is interpreted as write mask control (Z) field 2552C, the content of which distinguishes whether the write mask controlled by write mask field 2570 should be merged or zeroed.

[0274] In the case of instruction template 2505 for Class B non-memory access, a portion of the β field 2554 is interpreted as the RL field 2557A, the content of which distinguishes which of the different extended operation types to be performed (for example, the instruction template for the write mask control type operation 2512 for non-memory access and the instruction template for the write mask control type operation 2517 for non-memory access specify rounding 2557A.1 and vector length (VSIZE) 2557A.2, respectively), while the remaining portion of the β field 2554 distinguishes which of the specified types of operations to be performed. In the instruction template for non-memory access 2505, the scaling field 2560, the displacement field 2562A, and the displacement-scale field 2562B are absent.

[0275] In the instruction template of the write mask control section round control operation 2510 without memory access, the remainder of the β field 2554 is interpreted as the round operation field 2559A, and exception event reporting is disabled (the given instruction does not report any kind of floating-point exception flag and does not invoke any floating-point exception handler).

[0276] Rounding operation control field 2559A—like rounding operation control field 2558, its contents distinguish which of a set of rounding operations to perform (e.g., round up, round down, round to zero, and round to nearest). Thus, rounding operation control field 2559A allows the rounding mode to be changed instruction-by-instruction. In one embodiment where the processor includes a control register for specifying the rounding mode, the contents of rounding operation control field 2550 override that register value.

[0277] In the instruction template of the write mask control VSIZE type operation 2517 without memory access, the remainder of the β field 2554 is interpreted as the vector length field 2559B, the contents of which distinguish which of the multiple data vector lengths to be executed (e.g., 128 bytes, 256 bytes, or 512 bytes).

[0278] In the case of the instruction template for Class B memory access 2520, a portion of the β field 2554 is interpreted as a broadcast field 2557B, the content of which distinguishes whether a broadcast data manipulation operation is to be performed, while the remainder of the β field 2554 is interpreted as a vector length field 2559B. The instruction template for memory access 2520 includes a scaling field 2560 and optionally includes a displacement field 2562A or a displacement-scaling field 2562B.

[0279] For the general vector-friendly instruction format 2500, the complete opcode field 2574 is shown to include a format field 2540, a basic operation field 2542, and a data element width field 2564. Although one embodiment is shown in which the complete opcode field 2574 includes all of these fields, in embodiments that do not support all of these fields, the complete opcode field 2574 includes fewer than all of them. The complete opcode field 2574 provides the operation code (opcode).

[0280] The expanded operation field 2550, data element width field 2564, and write mask field 2570 allow these features to be specified on an instruction-by-instruction basis in a generic vector-friendly instruction format.

[0281] Combining the write mask field and the data element width field creates various types of instructions, as these instructions allow the mask to be applied based on different data element widths.

[0282] The various instruction templates appearing within classes A and B are advantageous in different situations. In some embodiments, different processors or different cores within a processor may support only class A, only class B, or both. For example, a high-performance general-purpose out-of-order core designed for general-purpose computing may support only class B, a core designed primarily for graphics and / or scientific (throughput) computing may support only class A, and a core designed for both general-purpose computing and graphics and / or scientific (throughput) computing may support both class A and class B (of course, cores with some mixture of templates and instructions from both classes, but not all templates and instructions from both classes, are within the scope of this invention). Similarly, a single processor may include multiple cores, all of which support the same class, or different cores that support different classes. For example, in a processor with separate graphics cores and general-purpose cores, one graphics core designed primarily for graphics and / or scientific computing may support only class A, while one or more general-purpose cores may be high-performance general-purpose cores with out-of-order execution and register renaming designed for general-purpose computing and supporting only class B. Another processor, which does not have a separate graphics core, may include one or more general-purpose ordered or out-of-order cores that support both Class A and Class B. Of course, in different embodiments, features from one class may also be implemented in other classes. This will enable programs written in high-level languages ​​to become (e.g., just-in-time compiled or statically compiled) various different executable forms, including: 1) forms with only instructions from the class(s) supported by the target processor(s) for execution; or 2) forms with alternative routines and control flow code that uses different combinations of instructions from all classes, the control flow code selecting these routines to execute based on instructions supported by the processor currently executing the code.

[0283] Exemplary dedicated vector-friendly instruction format

[0284] Figure 26A This is a block diagram illustrating an exemplary dedicated vector-friendly instruction format according to an embodiment. Figure 26A The diagram illustrates a dedicated vector-friendly instruction format 2600, which specifies the position, size, interpretation, and order of each field, as well as the values ​​of some of those fields. In this sense, the dedicated vector-friendly instruction format 2600 is dedicated. The dedicated vector-friendly instruction format 2600 can be used to extend the x86 instruction set, and thus some of its fields are similar to or identical to those used in existing x86 instruction sets and their extensions (e.g., AVX). This format maintains consistency with the prefix-coded fields, actual opcode byte fields, MOD R / M fields, SIB fields, shift fields, and immediate numeric fields of existing x86 instruction sets with extensions. The illustrations are from Figure 25. Figure 26A The fields are mapped to the fields from Figure 25.

[0285] It should be understood that although embodiments are described in the context of the general vector-friendly instruction format 2500 with reference to the dedicated vector-friendly instruction format 2600 for illustrative purposes, the invention is not limited to the dedicated vector-friendly instruction format 2600 unless otherwise stated. For example, the general vector-friendly instruction format 2500 envisions various possible sizes for various fields, while the dedicated vector-friendly instruction format 2600 is shown as a field with a specific size. As a specific example, although the data element width field 2564 is illustrated as a one-bit field in the dedicated vector-friendly instruction format 2600, the invention is not limited to this (i.e., other sizes for the data element width field 2564 are envisioned in the general vector-friendly instruction format 2500).

[0286] General Vector Friendly Instruction Format 2500 includes the following according to Figure 26A The fields in the order shown in the diagram are as follows.

[0287] EVEX prefix 2602 (bytes 0-3) — encoded in four-byte format.

[0288] Format field 2540 (EVEX byte 0, bits [7:0]) — The first byte (EVEX byte 0) is format field 2540, and it contains 0x62 (in one embodiment, a unique value used to distinguish vector-friendly instruction formats).

[0289] The second to fourth bytes (EVEX bytes 1-3) include multiple bit fields that provide specialized capabilities.

[0290] The REX field 2605 (EVEX byte 1, bits [7-5]) consists of the EVEX.R bit field (EVEX byte 1, bits [7]–R), the EVEX.X bit field (EVEX byte 1, bits [6]–X), and (2557BEX byte 1, bits [5]–B). The EVEX.R, EVEX.X, and EVEX.B bit fields provide the same functionality as the corresponding VEX bit fields and are encoded in two's complement form, i.e., ZMM0 is encoded as 1111B and ZMM15 is encoded as 0000B. The other fields of these instructions encode the lower three bits (rrr, xxx, and bbb) of the register index as known in the art, thereby forming Rrrr, Xxxx, and Bbbb by adding EVEX.R, EVEX.X, and EVEX.B.

[0291] REX' field 2510 – This is the first part of REX' field 2510 and is the EVEX.R' bit field (EVEX byte 1, bit [4] – R') used to encode the higher 16 or lower 16 registers of the extended 32-register set. In one embodiment, this bit is stored in a bit-inverted format along with other bits indicated below to distinguish it from the BOUND instruction (in the known x86 32-bit mode), which has an actual opcode byte of 62, but does not accept the value 11 in the MOD field in the MOD R / M field (described below); an alternative embodiment does not store the bit of this indication and the bits of the other indications below in an inverted format. The value 1 is used to encode the lower 16 registers. In other words, R'Rrrr is formed by combining EVEX.R', EVEX.R, and other RRRs from other fields.

[0292] Opcode mapping field 2615 (EVEX byte 1, bits [3:0] – mmmm) – its contents encode the implicit preamble opcode byte (0F, 0F 38, or 0F 3).

[0293] The width field of the data element is 2564 (EVEX byte 2, bits [7] – W) – represented by the notation EVEX.W. EVEX.W is used to define the granularity (size) of the data type (32-bit data element or 64-bit data element).

[0294] EVEX.vvvv 2620 (EVEX byte 2, bits [6:3]-vvvv) — The function of EVEX.vvvv can include the following: 1) EVEX.vvvv encodes the first source register operand specified in reverse (two's complement) form, and is valid for instructions with two or more source operands; 2) EVEX.vvvv encodes the destination register operand specified in two's complement form for a specific vector shift; or 3) EVEX.vvvv does not encode any operands; this field is reserved and should contain 1111b. Thus, the EVEX.vvvv field 2620 encodes the four lower-order bits of the first source register specifier stored in reverse (two's complement) form. Depending on the instruction, additional different EVEX bit fields are used to extend the specifier size to 32 registers.

[0295] EVEX.U 2568 Class Field (EVEX byte 2, bit [2]-U) — If EVEX.U = 0, it indicates Class A or EVEX.U0; if EVEX.U = 1, it indicates Class B or EVEX.U1.

[0296] The prefix-encoded field 2625 (EVEX byte 2, bits [1:0]-pp) provides additional bits for the underlying operation field. Besides supporting legacy SSE instructions in EVEX prefix format, this also has the benefit of compressing the SIMD prefix (EVEX prefixes require only 2 bits instead of bytes to represent SIMD prefixes). In one embodiment, to support legacy SSE instructions using SIMD prefixes (66H, F2H, F3H) in both legacy and EVEX prefix formats, these legacy SIMD prefixes are encoded into a SIMD prefix-encoded field; and at runtime, they are expanded to the legacy SIMD prefix before being provided to the PLA for the decoder (thus, the PLA can execute both legacy instructions in legacy format and legacy instructions in EVEX format without modification). While newer instructions can use the contents of the EVEX prefix-encoded field directly as opcode expansion, certain embodiments expand in a similar manner for consistency, but allow for different meanings specified by these legacy SIMD prefixes. Alternative embodiments may redesign the PLA to support 2-bit SIMD prefix encoding, thus eliminating the need for expansion.

[0297] α field 2552 (EVEX byte 3, bit [7] – EH, also known as EVEX.EH, EVEX.rs, EVEX.RL, EVEX.writemask control, and EVEX.N; also illustrated as α) – as previously stated, this field is context-sensitive.

[0298] β field 2554 (EVEX byte 3, bits [6:4] - SSS, also known as EVEX.s 2-0 EVEX.r 2-0 EVEX.rr1, EVEX.LL0, EVEX.LLB (also illustrated in βββ diagram) — as mentioned before, this field is context-sensitive.

[0299] REX' field 2510 — This is the remainder of the REX' field and is the EVEX.V' bit field (EVEX byte 3, bit [3] – V') that can be used to encode either the higher 16 or lower 16 registers of the extended 32-register set. This bit is stored in a bit-inverted format. A value of 1 is used to encode the lower 16 registers. In other words, V'VVVV is formed by combining EVEX.V' and EVEX.vvvv.

[0300] Write mask field 2570 (EVEX byte 3, bits [2:0]-kkk) – its contents specify the index of the register in the write mask register, as previously described. In one embodiment, the specific value EVEX.kkk = 000 has the special behavior of implying that no write mask is used for a particular instruction (this can be implemented in various ways, including using a write mask hardwired to all objects or hardware that bypasses the mask hardware).

[0301] The actual opcode field 2630 (byte 4) is also known as the opcode byte. Part of the opcode is specified in this field.

[0302] The MOD R / M field 2640 (byte 5) includes the MOD field 2642, the Reg field 2644, and the R / M field 2646. As previously described, the content of the MOD field 2642 distinguishes between memory access operations and non-memory access operations. The Reg field 2644 can be categorized into two cases: encoding the destination register operand or the source register operand; or it can be treated as an opcode extension and not used to encode any instruction operand. The R / M field 2646 can be categorized as encoding instruction operands that reference memory addresses; or encoding the destination register operand or the source register operand.

[0303] The Proportional, Index, Base (SIB) byte (byte 6) – as previously mentioned, the contents of SIB2650 are used for memory address generation. SIB.xxx 2654 and SIB.bbb 2656 – the contents of these fields have previously been mentioned for register indices Xxxx and Bbbb.

[0304] Displacement field 2562A (bytes 7-10) — When MOD field 2642 contains 10, bytes 7-10 are displacement field 2562A, and it works the same as a traditional 32-bit displacement (disp32), and works at the byte granularity.

[0305] Byte 7 (disp8) is the shift factor segment 2562B – when MOD field 2642 contains 0s and 1s, byte 7 is the shift factor segment 2562B. This field is located in the same position as the traditional x86 instruction set 8-bit shift (disp8) that operates at byte granularity. Because disp8 is sign-extended, it can only address between -128 and 127 byte offsets; in the case of a 64-byte cache line, disp8 uses 8 bits that can be set to only four truly useful values: -128, -64, 0, and 64; since a larger range is often required, disp32 is used; however, disp32 requires 4 bytes. In contrast to disp8 and disp32, the shift factor segment 2562B is a reinterpretation of disp8; when using the shift factor segment 2562B, the actual shift is determined by multiplying the contents of the shift factor segment by the size (N) of the memory operand access. This type of shift is called disp8*N. This reduces the average instruction length (a single byte is used for the shift, but with a much larger range). This type of compressed offset assumes that the effective offset is a multiple of the granularity of the memory access, and therefore the redundant low-order bits of the address offset do not need to be encoded. In other words, the offset is replaced by digital segment 2562B instead of the traditional x86 instruction set 8-bit offset. Thus, the offset is encoded by digital segment 2562B in the same way as the x86 instruction set 8-bit offset (therefore, there is no change in the ModRM / SIB encoding rules), the only difference being that disp8 is overloaded to disp8*N. In other words, there is no change in the encoding rules or encoding length, only in the hardware interpretation of the offset values ​​(which requires scaling the offset to the size of the memory operand to obtain a byte-level address offset). Immediate segment 2572 operates as previously described.

[0306] Full opcode field

[0307] Figure 26B This is a block diagram illustrating the fields constituting a complete opcode field 2574 with a dedicated vector-friendly instruction format 2600 according to one embodiment. Specifically, the complete opcode field 2574 includes a format field 2540, a basic opcode field 2542, and a data element width (W) field 2564. The basic opcode field 2542 includes a prefix encoding field 2625, an opcode mapping field 2615, and a real opcode field 2630.

[0308] Register index field

[0309] Figure 26C This is a block diagram illustrating the fields constituting register index field 2544 with a dedicated vector-friendly instruction format 2600 according to one embodiment. Specifically, register index field 2544 includes REX 2605 field, REX' 2610 field, MODR / M.reg field 2644, MODR / Mr / m field 2646, VVVV field 2620, xxx field 2654, and bbb field 2656.

[0310] Expand operation fields

[0311] Figure 26D This is a block diagram illustrating the fields of an extended operation field 2550 with a dedicated vector-friendly instruction format 2600 according to one embodiment. When the class (U) field 2568 contains 0, it indicates EVEX.U0 (Class A 2568A); when it contains 1, it indicates EVEX.U1 (Class B 2568B). When U = 0 and the MOD field 2642 contains 11 (indicating no memory access operation), the α field 2552 (EVEX byte 3, bits [7] – EH) is interpreted as the rs field 2552A. When the rs field 2552A contains 1 (rounded 2552A.1), the β field 2554 (EVEX byte 3, bits [6:4] – SSS) is interpreted as the rounding control field 2554A. The rounding control field 2554A includes a one-bit SAE field 2556 and a two-bit rounding operation field 2558. When rs field 2552A contains 0 (data transformation 2552A.2), β field 2554 (EVEX byte 3, bits [6:4] – SSS) is interpreted as a three-bit data transformation field 2554B. When U = 0 and MOD field 2642 contains 00, 01, or 10 (indicating a memory access operation), α field 2552 (EVEX byte 3, bits [7] – EH) is interpreted as an eviction hint (EH) field 2552B, and β field 2554 (EVEX byte 3, bits [6:4] – SSS) is interpreted as a three-bit data manipulation field 2554C.

[0312] When U = 1, α field 2552 (EVEX byte 3, bit [7] – EH) is interpreted as write mask control (Z) field 2552C. When U = 1 and MOD field 2642 contains 11 (indicating no memory access operation), a portion of β field 2554 (EVEX byte 3, bit [4] – S0) is interpreted as RL field 2557A; when it contains 1 (rounded to 2557A.1), the remainder of β field 2554 (EVEX byte 3, bit [6-5] – S0) is interpreted as RL field 2557A. 2-1The remainder of the β field 2554 (EVEX byte 3, bit [6-5]-S) is interpreted as a rounding operation field 2559A, while when the RL field 2557A contains 0 (VSIZE2557A.2), the remainder of the β field 2554 ... contains EVEX byte 3, bit [6-5]-S. 2-1 ) is interpreted as a vector length field 2559B (EVEX byte 3, bits [6-5] – L 1-0 When U=1 and MOD field 2642 contains 00, 01, or 10 (indicating a memory access operation), β field 2554 (EVEX byte 3, bits [6:4]–SSS) is interpreted as vector length field 2559B (EVEX byte 3, bits [6-5]–L). 1-0 ) and broadcast field 2557B (EVEX byte 3, bit [4]–B).

[0313] Exemplary Register Architecture

[0314] Figure 27 This is a block diagram of a register architecture 2700 according to one embodiment. In the illustrated embodiment, there are 32 512-bit wide vector registers 2710; these registers are referenced as zmm0 to zmm31. The lower-order 256 bits of the lower 16 zmm registers are overlaid on registers ymm0-16. The lower-order 128 bits of the lower 16 zmm registers (the lower-order 128 bits of the ymm registers) are overlaid on registers xmm0-15. The dedicated vector-friendly instruction format 2600 operates on these overlaid registers as illustrated in the following table.

[0315]

[0316] In other words, the vector length field 2559B selects between a maximum length and one or more other shorter lengths, each of which is half the length of the previous one, and instruction templates without the vector length field 2559B operate on the maximum vector length. Furthermore, in one embodiment, the Class B instruction template of the dedicated vector-friendly instruction format 2600 operates on compressed or scalar single / double-precision floating-point data and compressed or scalar integer data. Scalar operations are performed on the lowest-order data element positions in the zmm / ymm / xmm registers; depending on the embodiment, higher-order data element positions either remain the same as before the instruction or are zeroed out.

[0317] Write mask register 2715—In the illustrated embodiment, there are eight write mask registers (k0 to k7), each 64 bits in size. In an alternative embodiment, write mask register 2715 is 16 bits in size. As previously mentioned, in one embodiment, vector mask register k0 cannot be used as a write mask; when the encoding of the normal indicator k0 is used as the write mask, it selects the hardwired write mask 0xFFFF, effectively disabling the write mask for that instruction.

[0318] General Purpose Register 2725 — In the illustrated embodiment, there are sixteen 64-bit general purpose registers that are used in conjunction with existing x86 addressing modes for addressing memory operands. These registers are referenced by the names RAX, RBX, RCX, RDX, RBP, RSI, RDI, RSP, and R8 through R15.

[0319] A scalar floating-point stack register file (x87 stack) 2745 is overlaid with an MMX compact integer flat register file 2750. In the illustrated embodiment, the x87 stack is an octal stack used to perform scalar floating-point operations on 32 / 64 / 80-bit floating-point data using x87 instruction set extensions; while the MMX registers are used to perform operations on 64-bit compact integer data and to store operands for some operations performed between the MMX and XMM registers.

[0320] Alternative embodiments may use wider or narrower registers. Additionally, alternative embodiments may use more, fewer, or different register files and registers.

[0321] Exemplary core architectures, processors, and computer architectures

[0322] Processor cores can be implemented in different ways, for different purposes, and in different processors. For example, implementations of such cores may include: 1) general-purpose ordered cores intended for general-purpose computing; 2) high-performance general-purpose out-of-order cores intended for general-purpose computing; and 3) dedicated cores intended primarily for graphics and / or scientific (throughput) computing. Implementations of different processors may include: 1) CPUs, which include one or more general-purpose ordered cores and / or one or more general-purpose out-of-order cores intended for general-purpose computing; and 2) coprocessors, which include one or more dedicated cores intended primarily for graphics and / or scientific (throughput) computing. These different processors result in different computer system architectures, which may include: 1) a coprocessor on a separate chip from the CPU; 2) a coprocessor in the same package as the CPU but on a separate die; 3) a coprocessor on the same die as the CPU (in which case such a coprocessor is sometimes referred to as dedicated logic or a dedicated core, such as integrated graphics and / or scientific (throughput) logic); and 4) a system-on-a-chip that may include the described CPU (sometimes referred to as application cores or application processors), the coprocessors described above, and additional functionality on the same die. Exemplary core architectures are then described, followed by exemplary processors and computer architectures.

[0323] Exemplary core architecture

[0324] Ordered and disordered kernel diagrams

[0325] Figure 28A This is a block diagram illustrating an exemplary ordered pipeline and an exemplary out-of-order release / execution pipeline with register renaming according to various embodiments. Figure 28B This is a block diagram illustrating exemplary embodiments of ordered architecture cores to be included in a processor according to various embodiments and exemplary out-of-order release / execution architecture cores with register renaming. Figures 28A-28B The solid-line boxes in the diagram illustrate ordered pipelines and ordered cores, while the optional additions to the dashed boxes illustrate register renaming, out-of-order release / execution pipelines, and cores. Since the ordered aspect is a subset of the out-of-order aspect, the out-of-order aspect will be described.

[0326] exist Figure 28A In the processor pipeline 2800, there are fetch stage 2802, length decoding stage 2804, decoding stage 2806, allocation stage 2808, renaming stage 2810, scheduling (also known as dispatch or issue) stage 2812, register read / memory read stage 2814, execution stage 2816, write-back / memory write stage 2818, exception handling stage 2822, and commit stage 2824.

[0327] Figure 28BA processor core 2890 is shown, which includes a front-side unit 2830 coupled to an execution engine unit 2850, and both the front-side unit 2830 and the execution engine unit 2850 are coupled to a memory unit 2870. Core 2890 can be a Reduced Instruction Set Computing (RISC) core, a Complex Instruction Set Computing (CISC) core, a Very Long Instruction Word (VLIW) core, or a hybrid or alternative core type. Alternatively, core 2890 can be a dedicated core, such as, for example, a network or communication core, a compression engine, a coprocessor core, a general-purpose computing graphics processing unit (GPGPU) core, a graphics core, etc.

[0328] Front-end unit 2830 includes branch prediction unit 2832 coupled to instruction cache unit 2834 coupled to instruction translation lookup buffer (TLB) 2836 coupled to instruction fetch unit 2838 coupled to decoding unit 2840. Decoding unit 2840 (or decoder) decodes instructions and generates one or more micro-operations, microcode entry points, microinstructions, other instructions, or other control signals as output, which are decoded from, or otherwise reflect, the original instructions or derived from the original instructions. Decoding unit 2840 can be implemented using various different mechanisms. Examples of suitable mechanisms include, but are not limited to, lookup tables, hardware implementations, programmable logic arrays (PLAs), microcode read-only memory (ROM), etc. In one embodiment, core 2890 includes microcode ROM or other media (e.g., in decoding unit 2840, or otherwise within front-end unit 2830) storing microcode for certain macro instructions. The decoding unit 2840 is coupled to the rename / allocator unit 2852 in the execution engine unit 2850.

[0329] Execution engine unit 2850 includes a renaming / allocator unit 2852 coupled to a retirement unit 2854 and a set 2856 of one or more scheduler units. The scheduler units 2856 represent any number of different schedulers, including reservation stations, central instruction windows, etc. The scheduler units 2856 are coupled to physical register file units 2858. Each physical register file unit in the physical register file units 2858 represents one or more physical register files, where different physical register files store one or more different data types, such as scalar integers, scalar floating-point numbers, compressed integers, compressed floating-point numbers, vector integers, vector floating-point numbers, status (e.g., an instruction pointer as the address of the next instruction to be executed), etc. In one embodiment, the physical register file units 2858 include vector register units, write mask register units, and scalar register units. These register units can provide architectural vector registers, vector mask registers, and general-purpose registers. Multiple physical register file units 2858 are overlapped by retirement units 2854 to illustrate various ways in which register renaming and out-of-order execution can be implemented (e.g., using multiple reorder buffers and multiple retirement register files; using multiple future files, multiple history buffers, and multiple retirement register files; using register maps and register pools, etc.). Retirement units 2854 and multiple physical register file units 2858 are coupled to multiple execution clusters 2860. Multiple execution clusters 2860 include a set 2862 of one or more execution units and a set 2864 of one or more memory access units. Execution units 2862 can perform various operations (e.g., shift, addition, subtraction, multiplication) and can perform operations on various data types (e.g., scalar floating-point, compressed integer, compressed floating-point, vector integer, vector floating-point). While some embodiments may include multiple execution units dedicated to a particular function or set of functions, other embodiments may include only one execution unit or multiple execution units that all perform all functions. Multiple scheduler units 2856, multiple physical register file units 2858, and multiple execution clusters 2860 are shown as possibly multiple, because some embodiments create separate pipelines for certain types of data / operations (e.g., scalar integer pipelines, scalar floating-point / compact integer / compact floating-point / vector integer / vector floating-point pipelines, and / or memory access pipelines each having their own scheduler unit, multiple physical register file units, and / or execution clusters—and in the case of separate memory access pipelines, some embodiments are implemented where only the execution cluster of that pipeline has multiple memory access units 2864). It should also be understood that, in the case of using separate pipelines, one or more of these pipelines may be out-of-order deployment / execution, and the remaining pipelines may be ordered.

[0330] A set of memory access units 2864 is coupled to a memory unit 2870, which includes a data TLB unit 2872, which is coupled to a data cache unit 2874, which is coupled to a Level 2 (L2) cache unit 2876. In one exemplary embodiment, the memory access unit 2864 may include a load unit, a memory address unit, and a memory data unit, each of which is coupled to the data TLB unit 2872 in the memory unit 2870. An instruction cache unit 2834 is also coupled to the Level 2 (L2) cache unit 2876 in the memory unit 2870. The L2 cache unit 2876 is coupled to one or more other levels of cache and ultimately to main memory.

[0331] As an example, the exemplary out-of-order release / execution core architecture for register renaming can implement pipeline 2800 as follows: 1) Instruction fetch 2838 executes fetch stage 2802 and length decoding stage 2804; 2) Decoding unit 2840 executes decoding stage 2806; 3) Rename / allocator unit 2852 executes allocation stage 2808 and rename stage 2810; 4) (multiple) scheduler units 2856 execute scheduling stage 2812; 5) (multiple) physical register file units 2858 and memory unit 2870 execute register read / memory read stage 2814; execution cluster 2860 executes execution stage 2816; 6) memory unit 2870 and (multiple) physical register file units 2858 execute write-back / memory write stage 2818; 7) each unit may involve exception handling stage 2822; and 8) retirement unit 2854 and (multiple) physical register file units 2858 execute commit stage 2824.

[0332] Core 2890 may support one or more instruction sets (e.g., the x86 instruction set (with some extensions added with newer versions); the MIPS instruction set of MIPS Technologies, Inc., Sunnyvale, California; the ARM instruction set of ARM Holdings, Inc., Sunnyvale, California (with optional additional extensions such as NEON)), including the instructions described herein. In one embodiment, Core 2890 includes logic for supporting compressed data instruction set extensions (e.g., AVX1, AVX2), thereby allowing the use of compressed data to perform operations used by many multimedia applications.

[0333] It should be understood that a core can support multithreading (execution of two or more parallel operations or a set of threads), and this multithreading can be accomplished in various ways, including time-division multithreading, simultaneous multithreading (where a single physical core provides a logical core for each thread in the simultaneous multithreading of the physical core), or combinations thereof (e.g., time-division fetching and decoding, and subsequent operations such as...). Simultaneous multithreading in hyper-threading technology.

[0334] Although register renaming is described in the context of out-of-order execution, it should be understood that register renaming can also be used in ordered architectures. While the illustrated processor embodiment also includes separate instruction and data cache units 2834 / 2874 and a shared L2 cache unit 2876, alternative embodiments may have a single internal cache for both instructions and data, such as, for example, a Level 1 (L1) internal cache or multiple levels of internal caches. In some embodiments, the system may include a combination of internal caches and external caches located outside the core and / or processor. Alternatively, all caches may be located outside the core and / or processor.

[0335] Specific exemplary ordered core architecture

[0336] Figures 29A-29B The diagram illustrates a more specific exemplary ordered core architecture, where the core is one of several logic blocks within a chip (including other cores of the same and / or different types). Depending on the application, the logic block communicates with some fixed functional logic, memory I / O interfaces, and other necessary I / O logic via a high-bandwidth interconnect network (e.g., a ring network).

[0337] Figure 29A This is a block diagram of a single processor core according to an embodiment, its connection to the on-die interconnect network 2902, and a local subset 2904 of its second-level (L2) cache. In one embodiment, the instruction decoder 2900 supports the x86 instruction set with a compact data instruction set extension. The L1 cache 2906 allows low-latency access to cache memory into the scalar and vector units. Although in one embodiment (for design simplification), the scalar unit 2908 and the vector unit 2910 use separate sets of registers (scalar register 2912 and vector register 2914, respectively), and data transferred between these registers is written to memory and subsequently read back from the first-level (L1) cache 2906, alternative embodiments may use different methods (e.g., using a single set of registers or including a communication path that allows data to be transferred between these two register sets without being written and read back).

[0338] The local subset 2904 of the L2 cache is part of the global L2 cache, which is divided into multiple separate local subsets, one for each processor core. Each processor core has a direct access path to its own local subset 2904 of the L2 cache. Data read by a processor core is stored in its L2 cache subset 2904 and can be accessed quickly in parallel with other processor cores accessing their own local L2 cache subsets. Data written by a processor core is stored in its own L2 cache subset 2904 and is dumped from other subsets if necessary. A ring network ensures the consistency of shared data. The ring network is bidirectional to allow agents such as processor cores, L2 caches, and other logic blocks to communicate with each other within the chip. Each ring data path is 1012 bits wide in each direction.

[0339] Figure 29B According to the embodiments Figure 29A An expanded diagram of a portion of the processor core. Figure 29B This includes the L1 data cache 2906A portion of L1 cache 2904, and further details regarding vector unit 2910 and vector register 2914. Specifically, vector unit 2910 is a 16-wide vector processing unit (VPU) (see 16-wide ALU 2928) that executes one or more of integer, single-precision floating-point, and double-precision floating-point instructions. This VPU supports mixing of register inputs via mixing unit 2920, numerical conversion via value conversion units 2922A-B, and copying of memory inputs via copy unit 2924. Write mask register 2926 allows masked vector writes.

[0340] Figure 30 This is a block diagram of a processor 3000 according to an embodiment, which may have more than one core, an integrated memory controller, and an integrated graphics device. Figure 30 The solid-line box illustration shows a processor 3000 having a single core 3002A, a system agent 3010, and a set of one or more bus controller units 3016, while the optional additional illustration with dashed boxes shows an alternative processor 3000 having multiple cores 3002A-N, a set of one or more integrated memory controller units 3014 from the system agent units 3010, and dedicated logic 3008.

[0341] Therefore, different implementations of processor 3000 may include: 1) a CPU, wherein dedicated logic 3008 is integrated graphics and / or scientific (throughput) logic (which may include one or more cores), and cores 3002A-N are one or more general-purpose cores (e.g., general-purpose ordered cores, general-purpose out-of-order cores, or a combination of both); 2) a coprocessor, wherein cores 3002A-N are a large number of dedicated cores designed primarily for graphics and / or scientific (throughput); and 3) a coprocessor, wherein cores 3002A-N are a large number of general-purpose ordered cores. Thus, processor 3000 may be a general-purpose processor, coprocessor, or dedicated processor, such as, for example, a network or communication processor, a compression engine, a graphics processor, a GPGPU (General-Purpose Graphics Processing Unit), a high-throughput integrated many-core (MIC) coprocessor (including 30 or more cores), an embedded processor, etc. The processor may be implemented on one or more chips. Processor 3000 may be part of one or more substrates, and / or may be implemented on one or more substrates using any of a variety of process technologies (e.g., BiCMOS, CMOS, or NMOS).

[0342] The memory hierarchy includes one or more levels of cache within the core, a set 3006 of one or more shared cache units, and external memory (not shown) coupled to a set 3014 of integrated memory controller units. The set 3006 of shared cache units may include one or more intermediate-level caches, such as Level 2 (L2), Level 3 (L3), Level 4 (L4), or other levels of cache, Last Level Cache (LLC), and / or combinations thereof. While in one embodiment, ring-based interconnect units 3012 interconnect dedicated logic 3008 (integrated graphics logic is an example of dedicated logic and is also referred to herein as dedicated logic), the set 3006 of shared cache units, and system proxy units 3010 / (multiple) of integrated memory controller units 3014, alternative embodiments may use any number of known techniques to interconnect such units. In one embodiment, consistency is maintained between one or more cache units 3006 and cores 3002A-N.

[0343] In some embodiments, one or more cores 3002A-N can be multithreaded. System agent 3010 includes those components that coordinate and operate core 3002A-N. System agent unit 3010 may include, for example, a power control unit (PCU) and a display unit. The PCU may be, or may include, the logic and components required to regulate the power state of core 3002A-N and dedicated logic 3008. The display unit is used to drive one or more externally connected displays.

[0344] The 3002A-N cores can be homogeneous or heterogeneous in terms of their instruction set architecture; that is, two or more cores in the 3002A-N cores may be able to execute the same instruction set, while other cores may be able to execute only a subset of that instruction set or a different instruction set.

[0345] Exemplary computer architecture

[0346] Figures 31-34 This is a block diagram of an exemplary computer architecture. Other system designs and configurations known in the art for laptops, desktops, handheld PCs, personal digital assistants, engineering workstations, servers, network devices, network hubs, switches, embedded processors, digital signal processors (DSPs), graphics devices, video game devices, set-top boxes, microcontrollers, cellular phones, portable media players, handheld devices, and various other electronic devices are also suitable. Generally, a wide variety of systems or electronic devices capable of incorporating processors and / or other execution logic as disclosed herein are generally suitable.

[0347] Now for reference Figure 31 The diagram illustrates a system 3100 according to an embodiment of the present invention. System 3100 may include one or more processors 3110, 3115 coupled to a controller hub 3120. In one embodiment, the controller hub 3120 includes a graphics memory controller hub (GMCH) 3190 and an input / output hub (IOH) 3150 (which may be on separate chips); the GMCH 3190 includes memory and a graphics controller, to which a memory 3140 and a coprocessor 3145 are coupled; the IOH 3150 couples an input / output (I / O) device 3160 to the GMCH 3190. Alternatively, one or both of the memory and graphics controller may be integrated within a processor (as described herein), with the memory 3140 and coprocessor 3145 directly coupled to the processor 3110, and the controller hub 3120 and IOH 3150 residing on a single chip.

[0348] The optionality of the additional processor 3115 is Figure 31 The numbers are indicated by dashed lines. Each processor 3110, 3115 may include one or more of the processing cores described herein, and may be a version of processor 3000.

[0349] The memory 3140 may be, for example, dynamic random access memory (DRAM), phase-change memory (PCM), or a combination of both. In at least one embodiment, the controller hub 3120 communicates with the processors(s) 3110, 3115 via a multi-branch bus such as a front-side bus (FSB), a point-to-point interface such as a fast path interconnect (QPI), or a similar connection 3195.

[0350] In one embodiment, the coprocessor 3145 is a dedicated processor, such as, for example, a high-throughput MIC processor, a network or communication processor, a compression engine, a graphics processor, a GPGPU, an embedded processor, and so on. In one embodiment, the controller hub 3120 may include an integrated graphics accelerator.

[0351] There can be various differences between physical resources 3110 and 3115 in terms of a range of quality metrics, including architecture, microarchitecture, thermal and power consumption characteristics.

[0352] In one embodiment, processor 3110 executes instructions that control general-type data processing operations. Embedded within these instructions may be coprocessor instructions. Processor 3110 recognizes these coprocessor instructions as having a type that should be executed by an attached coprocessor 3145. Therefore, processor 3110 issues these coprocessor instructions (or control signals representing coprocessor instructions) to coprocessor 3145 on a coprocessor bus or other interconnect. Coprocessors 3145 (multiple) receive and execute the received coprocessor instructions.

[0353] See now Figure 32 The diagram shown is a block diagram of a first, more specific, exemplary system 3200 according to an embodiment of the present invention. Figure 32 As shown, the multiprocessor system 3200 is a point-to-point interconnect system and includes a first processor 3270 and a second processor 3280 coupled via a point-to-point interconnect 3250. Each of processors 3270 and 3280 may be a version of processor 3000. In one embodiment, processors 3270 and 3280 are processors 3110 and 3115, respectively, and coprocessor 3238 is coprocessor 3145. In another embodiment, processors 3270 and 3280 are processor 3110 and coprocessor 3145, respectively.

[0354] Processors 3270 and 3280 are shown as including integrated memory controller (IMC) units 3272 and 3282, respectively. Processor 3270 also includes point-to-point (PP) interfaces 3276 and 3278 as part of its bus controller unit; similarly, the second processor 3280 includes PP interfaces 3286 and 3288. Processors 3270 and 3280 can exchange information via PP interface 3250 using point-to-point (PP) interface circuits 3278 and 3288. Figure 32 As shown, IMC 3272 and 3282 couple the processor to the corresponding memory, namely memory 3232 and memory 3234, which may be portions of the main memory locally attached to the corresponding processor.

[0355] Processors 3270 and 3280 can each exchange information with chipset 3290 via their respective PP interfaces 3252 and 3254 using point-to-point interface circuits 3276, 3294, 3286, and 3298. Chipset 3290 can optionally exchange information with coprocessor 3238 via high-performance interface 3292. In one embodiment, coprocessor 3238 is a dedicated processor, such as, for example, a high-throughput MIC processor, a network or communication processor, a compression engine, a graphics processor, a GPGPU, an embedded processor, etc.

[0356] A shared cache (not shown) may be included in either processor or external to both processors but connected to them via a PP interconnect, such that if the processors are in a low-power mode, the local cache information of either or both processors may be stored in the shared cache.

[0357] Chipset 3290 can be coupled to first bus 3216 via interface 3296. In one embodiment, first bus 3216 may be a peripheral component interconnect (PCI) bus or a bus such as PCI Fast bus or another third-generation I / O interconnect bus, but the scope of the invention is not limited thereto.

[0358] like Figure 32As shown, various I / O devices 3214 may be coupled to a first bus 3216 together with a bus bridge 3218, which couples the first bus 3216 to a second bus 3220. In one embodiment, one or more additional processors 3215, such as a coprocessor, a high-throughput MIC processor, a GPGPU, an accelerator (such as, for example, a graphics accelerator or digital signal processing (DSP) unit), a field-programmable gate array, or any other processor, are coupled to the first bus 3216. In one embodiment, the second bus 3220 may be a low pin count (LPC) bus. In one embodiment, various devices may be coupled to the second bus 3220, including, for example, a keyboard and / or mouse 3222, a communication device 3227, and a storage unit 3228, such as a disk drive that may include instruction / code and data 3230, or other mass storage devices. Additionally, audio I / O 3224 may be coupled to the second bus 3220. Note that other architectures are possible. For example, instead of… Figure 32 The point-to-point architecture allows the system to implement multi-branch buses or other similar architectures.

[0359] Now for reference Figure 33 The diagram shown is a block diagram of a second, more specific, exemplary system 3300 according to an embodiment of the present invention. Figure 32 and 33 Similar elements in the figure use similar reference numerals, and from Figure 33 The middle part is omitted Figure 32 certain aspects to avoid confusion Figure 33 Other aspects.

[0360] Figure 33 The illustrated processors 3270 and 3280 may include integrated memory and I / O control logic (“CL”) 3272 and 3282, respectively. Therefore, CL 3272 and 3282 include an integrated memory controller unit and I / O control logic. Figure 33 The diagram shows that not only are the memories 3232 and 3234 coupled to CLs 3272 and 3282, but the I / O device 3314 is also coupled to the control logic 3272 and 3282. The conventional I / O device 3315 is coupled to the chipset 3290.

[0361] Now for reference Figure 34 The diagram shown is a block diagram of a SoC 3400 according to an embodiment of the present invention. Figure 30 Similar elements in the figure use similar reference numerals. Additionally, dashed boxes are optional features on more advanced SoCs. Figure 34In this embodiment, multiple interconnect units 3402 are coupled to: an application processor 3410, which includes a set of one or more cores 3002A-N and multiple shared cache units 3006, the set of one or more cores 3002A-N including cache units 3004A-N; a system proxy unit 3010; multiple bus controller units 3016; multiple integrated memory controller units 3014; a set of one or more coprocessors 3420, which may include integrated graphics logic, an image processor, an audio processor, and a video processor; a static random access memory (SRAM) unit 3430; a direct memory access (DMA) unit 3432; and a display unit 3440 for coupling to one or more external displays. In one embodiment, the multiple coprocessors 3420 include dedicated processors, such as, for example, network or communication processors, compression engines, GPGPUs, high-throughput MIC processors, or embedded processors, etc.

[0362] The various embodiments of the mechanisms disclosed herein can be implemented in hardware, software, firmware, or a combination of such implementations. Embodiments can be implemented as computer programs or program code executable on a programmable system including at least one processor, a storage system (including volatile and non-volatile memory and / or storage elements), at least one input device, and at least one output device.

[0363] Program code (such as, Figure 32 The code 3230 shown in the diagram is applied to input instructions to perform the functions described herein and generate output information. The output information can be applied to one or more output devices in a known manner. For the purposes of this application, the processing system includes any system having a processor, such as, for example, a digital signal processor (DSP), a microcontroller, an application-specific integrated circuit (ASIC), or a microprocessor.

[0364] The program code can be implemented using a high-level procedural or object-oriented programming language to communicate with the processing system. Assembly or machine language can also be used if needed. In fact, the mechanisms described herein are not limited to any particular programming language. In any case, the language can be a compiled or interpreted language.

[0365] One or more aspects of at least one embodiment can be implemented by representational instructions stored on a machine-readable medium, which represent various logics in a processor, and which, when read by a machine, cause the machine to manufacture logic for performing the techniques described herein. Such representations, referred to as “IP cores,” can be stored on tangible machine-readable media and can be supplied to various customers or production facilities for loading into manufacturing machines that actually manufacture the logic or processor.

[0366] Such machine-readable storage media can include, but are not limited to, non-transitory, tangible arrangements of articles made or formed by a machine or device, including storage media such as hard disks; any other type of disk, including floppy disks, optical disks, compact disc read-only memory (CD-ROM), rewritable compact discs (CD-RW), and magneto-optical disks; semiconductor devices such as read-only memory (ROM), random access memory (RAM) such as dynamic random access memory (DRAM) and static random access memory (SRAM), erasable programmable read-only memory (EPROM), flash memory, electrically erasable programmable read-only memory (EEPROM); phase-change memory (PCM); magnetic cards or optical cards; or any other type of medium suitable for storing electronic instructions.

[0367] Therefore, embodiments also include non-transitory tangible machine-readable media containing instructions or design data, such as a hardware description language (HDL), that defines the architectures, circuits, devices, processors, and / or system characteristics described herein. These embodiments are also referred to as program products.

[0368] Simulation (including binary transformation, code transformation, etc.)

[0369] In some cases, instruction translators can be used to translate instructions from a source instruction set to a target instruction set. For example, an instruction translator can transform instructions (e.g., using static binary transformation, including dynamically compiled binary transformation), modify, emulate, or otherwise convert them into one or more other instructions to be processed by the kernel. Instruction translators can be implemented in software, hardware, firmware, or a combination thereof. Instruction translators can be on the processor, off the processor, or partially on and partially off the processor.

[0370] Figure 35 This is a block diagram illustrating the use of a software instruction converter to convert binary instructions in a source instruction set into binary instructions in a target instruction set, according to an embodiment. In the illustrated embodiment, the instruction converter is a software instruction converter; however, alternatively, the instruction converter can be implemented using software, firmware, hardware, or various combinations thereof. Figure 35This illustrates the use of an x86 compiler 3504 to compile a program of the form of a high-level language 3502 to generate x86 binary code 3506 that can be natively executed by a processor 3516 having at least one x86 instruction set core. A processor 3516 having at least one x86 instruction set core refers to any processor that performs substantially the same function as an Intel processor having at least one x86 instruction set core by compatibly executing or otherwise processing: 1) a substantial portion of the instruction set of an Intel x86 instruction set core, or 2) a version of object code for an application or other software aimed at running on an Intel processor having at least one x86 instruction set core to achieve substantially the same results as an Intel processor having at least one x86 instruction set core. The x86 compiler 3504 refers to a compiler operable to generate x86 binary code 3506 (e.g., object code) that can be executed on a processor 3516 having at least one x86 instruction set core, with or without additional linking processing. Similarly, Figure 35 A program in the form of a high-level language 3502 can be compiled using an alternative instruction set compiler 3508 to generate an alternative instruction set binary 3510 that can be natively executed by a processor 3514 that does not have at least one x86 instruction set core (e.g., a processor with a core that executes the MIPS instruction set of MIPS Technologies, Inc., Sunnyvale, California, and / or the ARM instruction set of ARM Holdings, Inc., Sunnyvale, California). An instruction converter 3512 is used to translate the x86 binary code 3506 into code that can be natively executed by the processor 3514 that does not have an x86 instruction set core. This translated code is unlikely to be identical to the alternative instruction set binary code 3510 because an instruction converter capable of doing so would be difficult to manufacture; however, the translated code will perform general operations and consists of instructions from the alternative instruction set. Therefore, the instruction converter 3512 represents, through emulation, simulation, or any other process, software, firmware, hardware, or a combination thereof that allows a processor or other electronic device without an x86 instruction set processor or core to execute the x86 binary code 3506.

[0371] Further examples

[0372] Example 1 provides an exemplary processor including: fetching circuitry for fetching instructions having a format having fields for specifying an opcode, the position of a source matrix, and the position of a destination matrix, wherein the opcode instructs the processor to transform the specified source matrix into a specified destination matrix having a row-interleaved (RowInt) format; decoding circuitry for decoding the fetched instructions; and execution circuitry for responding to the decoded instructions by transforming the specified source matrix into a specified RowInt-formatted destination matrix via: interleaving J elements of each J-element subcolumn of the specified source matrix in a row-major or column-major order to form a K-width submatrix of the specified destination matrix, the K-width submatrix having K columns and enough rows to hold J elements.

[0373] Example 2 includes the substance of the exemplary processor of Example 1, wherein each of the specified source and destination matrices includes one of a set of vector registers, a set of slice registers, and a set of memory locations.

[0374] Example 3 includes the substantive content of the exemplary processor of Example 1, wherein the instruction format further includes fields for specifying J and K.

[0375] Example 4 includes the substance of the exemplary processor of Example 1, wherein the instruction format further includes fields for specifying the element size of the specified source destination matrix and the element size of the destination matrix, wherein the specified element size includes one of binary digits, nibble, byte, word, double word, and quadword.

[0376] Example 5 includes the substance of the exemplary processor of Example 1, wherein the instruction format further includes fields for specifying the element format of the specified source matrix and the element format of the destination matrix, the specified element format including compressed or scalar single-precision floating-point data, or compressed or scalar double-precision floating-point data, and compressed or scalar integer data.

[0377] Example 6 includes the substance of the exemplary processor of Example 1, wherein the specified source matrix comprises an M x N element array and the specified destination matrix comprises half rows and twice the columns, wherein the format of the instructions is further used to specify at least one of M and N.

[0378] Example 7 includes the substance of the exemplary processor of Example 1, wherein the instruction format further includes a field for specifying a mask, the specified mask being a multi-bit value with one bit for each destination element, the bit being used to control whether the destination element will be updated, or to control whether the destination element will be zeroed out or merged.

[0379] Example 8 provides an exemplary method executed by a processor, the method comprising: retrieving instructions having a format having fields for specifying an opcode, the position of a source matrix, and the position of a destination matrix using fetching circuitry, wherein the opcode instructs the processor to transform a specified source matrix into a specified destination matrix having a row-interleaved (RowInt) format; decoding the fetched instructions using decoding circuitry; and responding to the decoded instructions using execution circuitry by transforming the specified source matrix into a specified RowInt-formatted destination matrix via: interleaving J elements of each J-element subcolumn of the specified source matrix in a row-major or column-major order to form a K-width submatrix of the specified destination matrix, the K-width submatrix having K columns and enough rows to hold J elements.

[0380] Example 9 includes the substance of the exemplary method of Example 8, wherein each of the specified source matrix and destination matrix includes one of a set of vector registers, a set of slice registers, and a set of memory locations.

[0381] Example 10 includes the substance of the exemplary method of Example 8, wherein J and K are specified by one or more of the appended instruction operands, are passed as prefixes or suffixes of opcodes, are encoded in immediate values, and are specified by control registers that have been programmed by software.

[0382] Example 11 includes the substance of the exemplary method of Example 8, wherein the instruction format further includes fields for specifying the element size of the specified source matrix and the element size of the destination matrix, wherein the specified element size includes one of binary digits, nibble, byte, word, double word, and quadword.

[0383] Example 12 includes the substance of the exemplary method of Example 8, wherein the instruction format further includes fields for specifying the element format of the specified source matrix and the element format of the destination matrix, the specified element format including compressed or scalar single-precision floating-point data, or compressed or scalar double-precision floating-point data, and compressed or scalar integer data.

[0384] Example 13 includes the substance of the exemplary method of Example 8, wherein the specified source matrix is ​​used to include an array of M x N elements, and the specified destination matrix includes half rows and twice the columns, wherein the format of the instructions is further used to specify at least one of M and N.

[0385] Example 14 includes the substance of the exemplary method of Example 8, wherein the instruction format further includes a field for specifying a mask, the specified mask being a multi-bit value with one bit for each destination element, the bit being used to control whether the destination element will be updated, or to control whether the destination element will be zeroed out or merged.

[0386] Example 15 provides an exemplary non-transitory machine-readable medium containing instructions that, when executed by a processor, cause the processor to respond by: retrieving the instructions with a format having fields for specifying an opcode, the position of a source matrix, and the position of a destination matrix, wherein the opcode instructs the processor to transform the specified source matrix into a specified destination matrix having a row-interleaved (RowInt) format; decoding the retrieved instructions with a decoding circuit; and responding to the decoded instructions with an execution circuit by transforming the specified source matrix into a specified RowInt-formatted destination matrix by interleaving J elements of each J-element subcolumn of the specified source matrix in a row-major or column-major order to form a K-width submatrix of the specified destination matrix, the K-width submatrix having K columns and enough rows to hold J elements.

[0387] Example 16 includes the substantive content of the exemplary non-transitory machine-readable medium of Example 15, wherein each of the specified source matrix and destination matrix includes one of a set of vector registers, a set of slice registers, and a set of memory locations.

[0388] Example 17 includes the substantive content of the exemplary non-transitory machine-readable medium of Example 15, wherein J and K are specified by one or more of the appended instruction operands, are passed as prefixes or suffixes of opcodes, are encoded in immediate values, and are specified by control registers that have been programmed by software.

[0389] Example 18 includes the substantive content of the exemplary non-transitory machine-readable medium of Example 15, wherein the instruction format further includes fields for specifying the element size of the specified source matrix and the element size of the destination matrix, the specified element size including one of binary digits, nibble, byte, word, double word, and quadword.

[0390] Example 19 includes the substantive content of the exemplary non-transitory machine-readable medium of Example 15, wherein the instruction format further includes fields for specifying the element format of the specified source matrix and the element format of the destination matrix, the specified element format including compressed or scalar single-precision floating-point data, or compressed or scalar double-precision floating-point data, and compressed or scalar integer data.

[0391] Example 20 includes the substantive content of the exemplary non-transitory machine-readable medium of Example 15, wherein the specified source matrix comprises an M x N array of elements, and the specified destination matrix comprises half rows and twice the columns, wherein the format of the instructions is further used to specify at least one of M and N.

Claims

1. A processor, comprising: A decoding circuit is used to decode a single instruction into a decoded single instruction, the single instruction having a first field for specifying a source matrix, a second field for specifying a destination matrix, and an opcode for instructing the execution circuit to cause storage of the following: The first and second elements from the first column of the source matrix are respectively added to the first and second elements in the first row of the destination matrix. The first and second elements from the second column of the source matrix are respectively added to the third and fourth elements in the first row of the destination matrix. The third and fourth elements from the first column of the source matrix are respectively added to the first and second elements in the second row of the destination matrix, and The third and fourth elements from the second column of the source matrix are respectively added to the third and fourth elements in the second row of the destination matrix; and The execution circuit is used to execute the decoded single instruction according to the opcode.

2. The processor as claimed in claim 1, wherein, The second field specifies a two-dimensional register for storing the destination matrix.

3. The processor as claimed in claim 1, wherein, The second field specifies a two-dimensional register within the matrix accelerator that includes the execution circuitry, used to store the destination matrix.

4. The processor as claimed in claim 1, wherein, The second field specifies a first two-dimensional register for storing the destination matrix, and the single instruction further specifies a second source matrix and a second two-dimensional register for storing the second destination matrix.

5. The processor as claimed in claim 4, wherein, The opcode instructs the execution circuit to further cause the storage of the following: The first and second elements from the first column of the second source matrix are respectively added to the first and second elements in the first row of the second destination matrix. The first and second elements from the second column of the second source matrix are respectively added to the third and fourth elements in the first row of the second destination matrix. The third and fourth elements from the first column of the second source matrix are respectively added to the first and second elements in the second row of the second destination matrix, and The third and fourth elements from the second column of the second source matrix are respectively added to the third and fourth elements in the second row of the second destination matrix.

6. The processor as claimed in any one of claims 1 to 5, wherein, The first field includes the scale, index, and base address of the source matrix.

7. The processor as claimed in any one of claims 1 to 5, wherein, The single instruction further includes a field for specifying a hint that indicates the source matrix should be preferentially evicted from the processor's Level 1 cache.

8. The processor as claimed in any one of claims 1 to 5, wherein, The single instruction further includes a field for specifying zero-padding to be included in the destination matrix.

9. A method comprising: The processor's decoding circuitry decodes a single instruction into a decoded single instruction, which has a first field specifying a source matrix, a second field specifying a destination matrix, and an opcode instructing the processor's execution circuitry to cause storage of the following: The first and second elements from the first column of the source matrix are respectively added to the first and second elements in the first row of the destination matrix. The first and second elements from the second column of the source matrix are respectively added to the third and fourth elements in the first row of the destination matrix. The third and fourth elements from the first column of the source matrix are respectively added to the first and second elements in the second row of the destination matrix, and The third and fourth elements from the second column of the source matrix are respectively added to the third and fourth elements in the second row of the destination matrix; as well as The execution circuit executes the decoded single instruction according to the opcode.

10. The method of claim 9, wherein, The second field specifies a two-dimensional register for storing the destination matrix.

11. The method of claim 9, wherein, The second field specifies a two-dimensional register within the matrix accelerator that includes the execution circuitry, used to store the destination matrix.

12. The method of claim 9, wherein, The second field specifies a first two-dimensional register for storing the destination matrix, and the single instruction further specifies a second source matrix and a second two-dimensional register for storing the second destination matrix.

13. The method of claim 12, wherein, The opcode instructs the execution circuit to further cause the storage of the following: The first and second elements from the first column of the second source matrix are respectively added to the first and second elements in the first row of the second destination matrix. The first and second elements from the second column of the second source matrix are respectively added to the third and fourth elements in the first row of the second destination matrix. The third and fourth elements from the first column of the second source matrix are respectively added to the first and second elements in the second row of the second destination matrix, and The third and fourth elements from the second column of the second source matrix are respectively added to the third and fourth elements in the second row of the second destination matrix.

14. The method according to any one of claims 9 to 13, wherein, The first field includes the scale, index, and base address of the source matrix.

15. The method according to any one of claims 9 to 13, wherein, The single instruction further includes a field for specifying a hint that indicates the source matrix should be preferentially evicted from the processor's Level 1 cache.

16. The method according to any one of claims 9 to 13, wherein, The single instruction further includes a field for specifying zero-padding to be included in the destination matrix.

17. A non-transitory machine-readable medium storing code that, when executed by a machine, causes the machine to perform the method as described in any one of claims 9 to 16.

Images