Apparatus, method and system for 8-bit floating point matrix multiply instruction

Abstract

Apparatuses, methods, and systems for 8-bit floating point matrix multiply instructions are disclosed. Systems, methods, and apparatuses involving 8-bit floating point matrix multiply instructions are described. A processor embodiment includes fetch circuitry to fetch an instruction having a field to specify an opcode and a location of a destination matrix having single precision elements, a location of a first source matrix, and a location of a second source matrix, the source matrices having elements comprising quads of 8-bit floating point values, the opcode to indicate execution circuitry to cause, for each element of the first source matrix and a corresponding element of the second source matrix, conversion of the 8-bit floating point values to single precision values, multiplication of different pairs of converted single precision values to generate a plurality of results, and accumulation of the results with previous contents of a corresponding element of the destination matrix, decode circuitry to decode the fetched instruction, and execution circuitry to respond to the decoded instruction as specified by the opcode.

Description

[0001] This invention patent application is a divisional application of the invention patent application filed on December 8, 2021, with application number 202111490110.7 and title "Apparatus, Method and System for 8-bit Floating-Point Matrix Dot Product Instructions". Technical Field

[0002] This disclosure generally relates to computer processor architecture, and more specifically to systems and methods for executing 8-bit floating-point matrix dot product instructions. Background Technology

[0003] Matrices are becoming increasingly important in many computational tasks, such as machine learning and other batch data processing. Deep learning is a class of machine learning algorithms. Deep learning architectures, such as deep neural networks, have been applied in fields including computer vision, speech recognition, natural language processing, audio recognition, social network filtering, machine translation, bioinformatics, and drug design.

[0004] The two tools used for deep learning—inference and training—are trending towards low-precision arithmetic. Maximizing the throughput of deep learning algorithms and computations can help meet the needs of deep learning processors, such as those performing deep learning in data centers.

[0005] Matrix-matrix multiplication (also known as GEMM or generalized matrix multiplication) is a common computationally intensive operation on modern processors. Specialized hardware for matrix multiplication (e.g., GEMM) is a good option for improving peak computation (and energy efficiency) for certain applications such as deep learning.

[0006] Some of these applications, including deep learning, can operate on input data elements with relatively few bits without loss of accuracy, provided that the output element has enough bits (i.e., more than the input). Attached Figure Description

[0007] The present disclosure is illustrated by way of example and not by way of limitation in the accompanying drawings, in which similar reference numerals indicate similar elements, wherein:

[0008] Figure 1A The illustration shows an embodiment of the configured chips;

[0009] Figure 1B The illustration shows an embodiment of the configured chips;

[0010] Figure 2 Several examples of matrix storage are illustrated;

[0011] Figure 3 The illustration shows an example of a system that utilizes a matrix (slice) to operate an accelerator;

[0012] Figure 4 and Figure 5 Different embodiments of how to use a matrix operation accelerator to share memory are shown;

[0013] Figure 6 The illustration shows an example of using slice matrix multiplication accumulation operation (“TMMA”);

[0014] Figure 7 This illustration shows an example of an iterative execution subset of chained fused multiplication-accumulation instructions;

[0015] Figure 8 This illustration shows an example of an iterative execution subset of chained fused multiplication-accumulation instructions;

[0016] Figure 9 This illustration shows an example of an iterative execution subset of chained fused multiplication-accumulation instructions;

[0017] Figure 10 This illustration shows an example of an iterative execution subset of chained fused multiplication-accumulation instructions;

[0018] Figure 11 The illustration shows a SIMD implementation of a power of 2 according to an embodiment, wherein the accumulator uses an input size larger than the size of the input to the multiplier;

[0019] Figure 12 The diagram illustrates an embodiment of a system utilizing a matrix manipulation circuit;

[0020] Figure 13 The illustration shows an embodiment of a processor core pipeline that supports matrix operations using chips;

[0021] Figure 14 The illustration shows an embodiment of a processor core pipeline that supports matrix operations using chips;

[0022] Figure 15 The diagram shows examples of matrices expressed in row-major and column-major formats.

[0023] Figure 16 Illustrated examples of the use of matrices (slices);

[0024] Figure 17 An example of how to use a matrix (piece) is illustrated;

[0025] Figure 18 The illustration shows the configuration support for the use of chips according to an embodiment;

[0026] Figure 19 The illustration shows an embodiment describing the supported matrices (slices);

[0027] Figures 20(A)-20(D) An example of (multiple) registers;

[0028] Figure 21A This is a block diagram illustrating the use of the TDPBF8PS instruction to accelerate matrix multiplication according to some embodiments;

[0029] Figure 21B This is a block diagram illustrating an example execution circuit for executing TDPBF8PS instructions according to some embodiments;

[0030] Figure 22A This is pseudocode illustrating the execution of the TDPBF8PS instruction according to some embodiments;

[0031] Figure 22B The illustration is for use according to some embodiments of the invention. Figure 22A The pseudocode uses the helper function's pseudocode;

[0032] Figure 23 The illustration shows an embodiment of a processor executing a flow for processing TDPBF8PS instructions;

[0033] Figure 24 This is a block diagram illustrating the format of the TDPBF8PS instruction according to some embodiments;

[0034] Figures 25A-25B This is a block diagram illustrating a general vector-friendly instruction format and its instruction template according to an embodiment;

[0035] Figure 25A This is a block diagram illustrating a general vector-friendly instruction format and its Class A instruction template according to an embodiment;

[0036] Figure 25B This is a block diagram illustrating the general vector-friendly instruction format and its Class B instruction template according to an embodiment;

[0037] Figure 26A This is a block diagram illustrating an exemplary dedicated vector-friendly instruction format according to an embodiment;

[0038] Figure 26B This is a block diagram illustrating a field with a dedicated vector-friendly instruction format that constitutes a complete opcode field according to one embodiment;

[0039] Figure 26C This is a block diagram illustrating a field with a dedicated vector-friendly instruction format that constitutes a register index field according to one embodiment;

[0040] Figure 26D This is a block diagram illustrating a field with a dedicated vector-friendly instruction format that constitutes an extended operation field according to one embodiment;

[0041] Figure 27 This is a block diagram of a register architecture according to one embodiment;

[0042] Figure 28A This is a block diagram illustrating both an exemplary ordered pipeline and an exemplary out-of-order release / execution pipeline for register renaming, according to an embodiment.

[0043] Figure 28B This is a block diagram illustrating both an exemplary embodiment of an ordered architecture core to be included in a processor according to an embodiment and an exemplary out-of-order release / execution architecture core with register renaming.

[0044] Figures 29A-29B The diagram illustrates a more specific example of an ordered core architecture, which will be one of several logical blocks in a chip (including other cores of the same type and / or different types);

[0045] Figure 29A It is a block diagram of a single processor core according to an embodiment, its connection to the on-die interconnect network, and a local subset of its second-level (L2) cache;

[0046] Figure 29B According to the embodiments Figure 29A An expanded diagram of a portion of the processor core;

[0047] Figure 30 This is a block diagram of a processor according to an embodiment, which may have more than one core, may have an integrated memory controller, and may have an integrated graphics device;

[0048] Figures 31-34 This is a block diagram of an exemplary computer architecture;

[0049] Figure 31 A block diagram of a system according to an embodiment of the present disclosure is shown;

[0050] Figure 32 This is a block diagram of a first, more specific, exemplary system according to embodiments of the present disclosure;

[0051] Figure 33 This is a block diagram of a second, more specific, exemplary system according to embodiments of the present disclosure;

[0052] Figure 34 This is a block diagram of a System-on-a-Chip (SoC) according to embodiments of the present disclosure; and

[0053] Figure 35 This is a block diagram illustrating the use of a software instruction converter to convert binary instructions in a source instruction set into binary instructions in a target instruction set, according to an embodiment. Detailed Implementation

[0054] Numerous specific details are set forth in the following description. However, it should be understood that embodiments may be practiced without these specific details. In other instances, well-known circuits, structures, and techniques have not been shown in detail so as not to obscure the understanding of this description.

[0055] References to "an embodiment," "embodiment," "example embodiment," etc., in the specification indicate that the described embodiment may include a specific feature, structure, or characteristic; however, each embodiment may not necessarily include that specific feature, structure, or characteristic. Furthermore, such phrases do not necessarily refer to the same embodiment. Additionally, when a specific feature, structure, or characteristic is described in connection with an embodiment, it is believed that the influence of such feature, structure, or characteristic on such feature, structure, or characteristic in conjunction with other embodiments, whether explicitly described or not, is within the knowledge of those skilled in the art.

[0056] In many mainstream processors, handling matrices is a difficult and / or instruction-intensive task. For example, multiple rows of a matrix may be placed into multiple compact data (e.g., SIMD or vector) registers, and then operations can be performed on those rows individually. For instance, depending on the data size, adding two 8x2 matrices might require loading or pooling into four compact data registers. A first addition is then performed on the compact data registers corresponding to the first row of each matrix, and a second addition is performed on the compact data registers corresponding to the second row of each matrix. The resulting compact data registers are then distributed back into memory. While this scenario may be acceptable for small matrices, it is generally unacceptable for larger matrices.

[0057] discuss

[0058] This document describes mechanisms for supporting matrix operations in computer hardware such as central processing units (CPUs), graphics processing units (GPUs), and accelerators. Matrix operations utilize 2-dimensional (2-D) data structures representing one or more compact regions of memory (such as registers). Throughout this specification, these 2-D data structures are referred to as slices. Note that a matrix can be smaller than a slice (using less than the entirety of a slice), or multiple slices can be utilized (the matrix is ​​larger than any single slice). Throughout this specification, the matrix (slice) language is used to indicate operations performed using slices that affect the matrix; whether the matrix is ​​larger than any single slice is generally irrelevant.

[0059] Each slice can be operated on by different operations, such as those detailed herein, including but not limited to: matrix (slice) multiplication, slice addition, slice subtraction, slice diagonal, slice zeroing, slice transformation, slice dot product, slice broadcast, slice row broadcast, slice column broadcast, slice multiplication, slice multiplication and accumulation, slice movement, etc. Furthermore, in the future, support for operators such as those using scaling and / or biasing can be used in conjunction with these operations or to support non-numerical applications, such as OpenCL "local memory," data compression / decompression, etc. Instructions for performing matrix (slice) 8-bit floating-point slice dot product (TDPBF8PS) instructions are also described herein.

[0060] Multiple portions of storage (such as (non-volatile and volatile) memory, registers, caches, etc.) are arranged as slices with different horizontal and vertical scales. For example, a slice may have a horizontal scale of 4 (e.g., four rows of a matrix) and a vertical scale of 8 (e.g., eight columns of a matrix). Typically, the horizontal scale is related to the element size (e.g., 2-bit, 4-bit, 8-bit, 16-bit, 32-bit, 64-bit, 128-bit, etc.). Multiple data types (single-precision floating-point, double-precision floating-point, integers, etc.) can be supported.

[0061] Exemplary use of a configured chip

[0062] In some embodiments, slice parameters can be configured. For example, a given slice can be configured to provide slice options. Exemplary slice options include, but are not limited to, the number of rows in the slice, the number of columns in the slice, whether the slice is valid, and / or whether the slice consists of pairs of slices of equal size.

[0063] Figure 1A The illustration shows an embodiment of the configured chips. As shown, the 4kB of application memory 102 has four 1kB chips stored thereon—chip t0 104, chip t1 106, chip t2 108, and chip t3 110. In this example, these four chips are not composed of pairs, and each chip has elements arranged in rows and columns. Chips t0 104 and t1 106 have 4-byte elements (e.g., single-precision data) with K rows and N columns, where K = 8 and N = 32. Chips t2 108 and t3 110 have 8-byte elements (e.g., double-precision data) with K rows and N / 2 columns. Since the width of a double-precision operand is twice that of a single-precision operand, this configuration aligns with the palette used to provide chip options, providing at least 4kB of total storage for at least four names. In operation, chips can be loaded from and stored in memory using load and store operations. Depending on the instruction encoding scheme used, the amount of application memory available, as well as the size, number, and configuration of available chips, vary.

[0064] Figure 1BThe illustration shows an embodiment of the configured chips. As shown, 4kB of application memory 122 has two pairs of 1kB chips stored thereon. The first pair is chip t4L 124 and chip t4R 126, and the second pair is chip t5L 128 and chip t5R 130. As shown, the chip pairs are divided into left and right chips. In other embodiments, the chip pairs are divided into even-numbered and odd-numbered chips. In this example, each of the four chips has elements arranged in rows and columns. Chips t4L 124 and t4R 126 have 4-byte elements (e.g., single-precision floating-point data) with K rows and N columns, where K = 8 and N = 32. Chips t5L 128 and t5R 130 have 8-byte elements (e.g., double-precision floating-point data) with K rows and N / 2 columns. Since the width of a double-precision operand is twice that of a single-precision operand, this configuration aligns with the palette used to provide chip options, providing at least 4kB of total storage for at least two names. Figure 1A The four chips use four names, each name representing a 1kB chip, while Figure 1B Two chip pairs can be specified using two names. In some embodiments, chip instructions accept the names of the paired chips as operands. During operation, load and store operations can be used to load chips from memory and store chips into memory. The amount of available application memory, as well as the size, number, and configuration of available chips, vary depending on the instruction encoding scheme used.

[0065] In some embodiments, slice parameters are definable. For example, a "palette" is used to provide slice options. Exemplary options include, but are not limited to, the number of slice names, the number of bytes in the stored rows, the number of rows and columns in the slice, and so on. For example, the maximum "height" (number of rows) of a slice can be defined as:

[0066] Maximum number of rows per slice = number of constructed storage / (number of palette names * number of bytes per row).

[0067] This allows applications to write to names so that fixed usage can take advantage of different storage sizes across implementations.

[0068] The tilde configuration (“TILECONFIG”) directive is used to configure tildes, where specific tilde usage is defined within the selected palette. This declaration includes the number of tilde names to be used, the requested number of rows and columns for each name (tilde), and in some embodiments, the requested data type for each tilde. In some embodiments, a consistency check is performed during the execution of the TILECONFIG directive to determine its compliance with palette entries.

[0069] Exemplary slice storage type

[0070] Figure 2Several examples of matrix storage are illustrated. In (A), slices are stored in memory. As shown, each "row" consists of four compact data elements. A stride value is used to reach the next "row". Note that rows can be stored contiguously in memory. When slice storage does not map the row width of the underlying memory array, stride memory access allows access to a row and subsequently to the next row.

[0071] Loading a slice from memory and accessing a slice of memory typically involves step-by-step access from application memory to compressed data lines. Exemplary TILELOAD and TILESTORE instructions, or other instructions for application memory that serve as the TILE(slice) operand in load operation instructions, are in some embodiments reversible to handle page faults, unmasked floating-point exceptions, and / or interrupts for up to 2* lines per instruction.

[0072] In (B), the matrix is ​​stored in a slice of registers, such as compact data registers (Single Instruction Multiple Data (SIMD) or vector registers). In this example, the slice is stacked on three physical registers. Typically, contiguous registers are used; however, this is not always necessary.

[0073] In (C), the matrix is ​​stored in on-chip non-register memory accessible by fused multiply-accumulate (FMA) circuitry used in on-chip operations. This memory may be within or adjacent to the FMA. Furthermore, in some embodiments, as discussed below, this memory may be used for data elements rather than for entire rows or entire slices.

[0074] The CPUID reports the supported parameters of the TMMA architecture. In some embodiments, the information list includes the maximum height and the maximum SIMD scale. Configuring the TMMA architecture requires specifying the scale of each slice, the element size of each slice, and the palette identifier. This configuration is accomplished by executing the TILECONFIG directive.

[0075] Successful execution of the TILECONFIG instruction enables subsequent TILE operations. The TILERELEASEALL instruction clears the chip configuration and disables TILE operations (until the next TILECONFIG instruction is executed). In some embodiments, XSAVE, XSTORE, etc., are used during chip context switching. In some embodiments, two XCR0 bits are used in XSAVE: one for TILECONFIG metadata and one corresponding to the actual chip payload data.

[0076] TILECONFIG not only configures slice usage but also sets status variables that indicate whether the program is in the code region when the slice is configured. The implementation can enumerate restrictions on other instructions that can be used with the slice region, such as no use of existing register sets, etc.

[0077] Exiting a slice region is typically accomplished using the TILERELEASEALL command. This command takes no parameters and quickly invalidates all slices (indicating that no more data needs to be saved or recovered) and clears the internal state corresponding to the slice region.

[0078] In some embodiments, a slice operation will zero out any rows and columns that exceed the scale specified by the slice configuration. For example, as each row is written, the slice operation will zero out data exceeding the configured number of columns (taking into account the element size). For instance, for a 64-byte row and a slice configured with 10 rows and 12 columns, a write operation to an FP32 element will write output / result data in 12*4 bytes per row for the first 10 rows and zero out the remaining 4*4 bytes in each row. The slice operation also completely zeroes out any rows after the first 10 configured rows. When using a 1K slice with 64-byte rows, there will be 16 rows, so in this example, the last 6 rows will also be zeroed out.

[0079] In some embodiments, when loading data, a context recovery instruction (e.g., XRSTOR) forces that data exceeding the configured number of rows in the slice be maintained at zero. If no valid configuration exists, all rows are zeroed out. XRSTOR on slice data can load useless information from columns exceeding those configured columns. It should not be possible for XRSTOR to clear data exceeding the configured number of columns because there is no element width associated with the slice configuration.

[0080] When writing an entire TILE memory area to memory, a context save (e.g., XSAVE) exposes the entire TILE memory area. If XRSTOR loads useless data into the rightmost part of the slice, that data will be saved by XSAVE. For lines exceeding the number specified for each slice, XSAVE will write zeros.

[0081] In some embodiments, slice instructions are resumable. Memory access operations can be restarted after a page fault. Calculation instructions handling floating-point operations also allow unmasked floating-point exceptions, thanks to exception masks controlled by control and / or status registers.

[0082] To support the restart of instructions after these events, these instructions store information in the start register, which is detailed below.

[0083] Matrix (Slice) Operating System

[0084] Exemplary hardware support

[0085] Figure 3 The illustration depicts an embodiment of a system utilizing a matrix (slice) manipulation accelerator. In this illustration, a host processor / processing system 301 passes commands 311 (e.g., matrix manipulation operations, such as arithmetic or matrix manipulation operations, or load and store operations) to a matrix manipulation accelerator 307. However, this is shown in this manner for discussion purposes only. As detailed later, the accelerator 307 may be part of a processing core. Typically, commands 311, as slice manipulation operator instructions, refer to the slice in register-register (“reg-reg”) or register-memory (“reg-mem”) format. Other commands, such as TILESTORE, TILELOAD, TILECONFIG, etc., do not perform data manipulation on the slice. Commands may be decoded instructions (e.g., micro-operations) or macro instructions to be processed by the accelerator 307.

[0086] In this example, the coherent memory interface 303 is coupled to the host processor / processing system 301 and the matrix operation accelerator 307, enabling them to share memory. Figure 4 and Figure 5 Different embodiments of how to use a matrix operation accelerator to share memory are shown. For example... Figure 4 As shown, the host processor 401 and the matrix operation accelerator circuit 405 share the same memory 403. Figure 5 The illustration shows an embodiment where the host processor 501 and matrix operation accelerator 505 do not share memory but can access each other's memory. For example, processor 501 can access on-chip memory 507 and utilizes its host memory 503 as usual. Similarly, matrix operation accelerator 505 can access host memory 503, but more typically uses its own memory 507. Note that these memories can be of different types.

[0087] In some embodiments, a stacked structure on top of physical registers is used to support slices. For example, depending on the implementation, a slice may utilize 16 1024-bit registers, 32 512-bit registers, and so on. In some embodiments, matrix operations utilize 2-dimensional (2-D) data structures representing one or more compact regions of memory (such as registers). Throughout this specification, these 2-D data structures are referred to as slices or slice registers.

[0088] In some embodiments, the matrix operation accelerator 307 includes a plurality of FMAs 309 coupled to a data buffer 305 (in some implementations, one or more of these buffers 305 are stored in an FMA in a grid as shown). The data buffers 305 buffer slices loaded from memory and / or stored to memory (e.g., using slice load or slice store instructions). The data buffers may be, for example, a plurality of registers. Typically, these FMAs are arranged as a grid of linked FMAs 309s capable of reading and writing slices. In this example, the matrix operation accelerator 307 is used to perform matrix multiplication operations using slices T0, T1, and T2. At least one slice is accommodated in an FMA grid 309. In some embodiments, all slices in the operation are stored in an FMA grid 309. In other embodiments, only a subset is stored in an FMA grid 309. As shown, T1 is accommodated, while T0 and T2 are not. Note that A, B, and C refer to the matrices of these slices, which may or may not occupy the entire space of the slice.

[0089] Figure 6 The illustration shows an example of using slice matrix multiplication accumulation operation (“TMMA”).

[0090] The number of rows in the matrix (slice A 601) matches the number of cascaded (chained) FMAs, which include computational latency. The implementation is free to recycle on grids of smaller heights, but the computation remains the same.

[0091] The source / destination vectors come from a slice of N rows (slice C 605), and the grid 611 of the FMA performs N vector-matrix operations, resulting in a complete instruction to perform matrix multiplication of the slice. Slice B 603 is another vector source and provides "broadcast" items to the FMA at each level.

[0092] In operation, in some embodiments, the elements of matrix B (stored in slice B 603) are scattered across a rectangular grid of the FMA. Matrix B (stored in slice A 601) has its row elements transformed to match the column scale of the rectangular grid of the FMA. At each FMA in the grid, the elements of A and B are multiplied and added to the incoming addend (from the diagram above), and the outgoing sum is passed to the next row of the FMA (or the final output).

[0093] The latency of a single step is proportional to K (the row height of matrix B), and the dependent TMMA typically (in a single slice or across slices) has enough source-destination rows to hide this latency. Implementations can also segment the SIMD (compact data element) scale M (the row height of matrix A) across time steps, but this only changes the constant multiplied by K. When the program specifies K smaller than the maximum value enumerated by TMACC, the implementation is free to use either a "mask" or "early out".

[0094] The latency of the entire TMMA is proportional to N*K. The repetition rate is proportional to N. The number of MACs per TMMA instruction is N*K*M.

[0095] Figure 7 This illustration depicts an embodiment of a subset of the iterative execution of a chained fused multiply-accumulate instruction. Specifically, it illustrates the execution circuitry for iteratively executing a compressed data element position at a destination. In this embodiment, the chained fused multiply-accumulate operates against a signed source, where the accumulator is twice the size of the input data.

[0096] The first signed source (source 1 701) and the second signed source (source 2 703) each have four compact data elements. Each of these compact data elements stores signed data such as floating-point data. The third signed source (source 3 709) has two compact data elements, each of which stores signed data. The size of the first signed source 701 and the size of the second signed source 703 are half the size of the third signed source (initial value or previous result) 709. For example, the first signed source 701 and the second signed source 703 may have 32-bit compact data elements (e.g., single-precision floating-point), while the third signed source 709 may have 64-bit compact data elements (e.g., double-precision floating-point).

[0097] This diagram only shows the most valid two compressed data element positions of the first signed source 701 and the second signed source 703, as well as the most valid compressed data element position of the third signed source 709. Other compressed data element positions will, of course, be processed as well.

[0098] As shown, compacted data elements are processed in pairs. For example, multiplier circuit 705 multiplies the data from the most valid compacted data element positions of the first signed source 701 and the second signed source 703, and multiplier circuit 707 multiplies the data from the second most valid compacted data element positions of the first signed source 701 and the second signed source 703. In some embodiments, these multiplier circuits 705 and 707 are reused for other compacted data element positions. In other embodiments, additional multiplier circuitry is used to enable parallel processing of compacted data elements. In some contexts, parallel execution is accomplished using channels of the same size as the signed third source 709. The results of each of these multiplications are added using adder circuit 711.

[0099] (Using different adders 713 or the same adder 711) add the result of the addition of these multiplications to the data at the position of the most effective compacted data element from the signed source 3709.

[0100] Finally, the result of the second addition is stored in the signed destination 715 in the compacted data element position corresponding to the used compacted data element position from the signed third source 709, or if there is a next iteration, the result of the second addition is passed to that next iteration. In some embodiments, a write mask is applied to this storage such that storage occurs if the corresponding write mask (bit) is set, and storage does not occur if the corresponding write mask (bit) is not set.

[0101] Figure 8 This illustration depicts an embodiment of a subset of the iterative execution of a chained fused multiply-accumulate instruction. Specifically, it illustrates the execution circuitry for iteratively executing a compressed data element position at a destination. In this embodiment, the chained fused multiply-accumulate operates against a signed source, where the accumulator is twice the size of the input data.

[0102] The first signed source (source 1 801) and the second signed source (source 2 803) each have four compact data elements. Each of these compact data elements stores signed data such as integer data. The third signed source (source 3 809) has two compact data elements, each of which stores signed data. The size of the first signed source 801 and the size of the second signed source 803 are half the size of the third signed source 809. For example, the first signed source 801 and the second signed source 803 may have 32-bit compact data elements (e.g., single-precision floating-point), while the third signed source 809 may have 64-bit compact data elements (e.g., double-precision floating-point).

[0103] This diagram only shows the most valid two compressed data element positions of the first signed source 801 and the second signed source 803, as well as the most valid compressed data element position of the third signed source 809. Other compressed data element positions will, of course, be processed as well.

[0104] As shown, compacted data elements are processed in pairs. For example, multiplier circuit 805 multiplies the data from the most valid compacted data element positions of the first signed source 801 and the second signed source 803, and multiplier circuit 807 multiplies the data from the second most valid compacted data element positions of the first signed source 801 and the second signed source 803. In some embodiments, multiplier circuits 805 and 807 perform multiplication with infinite precision without saturation, and in case of overflow, use adder / saturation circuit 813 to saturate the accumulated result to positive or negative infinity, and in case of underflow, saturate the accumulated result to zero. In other embodiments, multiplier circuits 805 and 807 perform saturation themselves. In some embodiments, these multiplier circuits 805 and 807 are reused for other compacted data element positions. In other embodiments, additional multiplier circuits are used to enable parallel processing of compacted data elements. In some contexts, parallel execution is accomplished using channels of size 809 of the signed third source (initial value or result of previous iterations). The result of each of the multiple multiplications is added to the signed third source 809 using the addition / saturation circuit 813.

[0105] When addition results in an excessively large value, the adder / saturation (accumulator) circuit 813 preserves the sign of the operand. Specifically, saturation evaluation occurs for infinite-precision results between multiplexed additions and writes to the destination or the next iteration. When the accumulator 813 is floating-point and the input is an integer, the sum of the product and the floating-point accumulator input value are converted to an infinite-precision value (a fixed-point number with hundreds of digits), the multiplication result is added to the third input, and a single rounding to the actual accumulator type is performed.

[0106] Unsigned saturation means that the output value is limited to the maximum unsigned number (all 1s) of that element width. Signed saturation means that the value is limited to the range between the minimum negative number and the maximum positive number of that element width (e.g., for a byte, the range is from -128 (=-2^7) to 127 (=2^7-1)).

[0107] The result of addition and saturation check is stored in the signed result 815 in the corresponding compacted data element position from the signed third source 809, or the result is passed to the next iteration if there is one. In some embodiments, a write mask is applied to this storage such that storage occurs if the corresponding write mask (bit) is set, and storage does not occur if the corresponding write mask (bit) is not set.

[0108] Figure 9 This illustration shows an embodiment of a subset of the iterative execution of a chained fused multiplication-accumulation instruction. Specifically, it illustrates the iterative execution circuitry for a compact data element position at the destination. In this embodiment, the chained fused multiplication-accumulation operates on both signed and unsigned sources, where the accumulator is four times the size of the input data.

[0109] The first signed source (source 1 901) and the second unsigned source (source 2 903) each have four compacted data elements. Each of these compacted data elements contains data such as floating-point or integer data. The third signed source (initial value or result 915) has compacted data elements that store signed data. The size of the first source 901 and the second source 903 is one-quarter the size of the third signed source 915. For example, the first source 901 and the second source 903 may have 16-bit compacted data elements (e.g., words), while the third signed source 915 may have 64-bit compacted data elements (e.g., double-precision floating-point or 64-bit integer).

[0110] This diagram only shows the four most valid compressed data element positions of the first source 901 and the second source 903, as well as the most valid compressed data element position of the third signed source 915. Of course, if there are any other compressed data element positions, those will also be processed.

[0111] As shown in the figure, compacted data elements are processed as quadruples. For example, multiplier circuit 905 multiplies the data from the most significant compacted data element positions of the first source 901 and the second source 903, multiplier circuit 907 multiplies the data from the second most significant compacted data element positions of the first source 901 and the second source 903, multiplier circuit 909 multiplies the data from the third most significant compacted data element positions of the first source 901 and the second source 903, and multiplier circuit 911 multiplies the data from the least significant compacted data element positions of the first source 901 and the second source 903. In some embodiments, before multiplication, the signed compacted data elements of the first source 901 are sign-extended, and the unsigned compacted data elements of the second source 903 are zero-extended.

[0112] In some embodiments, these multiplier circuits 905-911 are reused for other compact data element locations. In other embodiments, additional multiplier circuitry is used to enable parallel processing of compact data elements. In some contexts, parallel execution is accomplished using channels of size 915 of the signed third source. The results of each of these multiplications are added using adder circuitry 913.

[0113] (Using different adders 917 or the same adder 913) add the result of the addition of these multiplications to the data at the position of the most effective compacted data element from the signed source 3 915.

[0114] Finally, the result 919 of the second addition is either stored in the signed destination at the compacted data element position corresponding to the position of the compacted data element used from the signed third source 915, or passed to the next iteration. In some embodiments, a write mask is applied to this storage such that storage occurs if the corresponding write mask (bit) is set, and storage does not occur if the corresponding write mask (bit) is not set.

[0115] Figure 10 This illustration depicts an embodiment of a subset of the iterative execution of a chained fused multiply-accumulate instruction. Specifically, it illustrates the execution circuitry for iteratively executing a compressed data element position at the destination. In this embodiment, the chained fused multiply-accumulate operates on both signed and unsigned sources, where the accumulator is four times the size of the input data.

[0116] The first signed source 1001 and the second unsigned source 1003 each have four compact data elements. Each of these compact data elements stores data such as floating-point data or integer data. The third signed source 1015 (initial or previous result) has compact data elements that store signed data. The size of the first source and the size of the second source are one-quarter the size of the third signed source 1015 (initial or previous result). For example, the first and second sources may have 16-bit compact data elements (e.g., words), while the third signed source 1015 (initial or previous result) may have 64-bit compact data elements (e.g., double-precision floating-point or 64-bit integer).

[0117] This diagram shows the four most valid compressed data element positions of the first signed source 1001 and the second unsigned source 1003, as well as the most valid compressed data element position of the third signed source 1015. Of course, if there are any other compressed data element positions, those will also be processed.

[0118] As shown in the figure, compacted data elements are processed as quadruples. For example, multiplier circuit 1005 multiplies the data from the most significant compacted data element positions of the first signed source 1001 and the second unsigned source 1003; multiplier circuit 1007 multiplies the data from the second most significant compacted data element positions of the first signed source 1001 and the second unsigned source 1003; multiplier circuit 1009 multiplies the data from the third most significant compacted data element positions of the first signed source 1001 and the second unsigned source 1003; and multiplier circuit 1011 multiplies the data from the least significant compacted data element positions of the first signed source 1001 and the second unsigned source 1003. In some embodiments, before multiplication, the signed compacted data elements of the first signed source 1001 are sign-extended, and the unsigned compacted data elements of the second unsigned source 1003 are zero-extended.

[0119] In some embodiments, these multiplier circuits 1005-1011 are reused for other compacted data element locations. In other embodiments, additional multiplier circuitry is used to enable parallel processing of compacted data elements. In some contexts, parallel execution is accomplished using a channel of size 1015 (the initial or previous result). The result of adding these multiplication results is added to the data from the highest valid compacted data element location of the third signed source 1015 (the initial or previous result) using the adder / saturator 1013 circuitry.

[0120] When addition results in a value that is too large or too small for signed saturation, the adder / saturation (accumulator) circuit 1013 preserves the sign of the operand. Specifically, saturation evaluation occurs for infinite-precision results between multiplexed additions and writes to the destination. When the accumulator 1013 is floating-point and the input is an integer, the sum of the product and the floating-point accumulator input value are converted to an infinite-precision value (a fixed-point number of several hundred digits), the multiplication result is added to the third input, and a single rounding to the actual accumulator type is performed.

[0121] The result 1019 of the addition and saturation check is either stored in the signed destination at the compacted data element position corresponding to the position of the compacted data element used from the third signed source 1015 (initial or previous result) or passed to the next iteration. In some embodiments, a write mask is applied to this storage such that storage occurs if the corresponding write mask (bit) is set, and storage does not occur if the corresponding write mask (bit) is not set.

[0122] Figure 11The illustration shows a SIMD implementation of a power of 2 according to an embodiment, where the accumulator uses an input size larger than the size of the input to the multiplier. Note that the source and accumulator values ​​(to the multiplier) can be signed or unsigned. Table 1101 illustrates different configurations for an accumulator with an input size of 2X (in other words, the size of the accumulator input value is twice the size of the compacted data element of the source). For a byte-sized source, the accumulator uses a 16-bit word or half-precision floating-point (HPFP) value. For a word-sized source, the accumulator uses a 32-bit integer or single-precision floating-point (SPFP) value. For an SPFP or 32-bit integer-sized source, the accumulator uses a 64-bit integer or double-precision floating-point (DPFP) value.

[0123] For an accumulator with a 4X input size (in other words, the size of the accumulator input value is four times the size of the compressed data element of the source), Table 1103 illustrates different configurations. For byte-sized sources, the accumulator uses 32-bit integer or single-precision floating-point (SPFP) values. In some embodiments, for word-sized sources, the accumulator uses 64-bit integer or double-precision floating-point (DPFP) values.

[0124] For an accumulator with an input size of 8X (in other words, the size of the accumulator input value is 8 times the size of the compressed data element of the source), Table 1105 illustrates the configuration. For a byte-sized source, the accumulator uses a 64-bit integer.

[0125] As previously mentioned, the matrix operation circuitry can be included in the core or used as an external accelerator. Figure 12 The illustration shows an embodiment of a system utilizing matrix operation circuitry. In this illustration, multiple entities are coupled to a ring interconnect 1245.

[0126] Multiple cores, core 0 1201, core 1 1203, core 2 1205, and core N 1207, provide non-chip-based instruction support. In some embodiments, matrix operation circuitry 1251 is located in core 1203, while in other embodiments, matrix operation circuitry 1211 and 1213 are accessible on ring interconnect 1245.

[0127] In addition, one or more memory controllers 1223-1225 are provided to communicate with memories 1233 and 1231 on behalf of the core and / or matrix operation circuitry.

[0128] Figure 13The illustration depicts an embodiment of a processor core pipeline that supports matrix operations using chips. Branch prediction and decoding circuitry 1303 performs branch prediction, decoding of instructions stored in instruction store 1301, and / or both branch prediction and decoding. For example, the instructions detailed herein may be stored in the instruction store. In some implementations, separate circuitry is used for branch prediction, and in some embodiments, at least some instructions are decoded into one or more micro-operations, microcode entry points, microinstructions, other instructions, or other control signals using microcode 1305. Branch prediction and decoding circuitry 1303 can be implemented using a variety of different mechanisms. Examples of suitable mechanisms include, but are not limited to, lookup tables, hardware implementations, programmable logic arrays (PLAs), microcode read-only memory (ROM), etc.

[0129] Branch prediction and decoding circuitry 1303 is coupled to allocation / renaming circuitry 1307, which in some embodiments is coupled to scheduler circuitry 1309. In some embodiments, these circuits provide register renaming, register allocation, and / or scheduling functionality by performing one or more of the following steps: 1) renaming logical operand values ​​to physical operand values ​​(e.g., a register alias table in some embodiments); 2) assigning status bits and flags to decoded instructions; and 3) (e.g., using a reserved station in some embodiments) scheduling the decoded instructions for execution on execution circuitry outside the instruction pool.

[0130] Scheduler circuitry 1309 represents any number of different schedulers, including reserved stations, central instruction windows, etc. Scheduler circuitry 1309 is coupled to or includes (multiple) physical register files 1315. Each of the (multiple) physical register files 1315 represents one or more physical register files, wherein different physical register files store one or more different data types, such as scalar integers, scalar floating-point numbers, compressed integers, compressed floating-point numbers, vector integers, vector floating-point numbers, status (e.g., an instruction pointer as the address of the next instruction to be executed), slices, etc. In one embodiment, (multiple) physical register files 1315 include vector register circuitry, write mask register circuitry, and scalar register circuitry. These register circuitry can provide architectural vector registers, vector mask registers, and general-purpose registers. Multiple physical register files 1315 are covered by retirement circuitry 1317 to illustrate various ways in which register renaming and out-of-order execution can be implemented (e.g., using multiple reordering buffers and multiple retirement register files, using multiple future files, multiple history buffers, multiple retirement register files, using register mappings and register pools, etc.). Retirement circuitry 1317 and multiple physical register files 1315 are coupled to execution circuitry 1311.

[0131] Although register renaming is described in the context of out-of-order execution, it should be understood that register renaming can be used in ordered architectures. While the illustrated embodiments of the processor may also include separate instruction and data cache units and a shared L2 cache unit, alternative embodiments may also have a single internal cache for both instructions and data, such as, for example, a Level 1 (L1) internal cache, or multiple levels of internal caches. In some embodiments, the system may include a combination of internal caches and external caches located outside the core and / or processor. Alternatively, all caches may be located outside the core and / or processor.

[0132] Execution circuitry 1311 is a collection of one or more execution units, including scalar circuitry 1321, vector / SIMD circuitry 1323, matrix operation circuitry 1327, and memory access circuitry 1325 for accessing cache 1313. The execution circuitry performs various operations (e.g., shift, addition, subtraction, multiplication) and on various data types (e.g., scalar floating-point, compressed integer, compressed floating-point, vector integer, vector floating-point). While some embodiments may include several execution units dedicated to a particular function or set of functions, other embodiments may include only one execution unit or multiple execution units that all perform all functions. Scalar circuitry 1321 performs scalar operations, vector / SIMD circuitry 1323 performs vector / SIMD operations, and matrix operation circuitry 1327 performs the matrix (slice) operations detailed herein.

[0133] As an example, an exemplary register-renaming, out-of-order release / execution core architecture can implement a pipeline as follows: 1) an instruction fetch circuit performs the fetch and length decoding stage; 2) a branch and decode circuit 1303 performs the decoding stage; 3) an allocation / rename circuit 1307 performs the allocation and rename stages; 4) a scheduler circuit 1309 performs the scheduling stage; 5) (multiple) physical register files coupled to or included in the scheduler circuit 1309 and the allocation / rename circuit 1307 and memory units) perform the register read / memory read stage; execution circuit 1311 performs the execution stage; 6) memory units and (multiple) physical register file units perform the write-back / memory write stage; 7) each unit may involve an exception handling stage; and 8) a retirement unit and (multiple) physical register file units perform the commit stage.

[0134] The core may support one or more instruction sets (e.g., the x86 instruction set (with some extensions added with newer versions); the MIPS instruction set of MIPS Technologies, Inc., Sunnyvale, California; the ARM instruction set of ARM Holdings, Inc., Sunnyvale, California (with optional additional extensions such as NEON)), including the instructions described herein. In one embodiment, core 1390 includes logic for supporting compressed data instruction set extensions (e.g., AVX1, AVX2), thereby allowing the use of compressed data to perform operations used by many multimedia applications.

[0135] It should be understood that a core can support multithreading (execution of two or more parallel operations or a set of threads), and this multithreading can be accomplished in various ways, including time-division multithreading, simultaneous multithreading (where a single physical core provides a logical core for each thread in the simultaneous multithreading of the physical core), or combinations thereof (e.g., time-division fetching and decoding, and subsequent operations such as...). Simultaneous multithreading in hyper-threading technology.

[0136] Figure 14 The illustration depicts an embodiment of a processor core pipeline that supports matrix operations using chips. Branch prediction and decoding circuitry 1403 performs branch prediction, decoding of instructions stored in instruction store 1401, and / or both branch prediction and decoding. For example, the instructions detailed herein may be stored in the instruction store. In some implementations, separate circuitry is used for branch prediction, and in some embodiments, at least some instructions are decoded into one or more micro-operations, microcode entry points, microinstructions, other instructions, or other control signals using microcode 1405. Branch prediction and decoding circuitry 1403 can be implemented using a variety of different mechanisms. Examples of suitable mechanisms include, but are not limited to, lookup tables, hardware implementations, programmable logic arrays (PLAs), microcode read-only memory (ROM), etc.

[0137] Branch prediction and decoding circuitry 1403 is coupled to allocation / renaming circuitry 1407, which in some embodiments is coupled to scheduler circuitry 1409. In some embodiments, these circuits provide register renaming, register allocation, and / or scheduling functionality by performing one or more of the following steps: 1) renaming logical operand values ​​to physical operand values ​​(e.g., a register alias table in some embodiments); 2) assigning status bits and flags to decoded instructions; and 3) (e.g., using a reserved station in some embodiments) scheduling the decoded instructions for execution on execution circuitry outside the instruction pool.

[0138] Scheduler circuitry 1409 represents any number of different schedulers, including reserved stations, central instruction windows, etc. Scheduler circuitry 1409 is coupled to or includes physical register files 1415. Each of the physical register files 1415 represents one or more physical register files, where different physical register files store one or more different data types, such as scalar integers, scalar floating-point numbers, compressed integers, compressed floating-point numbers, vector integers, vector floating-point numbers, status (e.g., an instruction pointer as the address of the next instruction to be executed), slices, etc. In one embodiment, physical register files 1415 include vector register circuitry, write mask register circuitry, and scalar register circuitry. These register circuitry can provide architectural vector registers, vector mask registers, and general-purpose registers. Multiple physical register files 1415 are covered by retirement circuitry 1417 to illustrate various ways in which register renaming and out-of-order execution can be implemented (such as using multiple reordering buffers and multiple retirement register files, using multiple future files, multiple history buffers, multiple retirement register files, using register mappings and register pools, etc.). Retirement circuitry 1417 and multiple physical register files 1415 are coupled to execution circuitry 1411.

[0139] Although register renaming is described in the context of out-of-order execution, it should be understood that register renaming can be used in ordered architectures. While the illustrated embodiments of the processor may also include separate instruction and data cache units and a shared L2 cache unit, alternative embodiments may also have a single internal cache for both instructions and data, such as, for example, a Level 1 (L1) internal cache, or multiple levels of internal caches. In some embodiments, the system may include a combination of internal caches and external caches located outside the core and / or processor. Alternatively, all caches may be located outside the core and / or processor.

[0140] The execution circuitry 1411 includes a set of one or more execution circuits 1427 and a set of one or more memory access circuits 1425 for accessing the cache 1413. The execution circuits 1427 perform the matrix (slice) operations detailed herein.

[0141] As an example, an exemplary register-renaming, out-of-order release / execution core architecture can implement a pipeline as follows: 1) an instruction fetch circuit performs the fetch and length decoding stage; 2) a branch and decode circuit 1403 performs the decoding stage; 3) an allocation / rename circuit 1407 performs the allocation and rename stages; 4) a scheduler circuit 1409 performs the scheduling stage; 5) (multiple) physical register files coupled to or included in the scheduler circuit 1409 and the allocation / rename circuit 1407 and memory units) perform the register read / memory read stage; execution circuit 1411 performs the execution stage; 6) memory units and (multiple) physical register file units perform the write-back / memory write stage; 7) each unit may involve an exception handling stage; and 8) a retirement unit and (multiple) physical register file units perform the commit stage.

[0142] The core may support one or more instruction sets (e.g., the x86 instruction set (with some extensions added with newer versions); the MIPS instruction set of MIPS Technologies, Inc., Sunnyvale, California; the ARM instruction set of ARM Holdings, Inc., Sunnyvale, California (with optional additional extensions such as NEON)), including the instructions described herein. In one embodiment, core 1490 includes logic for supporting compressed data instruction set extensions (e.g., AVX1, AVX2), thereby allowing the use of compressed data to perform operations used by many multimedia applications.

[0143] It should be understood that a core can support multithreading (execution of two or more parallel operations or a set of threads), and this multithreading can be accomplished in various ways, including time-division multithreading, simultaneous multithreading (where a single physical core provides a logical core for each thread in the simultaneous multithreading of the physical core), or combinations thereof (e.g., time-division fetching and decoding, and subsequent operations such as...). Simultaneous multithreading in hyper-threading technology.

[0144] layout

[0145] Throughout this specification, a row-based data layout is used to represent data. Column-based users should modify these items according to their orientation. Figure 15 The diagram illustrates examples of matrices expressed in row-major and column-major formats. As shown, matrix A is a 2x3 matrix. When this matrix is ​​stored in row-major format, the data elements in the rows are consecutive. When this matrix is ​​stored in column-major format, the data elements in the columns are consecutive. T *B T =(BA) T This is a well-known property of matrices, where the superscript T indicates a transformation. Reading data as row-major data results in a matrix that looks like a transformation matrix.

[0146] In some embodiments, the semantics of behavior-major are utilized in the hardware, and column-major data will swap the operand order and result in a matrix transformation, but for subsequent column-major reads from memory, it is a correct non-transformation matrix.

[0147] For example, if you have two columns-matrixes to be multiplied:

[0148] abgik ag+bh ai+bj ak+bl

[0149] cd* hjl= cg+dh ci+dj ck+dl

[0150] ef eg+fh ei+fj ek+fl

[0151] (3x2) (2x3) (3x3)

[0152] The input matrix will be stored in linear memory (column-major) as follows:

[0153] acebdf

[0154] as well as

[0155] ghijk l.

[0156] If those matrices are read as row-major matrices at scales of 2x3 and 3x2, they will be represented as follows:

[0157] ace and gh

[0158] bdfij

[0159] kl

[0160] Commutation and matrix multiplication:

[0161] ghace ag+bh cg+dh eg+fh

[0162] ij *bdf= ai+bj ci+dj ei+fj

[0163] kl ak+bl ck+dl ek+fl

[0164] The transformation matrix is ​​shifted out and can then be stored in row-major order:

[0165] ag+bh cg+dh eg+fh ai+bj ci+dj ei+fj ak+bl ck+dl ek+fl

[0166] And it is used in subsequent column-major calculations, and it is the correct untransformed matrix:

[0167] ag+bh ai+bj ak+bl

[0168] cg+dh ci+dj ck+dl

[0169] eg+fh ei+fj ek+fl

[0170] Exemplary use

[0171] Figure 16 This diagram illustrates an example of the use of matrices (e.g., slices). In this example, matrix C 1601 comprises two slices, matrix A 1603 comprises one slice, and matrix B 1605 comprises two slices. The diagram shows an example of the inner loop of an algorithm used to compute matrix multiplication. In this example, two resulting slices tmm0 and tmm1 from matrix C 1601 are used to accumulate intermediate results. When a slice (tmm2) from matrix A 1603 is multiplied by two slices from matrix B 1605, this slice is reused twice. Pointers are used to load a new A matrix (slice) and two new B matrices (e.g., slices) from the direction indicated by the arrow. The outer loop, not shown, adjusts the pointer for slice C.

[0172] The exemplary code shown in the figure includes the use of slice configuration instructions, which are executed to configure slice usage, load the slice, perform a slice loop, store the slice in memory, and release the slice usage.

[0173] Figure 17 The illustration shows an embodiment of the use of a matrix (e.g., a slice). At 1701, slice usage is configured. For example, the TILECONFIG instruction is executed to configure slice usage, including setting the number of rows and columns for each slice. Typically, at 1703, at least one matrix (slice) is loaded from memory. At 1705, the matrix (e.g., slice) is used to perform at least one matrix (slice) operation. At 1707, at least one matrix (slice) is stored out of memory, and at 1709, a context switch may occur.

[0174] Exemplary configuration

[0175] Hardware support for chip configuration

[0176] As discussed above, slice usage typically requires configuration before use. For example, it may not be necessary to use all rows and columns. In some embodiments, not configuring these rows and columns not only saves power but also allows the configuration to determine whether an operation will generate an error. For example, if M and L are not the same, matrix multiplication of the form (N x M)*(L x N) will generally not work.

[0177] In some embodiments, slice support is configured before using the slice matrix. For example, the number of rows and columns per slice, the slices to be used, and so on are configured. The TILECONFIG instruction is an improvement to the computer itself because it provides support for configuring the computer to use (as part of the processor core or as an external device) a matrix accelerator. Specifically, execution of the TILECONFIG instruction causes the configuration to be retrieved from memory and applied to the matrix (slice) settings within the matrix accelerator.

[0178] Chip usage configuration

[0179] Figure 18 The illustration shows the configuration for supporting the use of chips according to an embodiment. Memory 1801 contains chip descriptions 1803 of the matrices (e.g., chips) to be supported.

[0180] The instruction execution resource 1811 of the processor / core 1805 stores various aspects of the chip description 1803 into the chip configuration 1817. The chip configuration 1817 includes a palette table 1813 detailing which chips (number of rows and columns in each chip) are configured for the palette, and matrix support tags for use. Specifically, the instruction execution resource 1811 is configured to use the chip as specified in the chip configuration 1817. The instruction execution resource 1811 may also include machine-specific registers or configuration registers for indicating chip use. Additional values, such as usage midpoint and start values, are also set. The chip configuration 1817 utilizes register(s) 1819 to store chip usage and configuration information.

[0181] Figure 19 The illustration shows an embodiment of a description of the supported matrices (e.g., slices). This is a description that will be stored in response to the execution of the STTILECFG instruction. In this example, each field is a byte. In byte [0], the palette ID 1901 is stored. The palette ID is used to index the palette table 1813, which, as defined by the configuration, stores the number of bytes in the slice and the bytes of each row of the slice associated with that ID.

[0182] Byte 1 storage will be the value stored in the "startRow" register 1903, and byte 2 storage will be the value stored in the startP register 1905. These registers are used to support restart instructions following these events. The startRow value indicates the row that should be used for restarting. The startP value indicates the position within the row used for storage operations when the pair is used, and in some embodiments, the startP value indicates the lower half of the row (in the lower slice of the pair) or the upper half of the row (in the higher slice of the pair). Generally, this position within the row (column) is not required.

[0183] Successful execution of matrix (slice) instructions will set both startRow and startP to zero, with the exception of TILECONFIG and STTILECFG.

[0184] At any point without restarting the interrupted matrix (slice) instructions, it is the software's responsibility to zero out the values ​​of startRow and startP. For example, an unmasked floating-point exception handler might decide to complete the operation in software and change the program counter value for another instruction, typically the next one. In this case, the software exception handler must zero out the startRow and startP values ​​from the exception presented to it by the operating system before resuming the process. The operating system will then use recovery instructions to reload those values.

[0185] Indication of pairs of byte 3 memory slices (1 byte per slice) 1907.

[0186] Bytes 16-17 store the number of rows (1913) and columns (1915) of slice 0, bytes 18-19 store the number of rows and columns of slice 1, and so on. In other words, each 2-byte group specifies the number of rows and columns of the slice. If 2-byte groups are not used to specify slice parameters, they should have a value of zero. Specifying slice parameters for more slices than the implementation limits or palette limits will result in an error. Unconfigured slices are set to an initial state with 0 rows and 0 columns.

[0187] Ultimately, the configuration in memory usually ends with a description such as all zeros for several consecutive bytes.

[0188] Exemplary chips and chip configuration storage

[0189] Figures 20(A)-20(D)An example of multiple registers 1819 is illustrated in Figure 20(A). As shown, each slice (TMM0 2001...TMMN 2003) has separate registers, each storing the row and column dimensions for that particular slice. StartP 2011 and StartRow 2013 are stored in separate registers. One or more status registers 2015 are set (e.g., TILES_CONFIGURED = 1) to indicate that the slice is configured for use.

[0190] Figure 20(B) illustrates multiple registers 1819. As shown, each slice has separate registers for its rows and columns. For example, TMM0 row configuration 2021, TMM0 column configuration 2023, StartP 2011, and StartRow 2013 are stored in separate registers. One or more status registers 2015 are set (e.g., TILES_CONFIGURED = 1) to indicate that the slice is configured for use.

[0191] Figure 20(C) illustrates a single register 1819. As shown, this register stores the slice configuration (rows and columns per slice) 2031, StartP 2011, and StartRow 2013 in a single register as a compact data register. One or more status registers 2015 are set (e.g., TILES_CONFIGURED = 1) to indicate that the slice is configured for use.

[0192] Figure 20(D) illustrates multiple registers 1819. As shown, a single register stores slice configurations (rows and columns per slice) 2031. StartP and StartRow are stored in separate registers 2011 and 2013. One or more status registers 2015 are set (e.g., TILES_CONFIGURED = 1) to indicate that the slice is configured for use.

[0193] Other combinations were conceived, such as combining the start registers into a single register in which they are displayed separately, and so on.

[0194] TDPBF8PS

[0195] As mentioned above, specialized hardware for general matrix multiplication (known as GEMM) is a good choice for improving peak computational performance (and energy efficiency) for certain applications such as deep learning. Some of these applications, including deep learning, can operate on input data elements with relatively few bits without loss of accuracy, provided the output elements have enough bits (i.e., more than the input).

[0196] Therefore, the disclosed method and system perform an 8-bit floating-point matrix dot product operation TILEDPBF8PS (TDPBF8PS), which takes a source matrix (e.g., a slice) with 8-bit floating-point elements, performs dot product multiplication, and accumulates the resulting product with a 32-bit single-precision destination.

[0197] In some embodiments, the 8-bit floating-point format is an 8-bit wide binary floating-point format (BF8), which is the Institute of Electrical and Electronics Engineers (IEEE) (e.g., the IEEE 754 standard) half-precision binary floating-point format (IEEE FP16), in which the lower half (8 LSBs) is cut off. The BF8 format may include a sign field (one bit wide), an exponent field (five bits wide), and a mantissa (significant digit precision) field (two bits wide). In some embodiments, it is assumed that the mantissa (significant digit precision) field has an implicit leading bit with a value of one, unless the exponent field stores all zeros. Further, the 32-bit floating-point format may include binary32 (according to the IEEE standard), which is sometimes referred to herein as “single precision” or “fp32,” for example, having a sign field (one bit wide), an exponent field (eight bits wide), and a mantissa (significant digit precision) field (implicitly stored 24 bits, i.e., explicitly stored 23 bits wide).

[0198] In some embodiments, the disclosed TDPBF8PS instruction is executed by a processor including fetch circuitry for fetching the instruction, which has fields for specifying the opcode and the positions of an M x N destination matrix (slice), an M x K first source matrix (slice), and a K x N second source matrix (slice) having single-precision elements. Each element of the specified first and second source matrices includes a quadruple of 8-bit floating-point values, including a first 8-bit floating-point value (e.g., in...). Figure 21A The source element contains a (0,1,2,3) quadruple with a quadruple index of zero, and a second 8-bit floating-point value (e.g., in...). Figure 21A The source contains a (0,1,2,3) quadruple with a quadruple index of 1, and a third 8-bit floating-point value (e.g., in...). Figure 21A The source contains a (0,1,2,3) quadruple with index 2, and a fourth 8-bit floating-point value (e.g., in...). Figure 21AIn each element of the source matrix (e.g., the quadruples in the (0,1,2,3) quadruples are indexed as three), where the opcode instructs the execution circuit to: for each element (e.g., each of the M x N elements) of the specified destination matrix (e.g., a slice), convert the 8-bit floating-point values ​​of the K quadruples of rows M from the specified first source matrix (e.g., a slice) to the 8-bit floating-point values ​​of the K corresponding quadruples of columns N from the specified second source matrix (e.g., a slice) to single-precision values, and multiply the K pairs of converted first values ​​from the two specified source matrices (e.g., slices) to generate K first multiplications. The product is generated by multiplying K pairs of transformed second values ​​from two specified source matrices (e.g., slices) to produce K second products, multiplying K pairs of transformed third values ​​from two specified source matrices (e.g., slices) to produce K third products, multiplying K pairs of transformed fourth values ​​from two specified source matrices (e.g., slices) to produce K fourth products, summing the first and second products to produce a first accumulated sum, summing the third and fourth products separately to produce a second accumulated sum, and summing the first and second accumulated sums to produce a final accumulated sum to be added to the previous contents of the element (M,N).

[0199] In some embodiments, the disclosed TDPBF8PS instruction is executed by a processor including fetch circuitry for fetching the instruction, which has a field for specifying an opcode that instructs the execution circuitry to cause: for each element of the first source matrix and the corresponding element of the second source matrix, converting an 8-bit floating-point value to a single-precision value; multiplying the converted single-precision values ​​from the first value of the quaternion together to generate a first result; multiplying the converted single-precision values ​​from the second value of the quaternion together to generate a second result; multiplying the converted single-precision values ​​from the third value of the quaternion together to generate a third result; multiplying the converted single-precision values ​​from the fourth value of the quaternion together to generate a fourth result; and accumulating the first, second, third, and fourth results with the previous contents of the corresponding elements of the destination matrix.

[0200] In some embodiments, the processor will also include other supporting hardware, such as decoding circuitry for decoding the fetched instructions, and execution circuitry for responding to the decoded instructions as specified by the opcode, for example, enabling a matrix operation accelerator (e.g., Figure 3 The matrix operation accelerator 307 in the middle is an execution circuit that executes one or more (e.g., all) actions of the TDPBF8PS instruction.

[0201] Figure 21AThis is a block diagram illustrating the use of the TDPBF8PS instruction to accelerate matrix multiplication according to some embodiments. As shown, instruction 2101 includes fields for specifying the opcode 2102 (e.g., TDPBF8PS) and the positions 2104 of an M-by-N destination matrix (e.g., a slice) having single-precision elements, 2106 of an M-by-K first source matrix (e.g., a slice), and 2108 of a K-by-N second source matrix (e.g., a slice), wherein the specified source matrices have elements each comprising a quadruple of 8-bit (e.g., BF8) floating-point values. (At least referenced...) Figure 24 , Figures 25A-25B and Figures 26A-26D Further illustrations and descriptions are provided of the format of the TDPBF8PS instruction according to some embodiments.

[0202] Here, the size of the specified first source matrix (e.g., slice) 2112A is M = 4 multiplied by K = 3. The size of the specified second source matrix (e.g., slice) 2112B is K = 3 multiplied by N = 5. For illustrative purposes, K, M, and N are shown to have different values, but in other embodiments, they may be equal.

[0203] In one embodiment of operation, processor 2100 is configured to respond to opcode 2102 (TDPBF8PS) by: for each element (M, N) of a specified destination matrix (e.g., slice) 2122, using conversion circuit 2116A to convert the values ​​of K quadruplets of row M from a specified first source matrix (e.g., slice) 2112A to single precision, and using conversion circuit 2116B to convert the values ​​of K quadruplets of column N from a specified second source matrix (e.g., slice) 2112B to single precision, such as binary32 single-precision floating-point as specified in IEEE 794. Then, the processor 2100 uses the multiplication circuit 2118 to multiply K pairs of the transformed first quaternion values ​​together, K pairs of the transformed second quaternion values ​​together, K pairs of the transformed third quaternion values ​​together, and K pairs of the transformed fourth quaternion values ​​together, and uses the accumulation circuit 2120 to accumulate the products (4*K products) with the previous contents of the elements (M,N).

[0204] This illustration shows the execution of the TDPBF8PS instruction, used to set the destination element at a matrix (e.g., slice) location, for example, C. 1,0 The index of the (row, column) in the array is (1,0). Figure 21A In this context, ".0" refers to the first value of the quaternion, ".1" refers to the second value, ".2" refers to the third value, and ".3" refers to the fourth value. For example, to make A... 1,0.0 It is stored in element A 1,0The first value of the quadruple in B, and B 2,4.3 It is stored in element B 2,4 The fourth value of the quadruples in the matrix. In some embodiments, processor 2100 is used to convert the 8-bit floating-point values ​​of K (=3) quadruples from rows M (=1) of the specified first source matrix (e.g., slice) 2112A and the 8-bit floating-point values ​​of K (=3) quadruples from columns N (=0) of the specified second source matrix (e.g., slice) 2112B to single precision using conversion circuits 2116A and 2116B. In some embodiments, processor 2100 (e.g., matrix operation circuitry, for example, as part of a matrix operation accelerator) is then used to multiply K pairs of the converted first quadruples from the specified two source matrices (e.g., slices) to generate K first products, multiply K pairs of the converted second quadruples from the specified source matrix (e.g., slices) to generate K second products, and multiply K pairs of the converted third quadruples from the specified source matrix (e.g., slices) to generate K third products using multiplication circuit 2118. The product is generated by multiplying K pairs of transformed fourth quaternion values ​​from the specified source matrix (e.g., slice) to produce K fourth products, and then accumulating these products separately using accumulator circuit 2120, including the first sum of K first products, the second sum of K second products, the third sum of K third products, and the fourth sum of K fourth products, and accumulating the first, second, third, and fourth sums with the previous contents of element (M,N), for example, shown in the example here as the FP32 value from element C(1,0).

[0205] In some embodiments, processor 2100 is configured to use accumulator circuit 2120 to sum K first products with K second products, and separately sum K third products with K fourth products, and then sum the two sums with the previous contents of element (M,N).

[0206] As shown, three arrows emanate from each of the specified first and second source matrices (e.g., slices) to indicate that the transformations and multiplications occur in parallel. In some embodiments, the processor responds to the decoded instructions by generating results in parallel and storing the results in each element of the specified destination matrix (e.g., slice). In some embodiments, new values ​​are generated and stored in the destination one row at a time or one column at a time.

[0207] The disclosed embodiments improve upon alternative methods by allowing software to execute TDPBF8PS instructions with a reduced source element size. This allows for the use of less memory space and less memory bandwidth, and improves peak computational performance (and energy efficiency) for certain applications. Some applications, such as deep learning, can operate on input data elements with relatively few bits without loss of accuracy, provided the output element has enough bits (e.g., more than the input).

[0208] Figure 21B This is a block diagram illustrating an example execution circuit 2114 (e.g., matrix operation circuitry) for executing TDPBF8PS instructions according to some embodiments. The example execution circuit 2114 includes a first data-width data path (e.g., 8-bit wide, e.g., according to BF8 format) and a wider second data path width (e.g., 32-bit wide, e.g., according to full-precision format), for example, where row 2132 is 8-bit wide (e.g., BF8) and row 2134 is 32-bit wide (e.g., float32). For example, circuitry for converting BF8 to full precision (BF8 to F32), full-precision multiplication (F32 MUL), and full-precision addition (F32 ADD) circuitry are utilized. In some embodiments, the addition circuitry of certain precision (e.g., adder 2136) further includes a bit-aligned shifter, a bit-aligned adder, a bit-normalized subtractor, a bit incrementer, and / or exponentiation logic. References are at least... Figures 22A-22B , Figure 23 and Figures 28A-28B Further illustrations and descriptions are provided of the systems and methods used to execute the TDPBF8PS instructions.

[0209] In some embodiments, the TDPBF8PS instructions are part of a slice (e.g., AMX) architecture extension of an ISA that includes two-dimensional (2D) registers (e.g., where each slice register is identified as a single “slice register” (e.g., a single pointer to a single slice register), as opposed to vector (e.g., ZMM, YMM, or XMM) registers), and the ISA may include separate instructions for loading / storing 2D blocks (e.g., strides of consecutive locations) from memory, and instructions for performing matrix-matrix multiplications on three registers (e.g., matrix C). updated = matrix A x matrix B + matrix C previous ), and / or instructions for performing element-wise arithmetic operations on two (or three) source slices. In one embodiment, one or more source matrices are first loaded (e.g., via the host processor) into a cache (e.g., a level 1 (L1) data cache) and then loaded from the cache (e.g., via the execution of slice load instructions) into slice registers (e.g., via a matrix operation accelerator), for example via Figure 3The consistent memory interface 303 in the middle.

[0210] Exemplary execution

[0211] Figure 22A This is pseudocode illustrating exemplary execution of the TDPBF8PS instruction according to some embodiments. As shown, instruction 2201 includes opcode 2202 (e.g., TDPBF8PS) and positions 2204 for an M-by-N destination matrix (e.g., slice) having single-precision elements, positions 2206 for an M-by-K first source matrix (e.g., slice), and positions 2208 for a K-by-N second source matrix (e.g., slice), the specified source matrices having elements comprising quadruples of 8-bit floating-point values. Opcode 2202 (TDPBF8PS) instructs the processor to perform the following operations as shown in pseudocode 2200: For each element (M, N) of the specified destination matrix (e.g., slice), convert the 8-bit values ​​of K quadruplets from row M of the specified first source matrix (e.g., slice) and the 8-bit values ​​of K quadruplets from column N of the specified second source matrix (e.g., slice) to single precision; multiply the K pairs of converted first quadruplet values ​​from the two specified source matrices (e.g., slice) to generate a first product; and multiply the K pairs of converted first quadruplet values ​​from the two specified source matrices (e.g., slice) to generate a first product. For example, K from the specified source matrices (e.g., slices) is multiplied by the transformed second quaternion value to generate a second product; K from the specified source matrices (e.g., slices) is multiplied by the transformed third quaternion value to generate a third product; K from the specified source matrices (e.g., slices) is multiplied by the transformed fourth quaternion value to generate a fourth product; the first product and the second product are summed to generate a first accumulated sum; the third product and the fourth product are summed separately to generate a second accumulated sum; and the first and second accumulated sums are summed to obtain the final accumulated sum to be added to the previous contents of element (M, N). In other embodiments, the multiplication is performed before the transformation.

[0212] In one embodiment, the architecture machine-specific register (MSR) (e.g., as...) Figure 13 One of the registers in register 1315 (e.g., the MXCSR register storing control and / or status information for the SSE register) (e.g., as part of instruction execution) is read to determine, for example, exception information. DAZ can refer to "nonnormal zero" control (e.g., in the MSR). In some embodiments, 8-bit precision (e.g., BF8) values ​​can be processed to have nonnormal / subnormal values.

[0213] In one embodiment, the architecture machine-specific register (MSR) MXCSR register (e.g., the MXCSR register storing control and / or status information for the SSE register) (e.g., as part of instruction execution) is not read (e.g., not checked and / or not updated). In some embodiments, exception information for an instruction is implicit in the instruction, e.g., for a BF8 operation, it implies DAZ=1 (e.g., no need to consult the MXCSR) (e.g., for the TDPBF8PS instruction), and / or for a non-BF8 operation, it implies DAZ=0 (e.g., no need to consult the MXCSR).

[0214] In operation, M, K, and N can be specified in one or more of several ways: as operands of the TDPBF8PS instruction (e.g., shown here as slice register "t"), as suffixes or prefixes of the specified opcode (as an asterisk is used here as shorthand to refer to those optional suffixes and prefixes), as part of the immediate value provided by the instruction (e.g., K, M, and N are each specified as different (e.g., 8) bits of the (e.g., 32-bit) immediate value), as part of a software-programmed control register (e.g., XTILECONFIG is a register loaded by matrix accelerator configuration instructions (such as TILEFG) or XRSTORE* instructions and stored by matrix save instructions (such as XSAVE*), or even as architecture defaults.

[0215] Instruction 2201 further specifies a destination matrix (e.g., slice) location 2204, a first source matrix (e.g., slice) location 2206, and a second source matrix (e.g., slice) location 2208. Each specified matrix (e.g., slice) location may point to any of the following: a memory location, a set of vector registers, and a set of slice registers.

[0216] Figure 22B This is pseudocode 2220, which describes exemplary helper functions for use with the TDPBF8PS instruction according to some embodiments. As shown, pseudocode 2220 defines the functions convert_bf8_to_fp32(), write_row_and_zero(), zero_upper_rows(), and zero_tileconfig_start(), all of which can be configured by... Figure 22A The TDPBF8PS pseudocode is used.

[0217] Refer to Figure 21. Figures 22A-22B , Figure 23 , Figures 28A-28B and Figures 29A-29B Further illustrations and descriptions of the execution of the TDPBF8PS instruction are provided. (Reference) Figures 24-26DFurther illustrations and descriptions of the example format of the TDPBF8PS instruction are provided.

[0218] Execution of (multiple) exemplary methods

[0219] Figure 23 This is a block flowchart illustrating the processor's response to the TDPBF8PS instruction. As shown in flowchart 2300, at 2301, the processor uses the fetch circuit to fetch an instruction that has fields for specifying the opcode and the positions of the M x N destination matrix with single-precision elements, the positions of the M x K first source matrix, and the positions of the K x N second source matrix. The specified source matrices have elements consisting of quadruples of 8-bit floating-point values.

[0220] When using the processor's physical register file (or, for example, using one or more two-dimensional (2D) (e.g., AMX) slice registers, for example, from Figure 3 In some embodiments, the slice registers formed by the data buffer 305 in the matrix (e.g., slices) are separate from any scalar and / or vector (e.g., one-dimensional array) registers. In embodiments where the matrix (e.g., slice) is a collection of vector registers of the same type—whether a 128-bit xmm register, a 256-bit ymm register, or a 512-bit zmm register—quadruples with 8-bit floating-point formatted values ​​in the source allow for efficient use because the width of the destination element is four times that of the source element. Such efficient use can also be achieved when the matrix is ​​stored in slice registers (e.g., AMX). In other embodiments, a single source vector with 8-bit floating-point elements is converted to 32-bit elements stored in a destination vector with a width one-quarter that of the source vector.

[0221] In some embodiments, the specified opcode instructs the execution circuitry to: for each element (M, N) of the specified destination matrix, convert the elements of K quaternions from row M of the specified first source matrix and the elements of K quaternions from column N of the specified second source matrix to single precision; multiply K pairs of the converted single-precision elements from the first quaternion positions of the two specified source matrices (e.g., slices) to generate K first products; and multiply K pairs of the converted single-precision elements from the second quaternion positions of the two specified source matrices (e.g., slices) to generate K first products. Generate K second products by multiplying the transformed elements of the K pairs of the third quaternion positions from the two specified source matrices (e.g., slices) to generate K third products, multiplying the transformed elements of the K pairs of the fourth quaternion positions from the two specified source matrices (e.g., slices) to generate K fourth products, and then summing the first product with the second products to generate a first accumulated sum, and separately summing the third product with the fourth products to generate a second accumulated sum, and summing the first accumulated sum with the second accumulated sum to generate a final accumulated sum to be added to the previous contents of the element (m,n).

[0222] In some embodiments, the specified opcode is used to instruct the execution circuitry to cause the following operations: for each element of the first source matrix and the corresponding element of the second source matrix, converting an 8-bit floating-point value to a single-precision value; multiplying the converted single-precision values ​​from the first value of the quadruple together to generate a first result; multiplying the converted single-precision values ​​from the second value of the quadruple together to generate a second result; multiplying the converted single-precision values ​​from the third value of the quadruple together to generate a third result; multiplying the converted single-precision values ​​from the fourth value of the quadruple together to generate a fourth result; and accumulating the first, second, third, and fourth results with the previous contents of the corresponding elements of the destination matrix.

[0223] At 2303, the processor uses decoding circuitry to decode the fetched instruction. For example, the fetched TDPBF8PS instruction is decoded by decoding circuitry such as that detailed herein. In the context of the illustrated system, the decoding circuitry may be at least as described above. Figure 13 , Figure 14 and Figures 28A-28B The decoding circuit shown and described.

[0224] At 2305, the execution of the decoded instructions is scheduled (as needed), which is optional (as indicated by its dashed border), meaning it can occur at different times or not at all. At 2307, the processor responds to the decoded instructions using the execution circuitry specified by the opcode.

[0225] In some embodiments, at 2309, a submit or retire instruction is provided, which is optional (as indicated by its dashed border), and is reflected in the fact that it may occur at different times or not at all.

[0226] refer to Figures 3-14 Further illustrations and descriptions of example execution circuitry. In some embodiments, the execution circuitry causes execution of a matrix operation accelerator (e.g., migration to a matrix operation accelerator), such as accelerator 307 shown and described. Figure 3 An accelerator. In some embodiments, the execution circuit is a matrix operation circuit, such as matrix operation circuit 405. Figure 4 ), 505 Figure 5 ) or 1213 ( Figure 12 ) and 1327 ( Figure 13 ).

[0227] (Multiple) Exemplary Instruction Formats

[0228] Figure 24 This is a block diagram illustrating the format of the TDPBF8PS instruction according to some embodiments. As shown, the TDPBF8PS instruction 2400 includes a field for specifying opcode 2402 (TDPBF8PS*), which instructs the processor to: for each element (M, N) of the specified destination matrix, convert the elements of K quadruplets from row M of the specified first source matrix and the elements of K quadruplets from column N of the specified second source matrix to single precision; multiply K pairs of the converted first quaternion values ​​from the first quaternion positions of the specified two source matrices (e.g., slices) to generate K first products; and multiply the second quadruplet values ​​from the second quaternion positions of the specified two source matrices (e.g., slices) to generate K first products. The K pairs of transformed second quaternion values ​​at tuple positions are multiplied to generate K second products. The K pairs of transformed third quaternion values ​​at the third quaternion positions from the two specified source matrices (e.g., slices) are multiplied to generate K third products. The K pairs of transformed values ​​at the fourth quaternion positions from the two specified source matrices (e.g., slices) are multiplied to generate K fourth products. The first and second products are summed to generate a first accumulated sum. The third and fourth products are summed separately to generate a second accumulated sum. The first and second accumulated sums are summed to obtain the final accumulated sum to be added to the previous contents of element (m,n).

[0229] Instruction 2400 further includes a destination matrix (e.g., slice) location 2404, a first source matrix (e.g., slice) location 2406, and a second source matrix (e.g., slice) location 2408. Each of the specified source matrix location and destination matrix location may be in any of a memory location, a set of vector registers, and a set of slice registers (e.g., AMX).

[0230] The TDPBF8PS instruction 2400 further includes several optional parameters for controlling the behavior of the processor, including source element format 2410, k (mask control) and / or z (zeroing control) 2412, M 2414, and N 2416. In some embodiments, M and N are each any one of 4, 8, 16, and 32 (e.g., any number used for M or N, which can be 32, 64, or greater). In some embodiments, M and N are each integers greater than or equal to 4.

[0231] Opcode 2402 is shown as including an asterisk, which is used to convey that additional prefixes and / or suffixes can be added to specify instruction behavior. One or more of instruction modifiers 2410, 2412, 2414, and 2416 can be specified using prefixes or suffixes of opcode 2402, for example, prefixes and / or suffixes indicating that the instruction should be executed with a matrix manipulation accelerator (e.g., including an FMA grid).

[0232] In some embodiments, one or more of the optional instruction modifiers 2410, 2412, 2414, and 2416 are encoded in an immediate numeric field (not shown) optionally included in instruction 2400. In some embodiments, one or more of the optional instruction modifiers 2410, 2412, 2414, and 2416 are specified via a configuration / status register (e.g., XTILECONFIG).

[0233] In some embodiments, the instruction modifier 2410 includes a mask {k} (e.g., a write mask) and / or a zeroing control {z} 2412, for example, where the mask {k} controls which destination elements are to be updated and / or the zeroing control {z} controls whether zeroing (or merging) is applied to the masked destination elements.

[0234] When any one or more of the optional modifiers 2410, 2412, 2414, or 2416 are not specified by the instruction, they can use default values ​​or implicit parameters, such as parameters inherited from other parts of the slice architecture.

[0235] Detailed exemplary systems, processors, and simulations

[0236] This article details examples of hardware, software, etc., used to execute the instructions described above. For example, the following description details various aspects of instruction execution, including various pipeline stages such as fetch, decode, schedule, execute, and retire.

[0237] Instruction set

[0238] An instruction set may include one or more instruction formats. A given instruction format may define various fields (e.g., number of bits, bit positions) to specify the operation to be performed (e.g., opcode) and operand(s) and / or other data fields (e.g., mask) to which the operation will be performed, and so on. Some instruction formats are further decomposed by defining instruction templates (or subformats). For example, an instruction template for a given instruction format may be defined as a different subset of the fields of that instruction format (the included fields are generally in the same order, but at least some fields have different bit positions because fewer fields are included), and / or defined as a given field that is interpreted in a different way. Thus, each instruction of the ISA is expressed using a given instruction format (and, if defined, according to a given instruction template in the instruction template of that instruction format) and includes fields for specifying the operation and operands. For example, the exemplary ADD (addition) instruction has a specific opcode and instruction format, which includes an opcode field for specifying the opcode and an operand field for selecting operands (source 1 / destination and source 2); and the appearance of the ADD instruction in the instruction stream will cause the operand field to contain specific content for selecting the specific operand. Sets of SIMD extensions known as Advanced Vector Extensions (AVX) (AVX1 and AVX2) and utilizing Vector Extensions (VEX) encoding schemes have been introduced and / or released (see, for example, September 2014). See the IA-64 and IA-32 architecture software developer manual; and also see the October 2014 edition. (Advanced Vector Extension Programming Reference).

[0239] Exemplary instruction format

[0240] Examples of the instructions described herein can be embodied in different formats. Furthermore, exemplary systems, architectures, and pipelines are detailed below. Examples of the instructions can be executed on such systems, architectures, and pipelines, but are not limited to those detailed herein.

[0241] General Vector-Friendly Instruction Format

[0242] A vector-friendly instruction format is an instruction format suitable for vector instructions (e.g., the existence of specific fields dedicated to vector operations). Although an embodiment in which both vector and scalar operations are supported via a vector-friendly instruction format has been described, alternative embodiments use only vector operations via the vector-friendly instruction format.

[0243] Figures 25A-25B This is a block diagram illustrating a general vector-friendly instruction format and its instruction template according to an embodiment. Figure 25AThis is a block diagram illustrating a general vector-friendly instruction format and its Class A instruction template according to an embodiment; while Figure 25B This is a block diagram illustrating a general vector-friendly instruction format and its Class B instruction template according to an embodiment. Specifically, Class A and Class B instruction templates are defined for the general vector-friendly instruction format 2500, both of which include instruction templates for no memory access 2505 and instruction templates for memory access 2520. In the context of the vector-friendly instruction format, the term "general" means an instruction format that is not bound to any particular instruction set.

[0244] Although embodiments in which the vector-friendly instruction format supports the following will be described: 64-byte vector operand length (or size) with a 32-bit (4-byte) or 64-bit (8-byte) data element width (or size) (and thus, the 64-byte vector consists of 16 double-word-sized elements, or alternatively 8 quad-word-sized elements); 64-byte vector operand length (or size) with a 16-bit (2-byte) or 8-bit (1-byte) data element width (or size); 32-byte vector operand length (or size) with a 32-bit (4-byte) data element width (or size). ), 64-bit (8-byte), 16-bit (2-byte), or 8-bit (1-byte) data element widths (or dimensions); and 16-byte vector operand lengths (or dimensions) with 32-bit (4-byte), 64-bit (8-byte), 16-bit (2-byte), or 8-bit (1-byte) data element widths (or dimensions); however, alternative embodiments may support larger, smaller, and / or different vector operand sizes (e.g., 256-byte vector operands) with larger, smaller, or different data element widths (e.g., 128-bit (16-byte) data element widths).

[0245] Figure 25A The Class A instruction templates include: 1) within the instruction template for no memory access 2505, an instruction template for a fully round-control operation 2510 without memory access and an instruction template for a data transformation operation 2515 without memory access are shown; and 2) within the instruction template for memory access 2520, an instruction template for the timeliness of memory access 2525 and an instruction template for the non-timeliness of memory access 2530 are shown. Figure 25B The B-type instruction templates include: 1) within the instruction template for no memory access 2505, an instruction template showing a partial rounding control type operation 2512 for write mask control without memory access and an instruction template showing a vsize type operation 2517 for write mask control without memory access; and 2) within the instruction template for memory access 2520, an instruction template showing a write mask control 2527 for memory access.

[0246] The General Vector Friendly Instruction Format 2500 includes the following as listed in Figures 25A-25B The fields in the order shown in the diagram are as follows.

[0247] Format field 2540 – A specific value in this field (instruction format identifier value) uniquely identifies the vector-friendly instruction format, and thus indicates that the instruction appears in the instruction stream in the vector-friendly instruction format. Therefore, this field is not required for instruction sets that only have a general vector-friendly instruction format, and in this sense, it is optional.

[0248] Basic operation field 2542 – its content distinguishes different basic operations.

[0249] Register index field 2544—its contents, either directly or generated from addresses, specify the location of the source or destination operand in a register or in memory. These fields include a sufficient number of bits to select N registers from a PxQ (e.g., 32x512, 16x128, 32x1024, 64x1024) register file. While one embodiment supports up to three source registers and one destination register, alternative embodiments may support more or fewer source and destination registers (e.g., up to two sources, where one of these sources also serves as the destination; up to three sources, where one of these sources also serves as the destination; up to two sources and one destination).

[0250] The modifier field 2546 distinguishes between instructions in the generic vector instruction format that specify memory access and those that do not; that is, it differentiates between instruction templates with no memory access 2505 and instruction templates with memory access 2520. Memory access operations read and / or write to the memory hierarchy (in some cases, using values ​​in registers to specify the source and / or destination addresses), while non-memory access operations do not (e.g., the source and destination are registers). Although in one embodiment, this field also selects between three different ways to perform memory address calculations, alternative embodiments may support more, fewer, or different ways to perform memory address calculations.

[0251] Extended operation field 2550 – its content distinguishes which of the various operations to be performed in addition to the basic operation. This field is context-dependent. In one embodiment, this field is divided into class field 2568, α field 2552, and β field 2554. Extended operation field 2550 allows multiple sets of common operations to be performed in a single instruction instead of two, three, or four instructions.

[0252] The ratio field 2560—its contents allow for use in memory address generation (e.g., for use with (2...) 比例 The content of the index field is scaled proportionally by the address generated by the index + base address.

[0253] Displacement field 2562A—its contents are used as part of memory address generation (e.g., for use with (2 比例 Address generation (index + base address + offset).

[0254] The displacement factor field 2562B (note that the offset field 2562A directly indicates the use of one or the other on the offset factor field 2562B) – its contents are used as part of address generation; it specifies the displacement factor that will scale the size (N) of the memory access – where N is the number of bytes in the memory access (e.g., for use with (2... 比例 *Address generation (index + base address + scaled displacement). Redundant low-order bits are ignored, and therefore the contents of the displacement factor field are multiplied by the total memory operand size (N) to generate the final displacement that will be used in calculating the effective address. The value of N is determined by the processor hardware at runtime based on the full opcode field 2574 (described later herein) and the data manipulation field 2554C. Displacement field 2562A and displacement factor field 2562B are not used in the instruction template of no memory access 2505 and / or different embodiments may implement only one of them or neither of them; in this sense, displacement field 2562A and displacement factor field 2562B are optional.

[0255] The data element width field 2564—its content distinguishes which of the multiple data element widths will be used (in some embodiments for all instructions; in other embodiments for only some instructions). This field is unnecessary if only one data element width is supported and / or if one aspect of the opcode is used to support the data element width; in this sense, the field is optional.

[0256] The write mask field 2570—its contents control, element-by-element position, whether the position of data elements in the destination vector operand reflects the results of the base and augmentation operations. Type A instruction templates support merge-write masks, while Type B instruction templates support both merge-write masks and zero-write masks. When merging, the vector mask allows protecting any set of elements in the destination from updates during the execution of any operation (specified by the base and augmentation operations); in another embodiment, it preserves the old value of each element in the destination where the corresponding mask bit has a 0. Conversely, when zeroing, the vector mask allows zeroing any set of elements in the destination during the execution of any operation (specified by the base and augmentation operations); in one embodiment, the elements in the destination are set to 0 when the corresponding mask bit has a 0 value. A subset of this functionality is the ability to control the vector length of the operation being performed (i.e., the span from the first to the last element being modified), however, the modified elements do not necessarily have to be consecutive. Thus, the write mask field 2570 allows for partial vector operations, including load, store, arithmetic, logical, etc. Although an embodiment is described in which the content of the write mask field 2570 selects one of a plurality of write mask registers containing the write mask to be used (and thus the content of the write mask field 2570 indirectly identifies the mask to be performed), alternative embodiments alternatively or additionally allow the content of the mask write field 2570 to directly specify the mask to be performed.

[0257] The immediate number field 2572—its contents allow for the specification of an immediate number. This field is optional in implementations of generic vector-friendly formats that do not support immediate numbers and in instructions that do not use immediate numbers.

[0258] Class field 2568 – its content distinguishes instructions from different classes. (See reference) Figures 25A-25B The content of this field allows selection between Class A and Class B instructions. Figures 25A-25B In this context, rounded rectangles are used to indicate that a specific value exists in a field (e.g., in...). Figures 25A-25B The two fields are used for class field 2568, namely class A 2568A and class B 2568B.

[0259] Type A Instruction Template

[0260] In the case of the instruction template for Class A non-memory access 2505, the α field 2552 is interpreted as the RS field 2552A, whose content distinguishes which of the different extended operation types to be performed (e.g., for instruction templates specifying rounding 2552A.1 and data transformation 2552A.2 for non-memory access rounding operation 2510 and non-memory access data transformation operation 2515, respectively). The β field 2554 distinguishes which of the specified types of operations to be performed. In the instruction template for non-memory access 2505, the scaling field 2560, the shift field 2562A, and the shift factor field 2562B are absent.

[0261] Instruction template without memory access – Full round control operation

[0262] In the instruction template of a fully round-controlled operation 2510 without memory access, the β field 2554 is interpreted as a round control field 2554A that provides static rounding for its(plural) contents. Although the round control field 2554A in the described embodiment includes a suppress all floating-point exception (SAE) field 2556 and a round operation control field 2558, alternative embodiments may support both concepts, which may be encoded as the same field, or only one or the other of these concepts / fields may be present (e.g., only the round operation control field 2558 may be present).

[0263] SAE field 2556 – its content distinguishes whether exception event reporting is disabled; when the content of SAE field 2556 indicates that suppression is enabled, the given instruction does not report any kind of floating-point exception flag and does not invoke any floating-point exception handler.

[0264] The rounding operation control field 2558 distinguishes which of a set of rounding operations to perform (e.g., round up, round down, round to zero, and round to nearest). Thus, the rounding operation control field 2558 allows the rounding mode to be changed instruction-by-instruction. In one embodiment where the processor includes a control register for specifying the rounding mode, the content of the rounding operation control field 2550 overrides that register value.

[0265] Instruction template without memory access – data transformation operations

[0266] In the instruction template of the data transformation operation 2515 without memory access, the β field 2554 is interpreted as the data transformation field 2554B, the contents of which distinguish which of the multiple data transformations to be performed (e.g., no data transformation, mixing, broadcasting).

[0267] In the case of instruction template 2520 for Class A memory access, α field 2552 is interpreted as eviction hint field 2552B, the content of which distinguishes which eviction hint to use (in... Figure 25A In the instruction template for memory access timing 2525 and memory access non-timing 2530, timing 2552B.1 and non-timing 2552B.2 are specified respectively, while the β field 2554 is interpreted as a data manipulation field 2554C, the contents of which distinguish which of the multiple data manipulation operations (also called primitives) to be performed (e.g., no manipulation, broadcast, source upcast, and destination downcast). The instruction template for memory access 2520 includes a scaling field 2560 and optionally includes a displacement field 2562A or a displacement factor field 2562B.

[0268] Vector memory instructions use translation support to perform vector loads from and vector stores to memory. Like regular vector instructions, vector memory instructions transfer data to / from memory in an element-by-element manner, where the actual elements transferred are specified by the contents of a vector mask selected as the write mask.

[0269] Instruction templates for memory access – time-sensitive

[0270] Time-sensitive data is data that can be reused quickly enough to benefit from cache operations. However, this is a hint, and different processors can implement it in different ways, including ignoring the hint entirely.

[0271] Instruction templates for memory access – non-time-sensitive

[0272] Non-time-sensitive data is data that is unlikely to be reused quickly enough to benefit from cache operations in the L1 cache and should be given eviction priority. However, this is a hint, and different processors can implement it in different ways, including ignoring the hint entirely.

[0273] Type B Instruction Template

[0274] In the case of the B-type instruction template, α field 2552 is interpreted as write mask control (Z) field 2552C, the content of which distinguishes whether the write mask controlled by write mask field 2570 should be merged or zeroed.

[0275] In the case of instruction template 2505 for Class B non-memory access, a portion of the β field 2554 is interpreted as the RL field 2557A, the contents of which distinguish which of the different extended operation types to be performed (for example, the instruction template for the write mask control type operation 2512 for non-memory access and the instruction template for the write mask control type operation 2517 for non-memory access specify rounding 2557A.1 and vector length (VSIZE) 2557A.2, respectively), while the remaining portion of the β field 2554 distinguishes which of the specified types of operations to be performed. In the instruction template for non-memory access 2505, the scale field 2560, the shift field 2562A, and the shift factor field 2562B are absent.

[0276] In the instruction template of the write mask control section round control operation 2510 without memory access, the remainder of the β field 2554 is interpreted as the round operation field 2559A, and exception event reporting is disabled (the given instruction does not report any kind of floating-point exception flag and does not invoke any floating-point exception handler).

[0277] Rounding operation control field 2559A—like rounding operation control field 2558, its contents distinguish which of a set of rounding operations to perform (e.g., round up, round down, round to zero, and round to nearest). Thus, rounding operation control field 2559A allows the rounding mode to be changed instruction-by-instruction. In one embodiment where the processor includes a control register for specifying the rounding mode, the contents of rounding operation control field 2550 override that register value.

[0278] In the instruction template of write mask control VSIZE type operation 2517 without memory access, the remainder of β field 2554 is interpreted as vector length field 2559B, the contents of which distinguish which of the multiple data vector lengths to be executed (e.g., 128 bytes, 256 bytes, or 512 bytes).

[0279] In the case of the instruction template for Class B memory access 2520, a portion of the β field 2554 is interpreted as a broadcast field 2557B, the content of which distinguishes whether a broadcast data manipulation operation is to be performed, while the remainder of the β field 2554 is interpreted as a vector length field 2559B. The instruction template for memory access 2520 includes a scaling field 2560 and optionally includes a displacement field 2562A or a displacement factor field 2562B.

[0280] For the general vector-friendly instruction format 2500, the complete opcode field 2574 is shown to include a format field 2540, a basic operation field 2542, and a data element width field 2564. Although one embodiment is shown in which the complete opcode field 2574 includes all of these fields, in embodiments that do not support all of these fields, the complete opcode field 2574 includes fewer than all of them. The complete opcode field 2574 provides the operation code (opcode).

[0281] The expanded operation field 2550, data element width field 2564, and write mask field 2570 allow these features to be specified on an instruction-by-instruction basis in a generic vector-friendly instruction format.

[0282] Combining the write mask field and the data element width field creates various types of instructions, as these instructions allow the mask to be applied based on different data element widths.

[0283] The various instruction templates appearing within classes A and B are advantageous in different situations. In some embodiments, different processors or different cores within a processor may support only class A, only class B, or both. For example, a high-performance general-purpose out-of-order core designed for general-purpose computing may support only class B, a core designed primarily for graphics and / or scientific (throughput) computing may support only class A, and a core designed for both general-purpose computing and graphics and / or scientific (throughput) computing may support both class A and class B (of course, cores with some mixture of templates and instructions from both classes, but not all templates and instructions from both classes, are within the scope of this disclosure). Similarly, a single processor may include multiple cores, all of which support the same class, or different cores that support different classes. For example, in a processor with separate graphics cores and general-purpose cores, one graphics core designed primarily for graphics and / or scientific computing may support only class A, while one or more general-purpose cores may be high-performance general-purpose cores with out-of-order execution and register renaming designed for general-purpose computing and supporting only class B. Another processor, which does not have a separate graphics core, may include one or more general-purpose ordered or out-of-order cores that support both Class A and Class B. Of course, in different embodiments, features from one class may also be implemented in other classes. This will enable programs written in high-level languages ​​to become (e.g., just-in-time compiled or statically compiled) various different executable forms, including: 1) forms with only instructions from the class(s) supported by the target processor(s) for execution; or 2) forms with alternative routines and control flow code that uses different combinations of instructions from all classes, the control flow code selecting these routines to execute based on instructions supported by the processor currently executing the code.

[0284] Exemplary dedicated vector-friendly instruction format

[0285] Figure 26A This is a block diagram illustrating an exemplary dedicated vector-friendly instruction format according to an embodiment. Figure 26A This illustrates a dedicated vector-friendly instruction format 2600, which specifies the position, size, interpretation, and order of fields, as well as the values ​​of some of those fields. In this sense, the dedicated vector-friendly instruction format 2600 is dedicated. The dedicated vector-friendly instruction format 2600 can be used to extend the x86 instruction set, and thus some of its fields are similar to or identical to those used in existing x86 instruction sets and their extensions (e.g., AVX). This format maintains consistency with the prefix-coded fields, actual opcode byte fields, MOD R / M fields, SIB fields, shift fields, and immediate numeric fields of existing x86 instruction sets with extensions. (Illustration from...) Figures 25A-25B The field comes from Figure 26A Fields mapped to from Figures 25A-25B The field.

[0286] It should be understood that although embodiments are described in the context of the general vector-friendly instruction format 2500 with reference to the dedicated vector-friendly instruction format 2600 for illustrative purposes, this disclosure is not limited to the dedicated vector-friendly instruction format 2600 unless otherwise stated. For example, the general vector-friendly instruction format 2500 envisions various possible sizes for various fields, while the dedicated vector-friendly instruction format 2600 is shown as a field with a specific size. As a specific example, although the data element width field 2564 is illustrated as a one-bit field in the dedicated vector-friendly instruction format 2600, this disclosure is not limited to this (i.e., other sizes for the data element width field 2564 are envisioned in the general vector-friendly instruction format 2500).

[0287] Dedicated Vector Friendly Instruction Format 2600 includes the following according to Figure 26A The fields in the order shown in the diagram are as follows.

[0288] EVEX prefix 2602 (bytes 0-3) — encoded in four-byte format.

[0289] Format field 2540 (EVEX byte 0, bits [7:0]) — The first byte (EVEX byte 0) is format field 2540, and it contains 0x62 (in one embodiment, a unique value used to distinguish vector-friendly instruction formats).

[0290] The second to fourth bytes (EVEX bytes 1-3) include multiple bit fields that provide specialized capabilities.

[0291] The REX field 2605 (EVEX byte 1, bits [7-5]) consists of the EVEX.R bit field (EVEX byte 1, bits [7]–R), the EVEX.X bit field (EVEX byte 1, bits [6]–X), and the EVEX.B bit field (EVEX byte 1, bits [5]–B). The EVEX.R, EVEX.X, and EVEX.B bit fields provide the same functionality as the corresponding VEX bit fields and are encoded in two's complement form, i.e., ZMM0 is encoded as 1111B and ZMM15 is encoded as 0000B. The other fields of these instructions encode the lower three bits (rrr, xxx, and bbb) of the register index as known in the art, thereby forming Rrrr, Xxxx, and Bbbb by adding EVEX.R, EVEX.X, and EVEX.B.

[0292] REX' field 2510 – This is the first part of REX' field 2510 and is the EVEX.R' bit field (EVEX byte 1, bit [4] – R') used to encode the higher 16 or lower 16 registers of the extended 32-register set. In one embodiment, this bit is stored in a bit-inverted format along with other bits indicated below to distinguish it from the BOUND instruction (in the known x86 32-bit mode), which has an actual opcode byte of 62, but does not accept the value 11 in the MOD field in the MOD R / M field (described below); an alternative embodiment does not store the bit of this indication and the bits of the other indications below in an inverted format. The value 1 is used to encode the lower 16 registers. In other words, R'Rrrr is formed by combining EVEX.R', EVEX.R, and other RRRs from other fields.

[0293] Opcode mapping field 2615 (EVEX byte 1, bits [3:0] – mmmm) – its contents encode the implicit preamble opcode byte (0F, 0F 38, or 0F 3).

[0294] The width field of the data element is 2564 (EVEX byte 2, bits [7] – W) – represented by the notation EVEX.W. EVEX.W is used to define the granularity (size) of the data type (32-bit data element or 64-bit data element).

[0295] EVEX.vvvv 2620 (EVEX byte 2, bits [6:3]-vvvv) — The function of EVEX.vvvv can include the following: 1) EVEX.vvvv encodes the first source register operand specified in reverse (two's complement) form, and is valid for instructions with two or more source operands; 2) EVEX.vvvv encodes the destination register operand specified in two's complement form for a specific vector shift; or 3) EVEX.vvvv does not encode any operands; this field is reserved and should contain 1111b. Thus, the EVEX.vvvv field 2620 encodes the four lower-order bits of the first source register specifier stored in reverse (two's complement) form. Depending on the instruction, additional different EVEX bit fields are used to extend the specifier size to 32 registers.

[0296] EVEX.U 2568 Class Field (EVEX byte 2, bit [2]-U) — If EVEX.U = 0, it indicates Class A or EVEX.U0; if EVEX.U = 1, it indicates Class B or EVEX.U1.

[0297] The prefix-encoded field 2625 (EVEX byte 2, bits [1:0]-pp) provides additional bits for the underlying operation field. Besides supporting legacy SSE instructions in EVEX prefix format, this also has the benefit of compressing the SIMD prefix (EVEX prefixes require only 2 bits instead of bytes to represent SIMD prefixes). In one embodiment, to support legacy SSE instructions using SIMD prefixes (66H, F2H, F3H) in both legacy and EVEX prefix formats, these legacy SIMD prefixes are encoded into a SIMD prefix-encoded field; and at runtime, they are expanded to the legacy SIMD prefix before being provided to the PLA for the decoder (thus, the PLA can execute both legacy instructions in legacy format and legacy instructions in EVEX format without modification). While newer instructions can use the contents of the EVEX prefix-encoded field directly as opcode expansion, certain embodiments expand in a similar manner for consistency, but allow for different meanings specified by these legacy SIMD prefixes. Alternative embodiments may redesign the PLA to support 2-bit SIMD prefix encoding, thus eliminating the need for expansion.

[0298] α field 2552 (EVEX byte 3, bit [7] – EH, also known as EVEX.EH, EVEX.rs, EVEX.RL, EVEX.writemask control, and EVEX.N; also illustrated as α) – as previously stated, this field is context-sensitive.

[0299] β field 2554 (EVEX byte 3, bits [6:4] - SSS, also known as EVEX.s 2-0 EVEX.r 2-0 EVEX.rr1, EVEX.LL0, EVEX.LLB (also illustrated in βββ diagram) — as mentioned before, this field is context-sensitive.

[0300] REX' field 2510 — This is the remainder of the REX' field and is the EVEX.V' bit field (EVEX byte 3, bit [3] – V') that can be used to encode either the higher 16 or lower 16 registers of the extended 32-register set. This bit is stored in a bit-inverted format. A value of 1 is used to encode the lower 16 registers. In other words, V'VVVV is formed by combining EVEX.V' and EVEX.vvvv.

[0301] Write mask field 2570 (EVEX byte 3, bits [2:0]-kkk) – its contents specify the index of the register in the write mask register, as previously described. In one embodiment, the specific value EVEX.kkk = 000 has the special behavior of implying that no write mask is used for a particular instruction (this can be implemented in various ways, including using a write mask hardwired to all objects or hardware that bypasses the mask hardware).

[0302] The actual opcode field 2630 (byte 4) is also known as the opcode byte. Part of the opcode is specified in this field.

[0303] The MOD R / M field 2640 (byte 5) includes the MOD field 2642, the Reg field 2644, and the R / M field 2646. As previously described, the content of the MOD field 2642 distinguishes between memory access operations and non-memory access operations. The Reg field 2644 can be categorized into two cases: encoding the destination register operand or the source register operand; or it can be treated as an opcode extension and not used to encode any instruction operand. The R / M field 2646 can be categorized as encoding instruction operands that reference memory addresses; or encoding the destination register operand or the source register operand.

[0304] The Proportional, Index, Base (SIB) byte (byte 6) – as previously mentioned, the contents of SIB 2650 are used for memory address generation. SIB.xxx 2654 and SIB.bbb 2656 – the contents of these fields have previously been mentioned for register indices Xxxx and Bbbb.

[0305] Displacement field 2562A (bytes 7-10) — When MOD field 2642 contains 10, bytes 7-10 are displacement field 2562A, and it works the same as a traditional 32-bit displacement (disp32), and works at the byte granularity.

[0306] Byte 7 (disp8) is the shift factor segment 2562B – when MOD field 2642 contains 0s and 1s, byte 7 is the shift factor segment 2562B. This field is located in the same position as the traditional x86 instruction set 8-bit shift (disp8) that operates at byte granularity. Because disp8 is sign-extended, it can only be addressed between -128 and +127 byte offsets; in the case of a 64-byte cache line, disp8 uses 8 bits that can be set to only four truly useful values: -128, -64, 0, and 64; since a larger range is often required, disp32 is used; however, disp32 requires 4 bytes. In contrast to disp8 and disp32, the shift factor segment 2562B is a reinterpretation of disp8; when using the shift factor segment 2562B, the actual shift is determined by multiplying the contents of the shift factor segment by the size (N) of the memory operand access. This type of shift is called disp8*N. This reduces the average instruction length (a single byte is used for the offset, but with a much larger range). This type of compressed offset assumes the effective offset is a multiple of the granularity of the memory access, and therefore redundant low-order bits of the address offset do not need to be encoded. In other words, the offset is replaced by digital segment 2562B instead of the traditional x86 instruction set 8-bit offset. Thus, the offset is encoded in the same way as the x86 instruction set 8-bit offset (therefore, no change in the ModRM / SIB encoding rules), the only difference being that disp8 is overloaded to disp8*N. In other words, there is no change in the encoding rules or encoding length, only in the hardware interpretation of the offset values ​​(which requires scaling the offset to the size of the memory operand to obtain a byte-level address offset). Immediate segment 2572 operates as previously described.

[0307] Full opcode field

[0308] Figure 26B This is a block diagram illustrating the fields constituting a complete opcode field 2574 with a dedicated vector-friendly instruction format 2600 according to one embodiment. Specifically, the complete opcode field 2574 includes a format field 2540, a basic operation field 2542, and a data element width (W) field 2564. The basic operation field 2542 includes a prefix encoding field 2625, an opcode mapping field 2615, and a real opcode field 2630.

[0309] Register index field

[0310] Figure 26C This is a block diagram illustrating the fields constituting register index field 2544 with a dedicated vector-friendly instruction format 2600 according to one embodiment. Specifically, register index field 2544 includes REX 2605 field, REX' 2610 field, MODR / M.reg field 2644, MODR / Mr / m field 2646, VVVV field 2620, xxx field 2654, and bbb field 2656.

[0311] Expand operation fields

[0312] Figure 26D This is a block diagram illustrating the fields of an extended operation field 2550 with a dedicated vector-friendly instruction format 2600 according to one embodiment. When the class (U) field 2568 contains 0, it indicates EVEX.U0 (Class A 2568A); when it contains 1, it indicates EVEX.U1 (Class B 2568B). When U = 0 and the MOD field 2642 contains 11 (indicating no memory access operation), the α field 2552 (EVEX byte 3, bits [7] – EH) is interpreted as the rs field 2552A. When the rs field 2552A contains 1 (rounded 2552A.1), the β field 2554 (EVEX byte 3, bits [6:4] – SSS) is interpreted as the rounding control field 2554A. The rounding control field 2554A includes a one-bit SAE field 2556 and a two-bit rounding operation field 2558. When rs field 2552A contains 0 (data transformation 2552A.2), β field 2554 (EVEX byte 3, bits [6:4] – SSS) is interpreted as a three-bit data transformation field 2554B. When U = 0 and MOD field 2642 contains 00, 01, or 10 (indicating a memory access operation), α field 2552 (EVEX byte 3, bits [7] – EH) is interpreted as an eviction hint (EH) field 2552B, and β field 2554 (EVEX byte 3, bits [6:4] – SSS) is interpreted as a three-bit data manipulation field 2554C.

[0313] When U = 1, α field 2552 (EVEX byte 3, bit [7] – EH) is interpreted as write mask control (Z) field 2552C. When U = 1 and MOD field 2642 contains 11 (indicating no memory access operation), a portion of β field 2554 (EVEX byte 3, bit [4] – S0) is interpreted as RL field 2557A; when it contains 1 (rounded to 2557A.1), the remainder of β field 2554 (EVEX byte 3, bit [6-5] – S0) is interpreted as RL field 2557A. 2-1The remainder of the β field 2554 (EVEX byte 3, bit [6-5]-S) is interpreted as a rounding operation field 2559A, while when the RL field 2557A contains 0 (VSIZE2557A.2), the remainder of the β field 2554 (EVEX byte 3, bit [6-5]-S) is interpreted as a rounding operation field 2559A, and when the RL field 2557A contains 0 (VSIZE2557A.2), the remainder of the β field 2554 is interpreted as a rounding operation field 2559A, while when the RL field 2557A contains 0 (VSIZE2557A.2), the remainder of the β field 2554 (EVEX byte 3, bit [6-5]-S) is interpreted as a rounding operation field 2559A, and when the RL field 2 2-1 ) is interpreted as the vector length field 2559B (EVEX byte 3, bits [6-5] – L 1-0 When U=1 and MOD field 2642 contains 00, 01, or 10 (indicating a memory access operation), β field 2554 (EVEX byte 3, bits [6:4]–SSS) is interpreted as vector length field 2559B (EVEX byte 3, bits [6-5]–L). 1-0 ) and broadcast field 2557B (EVEX byte 3, bit [4]–B).

[0314] Exemplary Register Architecture

[0315] Figure 27 This is a block diagram of a register architecture 2700 according to one embodiment. In the illustrated embodiment, there are 32 512-bit wide vector registers 2710; these registers are referenced as zmm0 to zmm31. The lower-order 256 bits of the lower 16 zmm registers are overlaid on registers ymm0-16. The lower-order 128 bits of the lower 16 zmm registers (the lower-order 128 bits of the ymm registers) are overlaid on registers xmm0-15. The dedicated vector-friendly instruction format 2600 operates on these overlaid registers as illustrated in the following table.

[0316]

[0317] In other words, the vector length field 2559B selects between a maximum length and one or more other shorter lengths, each of which is half the length of the previous one, and instruction templates without the vector length field 2559B operate on the maximum vector length. Furthermore, in one embodiment, the Class B instruction template of the dedicated vector-friendly instruction format 2600 operates on compressed or scalar single / double-precision floating-point data and compressed or scalar integer data. Scalar operations are performed on the lowest-order data element positions in the zmm / ymm / xmm registers; depending on the embodiment, higher-order data element positions either remain the same as before the instruction or are zeroed out.

[0318] Write mask register 2715—In the illustrated embodiment, there are eight write mask registers (k0 to k7), each 64 bits in size. In an alternative embodiment, write mask register 2715 is 16 bits in size. As previously mentioned, in one embodiment, vector mask register k0 cannot be used as a write mask; when the encoding of the normal indicator k0 is used as the write mask, it selects the hardwired write mask 0xFFFF, effectively disabling the write mask for that instruction.

[0319] General Purpose Register 2725 — In the illustrated embodiment, there are sixteen 64-bit general purpose registers that are used in conjunction with existing x86 addressing modes for addressing memory operands. These registers are referenced by the names RAX, RBX, RCX, RDX, RBP, RSI, RDI, RSP, and R8 through R15.

[0320] A scalar floating-point stack register file (x87 stack) 2745 is overlaid with an MMX compact integer flat register file 2750. In the illustrated embodiment, the x87 stack is an octal stack used to perform scalar floating-point operations on 32 / 64 / 80-bit floating-point data using x87 instruction set extensions; while the MMX registers are used to perform operations on 64-bit compact integer data and to store operands for some operations performed between the MMX and XMM registers.

[0321] Alternative embodiments may use wider or narrower registers. Additionally, alternative embodiments may use more, fewer, or different register files and registers.

[0322] Exemplary core architectures, processors, and computer architectures

[0323] Processor cores can be implemented in different ways, for different purposes, and in different processors. For example, implementations of such cores may include: 1) general-purpose ordered cores intended for general-purpose computing; 2) high-performance general-purpose out-of-order cores intended for general-purpose computing; and 3) dedicated cores intended primarily for graphics and / or scientific (throughput) computing. Implementations of different processors may include: 1) CPUs, which include one or more general-purpose ordered cores and / or one or more general-purpose out-of-order cores intended for general-purpose computing; and 2) coprocessors, which include one or more dedicated cores intended primarily for graphics and / or scientific (throughput) computing. These different processors result in different computer system architectures, which may include: 1) a coprocessor on a separate chip from the CPU; 2) a coprocessor in the same package as the CPU but on a separate die; 3) a coprocessor on the same die as the CPU (in which case such a coprocessor is sometimes referred to as dedicated logic or a dedicated core, such as integrated graphics and / or scientific (throughput) logic); and 4) a system-on-a-chip that may include the described CPU (sometimes referred to as application cores or application processors), the coprocessors described above, and additional functionality on the same die. Exemplary core architectures are then described, followed by exemplary processors and computer architectures.

[0324] Exemplary core architecture

[0325] Ordered and disordered kernel diagrams

[0326] Figure 28A This is a block diagram illustrating an exemplary ordered pipeline and an exemplary out-of-order release / execution pipeline with register renaming according to various embodiments. Figure 28B This is a block diagram illustrating exemplary embodiments of ordered architecture cores to be included in a processor according to various embodiments and exemplary out-of-order release / execution architecture cores with register renaming. Figures 28A-28B The solid-line boxes in the diagram illustrate ordered pipelines and ordered cores, while the optional additions to the dashed boxes illustrate register renaming, out-of-order release / execution pipelines, and cores. Since the ordered aspect is a subset of the out-of-order aspect, the out-of-order aspect will be described.

[0327] exist Figure 28A In the processor pipeline 2800, there are fetch stage 2802, length decoding stage 2804, decoding stage 2806, allocation stage 2808, renaming stage 2810, scheduling (also known as dispatch or issue) stage 2812, register read / memory read stage 2814, execution stage 2816, write-back / memory write stage 2818, exception handling stage 2822, and commit stage 2824.

[0328] Figure 28BA processor core 2890 is shown, which includes a front-side unit 2830 coupled to an execution engine unit 2850, and both the front-side unit 2830 and the execution engine unit 2850 are coupled to a memory unit 2870. Core 2890 can be a Reduced Instruction Set Computing (RISC) core, a Complex Instruction Set Computing (CISC) core, a Very Long Instruction Word (VLIW) core, or a hybrid or alternative core type. Alternatively, core 2890 can be a dedicated core, such as, for example, a network or communication core, a compression engine, a coprocessor core, a general-purpose computing graphics processing unit (GPGPU) core, a graphics core, etc.

[0329] Front-end unit 2830 includes branch prediction unit 2832 coupled to instruction cache unit 2834 coupled to instruction translation lookup buffer (TLB) 2836 coupled to instruction fetch unit 2838 coupled to decoding unit 2840. Decoding unit 2840 (or decoder) decodes instructions and generates one or more micro-operations, microcode entry points, microinstructions, other instructions, or other control signals as output, which are decoded from, or otherwise reflect, the original instructions or derived from the original instructions. Decoding unit 2840 can be implemented using various different mechanisms. Examples of suitable mechanisms include, but are not limited to, lookup tables, hardware implementations, programmable logic arrays (PLAs), microcode read-only memory (ROM), etc. In one embodiment, core 2890 includes microcode ROM or other media (e.g., in decoding unit 2840, or otherwise within front-end unit 2830) storing microcode for certain macro instructions. The decoding unit 2840 is coupled to the rename / allocator unit 2852 in the execution engine unit 2850.

[0330] Execution engine unit 2850 includes a renaming / allocator unit 2852 coupled to a retirement unit 2854 and a set 2856 of one or more scheduler units. The scheduler units 2856 represent any number of different schedulers, including reservation stations, central instruction windows, etc. The scheduler units 2856 are coupled to physical register file units 2858. Each physical register file unit in the physical register file units 2858 represents one or more physical register files, where different physical register files store one or more different data types, such as scalar integers, scalar floating-point numbers, compressed integers, compressed floating-point numbers, vector integers, vector floating-point numbers, status (e.g., an instruction pointer as the address of the next instruction to be executed), etc. In one embodiment, the physical register file units 2858 include vector register units, write mask register units, and scalar register units. These register units can provide architectural vector registers, vector mask registers, and general-purpose registers. Multiple physical register file units 2858 are overlapped by retirement units 2854 to illustrate various ways in which register renaming and out-of-order execution can be implemented (e.g., using multiple reorder buffers and multiple retirement register files; using multiple future files, multiple history buffers, and multiple retirement register files; using register maps and register pools, etc.). Retirement units 2854 and multiple physical register file units 2858 are coupled to multiple execution clusters 2860. Multiple execution clusters 2860 include a set 2862 of one or more execution units and a set 2864 of one or more memory access units. Execution units 2862 can perform various operations (e.g., shift, addition, subtraction, multiplication) and can perform operations on various data types (e.g., scalar floating-point, compressed integer, compressed floating-point, vector integer, vector floating-point). While some embodiments may include multiple execution units dedicated to a particular function or set of functions, other embodiments may include only one execution unit or multiple execution units that all perform all functions. Multiple scheduler units 2856, multiple physical register file units 2858, and multiple execution clusters 2860 are shown as possibly multiple, because some embodiments create separate pipelines for certain types of data / operations (e.g., scalar integer pipelines, scalar floating-point / compact integer / compact floating-point / vector integer / vector floating-point pipelines, and / or memory access pipelines each having their own scheduler unit, multiple physical register file units, and / or execution clusters—and in the case of separate memory access pipelines, some embodiments are implemented where only the execution cluster of that pipeline has multiple memory access units 2864). It should also be understood that, in the case of using separate pipelines, one or more of these pipelines may be out-of-order deployment / execution, and the remaining pipelines may be ordered.

[0331] A set of memory access units 2864 is coupled to a memory unit 2870, which includes a data TLB unit 2872, which is coupled to a data cache unit 2874, which is coupled to a Level 2 (L2) cache unit 2876. In one exemplary embodiment, the memory access unit 2864 may include a load unit, a memory address unit, and a memory data unit, each of which is coupled to the data TLB unit 2872 in the memory unit 2870. An instruction cache unit 2834 is also coupled to the Level 2 (L2) cache unit 2876 in the memory unit 2870. The L2 cache unit 2876 is coupled to one or more other levels of cache and ultimately to main memory.

[0332] As an example, the exemplary out-of-order release / execution core architecture for register renaming can implement pipeline 2800 as follows: 1) Instruction fetch 2838 executes fetch stage 2802 and length decoding stage 2804; 2) Decoding unit 2840 executes decoding stage 2806; 3) Rename / allocator unit 2852 executes allocation stage 2808 and rename stage 2810; 4) (multiple) scheduler units 2856 execute scheduling stage 2812; 5) (multiple) physical register file units 2858 and memory unit 2870 execute register read / memory read stage 2814; execution cluster 2860 executes execution stage 2816; 6) memory unit 2870 and (multiple) physical register file units 2858 execute write-back / memory write stage 2818; 7) each unit may involve exception handling stage 2822; and 8) retirement unit 2854 and (multiple) physical register file units 2858 execute commit stage 2824.

[0333] Core 2890 may support one or more instruction sets (e.g., the x86 instruction set (with some extensions added with newer versions); the MIPS instruction set of MIPS Technologies, Inc., Sunnyvale, California; the ARM instruction set of ARM Holdings, Inc., Sunnyvale, California (with optional additional extensions such as NEON)), including the instructions described herein. In one embodiment, Core 2890 includes logic for supporting compressed data instruction set extensions (e.g., AVX1, AVX2), thereby allowing the use of compressed data to perform operations used by many multimedia applications.

[0334] It should be understood that a core can support multithreading (execution of two or more parallel operations or a set of threads), and this multithreading can be accomplished in various ways, including time-division multithreading, simultaneous multithreading (where a single physical core provides a logical core for each thread in the simultaneous multithreading of the physical core), or combinations thereof (e.g., time-division fetching and decoding, and subsequent operations such as...). Simultaneous multithreading in hyper-threading technology.

[0335] Although register renaming is described in the context of out-of-order execution, it should be understood that register renaming can also be used in ordered architectures. While the illustrated processor embodiment also includes separate instruction and data cache units 2834 / 2874 and a shared L2 cache unit 2876, alternative embodiments may have a single internal cache for both instructions and data, such as, for example, a Level 1 (L1) internal cache or multiple levels of internal caches. In some embodiments, the system may include a combination of internal caches and external caches located outside the core and / or processor. Alternatively, all caches may be located outside the core and / or processor.

[0336] Specific exemplary ordered core architecture

[0337] Figures 29A-29B The diagram illustrates a more specific exemplary ordered core architecture, where the core is one of several logic blocks within a chip (including other cores of the same and / or different types). Depending on the application, the logic block communicates with some fixed functional logic, memory I / O interfaces, and other necessary I / O logic via a high-bandwidth interconnect network (e.g., a ring network).

[0338] Figure 29A This is a block diagram of a single processor core according to an embodiment, its connection to the on-die interconnect network 2902, and a local subset 2904 of its second-level (L2) cache. In one embodiment, the instruction decoder 2900 supports the x86 instruction set with a compact data instruction set extension. The L1 cache 2906 allows low-latency access to cache memory into the scalar and vector units. Although in one embodiment (for design simplification), the scalar unit 2908 and the vector unit 2910 use separate sets of registers (scalar register 2912 and vector register 2914, respectively), and data transferred between these registers is written to memory and subsequently read back from the first-level (L1) cache 2906, alternative embodiments may use different methods (e.g., using a single set of registers or including a communication path that allows data to be transferred between these two register sets without being written and read back).

[0339] The local subset 2904 of the L2 cache is part of the global L2 cache, which is divided into multiple separate local subsets, one for each processor core. Each processor core has a direct access path to its own local subset 2904 of the L2 cache. Data read by a processor core is stored in its L2 cache subset 2904 and can be accessed quickly in parallel with other processor cores accessing their own local L2 cache subsets. Data written by a processor core is stored in its own L2 cache subset 2904 and is dumped from other subsets if necessary. A ring network ensures the consistency of shared data. The ring network is bidirectional to allow agents such as processor cores, L2 caches, and other logic blocks to communicate with each other within the chip. Each ring data path is 1012 bits wide in each direction.

[0340] Figure 29B According to the embodiments Figure 29A An expanded diagram of a portion of the processor core. Figure 29B This includes the L1 data cache 2906A portion of L1 cache 2904, and further details regarding vector unit 2910 and vector register 2914. Specifically, vector unit 2910 is a 16-wide vector processing unit (VPU) (see 16-wide ALU 2928) that executes one or more of integer, single-precision floating-point, and double-precision floating-point instructions. This VPU supports mixing of register inputs via mixing unit 2920, numerical conversion via value conversion units 2922A-B, and copying of memory inputs via copy unit 2924. Write mask register 2926 allows masked vector writes.

[0341] Figure 30 This is a block diagram of a processor 3000 according to an embodiment, which may have more than one core, an integrated memory controller, and an integrated graphics device. Figure 30 The solid-line box illustration shows a processor 3000 having a single core 3002A, a system agent 3010, and a set of one or more bus controller units 3016, while the optional additional illustration with dashed boxes shows an alternative processor 3000 having multiple cores 3002A-N, a set of one or more integrated memory controller units 3014 from the system agent units 3010, and dedicated logic 3008.

[0342] Therefore, different implementations of processor 3000 may include: 1) a CPU, wherein dedicated logic 3008 is integrated graphics and / or scientific (throughput) logic (which may include one or more cores), and cores 3002A-N are one or more general-purpose cores (e.g., general-purpose ordered cores, general-purpose out-of-order cores, or a combination of both); 2) a coprocessor, wherein cores 3002A-N are a large number of dedicated cores designed primarily for graphics and / or scientific (throughput); and 3) a coprocessor, wherein cores 3002A-N are a large number of general-purpose ordered cores. Thus, processor 3000 may be a general-purpose processor, coprocessor, or dedicated processor, such as, for example, a network or communication processor, a compression engine, a graphics processor, a GPGPU (General-Purpose Graphics Processing Unit), a high-throughput integrated many-core (MIC) coprocessor (including 30 or more cores), an embedded processor, etc. The processor may be implemented on one or more chips. Processor 3000 may be part of one or more substrates, and / or may be implemented on one or more substrates using any of a variety of process technologies (e.g., BiCMOS, CMOS, or NMOS).

[0343] The memory hierarchy includes one or more levels of cache within the core, a set 3006 of one or more shared cache units, and external memory (not shown) coupled to a set 3014 of integrated memory controller units. The set 3006 of shared cache units may include one or more intermediate-level caches, such as Level 2 (L2), Level 3 (L3), Level 4 (L4), or other levels of cache, Last Level Cache (LLC), and / or combinations thereof. While in one embodiment, ring-based interconnect units 3012 interconnect dedicated logic 3008 (integrated graphics logic is an example of dedicated logic and is also referred to herein as dedicated logic), the set 3006 of shared cache units, and system proxy units 3010 / (multiple) of integrated memory controller units 3014, alternative embodiments may use any number of known techniques to interconnect such units. In one embodiment, consistency is maintained between one or more cache units 3006 and cores 3002A-N.

[0344] In some embodiments, one or more cores 3002A-N can be multithreaded. System agent 3010 includes those components that coordinate and operate core 3002A-N. System agent unit 3010 may include, for example, a power control unit (PCU) and a display unit. The PCU may be, or may include, the logic and components required to regulate the power state of core 3002A-N and dedicated logic 3008. The display unit is used to drive one or more externally connected displays.

[0345] The 3002A-N cores can be homogeneous or heterogeneous in terms of their instruction set architecture; that is, two or more cores in the 3002A-N cores may be able to execute the same instruction set, while other cores may be able to execute only a subset of that instruction set or a different instruction set.

[0346] Exemplary computer architecture

[0347] Figures 31-34 This is a block diagram of an exemplary computer architecture. Other system designs and configurations known in the art for laptops, desktops, handheld PCs, personal digital assistants, engineering workstations, servers, network devices, network hubs, switches, embedded processors, digital signal processors (DSPs), graphics devices, video game devices, set-top boxes, microcontrollers, cellular phones, portable media players, handheld devices, and various other electronic devices are also suitable. Generally, a wide variety of systems or electronic devices capable of incorporating processors and / or other execution logic as disclosed herein are generally suitable.

[0348] Now for reference Figure 31 The diagram illustrates a block diagram of a system 3100 according to one embodiment of the present disclosure. System 3100 may include one or more processors 3110, 3115 coupled to a controller hub 3120. In one embodiment, the controller hub 3120 includes a graphics memory controller hub (GMCH) 3190 and an input / output hub (IOH) 3150 (which may be on separate chips); the GMCH 3190 includes memory and a graphics controller, to which a memory 3140 and a coprocessor 3145 are coupled; the IOH 3150 couples an input / output (I / O) device 3160 to the GMCH 3190. Alternatively, one or both of the memory and graphics controller may be integrated within a processor (as described herein), with the memory 3140 and coprocessor 3145 directly coupled to the processor 3110, and the controller hub 3120 and IOH 3150 residing on a single chip. The memory 3140 may include matrix acceleration code 3140A, for example, code that, when executed, causes the processor to perform any method of this disclosure.

[0349] The optionality of the additional processor 3115 is in Figure 31 The numbers are indicated by dashed lines. Each processor 3110, 3115 may include one or more of the processing cores described herein, and may be a version of processor 3000.

[0350] The memory 3140 may be, for example, dynamic random access memory (DRAM), phase-change memory (PCM), or a combination of both. In at least one embodiment, the controller hub 3120 communicates with the processors(s) 3110, 3115 via a multi-branch bus such as a front-side bus (FSB), a point-to-point interface such as a fast path interconnect (QPI), or a similar connection 3195.

[0351] In one embodiment, the coprocessor 3145 is a dedicated processor, such as, for example, a high-throughput MIC processor, a network or communication processor, a compression engine, a graphics processor, a GPGPU, an embedded processor, and so on. In one embodiment, the controller hub 3120 may include an integrated graphics accelerator.

[0352] There can be various differences between physical resources 3110 and 3115 in terms of a range of quality metrics, including architecture, microarchitecture, thermal and power consumption characteristics.

[0353] In one embodiment, processor 3110 executes instructions that control general-type data processing operations. Embedded within these instructions may be coprocessor instructions. Processor 3110 recognizes these coprocessor instructions as having a type that should be executed by an attached coprocessor 3145. Therefore, processor 3110 issues these coprocessor instructions (or control signals representing coprocessor instructions) to coprocessor 3145 on a coprocessor bus or other interconnect. Coprocessors 3145 (multiple) receive and execute the received coprocessor instructions.

[0354] See now Figure 32 The diagram shown is a block diagram of a first, more specific, exemplary system 3200 according to an embodiment of the present disclosure. Figure 32 As shown, the multiprocessor system 3200 is a point-to-point interconnect system and includes a first processor 3270 and a second processor 3280 coupled via a point-to-point interconnect 3250. Each of processors 3270 and 3280 may be a version of processor 3000. In one embodiment, processors 3270 and 3280 are processors 3110 and 3115, respectively, and coprocessor 3238 is coprocessor 3145. In another embodiment, processors 3270 and 3280 are processor 3110 and coprocessor 3145, respectively.

[0355] Processors 3270 and 3280 are shown as including integrated memory controller (IMC) units 3272 and 3282, respectively. Processor 3270 also includes point-to-point (PP) interfaces 3276 and 3278 as part of its bus controller unit; similarly, the second processor 3280 includes PP interfaces 3286 and 3288. Processors 3270 and 3280 can exchange information via PP interface 3250 using point-to-point (PP) interface circuits 3278 and 3288. Figure 32 As shown, IMC 3272 and 3282 couple the processor to the corresponding memory, namely memory 3232 and memory 3234, which may be portions of the main memory locally attached to the corresponding processor.

[0356] Processors 3270 and 3280 can each exchange information with chipset 3290 via their respective PP interfaces 3252 and 3254 using point-to-point interface circuits 3276, 3294, 3286, and 3298. Chipset 3290 can optionally exchange information with coprocessor 3238 via high-performance interface 3292. In one embodiment, coprocessor 3238 is a dedicated processor, such as, for example, a high-throughput MIC processor, a network or communication processor, a compression engine, a graphics processor, a GPGPU, an embedded processor, etc.

[0357] A shared cache (not shown) may be included in either processor or external to both processors but connected to them via a PP interconnect, such that if the processors are in a low-power mode, the local cache information of either or both processors may be stored in the shared cache.

[0358] Chipset 3290 can be coupled to first bus 3216 via interface 3296. In one embodiment, first bus 3216 may be a peripheral component interconnect (PCI) bus or a bus such as PCI Fast Bus or another third-generation I / O interconnect bus, but the scope of this disclosure is not limited thereto.

[0359] like Figure 32As shown, various I / O devices 3214 may be coupled to a first bus 3216 together with a bus bridge 3218, which couples the first bus 3216 to a second bus 3220. In one embodiment, one or more additional processors 3215, such as a coprocessor, a high-throughput MIC processor, a GPGPU, an accelerator (such as, for example, a graphics accelerator or digital signal processing (DSP) unit), a field-programmable gate array, or any other processor, are coupled to the first bus 3216. In one embodiment, the second bus 3220 may be a low pin count (LPC) bus. In one embodiment, various devices may be coupled to the second bus 3220, including, for example, a keyboard and / or mouse 3222, a communication device 3227, and a storage unit 3228, such as a disk drive that may include instruction / code and data 3230, or other mass storage devices. Additionally, audio I / O 3224 may be coupled to the second bus 3220. Note that other architectures are possible. For example, instead of… Figure 32 The point-to-point architecture allows the system to implement multi-branch buses or other similar architectures.

[0360] Now for reference Figure 33 The diagram shown is a block diagram of a second, more specific, exemplary system 3300 according to an embodiment of the present disclosure. Figure 32 and 33 Similar elements in the figure use similar reference numerals, and from Figure 33 The middle part is omitted Figure 32 certain aspects to avoid confusion Figure 33 Other aspects.

[0361] Figure 33 The illustrated processors 3270 and 3280 may include integrated memory and I / O control logic (“CL”) 3272 and 3282, respectively. Therefore, CL 3272 and 3282 include an integrated memory controller unit and I / O control logic. Figure 33 The diagram shows that not only are the memories 3232 and 3234 coupled to CLs 3272 and 3282, but the I / O device 3314 is also coupled to the control logic 3272 and 3282. The conventional I / O device 3315 is coupled to the chipset 3290.

[0362] Now for reference Figure 34 The diagram shown is a block diagram of a SoC 3400 according to an embodiment of the present disclosure. Figure 30 Similar elements in the figure use similar reference numerals. Additionally, dashed boxes are optional features on more advanced SoCs. Figure 34In this embodiment, multiple interconnect units 3402 are coupled to: an application processor 3410, which includes a set of one or more cores 3002A-N and multiple shared cache units 3006, the set of one or more cores 3002A-N including cache units 3004A-N; a system proxy unit 3010; multiple bus controller units 3016; multiple integrated memory controller units 3014; a set of one or more coprocessors 3420, which may include integrated graphics logic, an image processor, an audio processor, and a video processor; a static random access memory (SRAM) unit 3430; a direct memory access (DMA) unit 3432; and a display unit 3440 for coupling to one or more external displays. In one embodiment, the multiple coprocessors 3420 include dedicated processors, such as, for example, network or communication processors, compression engines, GPGPUs, high-throughput MIC processors, or embedded processors, etc.

[0363] The various embodiments of the mechanisms disclosed herein can be implemented in hardware, software, firmware, or a combination of such implementations. Embodiments can be implemented as computer programs or program code executable on a programmable system including at least one processor, a storage system (including volatile and non-volatile memory and / or storage elements), at least one input device, and at least one output device.

[0364] Program code (such as, Figure 32 The code 3230 shown in the diagram is applied to input instructions to perform the functions described herein and generate output information. The output information can be applied to one or more output devices in a known manner. For the purposes of this application, the processing system includes any system having a processor, such as, for example, a digital signal processor (DSP), a microcontroller, an application-specific integrated circuit (ASIC), or a microprocessor.

[0365] The program code can be implemented using a high-level procedural or object-oriented programming language to communicate with the processing system. Assembly or machine language can also be used if needed. In fact, the mechanisms described herein are not limited to any particular programming language. In any case, the language can be a compiled or interpreted language.

[0366] One or more aspects of at least one embodiment can be implemented by representational instructions stored on a machine-readable medium, which represent various logics in a processor, and which, when read by a machine, cause the machine to manufacture logic for performing the techniques described herein. Such representations, referred to as “IP cores,” can be stored on tangible machine-readable media and can be supplied to various customers or production facilities for loading into manufacturing machines that actually manufacture the logic or processor.

[0367] Such machine-readable storage media can include, but are not limited to, non-transitory, tangible arrangements of articles made or formed by a machine or device, including storage media such as hard disks; any other type of disk, including floppy disks, optical disks, compact disc read-only memory (CD-ROM), rewritable compact discs (CD-RW), and magneto-optical disks; semiconductor devices such as read-only memory (ROM), random access memory (RAM) such as dynamic random access memory (DRAM) and static random access memory (SRAM), erasable programmable read-only memory (EPROM), flash memory, electrically erasable programmable read-only memory (EEPROM); phase-change memory (PCM); magnetic cards or optical cards; or any other type of medium suitable for storing electronic instructions.

[0368] Therefore, embodiments also include non-transitory tangible machine-readable media containing instructions or design data, such as a hardware description language (HDL), that defines the architectures, circuits, devices, processors, and / or system characteristics described herein. These embodiments are also referred to as program products.

[0369] Simulation (including binary transformation, code transformation, etc.)

[0370] In some cases, instruction translators can be used to translate instructions from a source instruction set to a target instruction set. For example, an instruction translator can transform instructions (e.g., using static binary transformation, including dynamically compiled binary transformation), modify, emulate, or otherwise convert them into one or more other instructions to be processed by the kernel. Instruction translators can be implemented in software, hardware, firmware, or a combination thereof. Instruction translators can be on the processor, off the processor, or partially on and partially off the processor.

[0371] Figure 35 This is a block diagram illustrating the use of a software instruction converter to convert binary instructions in a source instruction set into binary instructions in a target instruction set, according to an embodiment. In the illustrated embodiment, the instruction converter is a software instruction converter; however, alternatively, the instruction converter can be implemented using software, firmware, hardware, or various combinations thereof. Figure 35 This demonstrates that a program in the form of a high-level language 3502 can be compiled using an x86 compiler 3504 to generate x86 binary code 3506 that can be natively executed by a processor 3516 having at least one x86 instruction set core. The processor 3516 having at least one x86 instruction set core indicates that it can execute programs with at least one x86 instruction set core by compatibly executing or otherwise processing the following... Any processor with essentially the same functionality as the processor: 1) The substantial portion of the instruction set of an x86 instruction set core, or 2) the objective is to have at least one x86 instruction set core. Running on the processor to achieve compatibility with at least one x86 instruction set core The target code version of the application or other software with substantially the same processor. x86 compiler 3504 refers to a compiler operable to generate x86 binary code 3506 (e.g., target code), which can be executed on a processor 3516 with at least one x86 instruction set core, with or without additional linking processing. Similarly, Figure 35 A program in the form of a high-level language 3502 can be compiled using an alternative instruction set compiler 3508 to generate an alternative instruction set binary 3510 that can be natively executed by a processor 3514 that does not have at least one x86 instruction set core (e.g., a processor with a core that executes the MIPS instruction set of MIPS Technologies, Inc., Sunnyvale, California, and / or the ARM instruction set of ARM Holdings, Inc., Sunnyvale, California). An instruction converter 3512 is used to translate the x86 binary code 3506 into code that can be natively executed by the processor 3514 that does not have an x86 instruction set core. This translated code is unlikely to be identical to the alternative instruction set binary code 3510 because an instruction converter capable of doing so would be difficult to manufacture; however, the translated code will perform general operations and consists of instructions from the alternative instruction set. Therefore, the instruction converter 3512 represents, through emulation, simulation, or any other process, software, firmware, hardware, or a combination thereof that allows a processor or other electronic device without an x86 instruction set processor or core to execute the x86 binary code 3506.

[0372] Further examples

[0373] At least some embodiments of the disclosed technology can be described based on the following examples:

[0374] Example 1: An apparatus comprising:

[0375] A fetching circuit is used to fetch a single instruction having fields specifying an opcode and the positions of an M x N destination matrix with single-precision elements, an M x K first source matrix, and a K x N second source matrix, the source matrices having elements each comprising a quadruple of 8-bit floating-point values, the opcode instructing the execution circuit to cause: for each element of the first source matrix and the corresponding element of the second source matrix, converting the 8-bit floating-point values ​​to single-precision values, multiplying the converted single-precision values ​​from the first value of the quadruple together to generate a first result, multiplying the converted single-precision values ​​from the second value of the quadruple together to generate a second result, multiplying the converted single-precision values ​​from the third value of the quadruple together to generate a third result, multiplying the converted single-precision values ​​from the fourth value of the quadruple together to generate a fourth result, and accumulating the first result, the second result, the third result, and the fourth result with the previous contents of the corresponding elements of the destination matrix;

[0376] Decoding circuitry, used to decode the retrieved instructions; and

[0377] An execution circuit is used to respond to a decoded instruction as specified by the opcode.

[0378] Example 2: The apparatus as described in Example 1, wherein the 8-bit floating-point format is specified by the opcode of the single instruction.

[0379] Example 3: The apparatus as described in Example 1, wherein M, N, and K are specified by the single instruction.

[0380] Example 4: The apparatus as described in Example 1, wherein the execution circuitry is configured to cause the matrix operation accelerator to perform at least multiplication and accumulation.

[0381] Example 5: The apparatus as described in Example 4, wherein M, N, and K are specified by the configuration of the matrix operation accelerator, which is used to program the execution of the instruction via the matrix accelerator configuration prior to the execution of the single instruction.

[0382] Example 6: The apparatus as described in Example 1, wherein the execution circuitry is further configured to induce saturation of the execution result when necessary.

[0383] Example 7: The apparatus as described in Example 1, wherein the single instruction is further configured to specify a write mask comprising M x N bits, each bit being configured to control whether to mask the corresponding element of the destination matrix.

[0384] Example 8: The apparatus as described in Example 1, wherein the execution circuitry is further configured to generate an error when an error condition occurs, the error condition being selectable from the following:

[0385] The destination matrix has fewer rows than the first source matrix; and

[0386] The destination matrix has fewer columns than the second source matrix.

[0387] Example 9: A method that includes:

[0388] A single instruction is fetched by the processor's fetch circuit. This single instruction has fields specifying the opcode and the positions of an M x N destination matrix with single-precision elements, an M x K first source matrix, and a K x N second source matrix. Each source matrix has elements comprising a quadruple of 8-bit floating-point values. The opcode instructs the execution circuit to: for each element of the first source matrix and the corresponding element of the second source matrix, convert the 8-bit floating-point value to a single-precision value; multiply the converted single-precision values ​​from the first quadruple to generate a first result; multiply the converted single-precision values ​​from the second quadruple to generate a second result; multiply the converted single-precision values ​​from the third quadruple to generate a third result; multiply the converted single-precision values ​​from the fourth quadruple to generate a fourth result; and accumulate the first, second, third, and fourth results with the previous contents of the corresponding elements of the destination matrix.

[0389] The processor's decoding circuit decodes the retrieved instructions into decoded single instructions; and

[0390] The processor's execution circuit executes the decoded single instruction according to the opcode.

[0391] Example 10: The method as described in Example 9, wherein the 8-bit floating-point format is specified by the opcode of the single instruction.

[0392] Example 11: The method described in Example 9, wherein M, N, and K are specified by the single instruction.

[0393] Example 12: The method as described in Example 9, wherein the execution circuit causes the matrix operation accelerator to perform at least multiplication and accumulation.

[0394] Example 13: The method of Example 12 further includes the execution circuitry of the processor executing a matrix accelerator configuration instruction before executing the single instruction, the matrix accelerator configuration instruction programming the configuration of the matrix operation accelerators specifying M, N, and K.

[0395] Example 14: The method as described in Example 9, wherein the execution includes saturating the execution result.

[0396] Example 15: The method as described in Example 9, wherein the single instruction further specifies a write mask comprising M x N bits, each bit controlling whether to mask the corresponding element of the destination matrix.

[0397] Example 16: The method as described in Example 9, wherein the execution generates an error when an error condition occurs, the error condition being selectable from the following:

[0398] The destination matrix has fewer rows than the first source matrix; and

[0399] The destination matrix has fewer columns than the second source matrix.

[0400] Example 17: A non-transitory machine-readable medium storing program code, which, when executed by a machine, causes the machine to perform a method comprising the following steps:

[0401] A single instruction is fetched by the processor's fetch circuit. This single instruction has fields specifying the opcode and the positions of an M x N destination matrix with single-precision elements, an M x K first source matrix, and a K x N second source matrix. Each source matrix has elements comprising a quadruple of 8-bit floating-point values. The opcode instructs the execution circuit to: for each element of the first source matrix and the corresponding element of the second source matrix, convert the 8-bit floating-point value to a single-precision value; multiply the converted single-precision values ​​from the first quadruple to generate a first result; multiply the converted single-precision values ​​from the second quadruple to generate a second result; multiply the converted single-precision values ​​from the third quadruple to generate a third result; multiply the converted single-precision values ​​from the fourth quadruple to generate a fourth result; and accumulate the first, second, third, and fourth results with the previous contents of the corresponding elements of the destination matrix.

[0402] The processor's decoding circuit decodes the retrieved instructions into decoded single instructions; and

[0403] The processor's execution circuit executes the decoded single instruction according to the opcode.

[0404] Example 18: A non-transitory machine-readable medium as described in Example 17, wherein the 8-bit floating-point format is specified by the opcode of the single instruction.

[0405] Example 19: A non-transitory machine-readable medium as described in Example 17, wherein M, N, and K are specified by the single instruction.

[0406] Example 20: A non-transitory machine-readable medium as described in Example 17, wherein the execution includes the execution circuit causing the matrix operation accelerator to perform at least multiplication and accumulation.

[0407] Example 21: A non-transitory machine-readable medium as described in Example 20, wherein the method further includes executing a matrix accelerator configuration instruction by the execution circuitry of the processor before executing the single instruction, the matrix accelerator configuration instruction programming the configuration of the matrix operation accelerators specifying M, N, and K.

[0408] Example 22: A non-transitory machine-readable medium as described in Example 17, wherein the execution includes saturating the execution result.

[0409] Example 23: A non-transitory machine-readable medium as described in Example 17, wherein the single instruction further specifies a write mask comprising M x N bits, each bit controlling whether to mask the corresponding element of the destination matrix.

[0410] Example 24: A non-transitory machine-readable medium as described in Example 17, wherein the execution generates an error when an error condition occurs, the error condition being selectable from the following:

[0411] The destination matrix has fewer rows than the first source matrix; and

[0412] The destination matrix has fewer columns than the second source matrix.

Claims

1. An apparatus comprising: Multiple memory controllers; The second-level L2 cache memory is coupled to the plurality of memory controllers; A processor, coupled to the plurality of memory controllers and coupled to the L2 cache memory, has multiple cores including circuitry for executing operations corresponding to instructions. The instructions specify a first matrix having M rows by 4*K columns of 8-bit floating-point data elements, a second matrix having 4*K rows by N columns of 8-bit floating-point data elements, and a third matrix having M rows by N columns of 32-bit single-precision floating-point data elements. The operations include: For each row m in the M rows of the first matrix, and for each column n in the N columns of the second matrix: The 4*K 8-bit floating-point data elements in row m of the first matrix are converted into 4*K higher-precision floating-point data elements with higher precision than 8-bit floating-point data elements, and the 4*K 8-bit floating-point data elements in column n of the second matrix are converted into 4*K higher-precision floating-point data elements with higher precision than 8-bit floating-point data elements. Multiply the 4*K higher precision floating-point data elements corresponding to row m of the first matrix by the corresponding data elements of the 4*K higher precision floating-point data elements corresponding to column n of the second matrix to generate 4*K products. The 4*K products are summed with the 32-bit single-precision floating-point data elements corresponding to row m in row M and column n in column N of the third matrix to generate the result 32-bit single-precision floating-point data elements. as well as The resulting 32-bit single-precision floating-point data elements are stored in row m and column n of the third matrix.

2. The apparatus of claim 1, wherein, The first matrix has 64 rows in its M row.

3. The apparatus of claim 2, wherein, The second matrix has more than 64 columns.

4. The apparatus of claim 1, wherein, The instruction has values ​​for specifying the N columns of the second matrix.

5. The apparatus of claim 1, wherein, The first matrix is ​​intended to be stored in a plurality of registers of the processor or in memory, and the second matrix is ​​intended to be stored in memory.

6. The apparatus of claim 1, wherein, The instruction has a source element format field.

7. The apparatus of claim 1, further comprising an interconnect interface coupled to the processor.

8. The apparatus of claim 1, further comprising a bus controller interface coupled to the processor.

9. The apparatus of claim 1, wherein, The kernel includes a graphics kernel.

10. The apparatus of claim 1, wherein, The nucleus includes heterogeneous nuclei.

11. The apparatus of claim 1, wherein, The core includes heterogeneous graphic cores.

12. The apparatus of claim 1, further comprising an instruction converter for converting the instructions into one or more instructions of different instruction sets executable by the core.

13. The apparatus of claim 1, wherein, Each of the 8-bit floating-point data elements has five exponent bits and two explicit mantissa bits.

14. The apparatus of claim 1, wherein, The higher precision floating-point data element includes 32 bits.

Images