Method and apparatus for performing reduction operations on plurality of data element values

Abstract

The invention discloses a method and apparatus for performing a reduction operation on a plurality of data element values. Embodiments detailed herein pertain to reduction operations on a plurality of data element values. In one embodiment, a processor includes a decode circuit to decode an instruction and an execution circuit to execute the decoded instruction. The instruction specifies a first input register including a plurality of data element values, a first index register including a plurality of indexes, where each index of the plurality of indexes maps to one unique data element position of the first input register, and an output register. Performing includes: identifying data element values associated with each other based on an index; performing one or more reduction operations on the associated data element values based on the identification; and storing results of the one or more reduction operations in an output register.

Description

This application is a continuation of the invention patent application entitled “Method and apparatus for performing a reduction operation on a plurality of data element values” having application number 202010112705.8 and filing date February 24, 2020. TECHNICAL FIELD

[0001] Embodiments of the invention relate to the field of computer processor architecture; more specifically to performing a reduction operation on a plurality of data element values. BACKGROUND

[0002] An instruction set or instruction set architecture (ISA) is the programming-oriented part of a computer architecture that includes the native data types, instructions, register architecture, addressing modes, memory architecture, interrupt and exception handling, and external input and output (I / O). It should be noted that the term “instruction” generally refers herein to macroinstructions — i.e., instructions provided to a processor for execution — as opposed to microinstructions or micro-ops — i.e., microinstructions or micro-ops are the result of a processor’s decoder decoding a macroinstruction. Microinstructions or micro-ops can be configured to instruct execution units on the processor to perform operations to implement the logic associated with the macroinstruction.

[0003] An ISA is distinct from microarchitecture, which is the collection of processor design techniques used to implement an instruction set. Processors with different microarchitectures can share a common instruction set. For example, Pentium 4 processors, Core TM (Core TM ) processors, and multiple processors from Advanced Micro Devices, Inc. of Sunnyvale, California implement nearly identical versions of the x86 instruction set (with some extensions added with updated versions), but have different internal designs. For example, the same register architecture of an ISA can be implemented in different microarchitectures using well-known techniques in different ways, including dedicated physical registers, one or more dynamically allocated physical registers using a register renaming mechanism (e.g., using a register alias table (RAT), a reorder buffer (ROB), and a retirement register file). Unless otherwise specified, the phrases “register architecture,” “register file,” and “register” are used herein to refer to the register architecture, register file, and registers that are visible to software / programmers and to the way in which registers are specified by instructions. Where differentiation is needed, the adjective “logical,” “architectural,” or “software-visible” will be used to indicate the registers / register file in the register architecture, while a different adjective will be used to specify the registers in a given microarchitecture (e.g., physical registers, reorder buffer, retirement registers, register pool).

[0004] An instruction set includes one or more instruction formats. A given instruction format defines various fields (number of bits, location of bits) to specify the operation to be performed and the operand(s) on which that operation is to be performed, etc. Some instruction formats are further decomposed through the definition of instruction templates (or subformats). For example, instruction templates of a given instruction format can be defined to have different subsets of the fields of the instruction format (the included fields are typically in the same order, but at least some of the fields have different bit locations because fewer fields are included), and / or to have given fields interpreted in different ways. A given instruction is expressed using a given instruction format (and, if defined, a given one of the instruction templates of the instruction format), and specifies the operation and the operands. An instruction stream is a particular sequence of instructions, where each instruction in the sequence is an occurrence of an instruction in the instruction format (and, if defined, a given one of the instruction templates of the instruction format). BRIEF DESCRIPTION OF DRAWINGS

[0005] The application can best be understood by referring to the following description and accompanying drawings that are used to illustrate embodiments of the application. In the drawings:

[0006] Figure 1A and Figure 1B is a block diagram illustrating a generic vector friendly instruction format and its instruction templates according to embodiments of the present application.

[0007] Figures 2A-2D is a block diagram illustrating an exemplary specific vector friendly instruction format according to embodiments of the present application.

[0008] Figure 3 is a block diagram of a register architecture according to one embodiment of the present application.

[0009] Figure 4A is a block diagram illustrating both an exemplary in-order pipeline and an exemplary register renaming, out-of-order issue / execution pipeline according to embodiments of the present application.

[0010] Figure 4B is a block diagram illustrating both an exemplary in-order architecture and an exemplary register renaming, out-of-order issue / execution architecture according to embodiments of the present application.

[0011] Figure 5A is a block diagram of a single processor core according to an embodiment of the present application, its connection to a network on a chip interconnect network, and its local subset of a level 2 (L2) cache.

[0012] Figure 5B is a block diagram of a Figure 5Aa partial disassembly of a portion of the processor cores in

[0013] Figure 6 is a block diagram of a single core processor and a multi-core processor with integrated memory controller and graphics according to embodiments of the present application.

[0014] Figure 7 shows a block diagram of a system according to one embodiment of the present application.

[0015] Figure 8 shows a block diagram of a second system according to embodiments of the present application.

[0016] Figure 9 shows a block diagram of a third system according to embodiments of the present application.

[0017] Figure 10 shows a block diagram of a system on a chip (SoC) according to embodiments of the present application.

[0018] Figure 11 shows a block diagram of converting binary instructions in a source instruction set to binary instructions in a target instruction set using a software instruction translator according to embodiments of the present application.

[0019] Figure 12 is a block diagram illustrating a computer system configured to implement one or more aspects of the embodiments described herein.

[0020] Figures 13A-13D illustrates a parallel processor component according to embodiments.

[0021] Figures 14A-14B is a block diagram illustrating a graphics processing unit according to embodiments.

[0022] Figures 15A-15F illustrates an exemplary architecture in which multiple GPUs are communicatively coupled to multiple multi-core processors.

[0023] Figure 16 illustrates a graphics processing pipeline according to embodiments.

[0024] Figure 17 shows a tree reduction operation on multiple vector elements stored in an input register when executing a single instruction according to one embodiment of the present application.

[0025] Figure 18 shows a reduction operation on multiple vector elements stored in multiple input registers when executing a single instruction according to one embodiment of the present application.

[0026] Figure 19A flow diagram illustrating a reduction operation on a plurality of vector elements when executing a single instruction using synchronization is shown.

[0027] Figure 20 A flow diagram illustrating a reduction operation on a plurality of vector elements when executing a single instruction is shown. DETAILED DESCRIPTION

[0028] In the following description, numerous specific details are set forth. However, it is understood that embodiments of the application can be practiced without the specific details. In other instances, well-known circuits, structures and techniques have not been shown to avoid obscuring the understanding of this description.

[0029] Bracketed text and blocks having dashed borders, such as long dashed lines, short dashed lines, dot dashed lines, and dot lines, can be used to illustrate optional operations that add additional features to embodiments of the application. However, such notations should not be taken to mean that these are the only options or the only optional operations, and / or that blocks having solid borders are not optional in some embodiments of the application.

[0030] References in the specification to "one embodiment", "an embodiment”, "an example embodiment”, etc. indicate that the embodiment described can include a particular feature, structure, or characteristic, but every embodiment can not necessarily include the particular feature, structure, or characteristic. Moreover, such phrases are not necessarily referring to the same embodiment. Furthermore, when a particular feature, structure, or characteristic is described in connection with an embodiment, it is submitted that it is within the knowledge of those skilled in the art to effect such feature, structure, or characteristic in connection with other embodiments whether or not explicitly described or claimed.

[0031] The following description and the accompanying claims can use the terms "coupled" and "connected" and their derivatives. These terms are not intended as synonyms for each other. "Coupled” is used to indicate that two or more elements, which can or can not be in direct physical or electrical contact with each other, co-operate or interact with each other. "Connected” is used to indicate the establishment of communication between two or more elements that are coupled with each other. As used herein, "set” means any positive whole integer including one.

[0032] An instruction set includes one or more instruction formats. A given instruction format defines various fields (number of bits, location of bits) to specify the operation to be performed (opcode) and the operand(s) on which that operation is to be performed, etc. Some instruction formats are further decomposed by the definition of instruction templates (or subformats). For example, instruction templates of a given instruction format can be defined as having different subsets of the fields of that instruction format (the included fields are typically in the same order, but at least some of the fields have different bit locations because fewer fields are included), and / or as having given fields interpreted in different ways. Thus, each instruction of an ISA is expressed using a given instruction format (and, if defined, in a given one of the instruction templates of that instruction format) and includes fields to specify the operation and operands. For example, an exemplary ADD (addition) instruction has a particular opcode and instruction format that includes an opcode field to specify that opcode and operand fields to select the operands (source 1 / destination and source 2); and an occurrence of that ADD instruction in an instruction stream will have particular contents in the operand fields that select particular operands. There have been advanced and / or published sets of SIMD extensions known as Advanced Vector Extensions (AVX) (AVX1 and AVX2) and extensions utilizing the Vector Extension (VEX) coding scheme. Exemplary instruction format

[0033] Embodiments of the instruction(s) described herein can be embodied in different formats. In addition, exemplary systems, architectures, and pipelines are detailed below. Embodiments of the instruction(s) can be executed on such systems, architectures, and pipelines, but are not limited to those detailed.

[0034] A vector friendly instruction format is an instruction format that is amenable to vector instructions (e.g., there are specific fields dedicated to vector operations). Although embodiments are described in which both vector and scalar operations are supported through the vector friendly instruction format, alternative embodiments use only vector operations through the vector friendly instruction format.

[0035] Figures 1A-1B is a block diagram illustrating a generic vector friendly instruction format and its instruction templates according to embodiments of the application. Figure 1A is a block diagram illustrating a generic vector friendly instruction format and its instruction templates according to embodiments of the application; whereas Figure 1Bis a block diagram illustrating a generic vector friendly instruction format and its class B instruction templates according to embodiments of the application. Specifically, class A and class B instruction templates are defined for the generic vector friendly instruction format 100, both of which include instruction templates with no memory access 105 and instruction templates with memory access 120. The term "generic" in the context of the vector friendly instruction format refers to an instruction format that is not tied to any specific instruction set.

[0036] Although embodiments of the application will be described in which the vector friendly instruction format supports a 64-byte vector operand length (or size) with a 32-bit (4-byte) or 64-bit (8-byte) data element width (or size) (and, as a result, a 64-byte vector is made up of 16 double word size elements, or, alternatively, 8 quad word size elements); a 64-byte vector operand length (or size) with a 16-bit (2-byte) or 8-bit (1 byte) data element width (or size); a 32-byte vector operand length (or size) with a 32-bit (4-byte), 64-bit (8-byte), 16-bit (2-byte) or 8-bit (1 byte) data element width (or size); and a 16-byte vector operand length (or size) with a 32-bit (4-byte), 64-bit (8-byte), 16-bit (2-byte), or 8-bit (1 byte) data element width (or size); alternative embodiments can support larger, smaller and / or different vector operand sizes (e.g., 256 byte vector operands) with larger, smaller or different data element widths (e.g., 128-bit (16 byte) data element widths).

[0037] Figure 1A The class A instruction templates in include: 1) within the no memory access 105 instruction templates, instruction templates for no memory access full-runder control type operations 110 and no memory access data transform type operations 115; and 2) within the memory access 120 instruction templates, instruction templates for memory access temporal 125 and memory access non-temporal 130. Figure 1B The class B instruction templates in include: 1) within the no memory access 105 instruction templates, instruction templates for no memory access write mask control partial-runder control type operations 112 and no memory access write mask control vsize type operations 117; and 2) within the memory access 120 instruction templates, instruction templates for memory access write mask control 127.

[0038] The generic vector friendly instruction format 100 includes the following fields listed in the order shown in Figures 1A-1B

[0039] ​Format field 140 - a particular value in this field (instruction format identifier value) uniquely identifies the vector friendly instruction format, and thereby identifies that the instruction appears in the instruction stream in the vector friendly instruction format. This field is thus optional in the sense that it is not needed for an instruction set that has only a general purpose vector friendly instruction format.

[0040] Base operation field 142 - its content distinguishes different base operations.

[0041] Register index field 144 - its content specifies the location of a source or destination operand in a register or in memory, either directly or through address generation. These fields include a sufficient number of bits to select N registers from a PxQ (e.g., 32x512, 16x128, 32x1024, 64x1024) register file. Although N can be up to three source registers and one destination register in one embodiment, alternative embodiments can support more or fewer source and destination registers (e.g., up to two sources, with one of the sources also used as a destination; up to three sources, with one of the sources also used as a destination; up to two sources and one destination).

[0042] Modifier field 146 - its content distinguishes instructions in the general purpose vector instruction format that specify memory access from those that do not; i.e., distinguishes between no memory access 105 instruction templates and memory access 120 instruction templates. Memory access operations read and / or write to the memory hierarchy (in some cases, using values in registers to specify source and / or destination addresses), while non-memory access operations do not (e.g., sources and / or destinations are registers). Although this field also selects between three different ways to perform memory address computation in one embodiment, alternative embodiments can support more, fewer, or different ways to perform memory address computation.

[0043] Augment operation field 150 - its content distinguishes which of a variety of different operations is to be performed in addition to the base operation. This field is context dependent. In one embodiment of the invention, this field is divided into class field 168, alpha field 152, and beta field 154. Augment operation field 150 allows a number of groups of common operations to be performed in a single instruction rather than 2, 3, or 4 instructions.

[0044] Scale field 160 - its content allows scaling of the content of the index field for memory address generation (e.g., for address generation using (2 比例 *index + base). The scale field 160 is optional in the sense that it is not needed for an instruction set that has only a general purpose vector friendly instruction format.

[0045] Displacement field 162A - whose content is used as part of memory address generation (e.g., for address generation using (2 比例 *index + base + displacement).

[0046] Displacement factor field 162B (note that collocation of displacement field 162A directly over displacement factor field 12B indicates use of one or the other) - whose content is used as part of address generation; it specifies a displacement factor by which to scale the size (N) of the memory access - where N is the number of bytes in the memory access (e.g., for address generation using (2 比例 *index + base + scaled displacement). Redundant low-order bits are ignored, and thus the content of the displacement factor field is multiplied by the total size of the memory operand (N) to generate the final displacement to be used in the calculation of the effective address. The value of N is determined by the processor hardware at runtime based on the full opcode field 174 (described herein) and the data manipulation field 154C. Displacement field 162A and displacement factor field 162B are optional in the sense that they are not used for instruction templates with no memory access 105 and / or different embodiments can implement only one of these two or neither of these two.

[0047] Data element width field 164 - whose content distinguishes which of a number of data element widths is to be used (in some embodiments for all instructions; in other embodiments for only some of the instructions). This field is not needed if only one data element width is supported and / or the use of a certain aspect of the opcode supports the data element width, in which case this field is optional.

[0048] Write mask field 170 - its content controls, on a data element position by data element position basis, whether a data element position in the destination vector operand reflects the result of the base operation and the augmentation operation. Class A instruction templates support merge-write mask, while class B instruction templates support both merge-write mask and zero- write mask. When merging, the vector mask allows any set of elements in the destination to be protected from update during the execution of any operation (specified by the base operation and the augmentation operation); in another embodiment, the old value of each element of the destination where the corresponding mask bit has a zero is preserved. Conversely, when zeroing, the vector mask allows any set of elements in the destination to be zeroed during the execution of any operation (specified by the base operation and the augmentation operation); in one embodiment, the elements of the destination are set to zero when the corresponding mask bit has a zero value. A subset of this functionality is the ability to control the vector length of the operation being performed (i.e., the span from the first to the last element being modified), however, the elements being modified do not have to be contiguous. Thus, the write mask field 170 allows partial vector operations, including loads, stores, arithmetic, logic, etc. While embodiments of the present application are described in which the content of the write mask field 170 selects one of a plurality of write mask registers containing the write mask to be used (and thus, the content of the write mask field 170 indirectly identifies the mask to be executed), alternative embodiments alternatively or additionally allow the content of the mask write field 170 to directly specify the mask to be executed.

[0049] Immediate field 172 - its content allows the specification of an immediate number. This field is not present in generic vector friendly format implementations that do not support immediate numbers and is not present in instructions that do not use an immediate number, in this sense, the field is optional.

[0050] Class field 168 - its content distinguishes between different classes of instructions. Referring to Figures 1A-1B , the content of this field selects between class A and class B instructions. In Figures 1A-1B , a rounded square is used to indicate that a particular value is present in the field (e.g., class A 168A and class B 168B for the class field 168 in Figure 1A , respectively). Class A instruction templates

[0051] In the case of class A non-memory access 105 instruction templates, the alpha field 152 is interpreted as an RS field 152A whose content distinguishes which of the different augmentation operation types are to be performed (e.g., round 152A.1 and data transform 152A.2 are specified for the no memory access round type operation 110 and no memory access data transform type operation 115 instruction templates, respectively), while the beta field 154 distinguishes which of the operations of the specified type are to be performed. In the no memory access 105 instruction templates, the scale field 160, the displacement field 162A, and the displacement factor field 162B (sometimes referred to as the displacement scale field) are not present.

[0052] No memory access instruction templates - full round control type operations

[0053] In the no memory access full round control type operation 110 instruction templates, the beta field 154 is interpreted as its content(s) providing a round control field 154A for static rounding. Although the round control field 154A includes a suppress all (floating point) exceptions (SAE) field 156 and a round operation control field 158 in the described embodiment of the invention, alternative embodiments can support both of these concepts, can encode both of these concepts as the same field, or have only one or the other of these concepts / fields (e.g., can have only the round operation control field 158).

[0054] SAE field 156 - whose content distinguishes whether or not exception event reporting is disabled; when the content of the SAE field 156 indicates that suppression is enabled, the given instruction does not report any kind of floating point exception flags, and does not invoke any floating point exception handlers.

[0055] Round operation control field 158 - whose content distinguishes which of a set of rounding operations are to be performed (e.g., round up, round down, round to zero, and round to nearest). As such, the round operation control field 158 allows the rounding mode to be changed on a per-instruction basis. In one embodiment of the invention in which the processor includes a control register for specifying the rounding mode, the content of the round operation control field 150 overrides that register value. No memory access instruction templates - data transform type operations

[0056] In the no memory access data transform type operation 115 instruction templates, the beta field 154 is interpreted as a data transform field 154B whose content distinguishes which of a number of data transforms are to be performed (e.g., no data transform, permute, broadcast).

[0057] In the case of class A memory access 120 instruction templates, the alpha field 152 is interpreted as an eviction hint field 152B whose content distinguishes which of the eviction hints are to be used (in the case of the load instruction templates, the eviction hint field 152B distinguishes between no hint, hint not loaded into cache, and hint loaded into cache).Figures 2A-2D In the case of the memory access non-temporal 130 instruction template, the alpha field 152 is interpreted as a data manipulation field 152C whose content distinguishes which of a number of data manipulation operations (also called primitives) is to be performed (e.g., no manipulation, broadcast, upcast of source, and downcast of destination). The memory access 120 instruction template includes a scale field 160 and optionally a displacement field 162A or a displacement factor field 162B.

[0058] Vector memory instructions use conversion support to perform vector loads from memory and vector stores to memory. Like ordinary vector instructions, vector memory instructions transfer data from / to memory in a data element-wise fashion, where the elements actually transferred are dictated by the contents of a vector mask selected as a write mask.

[0059] Memory access instruction templates - temporal

[0060] Temporal data is data that is likely to be reused quickly enough to benefit from cache operations. However, this is a hint, and different processors can implement it in different ways, including completely ignoring the hint. Memory access instruction templates - non-temporal

[0061] Non-temporal data is data that is not likely to be reused quickly enough to benefit from cache operations in a first level cache and should be given eviction priority. However, this is a hint, and different processors can implement it in different ways, including completely ignoring the hint. Class B instruction templates

[0062] In the case of the class B instruction templates, the alpha field 152 is interpreted as a write mask control (Z) field 152C whose content distinguishes whether the write mask controlled by the write mask field 170 should be merge or zeroed.

[0063] In the case of class B non-memory access 105 instruction templates, a portion of the beta field 154 is interpreted as an RL field 157A, whose content distinguishes which of the different augmentation operation types are to be performed (e.g., round 157A.1 and vector length (VSIZE) 157A.2 are specified for the write mask control partial round control type operation 112 and the write mask control VSIZE type operation 117, respectively, of the no memory access instruction templates), while the remainder of the beta field 154 distinguishes which of the operations of the specified type are to be performed. In the no memory access 105 instruction templates, the scale field 160, the displacement field 162A, and the displacement factor field 162B are not present.

[0064] In the write mask control partial round control type operation 110 instruction templates, the remainder of the beta field 154 is interpreted as a round control operation field 159A, and exception event reporting is disabled (the given instruction does not report any kind of floating point exception flags, and does not invoke any floating point exception handlers).

[0065] The round operation control field 159A, like the round operation control field 158, has a content that distinguishes which of a set of round operations are to be performed (e.g., round up, round down, round to zero, and round to nearest). As such, the round operation control field 159A allows the round mode to be changed on a per-instruction basis. In one embodiment of the application in which the processor includes a control register for specifying the round mode, the content of the round operation control field 150 overrides the register value.

[0066] In the write mask control VSIZE type operation 117 instruction templates, the remainder of the beta field 154 is interpreted as a vector length field 159B, whose content distinguishes which of a plurality of data vector lengths are to be performed (e.g., 128 bytes, 256 bytes, or 512 bytes).

[0067] In the case of class B memory access 120 instruction templates, a portion of the beta field 154 is interpreted as a broadcast field 157B, whose content distinguishes whether a broadcast type data manipulation operation is to be performed, while the remainder of the beta field 154 is interpreted as a vector length field 159B. The memory access 120 instruction templates include a scale field 160, and optionally include a displacement field 162A or a displacement scale field 162B.

[0068] For the generic vector friendly instruction format 100, the full opcode field 174 is shown to include the format field 140, the base operation field 142, and the data element width field 164. Although one embodiment is shown in which the full opcode field 174 includes all of these fields, in embodiments that do not support all of these fields, the full opcode field 174 includes fewer than all of these fields. The full opcode field 174 provides the operation code (opcode).

[0069] The extension operation field 150, the data element width field 164, and the writemask field 170 allow these features to be specified on a per-instruction basis with the generic vector friendly instruction format.

[0070] The combination of the writemask field and the data element width field create various types of instructions, as these instructions allow the mask to be applied based on different data element widths.

[0071] The various instruction templates that occur within class A and class B are beneficial in different scenarios. In some embodiments of the invention, different processors or different cores within a processor can support only class A, only class B, or both classes. For example, a high-performance general-purpose out-of-order core intended for general-purpose computing can support only class B, a core intended primarily for graphics and / or scientific (throughput) computing can support only class A, and a core intended for both general-purpose computing and graphics and / or scientific (throughput) computing can support both class A and class B (of course, a core that has some mixture of templates and instructions from both classes, but not all templates and instructions from both classes, is within the scope of the invention). Likewise, a single processor can include multiple cores, all of which support the same class, or where different cores support different classes. For example, in a processor with separate graphics and general-purpose cores, one core in the graphics core intended primarily for graphics and / or scientific computing can support only class A, while one or more of the general-purpose cores can be a high-performance general-purpose core with out-of-order execution and register renaming that supports only class B intended for general-purpose computing. Another processor without separate graphics cores can include one or more general-purpose in-order or out-of-order cores that support both class A and class B. Of course, features from one class can also be implemented in the other class in different embodiments of the invention. Programs written in a high-level language will be compiled into various different executable forms, including: 1) a form with only the instructions of the class(es) supported by the target processor for execution; or 2) a form with alternative routines written using different combinations of instructions of all classes, and control flow code that selects the routines to execute based on the instructions supported by the processor currently executing the code.

[0072] Figure 2Ais a block diagram illustrating an exemplary specific vector friendly instruction format according to embodiments of the application. Figures 1A-1B The specific vector friendly instruction format 200 is specific in the sense that it specifies the location, size, interpretation, and order of various fields, as well as the values of some of those fields. The specific vector friendly instruction format 200 can be used to extend the x86 instruction set, and as such some of the fields are similar or identical to those used as in the existing x86 instruction set and extensions thereof (e.g., AVX). The format remains consistent with the prefix encoding field, real opcode byte field, MOD R / M field, SIB field, displacement field, and immediate fields of the existing x86 instruction set with extensions. The fields from Figure 2A The fields from Figures 1A-1B map to the fields from Figure 2A

[0073] It should be appreciated that while embodiments of the application are described with reference to the specific vector friendly instruction format 200 in the context of the general vector friendly instruction format 100 for purposes of illustration, the application is not limited to the specific vector friendly instruction format 200 unless otherwise indicated. For instance, the general vector friendly instruction format 100 contemplates various possible sizes of various fields, while the specific vector friendly instruction format 200 is shown with fields of particular sizes. As a specific example, while the data element width field 164 is shown as a one-bit field in the specific vector friendly instruction format 200, the application is not so limited (i.e., the general vector friendly instruction format 100 contemplates other sizes for the data element width field 164).

[0074] The general vector friendly instruction format 100 includes the following fields listed in the order shown in Figure 2B

[0075] EVEX prefix (bytes 0-3) 202 - encoded in four bytes.

[0076] Format field 140 (EVEX byte 0, bits [7:0]) - the first byte (EVEX byte 0) is the format field 140 and it contains Ox62 (in one embodiment of the application, the only value used to distinguish the vector friendly instruction format).

[0077] The second-fourth bytes (EVEX bytes 1-3) include a number of bit fields that provide specific capabilities.

[0078] ​​REX field 205 (EVEX Byte 1, bits [7-5]) - made up of the EVEX.R bit field (EVEX Byte 1, bit [7] - R), the EVEX.X bit field (EVEX Byte 1, bit [6] - X), and the EVEX.B bit field (EVEX Byte 1, bit [5] - B). The EVEX.R, EVEX.X, and EVEX.B bit fields provide the same functionality as the corresponding VEX bit fields, and are encoded in ones' complement form, that is, ZMMO is encoded as 1111B, ZMM15 is encoded as 0000B. The other fields of these instructions encode the lower three bits of the register indexes as is known in the art (rrr, xxx, and bbb), from which the full 64 register indexes can be formed by prepending the EVEX.R, EVEX.X, and EVEX.B (Rrrr, Xxxx, and Bbbb, respectively).

[0079] REX' field 210 - this is the first part of the REX' field 210 and is the EVEX.R' bit field (EVEX Byte 1, bit [4] - R') that is used to encode either the upper 16 or lower 16 of the extended 32 register set. In one embodiment of the invention, this bit is stored in bit inverted format along with other bits indicated below to distinguish (in the well-known x86's 32-bit mode) from the BOUND instruction, whose real opcode byte is 62, but does not accept in the MOD field of the MOD R / M field (described below) the value of 11; alternative embodiments of the invention do not store this and the other indicated below in inverted format. A value of 1 is used to encode the lower 16 registers. In other words, R'Rrrr is formed by combining EVEX.R', EVEX.R, and the other RRR from other fields.

[0080] Opcode map field 215 (EVEX Byte 1, bits [3:0] - mmmm) - its content encodes the implied leading opcode byte (0F, 0F 38, or 0F 3).

[0081] Data element width field 164 (EVEX Byte 2, bit [7] - W) - represented by the notation EVEX.W. EVEX.W is used to define the granularity (size) of the data type either 32-bit data elements or 64-bit data elements.

[0082] EVEX.vvvv 220 (EVEX Byte 2, bits [6:3] - vvvl) - the role of EVEX.vvvv can include the following: 1) EVEX.vvvv encodes the first source register operand, specified in inverted (Is complement) form, and is valid for instructions with two or more source operands; 2) EVEX.vvvv encodes the destination register operand, specified in Is complement form for certain vector shifts; or 3) EVEX.vvvv does not encode any operand, the field is reserved and should contain 1111b. Thus, the EVEX.vvvv field 220 encodes the 4 lower-order bits of the first source register specifier, stored in inverted (Is complement) form. Depending on the instruction, an additional different EVEX bit field is used to extend the specifier size to 32 registers.

[0083] EVEX.U 168 class field (EVEX Byte 2, bit [2] - U) - if EVEX.U = 0, it indicates class A or EVEX.U0; if EVEX.U = 1, it indicates class B or EVEX.U1.

[0084] Prefix encoding field 225 (EVEX Byte 2, bits [1 :0] - pp) - provides additional bits for the base operation field. This has the benefit of compressing the SIMD prefix (EVEX prefix only needs 2 bits, instead of a byte to express the SIMD prefix), in addition to providing support for legacy SSE instructions in EVEX prefix format. In one embodiment, to support legacy SSE instructions using the SIMD prefixes in both legacy format and in EVEX prefix format (66H, F2H, F3H), these legacy SIMD prefixes are encoded into the SIMD prefix encoding field; and are expanded into the legacy SIMD prefixes before being provided to the PLA of the decoder at runtime (thus, without modification, the PLA can execute both the legacy format of these legacy instructions and the EVEX format of these legacy instructions). While newer instructions can use the contents of the EVEX prefix encoding field directly as an opcode extension, particular embodiments expand in a similar manner for consistency, but allow the different meanings designated by these legacy SIMD prefixes. Alternative embodiments can redesign the PLA to support 2-bit SIMD prefix encoding, and thus do not require expansion.

[0085] Alpha field 152 (EVEX Byte 3, bit [7] - EH, also known as EVEX.EH, EVEX.rs, EVEX.RL, EVEX.write mask control, and EVEX.N; also shown as a) - as previously described, this field is context specific.

[0086] Beta field 154 (EVEX byte 3, bit [6:4] - SSS, also referred to as EVEX.s 2-0 EVEX.r 2-0 EVEX.rr1, EVEX.LL0, EVEX.LLB, also shown as βββ) - as previously described, this field is context specific.

[0087] REX' field 210 - this is the remainder of the REX' field and is the EVEX.V' bit field (EVEX byte 3, bit [3] - V') that can be used to encode either the upper 16 or lower 16 of the extended 32 register set. The bit is stored in bit inverted format. A value of 1 is used to encode the lower 16 registers. In other words, VVVV is formed by the combination of EVEX.V', EVEX.vvvv.

[0088] Write mask field 170 (EVEX byte 3, bits [2:0] - kkk) - its content specifies the index of the register in the write mask register, as previously described. In one embodiment of the application, the special behavior of no write mask for the specific instruction is implied by the specific value EVEX.kkk = 000 (this can be implemented in a variety of ways, including using a hardwired write mask to all targets or hardware that bypasses the masking hardware).

[0089] Real opcode field 230 (byte 4) is also referred to as the opcode byte. Part of the opcode is specified in this field.

[0090] MOD R / M field 240 (byte 5) includes MOD field 242, Reg field 244 and R / M field 246. As previously described, the content of MOD field 242 distinguishes between memory access operations and non-memory access operations. The role of Reg field 244 can be summarized into two scenarios: encoding a destination register operand or a source register operand; or be considered as opcode extension and not used to encode any instruction operand. The role of R / M field 246 can include the following: encoding an instruction operand that references a memory address; or encoding a destination register operand or a source register operand.

[0091] Scale, index, base (SIB) byte (byte 6) - as previously described, the content of scale field 150 is used for memory address generation. SIB.xxx 254 and SIB.bbb 256 - the content of these fields has been previously mentioned with respect to register indexes Xxxx and Bbbb.

[0092] Displacement field 162A (bytes 7-10) - when MOD field 242 contains 10, bytes 7-10 are the displacement field 162A, and it works just like the traditional 32-bit displacement (disp32), and works in byte granularity.

[0093] Displacement factor field 162B (byte 7) - when MOD field 242 contains 01, byte 7 is the displacement factor field 162B. The location of this field is the same as the location of the traditional x86 instruction set 8-bit displacement (disp8) that works in byte granularity. Since disp8 is sign-extended, it can only address between -128 and 127 byte offsets; in terms of 64 byte cache lines, disp8 uses 8 bits that can be set to only four really useful values -128, -64, 0, and 64; since a larger range is often needed, disp32 is used; however, disp32 requires 4 bytes. In contrast to disp8 and disp32, the displacement factor field 162B is a reinterpretation of disp8; when the displacement factor field 162B is used, the actual displacement is determined by multiplying the contents of the displacement factor field by the size of the memory operand access (N). This type of displacement is referred to as disp8*N. This reduces the average instruction length (a single byte is used for the displacement, but with a much larger range). Such compressed displacements are based on the assumption that the effective displacement is a multiple of the granularity of the memory access, and thus the redundant low-order bits of the address offset do not need to be encoded. In other words, the displacement factor field 162B replaces the traditional x86 instruction set 8-bit displacement. Thus, the displacement factor field 162B is encoded in the same way as the x86 instruction set 8-bit displacement (so there are no changes in the ModRM / SIB encoding rules), the only difference being that disp8 is overloaded to disp8*N. In other words, there are no changes in the encoding rules or encoding length, but only in the interpretation of the displacement value by the hardware (which needs to scale the displacement by the size of the memory operand to obtain a byte-wise address offset).

[0094] The immediate field 172 operates as previously described. Full opcode field

[0095] Figure 2C is a block diagram showing the fields of the special-purpose vector friendly instruction format 200 that make up the full opcode field 174 according to one embodiment of the invention. Specifically, the full opcode field 174 includes the format field 140, the base operation field 142, and the data element width (W) field 164. The base operation field 142 includes the prefix encoding field 225, the opcode map field 215, and the real opcode field 230. Register index field

[0096] Figure 2Dis a block diagram illustrating the fields of the special vector friendly instruction format 200 that make up the register index field 144 according to one embodiment of the invention. Specifically, the register index field 144 includes the REX field 1405, the REX' field 210, the MODR / M.reg field 244, the MODR / M.r / m field 246, the VVVV field 220, the xxx field 254, and the bbb field 256. Augmenting the operation field

[0097] Figure 3 is a block diagram illustrating the fields of the special vector friendly instruction format 200 that make up the augmenting operation field 150 according to one embodiment of the invention. When the class (U) field 168 contains 0, it signifies EVEX.U0 (class A 168A); when it contains 1, it signifies EVEX.U1 (class B 168B). When U=0 and the MOD field 242 contains 11 (signifying a no memory access operation), the alpha field 152 (EVEX byte 3, bit [7] - EH) is interpreted as the rs field 152A. When the rs field 152A contains 1 (round 152A.1), the beta field 154 (EVEX byte 3, bits [6:4] - SSS) is interpreted as the round control field 154A. The round control field 154A includes a one bit SAE field 156 and a two bit round operation field 158. When the rs field 152A contains 0 (data transform 152A.2), the beta field 154 (EVEX byte 3, bits [6:4] - SSS) is interpreted as a three bit data transform field 154B. When U=0 and the MOD field 242 contains 00, 01, or 10 (signifying a memory access operation), the alpha field 152 (EVEX byte 3, bit [7] - EH) is interpreted as the eviction hint (EH) field 152B, and the beta field 154 (EVEX byte 3, bits [6:4] - SSS) is interpreted as a three bit data manipulation field 154C.

[0098] When U=1, the alpha field 152 (EVEX byte 3, bit [7] - EH) is interpreted as the write mask control (Z) field 152C. When U=1 and the MOD field 242 contains 11 (signifying a no memory access operation), a portion of the beta field 154 (EVEX byte 3, bit [4] - S0) is interpreted as the RL field 157A; when it contains 1 (round 157A.1), the remainder of the beta field 154 (EVEX byte 3, bits [6-5] - S 2-1 ) is interpreted as the round operation control field 159A, and when the RL field 157A contains 0 (VSIZE 157.A2), the remainder of the beta field 154 (EVEX byte 3, bits [6-5] - S 2-1 ) is interpreted as the vector length field 159B (EVEX byte 3, bits [6-5] - L1-0 ) When U=1 and the MOD field 242 contains 00, 01, or 10 (indicating a memory access operation), the beta field 154 (EVEX byte 3, bit [6:4] - SSS) is interpreted as a vector length field 159B (EVEX byte 3, bit [6-5] - L 1-0 ) and a broadcast field 157B (EVEX byte 3, bit [4] - B).

[0099] Exemplary processor architecture and data types is a block diagram of a register architecture 300 according to one embodiment of the application. In the illustrated embodiment, there are 32 512-bit wide vector registers 310; these registers are referenced as zmm0 through zmm31. The lower order 256 bits of the lower 16 zmm registers overlay on the registers ymm0-16. The lower order 128 bits of the lower 16 zmm registers (the lower order 128 bits of the ymm registers) overlay on the registers xmm0-15. The specific vector friendly instruction format 200 operates on these overlaid register files as shown in the following table.

[0100] In other words, the vector length field 159B selects between a maximum length and one or more other shorter lengths, where each such shorter length is half the preceding length, and the instruction templates of the specific vector friendly instruction format 200 that do not have a vector length field 159B operate on the maximum vector length. Furthermore, in one embodiment, the class B instruction templates of the specific vector friendly instruction format 200 operate on packed or scalar single / double-precision floating point data as well as packed or scalar integer data. Scalar operations are operations performed on the lowest order data element position in the zmm / ymm / xmm registers; depending on the embodiment, the higher order data element positions are either left the same as before the instruction or zeroed.

[0101] Write mask registers 315 - in the illustrated embodiment, there are 8 write mask registers (k0 through k7), each write mask register is 64 bits in size. In alternative embodiments, the write mask registers 315 are 16 bits in size. As previously noted, in one embodiment of the application, the vector mask register k0 cannot be used as a write mask; when the encoding that would normally indicate k0 is used as a write mask, it selects the hardwired write mask 0xFFFF, effectively disabling the write mask for that instruction.

[0102] General-purpose registers 325 - in the illustrated embodiment, there are sixteen 64-bit general-purpose registers that are used with existing x86 addressing modes to address memory operands. These registers are referenced by the names RAX, RBX, RCX, RDX, RBP, RSI, RDI, RSP, and R8 through R15.

[0103] Scalar floating point stack register file (x87 stack) 345, on top of which is overlaid MMX packed integer flat register file 350 - in the illustrated embodiment, the x87 stack is an eight-element stack used for performing scalar floating point operations using the x87 instruction set extensions on 32 / 64 / 80-bit floating point data; while the MMX registers are used for performing operations on 64-bit packed integer data, and to hold operands for some operations performed between MMX and XMM registers.

[0104] Alternative embodiments of the invention can use wider or narrower registers. In addition, alternative embodiments of the invention can use more, fewer, or different register files and registers. Figure 4A

[0105] Figure 4B is a block diagram illustrating an exemplary in-order pipeline and an exemplary register renaming, out-of-order issue / execution pipeline according to embodiments of the invention. Figures 4A-4B is a block diagram illustrating an exemplary in-order architecture core and an exemplary register renaming, out-of-order issue / execution architecture core to be included in a processor according to embodiments of the invention. Figure 4A The solid lined boxes in illustrate an in-order pipeline and an in-order core, while the optional addition of the dashed lined boxes illustrates a register renaming, out-of-order issue / execution pipeline and core. It is noted that some embodiments can not incorporate all elements of the depicted embodiments, and embodiments can incorporate elements of the depicted embodiments in different combinations.

[0106] In Figure 4B In, processor pipeline 400 includes fetch stage 402, length decode stage 404, decode stage 406, allocation stage 408, renaming stage 410, dispatch (also known as issue or launch) stage 412, register read / memory read stage 414, execution stage 416, write back / memory write stage 418, exception handling stage 422, and commit stage 424. Each stage can include circuitry to perform operations in the stage, and the circuitry can be referred to as fetch circuitry, decode circuitry, dispatch circuitry, execution circuitry, etc. to indicate that hardware circuitry is used to implement the stages.

[0107] Figures 5A-5BA processor core 490 is shown including a front end unit 430 coupled to an execution engine unit 450, and both the front end unit 430 and the execution engine unit 450 are coupled to a memory unit 470. The core 490 can be a reduced instruction set computing (RISC) core, a complex instruction set computing (CISC) core, a very long instruction word (VLIW) core, or a hybrid or alternative core type. As yet another option, the core 490 can be a special-purpose core, such as, for example, a network or communication core, a compression engine, a coprocessor core, a general-purpose computing graphics processing unit (GPGPU) core, a graphics core, and the like.

[0108] The front end unit 430 includes a branch prediction unit 432 coupled to an instruction cache unit 434, which is coupled to an instruction translation lookaside buffer (TLB) 436, which is coupled to an instruction fetch unit 438, which is coupled to a decode unit 440. The decode unit 440 (or decoder) can decode instructions, and generate one or more micro-operations, microcode entries, microinstructions, other instructions, or other control signals from the decoded instructions as output. The decode unit 440 can be implemented with one or more logic, logic arrays, arrays of logic formed using PLA, PAL, GAL, and / or GAL, hardwired circuitry, software, other implementations, or a combination of implementations. In one embodiment, the core 490 includes a microcode ROM or other medium that stores microcode for certain macroinstructions (e.g., in the decode unit 440, or otherwise within the front end unit 430). The decode unit 440 is coupled to a rename / allocator unit 452 in the execution engine unit 450.

[0109] The execution engine unit 450 includes a rename / allocator unit 452 that is coupled to a retirement unit 454 and a set of one or more scheduler units 456. The scheduler unit(s) 456 represents any number of different schedulers, including a reservation station, a central instruction window, etc. The scheduler unit(s) 456 is coupled to the physical register file(s) unit 458. Each physical register file unit 458 represents one or more physical register files, where different physical register files store one or more different types of data (e.g., scalar integer, scalar floating point, packed data (e.g., SIMD) integer, packed data (e.g., SIMD) floating point, state information (e.g., instruction pointer), etc.). In one embodiment, the physical register file(s) unit 458 includes a vector register unit, a write mask register unit, and a scalar register unit. These register units can provide architectural vector, mask, and scalar registers for the one or more instruction issue slots. The physical register file(s) unit 458 is overlapped with one or more execute units 460. The execute unit(s) 460 is coupled to the physical register file(s) unit 458 for the appropriate instruction set (e.g., SIMD and / or non-SIMD) and a set of one or more memory access units 464. The execute unit(s) 460 is the unit that performs all arithmetic, logical, move, and various other operations on data and / or values stored in registers and / or memory. The execute unit(s) 460 also includes a jump adder unit 461 that is coupled to the fetch unit 410 (see FIG. 4A) for determining a final effective address of an instruction for a branch instruction.

[0110] The set of memory access units 464 is coupled to a memory unit 470 that includes a data TLB unit 472 coupled to a data cache unit 474 that is coupled to a level 2 (L2) cache unit 476. In one exemplary embodiment, the memory access units 464 can include a load unit, a store address unit, and a store data unit, each of which is coupled to the data TLB unit 472 in the memory unit 470. The instruction cache unit 434 is also coupled to the level 2 (L2) cache unit 476 in the memory unit 470. The L2 cache unit 476 is coupled to one or more other levels of cache, and ultimately to main memory.

[0111] As an example, an exemplary register-renaming, out-of-order issue / execution core architecture can implement the pipeline 400 as follows: 1) the instruction fetch 438 performs fetch and length decode stages 402; 2) the decode unit 440 performs a decode stage 406; 3) the rename / allocator unit 452 performs a re- name stage 408 and a schedule stage 410; 4) the (multiple) scheduler unit(s) 456 perform a schedule stage 412; 5) the (multiple) physical register file(s) unit(s) 458 and memory unit 470 perform a register read / write stage 414; the execution cluster 460 performs an execute stage 416; 6) the memory unit 470 and (multiple) physical register file(s) unit(s) 458 perform a write back stage 418; 7) various units can provide partial results to an exception handling unit 420; and 8) the retirement unit 454 and (multiple) physical register file(s) unit(s) 458 perform a retire stage 424.

[0112] The core 490 can support one or more instruction sets (e.g., the x86 instruction set (with some extensions that have been added with newer versions); the MIPS instruction set of MIPS Technology, Inc. of Sunnyvale, California; the ARM instruction set (with optional additional extensions such as NEON) of ARM Holdings of San Jose, California), which can include the (multiple) instructions described herein. In one embodiment, the core 490 includes logic to support the Streaming

[0113] It should be understood that a core can support multithreading (execution of two or more parallel sets of instructions or threads), and can do so in a variety of ways including time-sliced multithreading, simultaneous multithreading (where a single physical core provides logical cores that are each contexts of the single physical core while the core is multithreading), or combinations thereof (for example, time-sliced fetching and decoding with simultaneous multithreading subsequent to fetch and decode). Simultaneous multithreading in hyperthreading technology).

[0114] Although register renaming is described in the context of out-of-order execution, it should be understood that register renaming can be used in in-order architectures. Although the illustrated processor includes separate instruction and data cache units 434 / 474 and a shared L2 cache unit 476, alternative embodiments can have a single internal cache for both instructions and data, such as for example, a unified cache or multiple levels of cache between a processor core and a level two (L2) cache. In some embodiments, the system can include a combination of internal caches and external caches outside of a core and / or processor. Alternatively, all of the caching can be external to the core and / or processor.

[0115] Figure 5A A block diagram of a more specific exemplary in-order core architecture is shown. The core will be one of several cores that will be included in a chip that will also include fixed function logic, memory I / O interface, and other necessary I / O logic.

[0116] Figure 5B is a block diagram of a single processor core according to an embodiment of the application and its connection to a network on a chip interconnect 502 and its local subset of a level two (L2) cache 504. In one embodiment, the instruction decoder 500 supports the x86 instruction set with a set of data compression instruction set extensions. The L1 cache 506 allows for low latency access to cache memory into and out of the scalar and vector units. Although in one embodiment (to simplify the design) the scalar unit 508 and vector unit 510 use separate register sets (scalar registers 512 and vector registers 514, respectively), and data transferred between these registers is written to memory and read back from the level one (L1) cache 506, alternative embodiments of the application can use a different approach (for example, using a single register set or including a communication path that allows data to be transferred between the two register files without being written to memory and read back).

[0117] The local subset 504 of the L2 cache is part of a global L2 cache that is partitioned into multiple separate local subsets, one for each processor core. Each processor core has a direct access path to its own local subset 504 of the L2 cache. Data read by a processor core is stored in its L2 cache subset 504 and can be quickly accessed in parallel with other processor cores accessing their own local L2 cache subsets. Data written by a processor core is stored in its own L2 cache subset 504 and is flushed from other subsets as necessary. A ring network ensures coherency of shared data. The ring network is bidirectional to allow agents such as processor cores, L2 caches, and other logic blocks to communicate with each other within the chip. Each ring data path is 1012 bits wide for each direction.

[0118] Figure 5A is a block diagram of a processor core in Figure 5B part of a system-on-a-chip (SoC) according to an embodiment of the application. Figure 6 The L1 data cache 506A portion includes the L1 cache 504, and more detail regarding the vector unit 510 and vector registers 514. In particular, the vector unit 510 is a 16-wide vector processing unit (VPU) (see 16-wide ALU 528) that executes one or more integer, single precision floating point, and double precision floating point instructions. The VPU supports a move operation for a register input via a mix unit 520, a numeric conversion via numeric conversion units 522A-B, and a replication to memory input via a replication unit 524. A writemask register 526 allows masking of the result vector write.

[0119] Figure 6 is a block diagram of a processor 600 that can have more than one core, that can have an integrated memory controller, and that can have integrated graphics according to embodiments of the application. Figures 7-10 The solid line boxes in illustrate a processor 600 with a single core 602A, system agent 610, set of one or more bus controller units 616, while the optional addition of the dashed line boxes illustrates an alternative processor 600 with multiple cores 602A-N (which can include one or more sets of caches 604A-N), one or more sets of integrated memory controller units 614 in the system agent unit 610, and specialized logic 608 (e.g., integrated graphics logic).

[0120] Thus, different implementations of the processor 600 can include: 1) a CPU with the special purpose logic 608 being integrated graphics and / or scientific (throughput) logic (which can include one or more cores), and the cores 602A-N being one or more general purpose cores (e.g., general purpose in-order cores, general purpose out-of-order cores, combinations thereof); 2) a coprocessor with the cores 602A-N being a large number of special purpose cores intended primarily for graphics and / or scientific (throughput); and 3) a coprocessor with the cores 602A-N being a large number of general purpose in-order cores. Thus, the processor 600 can be a general-purpose processor, coprocessor or a special-purpose processor, such as for example, a network or communication processor (NCP), compression engine, graphics processor, GPGPU (General Purpose Graphics Processing Unit), a High-Throughput (HT) integrated coprocessor (including 30 or more cores), an embedded processor, and the like. The processor can be implemented on one or more chips. The processor 600 can be part of one or more buses of a

[0121] The memory hierarchy includes one or more levels of cache within the cores, a set of shared cache units 606, and external memory coupled to the set of integrated memory controller units 614. The set of shared cache units 606 can include one or more levels of cache, such as Level 2 (L2), Level 3 (L3), Level 4 (L4) or other levels of cache, a last level cache (LLC), and / or combinations thereof. While in one embodiment, the ring-based interconnect unit 612 interconnects the special purpose logic 608, the set of shared cache units 606, and the system agent unit 610 / integrated memory controller unit(s) 614, alternative embodiments can use any number of well-known techniques for interconnecting such units. In one embodiment, coherency is maintained between the one or more shared cache units 606 and the cores 602A-N.

[0122] In some embodiments, one or more of the cores 602A-N are capable of multi-threading. The system agent 610 includes those components coordinating and operating cores 602A-N. The system agent unit 610 can include for example a power control unit (PCU) and a display unit. The PCU can be or include the logic and

[0123] The cores 602A-N can be homogeneous or heterogeneous in terms of architectural instruction sets; that is, two or more of the cores 602A-N can be capable of executing the same instruction set, while other cores can be capable of executing only a subset of that instruction set or a different instruction set.

[0124] Figure 7 is a block diagram of an exemplary computer architecture. Other system designs and configurations known in the arts for laptops, desktops, handheld PCs, personal digital assistants, engineering workstations, servers, network devices, network hubs, switches, embedded processors, digital signal processors (DSPs), graphics devices, video game devices, set-top boxes, micro controllers, cell phones, portable media players, hand held devices, and various other electronic devices, are also suitable. In general, a huge variety of systems or electronic devices capable of interfacing to a processor and / or other execution logic as disclosed herein are generally suitable.

[0125] Referring now to the Figure 7 , a block diagram of a system 700 according to one embodiment of the application is shown. The system 700 can include one or more processors 710, 715, which are coupled to a controller hub 720. In one embodiment the controller hub 720 includes a graphics memory controller hub (GMCH) 790 and an input / output hub (IOH) 750 (which can be on separate chips) or as described herein the memory and graphics controller are integrated within the processor and the memory 740 and co-processor 745 are coupled to the processor 710 directly and the controller hub 720 is on the same chip as the processor 710. The GMCH 790 includes memory and graphics controllers to which memory 740 and a co-processor 745 are coupled. The IOH 750 is coupled to the GMCH 790 and I / O devices 760. Alternatively, one or both of the memory and graphics controllers are integrated within the processor and the memory 740 and co-processor 745 are coupled directly to the processor 710 and the controller hub 720 is on the same chip as the processor 710.

[0126] The optional nature of additional processors 715 is denoted in Figure 8 by a dashed line. Each processor 710, 715 can include one or more of the processing cores described herein and can be some version of the processor 600.

[0127] The memory 740 can be, for example, a dynamic random access memory (DRAM) such as synchronous dynamic random access memory (SDRAM), a PCRAM, or a combination of such as these. For at least one embodiment, the controller hub 720 communicates with the processor(s) 710, 715 via a multi-limb bus, such as a frontside bus (FSB), a point-to-point interface such as a QuickPath Interconnect (QPI), or similar connection 795.

[0128] In one embodiment, coprocessor 745 is a special-purpose processor, such as, for example, a high-throughput MIC processor, network or communication processor, compression engine, graphics processor, GPGPU, embedded processor, etc. In one embodiment, controller hub 720 can include an integrated graphics processor.

[0129] There can be a variety of differences between the physical and operational characteristics of processors 710, 715.

[0130] In one embodiment, processor 710 executes instructions for processing tasks to generate operational control signals. Embedded within those instructions can be coprocessor instructions. The processor 710 recognizes these coprocessor instructions as being of a type that should be executed by a coprocessor 745. Accordingly, the processor 710 issues these coprocessor instructions (or control signals representing the coprocessor instructions) on a coprocessor bus or other interconnect, to coprocessor 745. Coprocessor(s) 745 accept and execute the received coprocessor instructions.

[0131] Referring now to the drawing figures in which is shown a block diagram of a first more specific exemplary system 800, in accordance with an embodiment of the present application. As Figure 8 shown in Figure 8 , multiprocessor system 800 is a point-to-point interconnect system, and includes a first processor 870 and a second processor 880 coupled via a point-to-point interconnect 850. Each of the processors 870, 880 can be some version of the processors 600. In one embodiment of the application, processors 870, 880 are processors 710, 715, respectively, while coprocessor 838 is coprocessor 745. In another embodiment, processors 870, 880 are processors 710, 745, respectively.

[0132] Processors 870, 880 are shown including integrated memory controller (IMC) units 872 and 882, respectively. Processor 870 also includes point-to-point (P-P) interfaces 876 and 878 as Figure 8 part of its bus controller units; similarly, second processor 880 includes P-P interfaces 886 and 888. Processors 870, 880 can exchange information via a point-to-point (P-P) interface 850 using P-P interface circuits 878, 888. As

[0133] Processors 870, 880 can each exchange information with a chipset 890 via individual P-P interfaces 852, 854 using point to point interface circuits 876, 894, 886, 898. Chipset 890 can optionally exchange information with a high-performance graphics processor 838, via a high-performance interface 839. In one embodiment, graphics processor 838 is a stand-alone processor that is not integrated into either processor 870 or 880. Chipset 890 can also exchange information with a network controller 841 via an high-performance network interface controller 840.

[0134] A shared cache (not shown) can be included in either processor or outside of both processors, connected via P-P interconnects, such that the local cache information for either or both processors can be stored in the shared cache if processors are placed into a low power mode.

[0135] Chipset 890 can be coupled to a first bus 816 via an interface 896. In one embodiment, first bus 816 can be a Peripheral Component Interconnect (PCI) bus, or a bus such as a PCI Express bus or another third generation I / O interconnect bus, although the scope of the application is not so limited.

[0136] As shown in Figure 8 , various I / O devices 814 can be coupled to first bus 816, along with a bus bridge 818, which couples first bus 816 to a second bus 820. In one embodiment, one or more additional processors 815 such as co-processors, high-speed MIC processors, GPGPU, accelerators, such as, for example, graphics accelerators or digital signal processing (DSP) units, field programmable gate arrays, or any other processor, can be coupled to first bus 816. In one embodiment, second bus 820 can be a low pin count (LPC) bus. Various devices can be coupled to second bus 820, shown in one embodiment as a memory slot 821 in which a memory unit 828, such as a SDRAM, can be coupled, and a bus bridge 822, an I / O device 823, and an audio I / O 824. Figure 9 Other architectures are possible. For example, instead of the point-to-point architecture of

[0137] Referring now to the drawing figures in which like reference numerals are used to Figure 8 identify like components throughout the drawing figures, there is shown in Figure 9 and 9 like elements using like reference numerals, and Figure 8 omitting Figure 9to avoid confusion Figure 9 other aspects of the application.

[0138] Figure 9 The processors 870, 880 are shown including integrated memory and I / O control logic ("CL") 872 and 882, respectively. Thus, the CL 872, 882 include integrated memory controller units and include I / O control logic. Figure 10 The memory 832, 2034 is shown coupled to the CL 872, 882, and I / O devices 914 are also coupled to the control logic 872, 882. Traditional I / O devices 915 are coupled to the chipset 890.

[0139] Referring now to Figure 6 A block diagram of a SoC 1000 according to an embodiment of the application is shown. Figure 10 Similar elements in FIGS. 1-9 are numbered similarly. Additionally, dashed lined boxes are optional features on more advanced SoCs. In Figure 8 In FIG. 10, interconnect units 1002 are coupled to: an application processor 1010 including a set of one or more cores 1002A-N and shared cache unit 1006; a system agent unit 1010; a bus controller unit 1016; an integrated memory controller unit(s) 1014; a set of one or more coprocessors 1020A-N, which can include integrated graphics logic, an image processor, an audio processor, and a video processor; a static random access memory (SRAM) unit 1030; a direct memory access (DMA) unit 1032; and a display unit 1040 for coupling to one or more external displays. In one embodiment, the coprocessor(s) 1020 include a special-purpose processor, such as, for example, a network or communication processor, compression engine, GPGPU, a high-throughput MIC processor, or embedded processor for audio, video, or

[0140] Embodiments of the mechanisms disclosed herein can be implemented in hardware, software, firmware, or combinations of such implementation. Embodiments of the application can be implemented as computer programs or program code executing on programmable systems comprising at least one processor, a storage system (including volatile and non-volatile memory and / or storage elements), at least one input device, and at least one output device.

[0141] Program code, such as code 1112 illustrated in FIG. 11, can be applied to input instructions to perform the functions described herein and generate output information. The output information can be applied to one or more output devices, in known fashion. Figure 11The code 830 shown in FIG. 8, when executed by the processing system, causes the processing system to perform various functions described herein and / or generates information for transmission or display. The code 830 can be written in any combination of one or more programming languages, including an object- oriented programming language, and / or a procedural programming language. The code 830 can execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer or server. In the latter scenario, the remote computer can be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or the connection can be made to an external computer (for example, through the Internet using an Internet Service Provider). In some embodiments, electronic circuitry including, for example, programmable logic circuitry, application specific circuitry, or field programmable gate array (FPGA) circuitry can execute the code 830 on the processing system. Generally, the code 830 can be implemented in hardware, software or a combination of hardware and software.

[0142] The program code can be implemented in a high level procedural or object oriented programming language to communicate with a processing system. The program code can also be implemented in assembly or machine language, if desired. In fact, the mechanisms described herein are not limited in scope to any particular programming language. In any case, the language can be a compiled or interpreted language.

[0143] One or more aspects of at least one embodiment can be implemented by firmware code executed in a machine accessible medium that directs the microprocessor to perform various functions as described herein. Such firmware code can be stored in a memory accessible medium, and can be supplied to various customers or manufacturing facilities to load into their manufacturing machines that fabricate the logic or microprocessor.

[0144] Such machine readable storage media can include, without limitation, non- transitory, tangible arrangements making up the article "program code" which is executed by a machine to perform functions described herein. For example, this can include recordable nonvolatile memory, such as optical, magnetic, or semiconductor memory such as Electrically Programmable Memory (EPROM), Electrically Erasable Programmable Memory (EEPROM), or flash memory. The above-listed memories are examples only, and other types of storage media that are readable by a machine and / or make up a tangible, non-transitory article of manufacture encompassed by "program code" can also be used. For example, instead of flash memory, a hard drive could be used, or other mass storage device.

[0145] Accordingly, embodiments of the present application also include non-transitory, tangible machine-readable media containing instructions or containing design data, such as Hardware Description Language (HDL), which defines structures, circuits, apparatuses, processors and / or system features described herein. Such embodiments can also be referred to as program products.

[0146] In some cases, an instruction translator can be used to translate instructions from a source instruction set to a target instruction set. For example, an instruction translator can transform (e.g., using static binary translation, dynamic binary translation including just-in-time compilation), morph, emulate, or otherwise convert instructions to one or more other instructions to be processed by a core. An instruction translator can be implemented in software, hardware, firmware, or combinations thereof. An instruction translator can be on a processor, off a processor, or partially on and partially off a processor.

[0147] Figure 11 is a block diagram of translating a binary instruction in a source instruction set to a binary instruction in a target instruction set in accordance with an embodiment of the present application using a software instruction translator. In the illustrated embodiment, the instruction translator is a software instruction translator, but the instruction translator can alternatively be implemented in software, firmware, hardware, or various combinations thereof. Figure 11 It is shown that a program in a high-level language 1102 can be compiled using an x86 compiler 1104 to generate x86 binary code 1106, which can be natively executed by a processor 1116 having at least one x86 instruction set core. A processor 1116 having at least one x86 instruction set core refers to any processor that Additional computing systems - overviewIt is shown that a program in the form of high-level language 1102 can be compiled using alternative instruction set compiler 1108 to generate alternative instruction set binary code 1110 that can be natively executed by a processor 1114 that does not have at least one x86 instruction set core (e.g., a processor with cores that execute the MIPS instruction set by MIPS Technologies of Sunnyvale, California, and / or that execute the ARM instruction set by ARM Holdings of Sunnyvale, California). Instruction translator 1112 is used to translate x86 binary code 1106 into code that can be natively executed by processor 1114 that does not have x86 instruction set cores. The translated code would not likely be the same as alternative instruction set binary code 1110 because an instruction translator capable of doing so would be difficult to make; however, the translated code would accomplish the general operation and be made up of instructions from the alternative instruction set. Thus, instruction translator 1112 represents the software, firmware, hardware, or combination thereof, to emulate, simulate or otherwise portray the original x86 binary code 1106 to the processor or other electronic device that does not have x86 instruction set processor or core. Figure 12

[0148] Figure 12 FIG. 12 is a block diagram illustrating a computing system 1200 configured to implement one or more aspects of the example embodiments described herein. The computing system 1200 includes a processing subsystem 1201 with one or more processor(s) 1202 and a system memory 1204 communicating via an interconnection path 1205 that can include a memory bus 1205. The memory bus 1205 can be in the form of a single bus, a combination of busses, or another interconnection medium. The memory bus 1205 can use one or more bus standards, including a PCI bus, a HyperTransport® bus, an industry standard architecture (ISA) bus, and the like.

[0149] In one embodiment, processing subsystem 1201 includes one or more parallel processor(s) 1212 coupled to memory hub 1205 via a bus or other communication link 1213. The communication link 1213 can be one of any number of standards-based communication links, such as a PCI Express, or can be a supplier- proprietary communication interface or communication structure. In one embodiment, one or more parallel processor(s) 1212 form a computationally-intensive, parallel, or vector processing system that can include a number of processing cores and / or a number of processing clusters, such as a many integrated core (MIC) processor. In one embodiment, one or more parallel processor(s) 1212 form a graphics processing subsystem that can output pixels to one or more display device(s) 1210A coupled via I / O hub 1207. One or more parallel processor(s) 1212 can also include a display controller(s) and a display interface (not shown) to enable a direct connection to one or more display device(s) 1210B.

[0150] Within I / O subsystem 1211, system storage unit 1214 can connect to I / O hub 1207 to provide storage

[0151] Computing system 1200 can include other components not explicitly shown, including USB or other port connections, an optical storage device, a video capture device, etc., which can also be connected to I / O hub 1207. The communication paths Figure 12 The communication paths among the components of computing system 1200 can use any suitable protocols, such as PCI (Peripheral Component Interconnect) based protocols (e.g., PCI-Express), or any other bus or point-to-point communication interfaces and / or protocols, such as NV-Link high-speed interconnect, or interconnect protocols known in the art.

[0152] In one embodiment, the one or more parallel processor(s) 1212 include circuitry optimized for graphics and video processing, including for example video output circuitry, and are configured for use in a gaming console, a personal computer, or other system. In another embodiment, the one or more parallel processor(s) 1212 include circuitry optimized for general use applications

[0153] It will be appreciated that the computing system 1200 illustrated herein is illustrative and that variations and modifications are possible. The connection topology, including the number and arrangement of bridges, the number and

[0154] Some of the particular components shown herein are optional and not included in all implementations of the computing system 1200. For example, any number of add-in cards or peripherals can be supported, or some components can be eliminated. Additionally, some architectures can use different terminology for components similar to those illustrated in FIG. 12A and 12B. For example, in some architectures a memory hub 1205 can be called a northbridge and an I / O hub 1207 can be called a southbridge. Figure 13A

[0155] Figure 12 ​Figure illustrates a parallel processor 1300, according to an embodiment. The components of parallel processor 1300 may Figure 13B variations of one or more parallel processors 1212 shown in FIG. 12B.

[0156] In one embodiment, parallel processor 1300 includes a parallel processing unit 1302. Parallel processing unit includes an I / O unit 1304 that enables communication with other devices, including other instances of parallel processing unit 1302. In one embodiment, I / O unit 1304 is directly connected to the other

[0157] When host interface 1306 receives a command buffer via I / O unit 1304, host interface 1306 can direct the work operations for executing those commands to front end 1308. In one embodiment, front end 1308 is coupled with scheduler 1310, which is configured to dispatch the commands or other work items to processing cluster array 1312. In one embodiment, scheduler 1310 ensures that processing cluster array 1312 is properly configured and in an active state before tasks are dispatched to processing clusters within processing cluster array 1312. In one embodiment, scheduler 1310 is implemented via firmware logic executing on a microcontroller. The microcontroller implemented scheduler 1310 can be configured to perform complex scheduling and work distribution operations at a coarse and fine granularity, enabling implementation of a wide range of parallel processing algorithms without support from host software. In one embodiment, host software can verify the workload for scheduling on processing cluster array 1312 via one of a number of graphics processing doorbell mechanisms. The workload can then be automatically distributed by scheduler 1310 logic within the scheduler microcontroller across the processing cluster array 1312.

[0158] The processing cluster array 1312 can include up to "N" processing clusters (e.g., cluster 1314A, 1314B, through 1314N). Each cluster 1314A-1314N in the processing cluster array 1312 can execute a plurality of concurrent threads. The scheduler 1310 can allocate work to the clusters 1314A-1314N of the processing cluster array 1312 using various scheduling and / or work distribution algorithms. The scheduling can be dynamic and can be handled by the scheduler 1310 or can be assisted in part by compiler logic during compilation of program logic configured for execution by the processing cluster array 1312. In one embodiment, different clusters 1314A-1314N of the processing cluster array 1312 can be allocated for processing different types of programs or for performing different types of computations.

[0159] The processing cluster array 1312 can be configured to perform a wide variety of parallel processing operations. In one embodiment, the processing cluster array 1312 is configured to perform general-purpose parallel compute operations. For example, the processing cluster array 1312 can include logic to perform processing tasks comprising filtering of video and / or audio data, performing modeling operations (including physics operations), and performing data transformations.

[0160] In one embodiment, the processing cluster array 1312 is configured to perform parallel graphics processing operations. In an embodiment in which the parallel processor 1300 is configured to perform graphics processing operations, the processing cluster array 1312 can include additional logic to support the execution of such graphics processing operations, including without limitation texture sampling logic to perform texture parameterization operations, and surface rendering logic and other vertex processing logic. In addition, the processing cluster array 1312 can be configured to execute shader programs, including, but not limited to, vertex shaders, tessellation shaders, geometry shaders, and pixel

[0161] In one embodiment, when using parallel processing unit 1302 to perform graphics processing, scheduler 1310 can be configured to divide processing workloads into approximately equal sized tasks to better enable distribution of graphics processing operations onto a plurality of clusters 1314A-1314N in processing cluster array 1312. In some embodiments, portions of processing cluster array 1312 can be configured to perform different types of processing. For example, a first portion can be configured to perform vertex shading and topology generation, a second portion can be configured to perform surface

[0162] During operation, processing cluster array 1312 can receive processing tasks to be executed via scheduler 1310, which receives commands defining the processing tasks from front end 1308. For graphics processing operations, the processing tasks can comprise indices of commands and state parameters defining how data is to be processed, e.g., what programs are to be executed, as well as the data to be processed, e.g., surface (patch) data, primitive data, vertex data, and / or pixel data. Scheduler 1310 can be configured to fetch indices corresponding to tasks, or can receive the indices from front end 1308. Front end 1308 can be configured to ensure that processing cluster array 1312 is configured in an effective state prior to a workload specified by an incoming command buffer (e.g., a batch buffer, a push buffer, etc.) being initiated.

[0163] Each of the one or more instances of parallel processing unit 1302 can be coupled with a parallel processor memory 1322. Parallel processor memory 1322 can be accessed by the processing clusters 1312 of the processing array 1312, as well as the I / O units 1304, via a memory crossbar 1316. The memory crossbar 1316 can receive memory requests from the processing clusters 1312 and the I / O units 1304 and sfransfer the memory requests to the parallel processor memory 1322. The memory crossbar 1316 can also include routing logic to sfransfer the memory requests to the appropriate memory unit of the parallel processor memory 1322. The parallel processor memory 1322 can include a number of memory units 1324A, 1324B, through 1324N. In one embodiment, the number of memory units 1324A-1324N can be equal to the number of partitions of the partition units 1320A-1320N. The memory units 1324A-1324N can store frame buffers, one or more of which can be assigned to each partition unit 1320A-1320N.

[0164] In embodiments, the memory units 1324A-1324N can include various types of memory devices including dynamic random access memory (DRAM) or graphics random access memory, such as synchronous graphics random access memory (SGRAM), including graphics double data rate (GDDR) memory. In one embodiment, the memory units 1324A-1324N can also include 3D stacked memory, including but not limited to high bandwidth memory (HBM). As will be appreciated, the specific implementation of the memory units 1324A-1324N can vary and can be selected from one of a variety of conventional designs. Render targets, such as frame buffers or texture maps, can be stored across the memory units 1324A-1324N, allowing the partition units 1320A-1320N to write portions of each render target in parallel for efficient use of the available bandwidth of the parallel processor memory 1322. In some embodiments, the local instance of the parallel processor memory 1322 can be excluded to facilitate a unified memory design that utilizes system memory in combination with local cache memory.

[0165] In one embodiment, any of the clusters 1314A-1314N in the processing cluster array 1312 can process data that is to be written into any of the memory units 1324A-1324N within the parallel processor memory 1322. The memory crossbar 1316 can be configured to transfer the output of each cluster 1314A-1314N to any of a wide variety of destination units 1320A-1320N or to another cluster 1314A-1314N that can perform additional processing operations on the output. Each cluster 1314A-1314N can communicate with the memory interface 1318 through the memory crossbar 1316 to perform read or write operations against various external memory devices. In one embodiment, the memory crossbar 1316 has a connection to the memory interface 1318 to communicate with the I / O unit 1304, and a connection to a local instance of the parallel processor memory 1322 enabling processing clusters 1314A-1314N within the different processing clusters 1314A-1314N to communicate with system memory or other memories not local to the parallel processor 1302. In one embodiment, the memory crossbar 1316 can use virtual channels to separate traffic streams between the clusters 1314A-1314N and the partition units 1320A-1320N.

[0166] Although a single instance of the parallel processor 1302 is shown in the parallel processor 1300, any number of instances of the parallel processor 1302 can be included. For example, a plurality of instances of the parallel processor 1302 can be provided on a single add-in card, or multiple add-in cards can be interconnected. The different instances of the parallel processor 1302 can be configured to inter-operate, even if the different instances have different numbers of processing cores, different amounts of local parallel processor memory, and / or other configuration differences.

[0167] Figure 13A is a block diagram of a partition unit 1320 according to an embodiment. In one embodiment, the partition unit 1320 is a Figure 13Aan instance of one of the partition units 1320A-1320N. As illustrated, partition unit 1320 includes an L2 cache 1321, a frame buffer interface 1325, and a ROP (Raster Operations unit) 1326. The L2 cache 1321 is a read / write cache memory configured to perform load and store operations received from the memory crossbar 1316 and the ROP 1326. Read misses and snoop misses are output to the frame buffer interface 1325 for processing. Updates are also sent to the frame buffer via the frame buffer interface 1325 for processing. In one embodiment, the frame buffer interface 1325 interfaces with one of the memory units in the parallel processor's memory, such as the memory unit 1324A-1324N, for example, within the parallel processor's memory 1322. Figure 13A

[0168] In graphics applications, the ROP 1326 is a processing unit that performs raster operations, such as stencil, z-test, blending, and so forth. The ROP 1326 then outputs processed graphics data that is stored in graphics memory. In some embodiments, the ROP 1326 includes compression logic to compress depth or color data that is written to memory and decompress depth or color data that is read from memory. The compression logic can be lossless compression logic that utilizes one or more of a variety of compression algorithms. The type of compression that is carried out by the ROP 1326 can vary based on the statistical characteristics of the data to be compressed. For example, in one embodiment, delta color compression is performed on depth and color data on a tile-by-tile basis.

[0169] In some embodiments, the ROP 1326 is included within each processing cluster (e.g., clusters 1314A-1314N) rather than in the partition unit 1320. In such embodiments, read and write requests for pixel data are transmitted over the memory crossbar 1316 rather than pixel fragment data. The processed graphics data can be displayed on a display device, such as one of the one or more display devices 1210, routed to the processor 1202 for further processing, or routed to one of the processing entities within the parallel processor 1300 for further processing. Figure 12 Figure 13A Figure 13C

[0170] Figure 13A is a block diagram of a processing cluster 1314 within a parallel processing unit according to an embodiment. In one embodiment, the processing cluster is a Figure 13A ​​​​an instance of one of the processing clusters 1314A-1314N. The processing clusters 1314 can be configured to perform a large number of threads in parallel, with the term "thread" referring to an instance of a particular program executing on a particular set of input data. In some embodiments, single-instruction, multiple-data (SIMD) instruction issue techniques are used to support parallel execution of a large number of threads with no or minimal support from the operating system or compiler. In other embodiments, single-instruction, multiple-thread (SIMT) techniques are used to support parallel execution of a large number of generally synchronous threads with or without hardware support, for example using a common instruction unit configured to issue instructions to a set of processing engines within each of the processing clusters. Unlike SIMD execution mechanisms in which all processing engines typically execute identical instructions, SIMT execution allows different threads to follow divergent execution paths through a given thread program. Those skilled in the art will appreciate the functional differences between SIMD and SIMT execution mechanisms.

[0171] The operation of the processing clusters 1314 can be controlled via a pipeline manager 1332 that distributes processing tasks to SIMT parallel processors. The pipeline manager 1332 receives instructions from the scheduler 1310 of the processing cluster 1300, and manages execution of those instructions via a graphics multiprocessor 1334 and / or a texture unit 1336. The illustrated graphics multiprocessor 1334 is an exemplary instance of a SIMT parallel processor. However, various types of SIMT parallel processors of differing architectures can be included within the processing cluster 1314. One or more instances of the graphics multiprocessor 1334 can be included within a processing cluster 1314. The graphics multiprocessor 1334 can process data and a data crossbar 1340 can be used to distribute the processed data to one of a number of possible destinations, including other shader units. The pipeline manager 1332 can facilitate the distribution of processed data by specifying destinations for processed data to be distributed via the data crossbar 1340. Figure 13A

[0172] Each graphics multiprocessor 1334 within the processing cluster 1314 can include the same functional execution logic (e.g., arithmetic logic units, load-store units, etc.). The functional execution logic can be configured in a pipelined manner in which new instructions can be issued before previous instructions are complete. The functional execution logic supports a variety of operations including integer and floating point arithmetic, comparison operations, Boolean operations, bit shifts, and compute operations. In one embodiment, different operations can be initiated in different ones of the functional execution units 1338. For example, one functional execution unit 1338 can be configured to perform integer

[0173] ​Instructions transmitted to processing cluster 1314 constitute a thread. A set of threads executing across a set of parallel processing engines is a thread group. A thread group performs the same program for different input data sets. Each thread within the thread group can be assigned to a different processing engine within graphics multiprocessor 1334. A thread group can include fewer threads than there are processing engines within graphics multiprocessor 1334. When a thread group includes fewer threads than the number of processing engines within graphics multiprocessor 1334, one or more of the processing engines can be idle during the period that the thread group is processing a cycle. A thread group can also include more threads than there are processing engines within graphics multiprocessor 1334. When a thread group includes more threads than the number of processing engines within graphics multiprocessor 1334, multiple threads can be executed by a single processing engine during one or more cycles. In one embodiment, multiple

[0174] In one embodiment, graphics multiprocessor 1334 includes an internal cache memory to perform load and store operations. In one embodiment, graphics multiprocessor 1334 can bypass an internal cache and use register file 1350 within processing cluster 1314. Each graphics multiprocessor 1334 also has access to L2 cache 1320A-1320N within partition unit 1320A-1320N, which is shared among all processing clusters 1314 and can be used to store data used by thread groups between threads. Graphics multiprocessor 1334 also has access to off-chip global memory, which can include one or more of local parallel processor memory and / or system memory. Any memory external to parallel processing unit 1302 can be accessed by graphics multiprocessor 1334 as global memory. Embodiments where processing cluster 1314 includes multiple instances of graphics multiprocessor 1334 can share common instructions and data, which can be stored in Ll cache 1348. Figure 13A

[0175] Each processing cluster 1314 can include an MMU 1345 (memory management unit). In other embodiments, one or more instances of MMU 1345 can reside within graphics multiprocessor 1334. In general, an MMU 1345 provides an interface between a processor (e.g., graphics multiprocessor 1334) and a memory, translating virtual addresses in memory requests to physical addresses used by a bus (e.g., bus 1316). During the translation, the MMU 1345 can perform virtual-to-physical Figure 13A ​The MMU 1345 is coupled to the processor's internal memory management and paging interfaces and is configured to handle addresses translated between in-memory addresses to physical addresses. The MMU 1345 also includes a memory management unit for a frame buffer, which is configured to provide improved data throughput between the memory interface 1318 and the graphics processor's cache memory. The MMU 1345 translates virtual addresses into physical addresses as instructions are passed through the pipeline. In one embodiment, the MMU 1345 includes address translation lookaside buffers (TLB) for passing dual address translations to the graphics processor's execution units. The MMU 1345 also provides a set of registers for machine state management and set of registers for set of interrupt controllers management.

[0176] In graphics and compute applications, the processing cluster 1314 can be configured such that each graphics multi-processor 1334 is coupled to a texture unit 1336 for performing texture mapping operations, e.g., determining texture sample positions, reading texture data, and filtering texture data. Texture data is read from an internal texture Ll cache (not shown) or in some embodiments, from an Ll cache within the graphics multi-processor 1334, and fetched from an L2 cache, local parallel processor memory, or system memory, as needed. Each graphics multi-processor 1334 outputs processed tasks to a data crossbar 1340 in order to provide the output to another processing cluster 1314 for further processing or to store the output in an L2 cache, local parallel processor memory or system memory via a memory crossbar 1316. A preROP 1342 (pre-raster operations unit) is configured to receive data from the graphics multi-processor 1334, direct data to a ROP unit which can be located within the processing cluster 1314 or within the GPC as described herein (e.g., within a partition unit 1320A-1320N of the GPC), in conjunction with a partition unit (e.g., partition unit 1320A-1320N) as described herein. The preROP 1342 unit can perform optimizations for color blending, organize pixel color data, and perform address translation. Figure 13D

[0177] It will be appreciated that the core architecture described herein is illustrative and variation and modification are possible. The number of processing units (e.g., graphics multi-processor 1334, texture unit 1336, preROP 1342, etc.) can be increased or decreased, and the illustrated depiction of each processing unit as a single processing unit is for illustration only. Further, while only one processing cluster 1314 is illustrated, any number of processing clusters 1314 can be included in the parallel processing unit as described herein. In one embodiment, each processing cluster 1314 can be configured to operate independently of the other processing clusters 1314 using separate and distinct processing units, Ll caches, etc.

[0178] Figures 14A-14B ​A graphics multiprocessor 1334 according to one embodiment is shown. In such embodiments, the graphics multiprocessor 1334 is coupled with a pipeline manager 1332 of the processing cluster 1314. The graphics multiprocessor 1334 has an execution pipeline that includes, in one embodiment, an instruction cache 1352, an instruction unit 1354, an address mapping unit 1356, a register file 1358, one or more general-purpose GPU (GPGPU) cores 1362, and one or more load / store units 1366. The GPGPU cores 1362 and load / store units 1366 are coupled with a shared memory 1370 and caches 1372 via a memory and cache interconnect 1368.

[0179] In one embodiment, the instruction cache 1352 receives a stream of instructions to execute from the pipeline manager 1332. The instructions are cached in the instruction cache 1352 and dispatched for execution by the instruction unit 1354. The instruction unit 1354 can dispatch the instructions as threads to different ones of the GPGPU cores 1362, where each thread is assigned a different one of the GPGPU cores 1362. The instructions can access the local, shared, or global address spaces defined by the executing program to retrieve data or store data. The address mapping unit 1356 can translate an address used by an instruction into a different address used to access a memory as defined by the type of address space identified in the instruction.

[0180] The register file 1358 provides register storage for functions executed by the graphics multiprocessor 1334. The register file 1358 can include a set of registers corresponding to the functional units of the graphics multiprocessor 1334 (e.g., GPGPU cores 1362, load / store units 1366). In one embodiment, the register file 1358 is divided into a set of registers reserved for atomic operations, which are available to the GPGPU cores 1362. In one embodiment, the register file 1358 is divided into a set of registers reserved for use only by the load / store units 1366. In one embodiment, the register file 1358 is allocated in a manner based on the needs of the underlying hardware of the graphics multiprocessor 1334.

[0181] The GPGPU cores 1362 can each include floating point, integer, and / or single instruction multiple data (SIMD) execution units supporting multiple data sizes. For example, the GPGPU cores 1362 can support 128-bit, 256-bit, and 512-bit data processing capabilities. The GPGPU cores 1362 can also include support for double precision and multiple precision floating point arithmetic units. In one embodiment, the GPGPU cores 1362 can execute single precision (32-bit) and double precision (64-bit) arithmetic, which can be used for complex matrix and vector operations or for other operations.

[0182] In one embodiment, the GPGPU cores 1362 include SIMD execution units 1368 to perform SIMD 4, SIMD 8, and SIMD 16 instructions. In one embodiment, the GPGPU cores 1362 can physically execute SIMD 4, SIMD 8, and SIMD 16 instructions and logically execute a SIMD 1, SIMD 2, and SIMD 32 instructions via micro- partitioning. In one embodiment, SIMD execution units 1368 can be configured to execute multiple execution lanes of 16-bit integer, 32-bit single and double precision floating point operations independently.

[0183] The memory and cache interconnect 1368 is an interconnect network that connects each of the functional units of graphics multiprocessor 1334 to the register file 1358 and to shared memory 1370. In one embodiment, the memory and cache interconnect 1368 is a crossbar switch interconnect that allows load / store units 1366 to implement load and store operations between shared memory 1370 and register file 1358. Register file 1358 is able to operate at the same frequency as GPGPU cores 1362, so that data transfers between GPGPU cores 1362 and register file 1358 are very low latency. Shared memory 1370 can be used to implement communications between threads executing on functional units within graphics multiprocessor 1334. Cache memory 1372 can be used as a cache for data stored in shared memory 1370, for example, to cache texture data transmitted between functional units and texture unit 1336. Shared memory 1370 can also be used to cache managed programs that are cached. In addition to automatically cached data stored within cache memory 1372, threads executing on GPGPU cores 1362 can store data in shared memory in a programmed manner.

[0184] Figure 13C An additional graphics multiprocessor is illustrated in accordance with an embodiment. The illustrated graphics multiprocessors 1425, 1450 are variants of graphics multiprocessor 1334 of Figure 14A FIG. 13A. The illustrated graphics multiprocessors 1425, 1450 can be configured as a streaming multiprocessor (SM) capable of executing a large number of execution threads concurrently.

[0185] Figure 13D A graphics multiprocessor 1425 in accordance with an additional embodiment is shown. Relative to graphics multiprocessor 1334 of Figure 14B FIG. 13A, graphics multiprocessor 1425 includes a number of additional instances of execution resource units. For example, graphics multiprocessor 1425 can include multiple instances of instruction units 1432A-1432B, register files 1434A-1434B, and texture unit(s) 1444A-1444B. Graphics multiprocessor 1425 also includes multiple sets of graphics or compute execution units (e.g., GPGPU cores 1436A-1436B, GPGPU cores 1437A-1437B, GPGPU cores 1438A-1438B) and multiple sets of load / store units 1440A-1440B. In one embodiment, the execution resource units have a common instruction cache 1430, texture and / or data cache memory 1442, and shared memory 1446.

[0186] The components can communicate via an interconnect fabric 1427. In one embodiment, the interconnect fabric 1427 includes one or more crossbars to enable communication between components of the graphics processing unit 1425. In one embodiment, the interconnect fabric 1427 is a separate, high-speed network structure layer on which each component of the graphics processing unit 1425 is stacked. The components of the graphics processing unit 1425 communicate with remote components via the interconnect fabric 1427. For example, the GPGPU cores 1436A-1436B, 1437A-1437B, and 1478A-1438B can each communicate with the shared memory 1446 via the interconnect fabric 1427. The interconnect fabric 1427 can arbitrate communications among the components within the graphics processing unit 1425 to ensure fair bandwidth allocation.

[0187] Figure 13D A graphics processing unit 1450 according to an additional embodiment is shown. The graphics processor includes a number of sets of execution resources 1456A-1456D, where each set of execution resources includes a number of instruction units, register file, GPGPU cores, and load store units, as illustrated in Figure 14A and Figure 14A In one embodiment, the execution resources 1456A-1456D can share a number of instances of the instruction cache 1454 and shared memory 1453 as well as the texture and / or data cache memory 1458A-1458B. The components can communicate via an interconnect fabric 1452 similar to the interconnect fabric 1427 of Figure 12 .

[0188] Those of skill in the art will understand that the architecture described in Figures 13A-13D , Figures 14A-14B and Figure 13A is descriptive and does not limit the scope of the embodiments. Thus, the techniques described herein can be implemented on any suitably configured processing unit, including but not limited to one or more mobile application processors, one or more desktop or server central processing units (CPUs) (including multi-core CPUs), one or more parallel processing units such as the parallel processing unit 1302 of Techniques for GPU-to-host processor interconnect , and one or more graphics processors or specialized processing units.

[0189] In some embodiments, the parallel processor or GPGPU described herein is communicatively coupled to a host / processor core to accelerate graphics operations, machine learning operations, pattern analysis operations, and various general purpose GPU (GPGPU) functions. The GPU can be communicatively coupled to the host processor / cores over a bus or other interconnect (e.g., a high-speed Figure 15A

[0190] Figure 15A FIG. 13 illustrates an exemplary architecture in which multiple GPUs 1510-1513 are communicatively coupled to multiple multi-core processors 1505-1506 over high-speed links 1540A-1540D (e.g., buses, point-to-point interconnects, etc.). In one embodiment, high-speed links 1540A-1540D support communication throughput of 4GB / s, 30GB / s, 80GB / s or higher, depending on implementation. Various interconnect protocols can be used including, but not limited to, PCIe 4.0 or 5.0 and NVLink 2.0. However, the underlying principles of the application are not limited to any particular communication protocol or throughput.

[0191] In addition, in one embodiment, two or more of GPUs 1510-1513 are interconnected over high-speed links 1542A-1542B, which can be implemented using the same or different protocols / links than those used for high-speed links 1540A-1540D. Similarly, two or more of multi-core processors 1505-1506 can be connected by high-speed link 1543, which can be an SMP bus operating at 20GB / s, 30GB / s, 120GB / s or higher. Alternatively, Figure 15B All communication between the various system components illustrated in FIG. 13 can be accomplished using the same protocol / links (e.g., over common interconnects). As mentioned, however, the underlying principles of the application are not limited to any particular type of interconnect technology.

[0192] In one embodiment, each multi-core processor 1505-1506 is communicatively coupled to processor memories 1501-1502 via memory interconnects 1530A-1530B, respectively, and each GPU 1510-1513 is communicatively coupled to GPU memories 1520-1523 by GPU memory interconnects 1550A-1550D, respectively. Memory interconnects 1530A-1530B and 1550A-1550D can utilize the same or different memory access technologies. As an example and not by way of limitation, processor memories 1501-1502 and GPU memories 1520-1523 can be volatile memory such as dynamic random access memory (DRAM), including stacked DRAM, graphics DDR SDRAM (GDDR) such as GDDR5, GDDR6, or high bandwidth memory (HBM), and / or can be non-volatile memory such as 3D XPoint or Nano-Ram. In one embodiment, some portion of these memories can be volatile memory and another portion can be non-volatile memory (e.g., using a two-level memory (2LM) hierarchy).

[0193] As described below, although processors 1505-1506 and GPUs 1510-1513 can be physically coupled to particular memories 1501-1502, 1520-1523, respectively, a unified memory architecture can be implemented in which the same virtual system address space (also referred to as “effective address” space) is distributed across all of the various physical memories. For example, processor memories 1501-1502 can each include 64 GB of system memory address space, and GPU memories 1520-1523 can each include 32 GB of system memory address space (resulting in a total of 256 GB of addressable memory in this example).

[0194] Figure 15C FIG. 16 illustrates additional details of the interconnect between multi-core processor 1507 and graphics acceleration module 1546, according to one embodiment. Graphics acceleration module 1546 can include one or more GPU chips integrated on a line card that is coupled with processor 1507 via high-speed link 1540. Alternatively, graphics acceleration module 1546 can be integrated on the same package or chip as processor 1507.

[0195] The illustrated processor 1507 includes multiple cores 1560A-1560D, each including a translation lookaside buffer 1561 A-1561 D and one or more caches 1562A-1562D. The cores can include various other components for executing instructions and processing data, which are not illustrated to avoid obscuring the essence of the application (e.g., instruction fetch unit, branch prediction unit, decoders, execution units, reorder buffer, etc.). The caches 1562A-1562D can include level one (LI) and level two (L2) caches. Additionally, one or more shared caches 1556 can be included in the cache hierarchy and shared by the core set 1560A-1560D. For example, one embodiment of the processor 1507 includes 24 cores, each with its own LI cache, twelve shared L2 caches, and twelve shared L3 caches. In this embodiment, one of the L2 and L3 caches are shared by two adjacent cores. The processor 1507 and graphics acceleration module 1546 are connected to system memory 1541, which can include processor memory 1501-1502.

[0196] Consistency for data and instructions stored in the caches 1562A-1562D, 1556, and system memory 1541 is maintained via inter-core communication over the coherence bus 1564. For example, each cache can have cache coherency logic / circuitry associated therewith to communicate over the coherence bus 1564 in response to a detected read or write of a particular cache line. In one implementation, a snoop protocol is implemented over the coherence bus 1564 to snoop cache accesses. Cache snoop / coherency techniques are well understood by those of ordinary skill in the art and will not be described in detail herein to avoid obscuring the essence of the application.

[0197] In one embodiment, the proxy circuit 1525 communicatively couples the graphics acceleration module 1546 to the coherence bus 1564, allowing the graphics acceleration module 1546 to participate in the cache coherence protocol as a peer to the cores. Specifically, the interface 1535 provides connectivity to the proxy circuit 1525 over a high-speed link 1540 (e.g., a PCIe bus, NVLink, etc.), and the interface 1537 connects the graphics acceleration module 1546 to the link 1540.

[0198] In one implementation, the accelerator integration circuit 1536 provides cache management, memory access, context management, and interrupt management services on behalf of the graphics processing engines 1531, 1532, N of the graphics processing module 1546. The graphics processing engines 1531, 1532, N can each comprise a separate graphics processing unit (GPU). Alternatively, the graphics processing engines 1531, 1532, N can comprise different types of graphics processing engines within a GPU such as graphics execution units, media processing engines (e.g., video encoders / decoders), samplers, and blit engines. In other words, the graphics processing module can be a GPU with a plurality of graphics processing engines 1531-1532, N or the graphics processing engines 1531-1532, N can be individual GPUs integrated on a common package, line card, or chip.

[0199] In one embodiment, the accelerator integration circuit 1536 includes a memory management unit (MMU) 1539 to translate virtual addresses into physical addresses, supply memory protection, and provide memory caching. In one embodiment, the MMU 1539 includes memory partitions to provide and / or isolate various memory resources. In one embodiment, the MMU 1539 includes a translation lookaside buffer (TLB) to perform translation of virtual addresses into physical addresses, which can be accessed by the graphics processing engines 1531-1532, N. In one embodiment, the MMU 1539 includes a memory management transaction queue to stage memory requests. In one embodiment, the MMU 1539 includes a memory protection unit to enforce memory protections.

[0200] The register set 1545 stores context data for threads executed by the graphics processing engines 1531-1532, N, and the context management circuit 1548 manages these thread contexts. For example, the context management circuit 1548 can perform save and restore operations for saving and restoring the context of each thread during a context switch (e.g., where a first thread is saved and a second thread is stored so that the second thread can be executed by the graphics processing engines). For example, upon context switch, the context management circuit 1548 can store the current register values into a designated area in memory (e.g., identified by a context pointer). When returning to this context, it can then restore the register values. In one embodiment, the interrupt management circuit 1547 receives and processes interrupts received from system devices.

[0201] In one implementation, virtual / effective addresses from the graphics processing engines 1531 are translated to real / physical addresses in system memory 1511 by the MMU 1539. One embodiment of the accelerator integration circuit 1536 supports multiple (e.g., 4, 8, 16) graphics processing modules 1546 and / or other accelerator devices. The graphics processing modules 446 can be dedicated to a single application executing on the processor 1507 or can be shared between multiple applications. In one embodiment, there is a virtualized graphics execution environment in which resources of the graphics processing engines 1531-1532, N are shared between multiple applications or virtual machines (VMs). The resources can be subdivided into “slices” that are allocated to different VMs and / or applications based on processing requirements and priority associated with the VMs and / or applications.

[0202] Thus, the accelerator integration circuit functions as a bridge to the system for the graphics processing module 1546 and provides address translation and system memory cache services. In addition, the accelerator integration circuit 1536 can provide virtualization facilities for a host processor, to manage virtualization of graphics processing engines, interrupts, and memory management.

[0203] Because the hardware resources of the graphics processing engines 1531-1532, N are explicitly mapped to the real address space viewed by the host processor 1507, any host processor can address these resources directly using effective address values. In one embodiment, one function of the accelerator integration circuit 1536 is to physically segregate the graphics processing engines 1531-1532, N so that they appear to the system as independent units.

[0204] As noted, in the illustrated embodiment, one or more graphics memory 1533-1534, M is coupled to each of the graphics processing engines 1531-1532, N, respectively. The graphics memory 1533-1534, M stores instructions and data for processing by each of the graphics processing engines 1531-1532, N. The graphics memory 1533-1534, M can be a volatile memory such as DRAM (including stacked DRAM), GDDR memory (e.g., GDDR5, GDDR6), or HBM, and / or can be a non-volatile memory such as 3D XPoint or Nano-Ram.

[0205] In one embodiment, to reduce the amount of data traffic on the link 1540, a biasing technique is used to ensure that data stored in the graphics memory 1533-1534, M is data that will be most frequently used by the graphics processing engines 1531-1532, N and preferably not used by the cores 1560A-1560D (at least not frequently). Similarly, the biasing mechanism attempts to keep data required by the cores (and preferably not the graphics processing engines 1531-1532, N) within the caches 1562A-1562D, 1556 and system memory 1511 of those cores.

[0206] Figure 15B Another embodiment is illustrated in which the accelerator integration circuit 1536 is integrated within the processor 1507. In this embodiment, the graphics processing engines 1531-1532, N communicate directly over high-speed link 1540 to the accelerator integration circuit 1536 via the interface 1537 and interface 1535 (which can likewise be utilized for communication between processor 1507 and accelerator integration circuit 1536). The accelerator integration circuit 1536 can execute same operations as those described with reference to Figure 15D those described, but with the consideration that the accelerator integration circuit 1536 is in close proximity to the coherent bus 1564 and caches 1562A-1562D, 1556, it potentially can perform operations at a higher throughput.

[0207] One embodiment supports different programming models, including a dedicated process programming model (no graphics acceleration module virtualization) and a shared programming model (with virtualization). The latter can include a programming model controlled by the accelerator integration circuit 1536 as well as a programming model controlled by the graphics acceleration module 1546.

[0208] In one embodiment of the dedicated process model, the graphics processing engines 1531-1532, N are dedicated to a single application or process under a single operating system. The single application can funnel other application requests to the graphics processing engines 1531-1532, N, providing virtualization within the VM / partition.

[0209] In a dedicated process programming model, graphics processing engines 1531-1532, N can be shared by multiple VM / application partitions. The shared model requires the system hypervisor to virtualize the graphics processing engines 1531-1532, N to allow access by each operating system. For a single-partition system without a hypervisor, the graphics processing engines 1531-1532, N are owned by the operating system. In both cases, the operating system can virtualize the graphics processing engines 1531-1532, N to provide access to each process or application.

[0210] For the shared programming model, the graphics acceleration module 1546 or individual graphics processing engines 1531-1532, N use a process handle to select a process element. In one embodiment, the process element is stored in system memory 1511 and is addressable using the effective to real address translation techniques described herein. The process handle can be an implementation specific value provided to the host process when it registers its context with the graphics processing engines 1531-1532, N (i.e., calls system software to add the process element to a process element linked list). The lower 16 bits of the process handle can be the offset of the process element within the process element linked list.

[0211] Slice control register An exemplary accelerator integration slice 1590 is shown. As used herein, a "slice" includes a specified portion of the processing resources of the accelerator integration circuit 1536. An application effective address space 1582 within system memory 1511 stores process elements 1583. In one embodiment, the process elements 1583 are stored in response to GPU calls 1581 from an application 1580 executing on a processor 1507. The process elements 1583 contain process state for the corresponding application 1580. A work descriptor (WD) 1584 contained in the process element 1583 can be a single job requested by the application or can contain a pointer to a queue of jobs. In the latter case, the WD 1584 is a pointer to a job request queue in the application's address space 1582.

[0212] The graphics acceleration module 1546 and / or individual graphics processing engines 1531-1532, N can be shared by all or a subset of the processes in a system. Embodiments of the invention include infrastructure for setting up process state and sending WDs 1584 to the graphics acceleration module 1546 to start jobs in a virtualized environment.

[0213] In one implementation, the dedicated process programming model is implementation specific. In this model, a single process owns a graphics acceleration module 1546 or individual graphics processing engines 1531. Because the graphics acceleration module 1546 is owned by a single process, the hypervisor initializes the accelerator integration circuit 1536 for the owning partition when assigning the graphics acceleration module 1546, and the operating system initializes the accelerator integration circuit 1536 for the owning process.

[0214] In operation, a WD fetch unit 1591 in the accelerator integration slice 1590 fetches a next WD 1584 that includes an indication of work to be done by one of the graphics processing engines of the graphics acceleration module 1546. As illustrated, data from the WD 1584 can be stored in registers 1545 and used by the MMU 1539, interrupt management circuit 1547, and / or context management circuit 1548. For example, one embodiment of the MMU 1539 includes segment / page walk circuitry to access a segment / page table 1586 within an OS virtual address space 1585. The interrupt management circuit 1547 can handle interrupt events 1592 received from the graphics acceleration module 1546. When performing graphics operations, effective addresses 1593 generated by the graphics processing engines 1531-1532, N are translated to real addresses by the MMU 1539.

[0215] In one embodiment, the same set of registers 1545 is replicated for each graphics processing engine 1531-1532, N and / or graphics acceleration module 1546 and can be initialized by the hypervisor or operating system. Each of these replicated registers can be included in the accelerator integration slice 1590. Exemplary registers that can be initialized by the hypervisor are shown in Table 1. Table 1 - Hypervisor-Initialized Registers 1 Real address (RA) scheduling process region pointer 2 Privilege mask override register 3 Interrupt vector table entry offset 4 Interrupt vector table entry limit 5 Status register 6 Logical partition ID 7 Real address (RA) hypervisor accelerator utilization record pointer 8 Storage description register 9 Process and thread identification

[0216] Exemplary registers that can be initialized by the operating system are shown in Table 2. Table 2 - Operating System-Initialized Registers 1 Effective address (EA) context save / restore pointer 2 Virtual address (VA) accelerator utilization record pointer 3 Virtual address (VA) storage segment table pointer 4 Privilege mask 5 Work descriptor 6 Figure 15E

[0217] In one embodiment, each WD 1584 is specific to a particular graphics acceleration module 1546 and / or graphics processing engine 1531-1532, N. It contains all information required by the graphics processing engines 1531-1532, N to do its work, or it can be a pointer to a memory location where the application has set up a command queue of work to be done.

[0218] Work descriptor (WD)Figure illustrates additional details of one embodiment of the shared model. This embodiment includes a hypervisor real address space 1598 in which the process element list 1599 is stored. The hypervisor real address space 1598 is accessible via a hypervisor 1596 that uses a graphics acceleration module engine virtualization for operating system 1595.

[0219] The shared programming model allows all or a subset of processes from all or a subset of the partitions in the system to use the graphics acceleration module 1546. There are two programming models in which the graphics acceleration module 1546 is shared by multiple processes and partitions: time-sliced sharing and graphics-directed sharing.

[0220] In this model, the system hypervisor 1596 owns the graphics acceleration module 1546 and makes its functionality available to all operating systems 1595. In order for the graphics acceleration module 1546 to support virtualization by the system hypervisor 1596, the graphics acceleration module 1546 can adhere to the following requirements: 1) Application job requests must be autonomous (i.e., state does not need to be maintained between jobs), or the graphics acceleration module 1546 must provide a context save and restore mechanism. 2) Application job requests are guaranteed to complete in a specified amount of time by the graphics acceleration module 1546, including any translation faults, or the graphics acceleration module 1546 provides the ability to preempt processing of a job. 3) The graphics acceleration module 1546 must guarantee fairness between processes when operating under the directed shared programming model.

[0221] In one embodiment, for a shared model, application 1580 is required to make an operating system 1595 system call with a graphics acceleration module 1546 type, a work descriptor (WD), an authority mask register (AMR) value, and a context save / restore area pointer (CSRP). The graphics acceleration module 1546 type describes the target acceleration function for the system call. The graphics acceleration module 1546 type can be a system specific value. The WD is formatted specifically for the graphics acceleration module 1546 and can take the form of a graphics acceleration module 1546 command, a valid address pointer to a user defined structure, a valid address pointer to a command queue, or any other data structure used to describe work to be done by the graphics acceleration module 1546. In one embodiment, the AMR value is the AMR state for the current process. The value passed to the operating system is similar to an application setting the AMR. If the accelerator integration circuit 1536 and graphics acceleration module 1546 implementation does not support a user authority mask override register (UAMOR), the operating system can apply the current UAMOR value to the AMR value before passing the AMR in a hypervisor call. The hypervisor 1596 can optionally apply the current authority mask override register (AMOR) value before placing the AMR in the process element 1583. In one embodiment, the CSRP is a register 1545 containing a valid address of an area in the application address space 1582 for the graphics acceleration module 1546 to save and restore context state. The pointer is optional if state is not required to be saved between jobs or when a job is preempted. The context save / restore area can be fixed system memory.

[0222] Upon receiving the system call, the operating system 1595 can verify that the application 1580 is registered and has been given permission to use the graphics acceleration module 1546. The operating system 1595 then calls the hypervisor 1596 with the information shown in Table 3. Table 3 - OS to hypervisor call parameters 1 Privilege mask register (AMR) value (potentially masked). 2 Effective address (EA) context save / restore region pointer (CSRP) 3 Process ID (PID) and optional thread ID (TID) 4 Virtual address (VA) accelerator utilization record pointer (AURP) 5 Virtual address of storage segment table pointer (SSTP) 6 Logical interrupt service number (LISN) 7 Figure 15F

[0223] Upon receiving the hypervisor call, the hypervisor 1596 verifies that the operating system 1595 is registered and has been given permission to use the graphics acceleration module 1546. The hypervisor 1596 then places the process element 1583 in a process element linked list for the corresponding graphics acceleration module 1546 type. The process element can include the information shown in Table 4. Table 4 - process element information

[0224] In one embodiment, the hypervisor causes the plurality of accelerator integration slices 1590 registers 1545 to initialize.

[0225] As Figure 15F As illustrated in FIG. 15, one embodiment of the present application employs a unified memory that is addressable via a common virtual memory address space that is used to access physical processor memory 1501-1502 and GPU memory 1520-1523. In this implementation, operations performed on GPUs 1510-1513 utilize the same virtual / effective memory address space to access processor memory 1501-1502 and vice versa, thereby simplifying programmability. In one embodiment, a first portion of the virtual / effective address space is allocated to processor memory 1501, a second portion is allocated to second processor memory 1502, a third portion is allocated to GPU memory 1520, and so on. The entire virtual / effective memory space (sometimes referred to as the effective address space) is thereby distributed across each of processor memory 1501-1502 and GPU memory 1520-1523, allowing any processor or GPU to access any physical memory using a virtual address that maps to that memory.

[0226] In one embodiment, bias / coherence management circuitry 1594A-1594E within one or more of MMUs 1539A-1539E ensures cache coherence between the caches of host processors (e.g., 1505) and the caches of GPUs 1510-1513, and implements bias techniques that dictate certain types of data should be stored in which physical memory. Although multiple instances of bias / coherence management circuitry 1594A-1594E are illustrated in FIG. 15, bias / coherence circuitry can be implemented within the MMU(s) of host processor(s) 1505 and / or within accelerator integration circuit 1536. Graphics processing pipeline

[0227] ​One embodiment allows GPU-attached memory 1520-1523 to be mapped as part of system memory and accessed using shared virtual memory (SVM) technology, but without the typical performance penalties associated with full system cache coherency. The ability for GPU-attached memory 1520-1523 to be accessed as system memory, and without the heavy cache coherency overhead, provides a beneficial operating environment for GPU offload. Such an arrangement allows host processor 1505 software to set operands and access computation results without the overhead of traditional I / O DMA data copies. Such traditional copies involve driver calls, interrupts, and memory-mapped I / O (MMIO) accesses, which are all inefficient relative to simple memory accesses. At the same time, the ability to access GPU-attached memory 1520-1523 without cache coherency overhead can be critical to the execution time of the offloaded computation. For example, in the case of a write memory traffic with a lot of streaming, cache coherency overhead can significantly reduce the effective write bandwidth seen by GPU 1510-1513. The efficiency of operand setup, the efficiency of result access, and the efficiency of GPU computation all play a role in determining the effectiveness of GPU offload.

[0228] In one implementation, the selection between GPU bias and host processor bias is driven by a bias tracking program data structure. For example, a bias table can be used, which can be a page-granular structure (i.e., controlled at the memory page granularity), including 1 or 2 bits per GPU-attached memory page. The bias table can be implemented in the stolen memory range of one or more GPU-attached memories 1520-1523, with or without a bias cache in GPU 1510-1513 (e.g., to cache frequently / recently used bias table entries). Alternatively, the entire bias table can be maintained within the GPU.

[0229] In one implementation, the bias table entry associated with each access to GPU-attached memory 1520-1523 is accessed prior to the actual access to GPU memory, resulting in the following operations. First, local requests from GPU 1510-1513 that find their pages in the GPU bias are forwarded directly to the corresponding GPU memory 1520-1523. Local requests from the GPU that find their pages in the host bias are forwarded to processor 1505 (e.g., over the high-speed link, as discussed above). In one embodiment, requests from processor 1505 that find the requested pages in the host processor bias complete the request as a normal memory read. Alternatively, requests involving pages in the GPU bias can be forwarded to GPU 1510-1513. If the GPU is not currently using the page, the GPU can then transition the page to the host processor bias.

[0230] The bias state of a page may be changed by a software-based mechanism, a hardware-assisted software-based mechanism, or, for a limited set of circumstances, by a purely hardware-based mechanism.

[0231] One mechanism for changing the bias state employs an API call (e.g., OpenCL), which in turn calls the GPU's device driver, which in turn sends a message (or queues a command descriptor) to the GPU instructing it to change the bias state and, for some transitions, performs a cache flush operation in the host. The cache flush operation is required for transitions from host processor 1505 bias to GPU bias, but not for the reverse transition.

[0232] In one embodiment, cache coherence is maintained by temporarily rendering GPU-biased pages that are not cacheable by host processor 1505. To access these pages, processor 1505 may request access from GPU 1510, which may or may not immediately grant access, depending on the implementation. Therefore, to reduce communication between host processor 1505 and GPU 1510, it is beneficial to ensure that GPU-biased pages are those requested by the GPU rather than host processor 1505, and vice versa. Figure 16

[0233] Figure 13A The diagram illustrates a graphics processing pipeline 1600 according to an embodiment. In one embodiment, a graphics processor may implement the illustrated graphics processing pipeline 600. The graphics processor may be included in a parallel processing subsystem as described herein, such as a graphics processor. Figure 12 The parallel processor 1300, in one embodiment, is Figure 13A Various parallel processing systems can be implemented by parallel processing units (e.g., Figure 13C The graphics processing pipeline 1600 is implemented using one or more instances of a parallel processing unit 1302 of the graphics processing unit. For example, a shader unit (e.g., Figure 13A The graphics multiprocessor 1334 of the graphics multiprocessor 1334 may be configured to perform the functionality of one or more of the vertex processing unit 1604, the tessellation control processing unit 1608, the tessellation evaluation processing unit 1612, the geometry processing unit 1616, and the fragment / pixel processing unit 1624. The functionality of the data assembler 1602, the primitive assemblers 1606, 1614, 1618, the tessellation unit 1610, the rasterizer 1622, and the raster operation unit 1626 may also be performed by a processing cluster (e.g., Figure 13Aother processing engines and corresponding partition units within the processing cluster 1314) of the graphics processing unit 1312. The processing cluster 1314 can process multiple portions of the data in parallel. The partition units can be configured to process one or more of the portions of the data in parallel. The processing cluster 1314 can include general-purpose processing units (GPGPU) to perform general-purpose computing operations, machine-learning operations, or graph analytics operations. The GPGPU can be configured to execute instructions that are not directly related to graphics processing. Figure 13A The graphics processing pipeline 1600 can also be implemented using dedicated processing units for one or more functions. In one embodiment, one or more portions of the graphics processing pipeline 1600 can be performed by parallel processing logic within a general-purpose processor (e.g., a CPU). In one embodiment, one or more portions of the graphics processing pipeline 1600 can access on-chip memory (e.g., parallel processor memory 1322 in FIG. 13) via a memory interface 1628, which can be an instance of the memory interface 1318 of FIG. 13. The memory interface 1628 can be used to exchange data with the parallel processor memory 1322, which can be shared by one or more graphics processing pipelines 1600. Figure 13A Figure 13A The memory interface 1628 can be an instance of the memory interface 1318 of FIG. 13.

[0234] In one embodiment, the data assembler 1602 is a processing unit that collects vertex data for surfaces and primitives. The data assembler 1602 then outputs vertex data including vertex attributes to the vertex processing unit 1604. The vertex processing unit 1604 is a programmable execution unit that is configured to execute vertex shader programs to transform vertex data as specified by the vertex shader programs. The vertex processing unit 1604 reads data stored in the cache, local, or system memory to perform the operations specified by the vertex shader programs. The vertex processing unit 1604 can be programmed to transform vertex data from object-based coordinates to world space coordinates, or normalized device coordinates space.

[0235] A first instance of the primitive assembler 1606 receives vertex attributes from the vertex processing unit 1604. The primitive assembler 1606 reads stored vertex attributes as needed, and constructs graphics primitives for processing by the tessellation control processing unit 1608. The graphics primitives include triangles, lines, points, patches, and so forth, as supported by various graphics processing application programming interfaces (APIs).

[0236] ​The tessellation control processing unit 1608 treats the input vertices as control points for a geometric patch. The control points are transformed from an input representation from the patch (e.g., a basis for the patch) to a representation suitable for use in surface evaluation by the tessellation evaluation processing unit 1612. The tessellation control processing unit 1608 can also compute tessellation factors for edges of the geometric patch. The tessellation factors are applied to individual edges and quantify a view-dependent level of detail associated with that edge. The tessellation unit 1610 is configured to receive the tessellation factors for the edges of the patch and to tessellate the patch into a plurality of geometric primitives (such as line, triangle, or quadrilateral primitives) that are passed to the tessellation evaluation processing unit 1612. The tessellation evaluation processing unit 1612 operates on the parametric coordinates of the tessellated patch to generate surface representations and vertex attributes for each vertex associated with the geometric primitives.

[0237] A second instance of the primitive assembler 1614 receives the vertex attributes from the tessellation evaluation processing unit 1612, reads the stored vertex attributes as needed, and constructs graphics primitives for processing by the geometry processing unit 1616. The geometry processing unit 1616 is a programmable execution unit that executes a geometry shader program to transform the graphics primitives received from the primitive assembler 1614 as specified by the geometry shader program. In one embodiment, the geometry processing unit 1616 is programmed to subdivide the graphics primitives into one or more new graphics primitives and calculate parameters used to rasterize the new graphics primitives.

[0238] In some embodiments, the geometry processing unit 1616 can add or remove elements in the geometry stream. The geometry processing unit 1616 outputs parameters and vertices that specify new graphics primitives to the primitive assembler 1618. The primitive assembler 1618 receives the parameters and vertices from the geometry processing unit 1616 and constructs graphics primitives for processing by the viewport scale, cull, and clip unit 1620. The geometry processing unit 1616 reads data stored in the parallel processor memory or system memory for use while processing the geometry data. The viewport scale, cull, and clip unit 1620 performs clipping, culling, and viewport scaling and outputs the processed graphics primitives to the rasterizer 1622.

[0239] The rasterizer 1622 can perform depth culling and other depth-based optimizations. The rasterizer 1622 also performs scan conversion of the new graphics primitives to generate fragments and outputs those fragments and associated coverage data to the fragment / pixel processing units 1624. The fragment / pixel processing units 1624 are programmable execution units that are configured to perform fragment shader programs or pixel shader programs. The fragment / pixel processing units 1624 transform the fragments or pixels received from the rasterizer 1622 as specified by the fragment or pixel shader programs. For example, the fragment / pixel processing units 1624 can be programmed to perform operations including, but not limited to, texture mapping, shading, blending, texture correction, and perspective correction to produce a shaded fragment or pixel that is output to the raster operations unit 1626. The fragment / pixel processing units 1624 can read data stored in the parallel processor memory or system memory for use while processing the fragment data. The fragment or pixel shader programs can be configured to shade with samples, pixels, tiles, or other granularity depending on the sampling rate configured for the processing units.

[0240] The raster operations unit 1626 is a processing unit that performs raster operations and outputs processed graphics data in memory such as parallel processor memory 1322 in FIG. 13. The raster operations include, for example, copy operations, texture mapping, blending, z test, z bumping, and fogging. The raster operations are performed on the outgoing primitive or pixel data generated by the rendering engine 1620. The data can be processed for each pixel to be output to a display device 1210 (FIG. 12) or to an image encoder. Figure 12 Performing a reduction operation on vector elements The raster operations unit 1626 is a processing unit that performs raster operations and outputs processed graphics data in memory such as parallel processor memory 1322 in FIG. 13. The raster operations include, for example, copy operations, texture mapping, blending, z test, z bumping, and fogging. The raster operations are performed on the outgoing primitive or pixel data generated by the rendering engine 1620. The data can be processed for each pixel to be output to a display device 1210 (FIG. 12) or to an image encoder. Performing a reduction operation on vector elements in a single register

[0241] Vectors can contain multiple data elements that share the same characteristic, and vectors can be combined through reduction operations. For example, a vector can contain data elements that have the same value (a copy). For certain computations, one data element within a copy set has a data dependency that involves another data element within the same copy set. The copy set needs to be identified in order to allow the computation to be vectorized when processing the vector operation. Copies can be detected using a conflict detection instruction such as VPCONFLICT.

[0242] ​In particular, an instruction can test each element of its vector register input for equality with all earlier elements of that input (e.g., all elements closer to the least significant bit (LSB)) and output the results of these comparisons as a set of bit vectors. The conflict detection instruction provides a way to determine whether an element has a data dependency involving other elements within the same register (e.g., a SIMD register). In cases where possible data dependencies across register elements might otherwise prevent vectorization, the ability to identify duplicate values within multiple individual elements of a register allows scalar code to be vectorized.

[0243] Prior instruction set architectures (ISAs) use multiple instructions to perform reduction operations in multiple steps. For example, one instruction can detect duplicates within a vector (e.g., by determining a subset of elements with unique indices) and can perform that detection in SIMD fashion. Subsequently, another instruction can perform a reduction operation on the duplicates with the same index.

[0244] However, it is desirable for certain operations to be implemented as a single instruction for a reduction operation on one or more vectors stored in one or more registers (or other storage units) such that once the single instruction is decoded, it causes the processor to perform the required reduction operation without further instructions. A single instruction for performing a reduction operation yields better compilation and computational efficiency. Figure 17

[0245] Embodiments of the invention include techniques for performing a reduction operation on vector elements in one register using a single instruction. Upon execution of the single instruction, a reduction operation can be performed using multiple tree reductions (one for each unique index value) on elements within a register (e.g., a SIMD register) in parallel. The single instruction can combine operations performed by multiple earlier operations. For example, the single instruction can sort each of the elements sharing an index (previously performed by an instruction such as VPOPCNT). The single instruction can also iteratively identify and combine partial results from pairs of data elements within the same index (previously performed by an instruction such as VPCONFLICT).

[0246] Within a register, there can be multiple values that need to be combined via one or more reduction modes. For example, an application can have a set of values {a0, b0, a1, a2, b1, a3, a4, b2} within a single register that need to be combined such that all 'a' values are summed and all 'b' values are summed, resulting in only two values {a0+a1+a2+a3+a4, b0+b1+b2}. Given a reduction operation with only two inputs (e.g., by a'sum' in a processor), an efficient way to sum all 'a' values and all 'b' values is to perform multiple binary tree reductions in parallel across elements of a register.

[0247] Some embodiments of the present application generate: (1) a first output that identifies independent reductions; and (2) a second output that identifies an order (e.g., left child vs. right child) of a binary reduction tree. The former is used to parallelize the reduction operations for independent reductions, and the latter is used to count the number of instances of each unique index value within the associated index vector.

[0248] Figures 1A-1B A tree reduction operation on multiple vector elements stored in input registers is shown when a single instruction is executed according to one embodiment of the present application. The single instruction can take the format of "vmatchindx zmmO{kO}, zmm l, zmm2" as shown at label 1700, although many variants of the instruction format can be implemented in which one or more operands and / or immediate values and opcode(s) as discussed herein above can be specified for the reduction operation according to embodiments of the present application. In some embodiments, the single instruction can be issued as a single instruction multiple data (SIMD) instruction, additionally / alternatively, the single instruction can be issued as a single instruction multiple thread (SIMT) instruction as discussed herein above with reference to Figure 5A .

[0249] The reduction operation takes values from an input register (zmm2) 1710. The input register 1710 stores a set of data elements d0 through d 15 , and the set of data elements can belong to a vector / array / matrix to which the reduction operation can be applied. In one embodiment, the input register can be one of the vector registers 514 in Figure 13D . Each value within the input register 1710 can be considered to be within a lane to be executed using SIMD techniques. The input register 1710 can also be within the register file 1358 of Performing a reduction operation on vector elements in multiple registers , in which the input register is a data element can be vectorized for execution by a thread group (e.g., a packet block).

[0250] The reduction operation also takes values from the index register (zmmi) 1720. The index values indicate which values within the input register are associated with each other. If two data elements in the input register correspond to the same index value, they participate in the same tree reduction; otherwise, they participate in different tree reductions. For example, do, d3, d8, d 14 and d 15 have the same index value 'A'; di, d4-d6, d9-d 11 and d 13 have the same index value 'B'; and d2 and d7 have the same index value 'D'. These data elements within the same index value participate in the corresponding tree reduction.

[0251] The reduction operation is shown at label 1730, and these element operations can be one or more of the following based on the index value to obtain the associated data elements: sum (by "addition"), minimum, maximum, mean, median, product, difference (by "subtraction"), count of occurrences. The reduction operation can be one or more of the following: accumulate the associated data element values (via addition, subtraction, or multiplication); select the maximum or minimum of the associated data element values; and compute a mean or median value of the associated data elements. Accumulation (specifically, sum) is used as an example for the following discussion.

[0252] The reduction operation begins with an initial condition 1732 of the input register values. A set of iterations can be performed based on the index values on the associated data elements. For example, do, d3, d8, d 14 and d 15 have the same index value 'A', and these values are shown as being reduced by three iterations. For the first iteration at label 1734, the pairs of data elements having the value 'A' (in this embodiment, from the least significant bit position) are summed together to obtain an intermediate result 1734. The values at the odd-numbered data elements will remain the same (e.g., the value of d 15 in this embodiment). The other data elements having the same index value 'B' and the same index value 'D' will go to the same iteration, respectively. At the second iteration, the pairs of the intermediate result 1734 are again summed together to obtain an intermediate result 1736. This process continues to obtain a single value for each unique index as the result at the final result 1738.

[0253] At label 1760, the result of the reduction operation is stored within the output register (zmm0) 1750. The output register 1750 can store the final result 1738, with each of the final result 1738 for a unique index value at an associated data element position of zmm0. For example, the final result for the index value 'A' (for 'A') sum = do + d3 + d8 + d 14 + d15 All associated bits that can be stored in zmmO at positions R0, R3, R8, R 14 and R 15 . Alternatively, the output register can store only the final result for a unique index value to the left-most (or right-most, or another predefined) data element position of zmmO associated with the unique index value, e.g., to R 15 for index 'A'. Furthermore, the output register 1750 can alternatively store results including intermediate results in the associated data element positions of zmmO. For example, R0 can store dO, R3 store dO+d3, R8 store d8, R 14 store dO+d3+d8+d 14 , and R 15 store dO+d3+d8+d 14 +d 15 .

[0254] Note that while the tree reduction operation uses a binary tree as an example, other types of tree reductions including more data elements per iteration can also be implemented. Furthermore, while the pairing within the tree reduction is from left-most to right, the reverse direction can be implemented in alternative embodiments.

[0255] In some embodiments, the reduction operation can additionally take values of a mask vector (kO). These values can be taken from a mask register or another storage location (e.g., any of the levels in the cache or memory hierarchy). The mask values can be set for the input data elements of the input register 1710 or the output data elements of the output register 1750. When the mask vector is used for the input register and the mask values are cleared (e.g., set to '0'), the corresponding data elements in the input register are ignored in the reduction operation. Alternatively, the mask vector can be set for the output register, and when the mask values are cleared, the corresponding data elements in the output register can be set to 0 ("zero mask") or the original value before the reduction operation ("merge mask"). Alternatively, in embodiments, the mask values can be set to '1' to ignore the corresponding data elements in the input register and / or output register.

[0256] By using a tree reduction, the reduction is at most log2N iterations (N is the number of data elements, vector length). For example, when all 16 data elements of the input register 1710 have the same index value, the reduction operation iterates log2N = 4 times. This is advantageous over existing approaches in which the reduction operation would iterate N-1 times, once for each serialized reduction iteration.

[0257] Further, by using a single instruction for performing a reduction operation, the instruction set can be significantly more efficient. The instruction itself can specify the particular reduction operation to be performed (e.g., obtain one or more of the following for the associated data elements: sum, minimum value, maximum value, mean value, median value, product, occurrence count), with a particular mask value applied using a mask vector, and the intermediate result can be stored in an output register for future operations. Based on the values in the index registers, the reduction operation can be applied to any data elements of the input vectors loaded into the input registers, and the reduction can be performed in parallel using SIMD and / or SIMT techniques, thereby significantly improving the computational efficiency of the processor (e.g., CPU, GPU, or other processing unit). Figure 18

[0258] The reduction operation can be extended to vector values stored in multiple registers. Figure 18 A reduction operation on multiple vector elements stored in multiple input registers is shown when a single instruction is executed according to one embodiment of the application. Figure 17 With Figure 18 Similar, with some details omitted to focus on additional details for a multiple input register implementation, and same or similar labels indicating elements or components with the same or similar functionality.

[0259] In one embodiment, the single instruction takes the format shown at label 1800 as "vmatchindx2zmm0{k0}, zmm1{k1}, zmm2, zmm3, zmm4, zmm5". The reduction operation takes values from input register (zmm4) 1810 and input register (zmm5) 1812, which store values for a set of data elements belonging to one or more vectors to which the reduction operation can be applied. In one embodiment, each input register 1810 / 1812 can operate in a similar manner to input register 1710. The reduction operation also takes values from index register (zmm2) 1820 and index register (zmm3) 1822. Each index register can operate in a similar manner to index register 1720, and if two data elements within an input register correspond to the same index value, they can participate in the same tree reduction. The reduction operation is shown at label 1830, with operations similar to those discussed above with reference to label 1730.

[0260] At label 1860, the results of the reduction operation are stored within output register (zmm0) 1850 and output register (zmm1) 1852. The storage of the results is similar to the operation at label 1760 as discussed above herein.

[0261] Note that many various instruction formats can be implemented for reduction operations on vector values stored in multiple registers. In an alternative embodiment, a single instruction can take the format of "vmatchindx2zmm0{k0}, register 1, register 2," where register 1 corresponds to zmm2 in Figure 18 and register 2 corresponds to zmm4 in Performing a reduction operation on vector elements in multiple registers of different compute units In this way, a single instruction specifies only one register location for each type of register, but the processor will operate on two registers for each type of register. For example, while zmmO can be specified as the output register (e.g., with a register number of "10"), the processor will also operate on the next output register zmm 1 (e.g., with a register number of "11") as well. Thus, zmm 1 is an output register that is not explicitly indicated in the single instruction as shown at label 1822. The explicitly indicated registers and the implicitly indicated registers can be paired as odd / even pairs, such that if one register of a pair is specified in the instruction, the value of the other register of the pair will be operated on.

[0262] Similarly, in a single instruction, zmm4 can be specified as an input register, zmm2 can be specified as an index register, and kO can be specified as a mask vector; but the processor will also fetch input values from zmm5, index values from zmm3, and mask values from mask vector k 1, respectively. These values are not explicitly indicated in the single instruction as shown at label 1880.

[0263] Reduction operations on vector elements are not limited to values within two input registers, and can operate on values in additional registers by a single instruction. Reduction operations on vector values on multiple registers can be performed in a similar manner as reduction operations on vector values on two registers, such as input registers 1810 and 1812 discussed herein. Figure 19

[0264] A register file of a multi-processor can be partitioned among different compute units. For example, a register file can be partitioned among different compute units within a multi-processor for a packet or thread (e.g., register file 1358 can be partitioned among different packets executed by graphics multi-processor 1334, as discussed above herein). A single instruction can explicitly specify each operand (e.g., input register(s), index register(s), mask vector(s), and / or output register(s)) as discussed herein for vector elements stored in different registers of different compute units. When the software / multi-processor is aware of the complete set of registers available to all threads / packets (e.g., as is typical for graphics multi-processors), the instruction can explicitly identify all operands (e.g., registers and mask vectors).

[0265] Alternative embodiments can be implemented when the software / multi-processor is not aware. For example, a single instruction can take the format of "vmatchindx2 zmmO{kO}, zmm l, zmm2, reg / imm", where the "reg / imm" operand specifies a register (or another storage location, such as a cache / memory in the memory hierarchy) or an immediate value that identifies a value for a second thread or packet. The software / multi-processor is aware of the case with respect to the earlier operands (zmmO, zmm l, zmm2, kO) for a first thread or packet, and the "reg / imm" can hold an identifier for a second thread or packet for which another input / index / output register and mask vector hold values for the reduction operation. Alternatively, the "reg / imm" can specify an offset from the first thread or packet. For example, if thread X executes the single instruction, the single instruction can be executed with combined information from thread X + reg / imm.

[0266] In addition, a virtual register number for a register can be used to specify a register for a single instruction, and the software / multi-processor converts the virtual register number to a physical register number for each thread or warp. Today, multi-processors have the concept of both logical / architectural registers and physical registers. If a single instruction specifies an output register number "10," the multi-processor can identify the physical register "10" for each thread / warp. Such conversion is straightforward for some multi-processors (e.g., CPUs). For other multi-processors (e.g., GPUs), the multi-processor knows the total number of registers needed by the threads / warps, and accordingly allocates a contiguous block of registers for each thread / warp. For example, the multi-processor can allocate registers 0 through N-1 to thread / warp 0, registers N through 2N-1 to warp 1, and so on. Each thread / warp has a starting point in the register file. The multi-processor can find the physical register corresponding to a specified virtual register for a warp by adding an offset for the thread / warp (e.g., thread / warp number * offset + specified virtual register). Thus, even when multiple input registers are not explicitly identified in a single instruction, the single instruction can be used for a reduction operation across input values in multiple input registers.

[0267] When performing a reduction operation by using multiple compute units within a multi-processor for warps or threads, synchronization between the multiple compute units can be needed. Synchronization can be achieved using barriers to ensure that the warps and threads have their information ready for combination. For example, in some embodiments, instructions such as DMB (data memory barrier), DSB (data synchronization barrier), or ISB (instruction synchronization barrier) can be used.

[0268] Alternatively, a single instruction can implement synchronization without using any additional instructions. For example, a single instruction can take the format of "vmatchindx2 zmmO{kO}, zmm l, zmm2, sync_reg / imm", where the "sync_reg / imm" operand specifies a location (register or another storage location, such as a cache / memory in the memory hierarchy) that the packet / thread will use for synchronization. Assume that two packets / threads participate in the reduction operation. The two packets / threads will execute the single instruction, and they can have the same value for "sync_reg / imm". Before the packets / threads involved execute the instruction, the specified location will be initialized to an initial value. When the first thread / packet executes the instruction, it can check the specified location. When the first thread / packet sees that the specified location contains the initial value, it overwrites the location with another value, indicating that the first thread / packet has arrived and is waiting for the second packet / thread. Subsequently, when the second thread / packet executes the instruction, it again checks the specified location. When the second thread / packet determines that the specified location does not contain the initial value, the determination triggers the execution of a combining operation across different compute units. When the combining operation is complete, the two threads / packets are allowed to proceed.

[0269] When more than two packets / threads participate in the reduction operation, the value in the specified location can indicate to the packets / threads that the packet / thread is the last to arrive, and the value triggers the execution of a combining operation across different compute units, and when the combining operation is complete, all threads / packets are allowed to proceed.

[0270] Figure 19 A reduction operation on a plurality of vector elements stored in a plurality of input registers when executing a single instruction using synchronization is shown in accordance with one embodiment of the present application. Figure 18 With Some embodiments of the invention Similarities, with some details omitted to focus on additional details for the multiple input register implementation, and same or similar labels indicate elements or components with the same or similar functionality.

[0271] At label 1970, a portion of the vector elements are executed by a first set of compute units, while at label 1972, other portions of the vector elements are executed by other sets of compute units. Different sets of compute units can use different portions of a register file of a multi-processor (e.g., CPU / GPU). In addition, different portions of the register file can also be used to store optional mask vectors 1940 and 1942 used to execute the single instruction for the reduction operation.

[0272] A single instruction (e.g., "vmatchindx2 zmmO{kO}, zmm l, zmm2, sync_reg / imm" as discussed above) can specify a synchronization location 1900 that indicates to the computing units participating in the reduction operation whether synchronization between the computing units involved is achieved. Once synchronization is achieved, a combination operation can be performed at marker 1934, where the combined results from the computing units can be completed. Once combination is complete, as shown at marker 1960, the results can be stored in output registers 1950 and 1952.

[0273] While a synchronization indication is used in a single instruction, embodiments of the application do not require additional instructions to achieve a synchronized reduction around vector elements stored in multiple registers at different parts of a register file. Thus, a single instruction is more efficient in achieving synchronization than multiple instructions. Figure 20

[0274] Figures 17-19 is a flowchart illustrating a reduction operation on multiple vector elements when executing a single instruction according to one embodiment of the application. The method 2000 can be performed on a processor that includes a decode circuit and an execution circuit. At marker 2002, the decode circuit decodes an instruction, where the instruction specifies a first input register containing a plurality of data element values, a first index register containing a plurality of indices, and an output register. Each index of the plurality of indices maps to a unique data element position of the first input register. While the present discussion refers to a single instruction, it is contemplated that a plurality of instructions can be executed to perform the reduction operation. Figure 18 While several instruction examples are given, many variations of the instruction format can be implemented, where one or more operands and / or immediate values and opcode(s) as discussed above herein can be specified for a reduction operation according to embodiments of the application.

[0275] The instruction is used to perform a reduction operation on a plurality of vector elements that can belong to an array / matrix. The operation code (opcode) of the instruction specifies one or more reduction operations. The reduction operation can be one or more of: accumulating associated data element values; selecting a maximum or minimum value among associated data element values; and computing a mean or median value of associated data element values. Accumulation (fetch and add) is used as an example discussed below.

[0276] In some embodiments, the instruction further specifies a mask vector containing a plurality of masks, where each mask indicates that a data element position of the output register is active or inactive, and where the result is not written to inactive data element positions. In alternative embodiments, each mask indicates that a data element position of the input register is active or inactive, and inactive data elements do not participate in the reduction operation caused by the instruction.

[0277] At label 2004, the execution circuit executes the decoded instruction. The execution includes: at label 2012, identifying data element values associated with each other based on the indices. Subsequently, at label 2014, the execution circuit executes one or more reduction operations on the associated data element values based on the identifying; and at label 2016, results of the one or more reduction operations are stored in the output register.

[0278] In one embodiment, the group of data element values are associated with each other when they have the same index value. In one embodiment, the one or more reduction operations are executed to: combine the group of data element values for the group of data element values sharing the same index value to generate an arithmetic combination as a result. The arithmetic combination can include the following results based on the index value of obtaining the associated data elements: sum (by "addition"), difference (by "subtraction"), minimum, maximum, mean, median value, product, count of occurrences.

[0279] In one embodiment, the result is stored in a plurality of data element locations of the output register, each data element location corresponding to one of the corresponding associated data element values. In one embodiment, the one or more reduction operations are executed on the group of associated data element values in a plurality of iterations, and intermediate results of the plurality of iterations are stored in data element locations corresponding to the plurality of data element values of the corresponding associated data element values that participated in obtaining the intermediate results.

[0280] In one embodiment, the execution of the decoded instruction is additionally performed on a second input register, and a further plurality of data element values within the second input register and a further plurality of indices within a second index register are used in performing the one or more reduction operations. In one embodiment, the instruction further specifies a second input register containing the further plurality of data element values and a second index register containing the further plurality of indices, each index of the further plurality of indices maps to a unique data element location of the second input register, and the one or more reduction operations are performed on the data element values of the first input register and the second input register based on the indices of the first index register and the second index register. These embodiments are discussed in more detail with respect to Figure 19 More details are discussed.

[0281] In one embodiment, the instructions are executed by two or more compute units. In one embodiment, each of the two or more compute units is a warp or thread. In one embodiment, the two or more compute units are synchronized when performing one or more reduction operations. In one embodiment, the instructions further specify a location to store a value indicating whether the two or more compute units are synchronized. In one embodiment, the processor is a graphics processing unit (GPU). These embodiments are discussed herein with reference to the following clauses: ​ Further discussion is provided.

[0282] In the foregoing specification, embodiments of the application have been described with reference to specific exemplary embodiments thereof. It will, however, be evident that various modifications and changes can be made to these embodiments without departing from the broader spirit and scope of the application as set forth in the appended claims. The Specification and drawings are, accordingly, to be regarded in an illustrative rather than a restrictive sense.

[0283] Embodiments of the application can include each of the steps described above. These steps can be embodied in machine-executable instructions which can be used to cause a general-purpose or special-purpose processor to perform the steps. Alternatively, these steps can be performed by specific hardware components that contain hardwired logic for performing the steps, or by any combination of programmed computer components and custom hardware components.

[0284] As described herein, instructions can refer to specific configurations of hardware such as one or more application specific integrated circuits (ASICs) configured to perform certain operations or having a predetermined functionality or, more generally, to a form of logic that is implemented in hardware or software firmware being stored in memory that results in a machine when implemented on hardware having an architecture that supports that machine. Thus, a technology illustrated in the figures can be implemented using code and data stored and executed on one or more electronic devices (e.g., an end station, a network element, etc.). Such electronic devices store and communicate (internally and / or with other electronic devices over a network) code and data using computer machine-readable media such as non-transitory computer machine-readable storage media (e.g., magnetic disks; optical disks; random access memory; read only memory; flash memory devices; phase-change memory) and transitory computer machine-readable communication media (e.g., electrical, optical, acoustical or other form of propagated signals - such as carrier waves, infrared signals, digital signals, etc.). In addition, such electronic devices typically include a set of one or more processors coupled to one or more other components, such as one or more storage devices (non-transitory machine-readable storage media), user input / output devices (e.g., a keyboard, a touchscreen, and / or a display), and network connections. The coupling of the set of processors and other components is typically through one or more busses and bridges (also termed as bus controllers). The storage device and signals are examples of machine-readable storage media and machine-readable communication media respectively. Thus, a storage device of a given electronic device typically stores code and / or data for execution on the set of one or more processors of that electronic device. Of course, one or more parts of an embodiment of the application can be implemented using different combinations of software, firmware, and / or hardware. Throughout this detailed description, for the purposes of explanation, numerous specific details were set forth in order to provide a thorough understanding of the present application. It will be apparent, however, to one skilled in the art that the present application can be practiced without some or all of these specific details. In certain instances, well known structures and functions have not been described in elaborate detail in order to avoid obscuring the subject matter of the present application. Thus, the scope and spirit of the present application should be judged in terms of the claims and the equivalents thereof, rather than the details of the descriptions.

Claims

1. A device comprising: multiple memory controllers; a second level L2 cache memory coupled to the plurality of memory controllers; a processor coupled to the plurality of memory controllers and to the L2 cache memory, the processor configured to execute an operation corresponding to a single instruction multiple thread (SIMT) instruction, the SIMT instruction configured to indicate a reduction operation to be performed on a plurality of data elements, a plurality of registers to store the plurality of data elements, a memory location for synchronizing a thread group, and a destination location, wherein the reduction operation is one reduction operation selected from the group consisting of a maximum reduction operation and a minimum reduction operation, the processor comprising: The plurality of registers are configured to store the plurality of data elements; and Multiple computing units, for executing the SIMT instructions include the following steps: performing the indicated reduction operation on the plurality of data elements from the plurality of registers to generate result data elements for storage in the destination location, wherein each of the plurality of computation units is configured to perform the indicated reduction operation on a different corresponding data element of the plurality of data elements from a different corresponding register of the plurality of registers; synchronizing threads of the thread group using the memory location, wherein the memory location is to be initialized to an initial value before the plurality of compute units execute the SIMT instruction, wherein the plurality of compute units are to update a value in the memory location to indicate that the thread of the thread group has arrived, and wherein the threads in the thread group are to be synchronized when all threads in the thread group have indicated arrival; and storing the resultant data element in the destination location; and An interconnect interface is coupled to the processor.

2. The device according to claim 1, wherein The reduction operation is the one reduction operation selected from the group that also includes an additive reduction operation.

3. The device according to claim 1, wherein The indicated reduce operation is the maximum reduce operation, and wherein performing the maximum reduce operation includes selecting a maximum data element of the plurality of data elements.

4. The device according to claim 1, wherein The indicated reduce operation is the minimum reduce operation, and wherein performing the minimum reduce operation comprises selecting a minimum data element of the plurality of data elements.

5. The device according to claim 1, wherein The SIMT instruction is used to indicate the reduction operation as any one of at least six reduction operations supported by the SIMT instruction.

6. The device according to claim 1, wherein The plurality of data elements are part of a matrix.

7. The device according to claim 1, wherein The threads in the thread group include threads of multiple package blocks.

8. The device according to claim 1, wherein The processor includes a plurality of graphics cores.

9. The device according to claim 1, wherein The processor includes multiple heterogeneous graphics cores.

10. The apparatus of claim 1, further comprising an instruction converter for converting the SIMT instruction into one or more instructions of a different instruction set executable by the processor.

11. The device according to claim 1, wherein The reduce operation is the one reduce operation selected from the group that also includes an addition reduce operation, wherein the threads of the thread group include threads of a plurality of packet blocks, and wherein the processor includes a plurality of heterogeneous graphics cores.

12. A device comprising: a conversion circuit configured to convert a single instruction multiple thread (SIMT) instruction into one or more other instructions, the SIMT instruction indicating a reduction operation to be performed on a plurality of data elements, a plurality of registers to store the plurality of data elements, a memory location to be used for synchronizing a thread group, and a destination location, wherein the reduction operation is a reduction operation selected from the group consisting of a maximum reduction operation and a minimum reduction operation; The plurality of registers are configured to store the plurality of data elements; and A plurality of computing units are configured to execute the one or more other instructions for different corresponding threads of the thread group, comprising performing the following steps: performing an indicated reduction operation on the plurality of data elements from the plurality of registers to generate a result data element for storage in the destination location, wherein each computation unit of the plurality of computation units is to perform the indicated reduction operation on a different corresponding data element of the plurality of data elements from a different corresponding register of the plurality of registers; synchronizing threads of the thread group using the memory location, wherein the memory location is to be initialized to an initial value before the plurality of compute units execute the one or more other instructions, wherein the plurality of compute units are to update a value in the memory location to indicate that the thread of the thread group has arrived, and wherein the threads in the thread group are to be synchronized when all threads in the thread group have indicated arrival; and The resultant data element is stored in the destination location.

13. The device of claim 12, wherein: The reduction operation is the one reduction operation selected from the group that also includes an additive reduction operation.

14. The apparatus of claim 12, wherein: The indicated reduce operation is the maximum reduce operation, and wherein performing the maximum reduce operation includes selecting a maximum data element of the plurality of data elements.

15. The apparatus of claim 12, wherein: The SIMT instruction is used to indicate the reduction operation as any one of at least six reduction operations supported by the SIMT instruction.

16. The apparatus of claim 12, wherein: The plurality of data elements are part of a matrix.

17. The apparatus of claim 12, wherein: The threads in the thread group include threads of multiple package blocks.

18. The apparatus of claim 12, wherein: The plurality of computing units are within a processor including a plurality of heterogeneous graphics cores.

19. The apparatus of claim 12, wherein: The SIMT instruction is used to indicate the reduction operation as any one of at least six reduction operations supported by the SIMT instruction, wherein the threads of the thread group include threads of a plurality of package blocks.

20. An apparatus comprising: an instruction converter configured to convert a single-instruction multiple-thread (SIMT) instruction into one or more other instructions, the SIMT instruction indicating a reduction operation to be performed on a plurality of data elements, a plurality of registers to store the plurality of data elements, a memory location to synchronize thread groups, and a destination location, wherein the reduction operation is a reduction operation selected from the group consisting of a maximum reduction operation and a minimum reduction operation; The plurality of registers are configured to store the plurality of data elements; and A plurality of computing units are configured to execute operations corresponding to the one or more other instructions for different corresponding threads of the thread group, comprising executing the following steps: performing an indicated reduction operation on the plurality of data elements from the plurality of registers to generate a result data element for storage in the destination location, wherein each computation unit of the plurality of computation units is to perform the indicated reduction operation on a different corresponding data element of the plurality of data elements from a different corresponding register of the plurality of registers; synchronizing threads of the thread group using the memory location, wherein the memory location is to be initialized to an initial value before the plurality of compute units execute the one or more other instructions, wherein the plurality of compute units are to update a value in the memory location to indicate that the thread of the thread group has arrived, and wherein the threads in the thread group are to be synchronized when all threads in the thread group have indicated that they have arrived; The resultant data element is stored in the destination location.

21. The apparatus of claim 20, wherein: The reduction operation is the one reduction operation selected from the group that also includes an additive reduction operation.

22. The apparatus of claim 20, wherein: The SIMT instruction is used to indicate the reduction operation as any one of at least six reduction operations supported by the SIMT instruction.

23. The apparatus of claim 20, wherein: The threads in the thread group include threads of multiple package blocks.

24. The apparatus of claim 20, wherein: The SIMT instruction is used to indicate the reduction operation as any one of at least six reduction operations supported by the SIMT instruction, wherein the threads of the thread group include threads of a plurality of package blocks.

25. A non-transitory machine-readable storage medium storing instructions that, when executed by a machine, cause the machine to perform operations comprising: A single instruction multiple thread (SIMT) instruction is received, the SIMT instruction indicating a reduction operation to be performed on a plurality of data elements, a plurality of registers to store the plurality of data elements, a memory location for synchronizing a thread group, and a destination location, wherein: The reduction operation is a reduction operation selected from the group consisting of a maximum reduction operation and a minimum reduction operation; as well as Enabling multiple computing units of the machine to execute operations corresponding to the SIMT instructions for different corresponding threads of the thread group includes executing the following steps: performing an indicated reduction operation on the plurality of data elements from the plurality of registers to generate a result data element for storage in the destination location, wherein each computation unit of the plurality of computation units is to perform the indicated reduction operation on a different corresponding data element of the plurality of data elements from a different corresponding register of the plurality of registers; synchronizing threads of the thread group using the memory location, wherein the memory location is to be initialized to an initial value before the plurality of compute units execute operations corresponding to the SIMT instruction, wherein the plurality of compute units are to update a value in the memory location to indicate that the thread of the thread group has arrived, and wherein the threads in the thread group are to be synchronized when all threads in the thread group have indicated arrival; and The resultant data element is stored in the destination location.

26. The non-transitory machine-readable storage medium of claim 25, wherein: The reduction operation is the one reduction operation selected from the group that also includes an additive reduction operation.

27. The non-transitory machine-readable storage medium of claim 25, wherein: The SIMT instruction is used to indicate the reduction operation as any one of at least six reduction operations supported by the SIMT instruction.

28. The non-transitory machine-readable storage medium of claim 25, wherein: The threads of the thread group include threads of a plurality of package blocks.

29. The non-transitory machine-readable storage medium of claim 25, wherein: The SIMT instruction is used to indicate the reduction operation as any one of at least six reduction operations supported by the SIMT instruction, wherein the threads of the thread group include threads of a plurality of package blocks, and wherein the machine includes a plurality of heterogeneous graphics cores.

30. A processor core, comprising: Instruction decoder; a scalar unit coupled to the instruction decoder, the scalar unit utilizing a scalar register; a vector unit coupled to the instruction decoder, the vector unit utilizing vector registers; and an L1 cache that allows low-latency access to the scalar registers and the vector registers, The processor core uses a local subset of the global L2 cache and has a direct access path to the local subset.

31. The processor core according to claim 30, wherein: The vector unit is a 16-wide vector processing unit (VPU) that executes one or more of integer, single-precision floating point, and double-precision floating point instructions.

32. The processor core of claim 31, wherein: The VPU supports mixing of register inputs through a mixing unit, supports value conversion through a value conversion unit, and supports copying of memory inputs through a copy unit.

33. A method comprising: Compiling a program in a high-level programming language using an x86 compiler to generate x86 binary code that is natively executed by a first processor having at least one x86 instruction set core; Using an instruction converter, the x86 binary code is converted into an alternative binary code that can be natively executed by a second processor that does not have an x86 instruction set core.

Images