A mechanism for partitioning shared local memory

Abstract

To provide a device which facilitates partitioning of local memory.SOLUTION: A device provided herein comprises a plurality of execution units configured to execute a plurality of execution threads, a memory coupled to share access between the plurality of execution units, and partitioning hardware for partitioning the memory to be used as a cache and as shared local memory (SLM). The partitioning hardware partitions the memory based on the number of the plurality of execution threads executing on the execution units that are active.SELECTED DRAWING: Figure 15

Description

[Background technology]

[0001] Graphics Processing Units (GPUs) are highly threaded machines where hundreds of threads of a program are executed in parallel to achieve high throughput. GPU thread groups are implemented to perform three-dimensional (3D) rendering and mesh shading applications. With the increasingly complex GPUs requiring heavy computations, there is a challenge to keep up with the memory bandwidth requirements.

[0002] So that the above-mentioned features of the present invention may be understood in detail, a more particular description of the present invention briefly summarized above can be made with reference to embodiments, some of which are illustrated in the accompanying drawings. It should be noted, however, that the accompanying drawings merely represent exemplary embodiments of the present invention and should not be considered as limiting its scope, as the present invention may admit of other equally effective embodiments. [Brief description of the drawings]

[0003] [Figure 1] FIG. 2 is a block diagram of a processing system according to an embodiment. [Figure 2A] 1 illustrates a computer system and graphics processor provided by embodiments described herein. [Figure 2B] 1 illustrates a computer system and graphics processor provided by embodiments described herein. [Figure 2C] 1 illustrates a computer system and graphics processor provided by embodiments described herein. [Figure 2D] 1 illustrates a computer system and graphics processor provided by embodiments described herein. [Figure 3A] 4 illustrates a block diagram of a further graphics processor and computation accelerator architecture provided by an embodiment. [Figure 3B]4 illustrates a block diagram of a further graphics processor and computation accelerator architecture provided by an embodiment. [Figure 3C] 4 illustrates a block diagram of a further graphics processor and computation accelerator architecture provided by an embodiment. [Figure 4] FIG. 2 is a block diagram of a graphics processing engine of a graphics processor according to some embodiments. [Figure 5A] 5 illustrates thread execution logic 500 including an array of processing elements for use in a graphics processor according to an embodiment. [Figure 5B] 5 illustrates thread execution logic 500 including an array of processing elements for use in a graphics processor according to an embodiment. [Figure 6] 6 illustrates a further execution unit 600 according to an embodiment. [Figure 7] FIG. 2 is a block diagram illustrating a graphics processor instruction format according to some embodiments. [Figure 8] FIG. 2 is a block diagram of a graphics processor according to another embodiment. [Figure 9A] 4 illustrates a graphics processor command format and command sequence according to some embodiments. [Figure 9B] 4 illustrates a graphics processor command format and command sequence according to some embodiments. [Figure 10] 1 illustrates an example of a graphics software architecture for a data processing system according to some embodiments. [Figure 11A] 1 illustrates an integrated circuit package assembly according to an embodiment. [Figure 11B] 1 illustrates an integrated circuit package assembly according to an embodiment. [Figure 11C] 1 illustrates an integrated circuit package assembly according to an embodiment. [Figure 11D] 1 illustrates an integrated circuit package assembly according to an embodiment. [Figure 12]FIG. 1 is a block diagram illustrating an example of a system on a chip integrated circuit according to an embodiment. [Figure 13A] FIG. 2 is a block diagram illustrating a further exemplary graphics processor. [Figure 13B] FIG. 2 is a block diagram illustrating a further exemplary graphics processor. [Figure 14] 1 illustrates one embodiment of a computing device. [Figure 15] 1 illustrates one embodiment of a sub-slice. [Figure 16] 1 represents one embodiment of thread spawner logic. [Figure 17A] 1 illustrates an embodiment of an SLM allocation. [Figure 17B] 1 illustrates an embodiment of an SLM allocation. [Figure 18] FIG. 1 is a flow diagram of one embodiment of a process for partitioning shared local memory. [Figure 19] 1 illustrates one embodiment of pseudocode for partitioning a shared local memory. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0004] In the following description, numerous specific details are set forth in order to provide a more thorough understanding of the present invention. However, it will be apparent to one skilled in the art that the present invention may be practiced without one or more of these specific details. In other instances, well-known techniques have not been described in order to avoid obscuring the present invention.

[0005] In an embodiment, a shared local memory (SLM) may be implemented to act as a cache in addition to the SLM, and in such an embodiment, the SLM may be partitioned based on the number of execution threads executing on the active execution units.

[0006] [System Overview] 1 is a block diagram of a processing system 100 according to an embodiment. System 100 may be used in a single processor desktop system, a multiprocessor workstation system, or a server system with multiple processors 102 or processor cores 107. In one embodiment, system 100 is a processing platform integrated into a system-on-a-chip (SoC) integrated circuit for use in mobile, portable, or embedded devices, such as in Internet of Things (IoT) devices with wired or wireless connections to local or wide area networks.

[0007] In one embodiment, the system 100 may include, be coupled to, or be incorporated in a server-based gaming platform; a gaming machine, including a game and media console; a mobile gaming machine, a handheld gaming machine, or an online gaming machine. In some embodiments, the system 100 is part of a mobile phone, a smart phone, a tablet computing device, or a mobile Internet-connected device, such as a laptop with low internal storage capacity. The processing system 100 may also include, be coupled to, or be incorporated in a wearable device, such as a smart watch wearable device; an augmented reality (AR) or virtual reality (VR) augmented smart eyewear or clothing that provides visual, audio, or haptic output to supplement a real-world visual, audio, or haptic experience, or otherwise provides text, audio, graphics, video, holographic images or video, or haptic feedback; other augmented reality (AR) devices; or other virtual reality (VR) devices. In some embodiments, the processing system 100 includes, or is part of a television or set-top box device. In one embodiment, system 100 may include, be coupled to, or be incorporated into an autonomous vehicle, such as a bus, a tractor trailer, an automobile, a motorcycle or electric bicycle, an airplane or glider (or any combination thereof), etc. The autonomous vehicle may use system 100 to process the sensed environment around the vehicle.

[0008] In some embodiments, the one or more processors 102 each include one or more processor cores 107 to process instructions that, when executed, perform operations for system or user software. In some embodiments, at least one of the one or more processor cores 107 is configured to process a particular instruction set 109. In some embodiments, the instruction set 109 may facilitate Complex Instruction Set Computing (CISC), Reduced Instruction Set Computing (RISC), or Very Long Instruction Word (VLIW) computations. One or more processor cores 107 may process another instruction set 109, which may include instructions that facilitate emulation of other instruction sets. The processor cores 107 may also include other processing devices, such as a digital signal processor (DSP).

[0009] In some embodiments, the processor 102 includes a cache memory 104. Depending on the architecture, the processor 102 may have a single internal cache or multiple levels of internal cache. In some embodiments, the cache memory is shared among various components of the processor 102. In some embodiments, the processor 102 also uses an external cache (e.g., a level 3 (L3) cache or a last level cache (LLC)) that may be shared among the processor cores 107 using known cache coherency techniques. The processor 102 may further include a register file 106. The register file 106 may include various types of registers that store various types of data (e.g., integer registers, floating point registers, status registers, and an instruction pointer register). Some registers may be general purpose registers, while other registers may be specific to the design of the processor 102.

[0010] In some embodiments, the one or more processors 102 are coupled to one or more interface buses 110 to carry communication signals, such as address, data, or control signals, between the processors 102 and other components in the system 100. The interface bus 110, in one embodiment, can be a processor bus, such as a version of a Direct Media Interface (DMI) bus. However, the processor bus is not limited to a DMI bus and may include one or more peripheral component interconnect buses (e.g., PCI, PCI Express), memory buses, or other types of interface buses. In one embodiment, the processor 102 includes an integrated memory controller 116 and a platform controller hub 130. The memory controller 116 facilitates communication between memory devices and other components of the system 100, while the platform controller hub (PCH) 130 provides connectivity to I / O devices via a local I / O bus.

[0011] The memory device 120 can be a dynamic random access memory (DRAM) device, a static random access memory (SRAM) device, a flash memory device, a phase change memory device, or other memory device with suitable performance to act as a process memory. In one embodiment, the memory device 120 can operate as a system memory for the system 100 to store data 122 and instructions 121 used when the one or more processors 102 execute applications or processes. The memory controller 116 also couples to an optional external graphics processor 118. The external graphics processor 118 may communicate with one or more graphics processors 108 in the processor 102 to perform graphics and media operations. In some embodiments, graphics, media, and computation operations may be assisted by an accelerator 112, which is a co-processor that may be configured to perform a specialized set of graphics, media, or computation operations. For example, in one embodiment, the accelerator 112 is a matrix multiplication accelerator used to optimize machine learning or computation operations. In one embodiment, accelerator 112 is a ray tracing accelerator that may be used to perform ray tracing operations in conjunction with graphics processor 108. In one embodiment, external accelerator 119 may be used instead of or in conjunction with accelerator 112.

[0012] In some embodiments, a display device 111 may be connected to the processor 102. The display device 111 may be one or more of an internal display device such as found in a mobile electronic device or laptop device, or an external display device attached via a display interface (e.g., DisplayPort, etc.). In one embodiment, the display device 111 may be a head mounted display (HMD), such as a stereoscopic display device used in virtual reality (VR) or augmented reality (AR) applications.

[0013] In some embodiments, the platform controller hub 130 allows peripherals to connect to the memory device 120 and the processor 102 via a high-speed I / O bus. The I / O peripherals include, without limitation, an audio controller 146, a network controller 134, a firmware interface 128, a wireless transceiver 126, a touch sensor 125, and a data storage device 124 (e.g., non-volatile memory, volatile memory, hard disk drive, flash memory, NAND, 3D NAND, 3D XPoint, etc.). The data storage device 124 can be connected via a storage interface (e.g., SATA) or via a peripheral bus such as a Peripheral Component Interconnect bus (PCI, PCI Express). The touch sensor 125 can include a touch screen sensor, a pressure sensor, or a fingerprint sensor. The wireless transceiver 126 can be a Wi-Fi transceiver, a Bluetooth transceiver, or a mobile network transceiver such as a 3G, 4G, 5G, or Long Term Evolution (LTE) transceiver. Firmware interface 128 allows for communication with system firmware and can be, for example, a Unified Extensible Firmware Interface (UEFI). Network controller 134 can allow for network connectivity to a wired network. In some embodiments, a high performance network controller (not shown) couples to interface bus 110. Audio controller 146, in one embodiment, is a multi-channel high definition audio controller. In one embodiment, system 100 includes an optional legacy I / O controller 140 that couples legacy (e.g., Personal System 2 (PS / 2)) devices to the system. Platform controller hub 130 can also connect to one or more universal serial bus (USB) controllers 142 connected input devices, such as a keyboard and mouse 143 combination, a camera 144, or other USB input devices.

[0014] It will be understood that the illustrated system 100 is exemplary and not limiting, as other types of data processing systems configured differently may also be used. For example, instances of memory controller 116 and platform controller hub 130 may be incorporated into a separate external graphics processor, such as external graphics processor 118. In one embodiment, platform controller hub 130 and / or memory controller 116 may be external to one or more processors 102. For example, system 100 may include external memory controller 116 and platform controller hub 130, which may be configured as a memory controller hub and peripheral controller hub within a system chipset that communicates with the processor.

[0015] For example, the circuit boards ("sleds") upon which components such as CPUs, memory, and other components are located are designed for improved thermal performance. In some examples, processing components such as processors are located on the top side of the sled, while near memory such as DIMMs are located on the bottom side of the sled. As a result of the enhanced airflow provided by this design, the components can operate at higher frequencies and power levels in a typical system, improving performance. Additionally, the sleds are configured to blindly mate with power and data communication cables in the rack, thereby enhancing their ability to be quickly removed, upgraded, reinstalled, and / or replaced. Similarly, the individual components located on the sleds, such as the processors, accelerators, memory, and data storage drives, are configured to be easily upgraded by being widely spaced from one another. In an illustrative embodiment, the components further include hardware attestation mechanisms to attest to their authenticity.

[0016] Data centers can utilize a single network architecture ("fabric") that supports multiple other network architectures, including Ethernet and Omni-Path. Threads can be coupled to switches via optical fibers. Optical fibers provide higher bandwidth and lower latency than typical twisted pair cables (Cat 5, Cat 5e, Cat 6, etc.). With a high bandwidth, low latency interconnect and network architecture, data centers can pool resources such as memory, accelerators (e.g., GPUs, graphics accelerators, FPGAs, ASICs, neural network and / or artificial intelligence accelerators, etc.), and data storage drives that are physically disaggregated during use and provide them to computing resources (e.g., processors) as needed, allowing the computing resources to access the pooled resources as if they were local.

[0017] A power supply or source can provide voltage and / or current to system 100 or any component or system described herein. In one example, the power supply includes an AC / DC (alternating current to direct current) adapter that plugs into a wall outlet input. Such AC power can be a renewable energy (e.g., solar) source. In one example, the power source includes a DC power source, such as an external AC / DC converter. In one example, the power source or power supply includes wireless charging hardware that charges due to proximity to a charging field. In one example, the power source can include an internal battery, an AC supply, a motion-based power supply, a solar power source, or a fuel cell power source.

[0018] 2A-2D depict a computer system and graphics processor provided by embodiments described herein. Elements of FIGs. 2A-2D having the same reference numbers (or names) as elements of any other figure in the present application may operate or function similarly as described elsewhere herein, but are not limited to such.

[0019] 2A is a block diagram of an embodiment of a processor 200 including one or more processor cores 202A-202N, an integrated memory controller 214, and an integrated graphics processor 208. The processor 200 may include additional cores up to additional core 202N, represented by a dashed box. The processor cores 202A-202N include one or more internal cache units 204A-204N. In some embodiments, each processor core also has access to one or more shared cache units 206. The internal cache units 204A-204N and the shared cache unit 206 represent a cache memory hierarchy within the processor 200. The cache memory hierarchy may include at least one level of instruction and data cache within each processor core, and one or more levels of shared mid-level cache, such as a level 2 (L2), level 3 (L3), level 4 (L4), or other level cache, with the highest level of cache before the external memory being classified as LLC. In some embodiments, cache coherency logic maintains coherency between the various cache units 206 and 204A-204N.

[0020] In some embodiments, processor 200 may also include a set of one or more bus controller units 216 and a system agent core 210. The one or more bus controller units 216 manage a set of peripheral buses, such as one or more PCI or PCI Express buses. System agent core 210 provides management functions for the various processor components. In some embodiments, system agent core 210 includes one or more integrated memory controllers 214 to manage access to various external memory devices (not shown).

[0021] In some embodiments, one or more of the processor cores 202A-202N include support for synchronous multithreading. In such embodiments, the system agent core 210 includes components that coordinate and operate the cores 202A-202N during multithreaded processing. The system agent core 210 may further include a power control unit (PCU). The PCU includes logic and components that regulate the power state of the processor cores 202A-202N and the graphics processor 208.

[0022] In some embodiments, processor 200 further includes a graphics processor 208 to perform graphics processing operations. In some embodiments, graphics processor 208 couples to a set of shared cache unit 206 and a system agent core 210 that includes one or more integrated memory controllers 214. In some embodiments, system agent core 210 also includes a display controller 211 to drive the graphics processor output to one or more coupled displays. In some embodiments, display controller 211 may be a separate module coupled to the graphics processor via at least one interconnect, or may be incorporated within graphics processor 208.

[0023] In some embodiments, a ring-based interconnect unit 212 is used to couple the internal components of processor 200. However, alternative interconnect units such as point-to-point interconnects, switched interconnects, or other technologies, including those well known in the art, may be used. In some embodiments, graphics processor 208 couples to ring interconnect 212 via I / O link 213.

[0024] The example I / O link 213 represents at least one of a wide variety of I / O interconnects, including a package I / O interconnect that facilitates communication between various processor components and a high performance embedded memory module 218, such as an eDRAM module. In some embodiments, each of the processor cores 202A-202N and the graphics processor 208 can use the embedded memory module 218 as a shared last level cache.

[0025] In some embodiments, the processor cores 202A-202N are homogenous cores that execute the same instruction set architecture. In other embodiments, the processor cores 202A-202N are heterogeneous with respect to instruction set architecture (ISA), where one or more of the processor cores 202A-202N execute a first instruction set, while at least one of the remaining cores executes a subset of the first instruction set or another instruction set. In one embodiment, the processor cores 202A-202N are heterogeneous with respect to microarchitecture, where one or more cores having relatively higher power consumption are combined with one or more power cores having relatively lower power consumption. In one embodiment, the processor cores 202A-202N are heterogeneous with respect to computational capability. Additionally, the processor 200 may be implemented on one or more chips or as a SoC integrated circuit having the depicted components in addition to other components.

[0026] FIG. 2B is a block diagram of hardware logic of a graphics processor core 219 according to some embodiments described herein. Elements of 2B having the same reference numbers (or names) as elements of any other figures of the present application may operate or function similarly as described elsewhere herein, but are not limited to such. Graphics processor core 219, sometimes referred to as a core slice, can be one or more graphics cores in a modular graphics processor. Graphics processor core 219 is an example of one graphics core slice, and graphics processors described herein may include multiple graphics core slices based on target power and performance envelopes. Each graphics processor core 219 may include a fixed function block 230 coupled with multiple sub-cores 221A-221F, also referred to as sub-slices, that include modular blocks of general purpose fixed function logic.

[0027] In some embodiments, fixed function block 230 includes a geometry / fixed function pipeline 231 that may be shared by all sub-cores in graphics processor core 219, for example, in lower performance and / or lower power graphics processor implementations. In various embodiments, geometry / fixed function pipeline 231 includes a 3D fixed function pipeline (e.g., 3D pipeline 312 seen in Figures 3 and 4, described below), a video front end unit, a thread spawner and thread dispatcher, and a unified return buffer manager that manages a unified return buffer (e.g., unified return buffer 418 in Figure 4, described below).

[0028] In one embodiment, the fixed function block 230 also includes a graphics SoC interface 232, a graphics microcontroller 233, and a media pipeline 234. The graphics SoC interface 232 provides an interface between the graphics processor core 219 and other processor cores in the SoC integrated circuit. The graphics microcontroller 233 is a programmable sub-processor that is configurable to manage various functions of the graphics processor core 219, including thread dispatch, scheduling, and pre-emption. The media pipeline 234 (e.g., media pipeline 316 of FIGS. 3 and 4) includes logic that aids in decoding, encoding, pre-processing, and / or post-processing of multimedia data, including image and video data. The media pipeline 234 implements media operations by requesting computation or sampling logic in the sub-cores 221A-221F.

[0029] In one embodiment, SoC interface 232 enables graphics processor core 219 to communicate with general purpose application processor cores (e.g., CPUs) and / or other components in the SoC, including memory hierarchy elements such as shared last level cache memory, system RAM, and / or embedded on-chip or on-package DRAM. SoC interface 232 may also enable communication with fixed function devices in the SoC, such as a camera imaging pipeline, and enable and / or implement the use of global memory atomics that may be shared between graphics processor core 219 and CPUs in the SoC. SoC interface 232 may also implement power management controls for graphics processor core 219 and enable an interface between the clock domain of graphics processor core 219 and other clock domains in the SoC. In one embodiment, SoC interface 232 enables receipt of command buffers from a command streamer and global thread dispatcher configured to provide commands and instructions to each of one or more graphics cores in the graphics processor. Commands and instructions may be dispatched to the media pipeline 234 when a media operation is to be performed, or to the geometry and fixed function pipelines (e.g., geometry and fixed function pipelines 231, geometry and fixed function pipelines 237) when a graphics processing operation is to be performed.

[0030] Graphics microcontroller 233 may be configured to perform various scheduling and management tasks for graphics processor core 219. In one embodiment, graphics microcontroller 233 may perform graphics and / or compute workload scheduling to various graphics parallel engines within execution unit (EU) arrays 222A-222F, 224A-224F in sub-cores 221A-221F. In this scheduling model, host software executing on a CPU core of an SoC including graphics processor core 219 may submit a workload to one of multiple graphics processor doorbells, which invokes a scheduling operation on the appropriate graphics engine. Scheduling operations include determining which workload to run next, submitting the workload to a command streamer, preempting existing workloads running on the engines, monitoring the progress of the workloads, and notifying the host software when the workloads are complete. In one embodiment, graphics microcontroller 233 can also facilitate low power or idle states for graphics processor core 219, while giving graphics processor core 219 the ability to save and restore registers within graphics processor core 219 across low power state transitions independent of the operating system and / or graphics driver software on the system.

[0031] Graphics processor core 219 may have up to N modular sub-cores, more or less than the depicted sub-cores 221A-221F. For each set of N sub-cores, graphics processor core 219 may also include shared function logic 235, shared and / or cache memory 236, geometry / fixed function pipeline 237, and additional fixed function logic 238 to accelerate various graphics and computational operations. Shared function logic 235 may include logic units associated with the shared function logic of FIG. 4 (e.g., sampler, math, and / or inter-thread communication logic) that may be shared by each of the N sub-cores in graphics processor core 219. Shared and / or cache memory 236 may be a last level cache for the set of N sub-cores 221A-221F in graphics processor core 219, and may also function as a shared memory that is accessible by multiple sub-cores. Geometry / fixed-function pipeline 237 may be included in place of geometry / fixed-function pipeline 231 in fixed-function block 230 and may include the same or similar logic units.

[0032] In one embodiment, graphics processor core 219 includes additional fixed function logic 238, which may include various fixed function acceleration logic used by graphics processor core 219. In one embodiment, additional fixed function logic 238 includes an additional geometry pipeline used in position only shading. In position only shading, there are two geometry pipelines: a full geometry pipeline in geometry / fixed function pipelines 237, 231, and a cull pipeline, which is an additional geometry pipeline that may be included in additional fixed function logic 238. In one embodiment, the cull pipeline is a fragmented version of the full geometry pipeline. The full pipeline and the cull pipeline may run different instances of the same application, with each instance having a separate context. Position only shading may complete shading faster in some cases because it can hide long cull runs of discarded triangles. For example, in one embodiment, the cull pipeline logic in additional fixed function logic 238 may run position shaders in parallel with the main application, generally producing significant results faster than the full pipeline. This is because the cull pipeline only fetches and shades the position attributes of vertices without performing pixel rasterization and rendering to the framebuffer. The cull pipeline can use the generated material results to compute visibility information for all triangles, regardless of whether they are culled or not. The full pipeline (which in this example may also be called the replay pipeline) can consume the visibility information to skip culled triangles so as to shade only visible triangles that are eventually passed to the rasterization phase.

[0033] In one embodiment, the additional fixed function logic 238 may also include machine learning acceleration logic, such as fixed function matrix multiplication logic, for implementations that include optimizations for machine learning training or inference.

[0034] Within each graphics sub-core 221A-221F is a set of execution resources that can be used to perform graphics, media, and compute operations in response to requests by the graphics pipeline, media pipeline, or shader pipeline. The graphics sub-cores 221A-221F include a number of EU arrays 222A-222F, 224A-224F, thread dispatch and inter-thread communication (TD / IC) logic 223A-223F, 3D (e.g., texture) samplers 225A-225F, media samplers 206A-206F, shader processors 227A-227F, and shared local memories (SLMs) 228A-228F. The EU arrays 222A-222F, 224A-224F each include a number of execution units that are general purpose graphics processing units capable of performing floating point and integer / fixed point logic operations in service of graphics, media, or compute operations including graphics, media, or compute shader programs. The TD / IC logic 223A-223F performs local thread dispatch and thread control operations for the execution units in the sub-cores and facilitates communication between threads executing on the execution units of the sub-cores. The 3D samplers 225A-225F can read textures or other 3D graphics related data into memory. The 3D samplers can read texture data differently based on the texture format and configured sample state associated with a given texture. The media samplers 206A-206F can perform similar read operations based on the type and format associated with the media data. In one embodiment, each graphics sub-core 221A-221F can alternatively include an integrated 3D and media sampler. Threads executing in each execution unit of sub-cores 221A-221F can use shared local memory 228A-228F within each sub-core, such that threads executing within a thread group can execute using a common pool of on-chip memory.

[0035] 2C depicts a graphics processing unit (GPU) 239 that includes a dedicated set of graphics processing resources arranged in multi-core groups 240A-240N. Although details of a single multi-core group 240A are given, it will be understood that the other multi-core groups 240B-240N may be equipped with the same or similar sets of graphics processing resources.

[0036] As depicted, multi-core group 240A may include a set of graphics cores 243, a set of tensor cores 244, and a set of ray tracing cores 245. Scheduler / dispatcher 241 schedules and dispatches the graphics threads to execute on the various cores 243, 244, 245. A set of register files 242 store operand values ​​used by cores 243, 244, 245 when executing the graphics threads. They may include, for example, integer registers to store integer values, floating point registers to store floating point values, vector registers to store packed data elements (integer and / or floating point data elements), and tile registers to store tensor / matrix values. In one embodiment, the tile registers are implemented as a combined set of vector registers.

[0037] One or more combined level 1 (L1) cache and shared memory units 247 store graphics data such as texture data, vertex data, pixel data, ray data, bounding volume data, etc. locally within each multicore group 240A. One or more texture units 247 may also be used to perform texturing operations such as texture mapping and sampling. A level 2 (L2) cache 253 shared by all or some of the multicore groups 240A-240N stores graphics data and / or instructions for multiple concurrent graphics threads. As shown, the L2 cache 253 may be shared across multiple multicore groups 240A-240N. One or more memory controllers 248 couple the GPU 239 to memory 249. The memory 249 may be system memory (e.g., DRAM) and / or dedicated graphics memory (e.g., GDDR6 memory).

[0038] Input / output (I / O) circuitry 250 couples GPU 239 to one or more I / O devices 252, such as a digital signal processor (DSP), a network controller, or a user input device. An on-chip interconnect may be used to couple I / O device 252 to GPU 239 and memory 249. One or more I / O memory management units (IOMMUs) 251 of I / O circuitry 250 directly couple I / O device 252 to system memory 249. In one embodiment, IOMMU 251 manages sets of page tables to map virtual addresses to physical addresses in system memory 249. In this embodiment, I / O device 252, CPU 246, and GPU 239 may share the same virtual address space.

[0039] In one implementation, IOMMU 251 supports virtualization. In this case, it may manage a first set of page tables that map guest / graphics virtual addresses to guest / graphics physical addresses and a second set of page tables that map guest / graphics physical addresses to system / host physical addresses (e.g., in system memory 249). The base addresses of each of the first and second sets of page tables are stored in control registers and may be swapped out on a context switch (e.g., so that a new context is given access to the associated set of page tables). Although not shown in FIG. 2C, each of cores 243, 244, 245 and / or multicore groups 240A-240N may include a translation lookaside buffer (TLB) that caches guest virtual to guest physical, guest physical to host physical, and guest virtual to host physical translations.

[0040] In one embodiment, CPU 246, GPU 239, and I / O devices 252 are integrated onto a single semiconductor chip and / or chip package. Memory 249 is depicted as being integrated onto the same chip or may be coupled to memory controller 248 via an off-chip interface. In one embodiment, memory 249 comprises GDDR6 memory that shares the same virtual address space as other physical system level memory. However, the underlying principles of the present invention are not limited to this specific implementation.

[0041] In one embodiment, tensor cores 244 include multiple execution units specifically designed to perform matrix operations, which are fundamental computational operations used to perform deep learning computations. For example, simultaneous matrix multiplication operations may be used for neural network training and inference. Tensor cores 244 may perform matrix operations using a variety of operand precisions, including single-precision floating point (e.g., 32 bits), half-precision floating point (e.g., 16 bits), integer word (16 bits), byte (8 bits), and half-byte (4 bits). In one embodiment, the neural network implementation extracts features of each rendered scene to potentially combine details from multiple frames to compose a high-quality final image.

[0042] In a deep learning implementation, parallel matrix computation work may be scheduled for execution on tensor cores 244. Training neural networks, among other things, requires efficient matrix dot product operations. To process the inner product formulation of N×N×N matrix multiplication, tensor cores 244 may include at least N dot product processing elements. Before the matrix multiplication begins, an entire matrix is ​​loaded into a tile register, and at least one column of a second matrix is ​​loaded each of the N cycles. There are N dot products to be processed per cycle.

[0043] Matrix elements may be stored in different precisions depending on the particular implementation, including 16-bit words, 8-bit bytes (e.g., INT8), and 4-bit half-bytes (e.g., INT4). Different precision modes may be specified for tensor cores 244 to ensure that the most efficient precision is used for different workloads (e.g., inferring workloads that can tolerate quantization to bytes and half-bytes, etc.).

[0044] In one embodiment, ray tracing core 245 accelerates ray tracing operations for both real-time and non-real-time ray tracing implementations. In particular, ray tracing core 245 includes ray traversal / intersection circuitry to perform ray traversal using a bounding volume hierarchy (BVH) and identify intersections between rays and primitives in the BVH volume. Ray tracing core 245 may also include circuitry to perform depth testing and culling (e.g., using a Z-buffer or similar arrangement). In one implementation, ray tracing core 245 performs traversal and intersection operations in conjunction with the image denoising techniques described herein. At least a portion of the image denoising techniques may be performed in tensor core 244. For example, in one embodiment, tensor core 244 implements a deep learning neural network to perform denoising of frames generated by ray tracing core 245. It should be noted that the CPU 246, the graphics core 243, and / or the ray tracing core 245 may also implement all or part of the denoising and / or deep learning algorithms.

[0045] Additionally, as described above, a distributed approach to denoising may be used, where the GPU 239 is in a computing device that is coupled to other computing devices via a network or high speed interconnect. In this embodiment, the interconnected computing devices may share neural network learning / training data to perform denoising for different types of image frames and / or different graphics applications, and to improve the speed at which the entire system learns.

[0046] In one embodiment, the ray tracing core 245 handles all BVH traversal and ray primitive intersections to keep the graphics core 243 from being overloaded with thousands of instructions per ray. In one embodiment, each ray tracing core 245 includes a first set of dedicated circuitry to perform bounding box tests (e.g., for traversal operations) and a second set of dedicated circuitry to perform ray triangle intersection tests (traversed intersecting rays). Thus, in one embodiment, the multi-core group 240A can simply launch ray probes, and the ray tracing cores 245 independently perform ray traversal and intersections and return hit data (e.g., hit, no hit, multiple hits, etc.) to the thread context. While the ray tracing core 245 performs traversal and intersection operations, the other cores 243, 244 are freed to perform other graphics or computational work.

[0047] In one embodiment, each ray tracing core 245 includes a traversal unit that performs BVH test operations and an intersection unit that performs ray primitive intersection tests. The intersection unit generates a "hit," "no hit," or "multi-hit" response, which is delivered to the appropriate thread. During traversal and intersection operations, the execution resources of other cores (e.g., graphics core 243 and tensor core 244) are freed to perform other forms of graphics work.

[0048] In one particular embodiment described below, a hybrid rasterization / ray tracing approach is used, with work distributed between the graphics core 243 and the ray tracing core 245.

[0049] In one embodiment, the ray tracing core 245 (and / or other cores 243, 244) includes hardware support for a ray tracing instruction set such as Microsoft's DirectX Ray Tracing (DXR), including ray-generation, closest-hit, any-hit, and miss shaders and DispatchRays commands that allow for the allocation of a unique set of shaders and textures per object. Another ray tracing platform that may be supported by the ray tracing core 245, graphics core 243, and tensor core 244 is Vulkan 1.1.85. It should be noted that the principles underlying the present invention are not limited to any particular ray tracing ISA.

[0050] In general, the various cores 245, 244, 243 may support a ray tracing instruction set that includes instructions / functions for ray-generation, closest-hit, any-hit, ray-primitive intersection, per-primitive and hierarchical bounding box construction, miss, visit, and exceptions. More specifically, one embodiment includes ray tracing instructions that perform the following functions:

[0051] Ray-generation: Ray-generation commands can be performed on each pixel, sample, or other user-defined work allocation.

[0052] closest-hit: The closest-hit command can be executed to locate the closest intersection point of a ray with a primitive in the scene.

[0053] any-hit: The any-hit command identifies multiple intersections between primitives in the scene and rays to potentially identify a new closest intersection point.

[0054] The intersection:intersection command performs a ray-primitive intersection test and outputs the result.

[0055] per-primitive bounding box construction: This instruction creates a bounding box around a given primitive or group of primitives (e.g., when building a new BVH or other acceleration data structure).

[0056] miss: indicates that the ray misses all geometry in the scene or a specialized region of the scene.

[0057] visit: indicates the child volume that the ray traverses.

[0058] exceptions: Indicates different types of exception handlers (e.g., called for different error conditions).

[0059] FIG. 2D illustrates a general purpose graphics processing unit (GPGPU), which may be configured as a graphics processor and / or computational accelerator in accordance with embodiments described herein. 2 is a block diagram of a GPGPU 270. GPGPU 270 may be interconnected with a host processor (e.g., one or more CPUs 246) and memories 271, 272 via one or more system and / or memory buses. In one embodiment, memory 271 is system memory that may be shared with one or more CPUs 246, while memory 272 is device memory that is dedicated to GPGPU 270. In one embodiment, components within GPGPU 270 and device memory 272 may be mapped to memory addresses accessible to one or more CPUs 246. Access to memories 271 and 272 may be facilitated via memory controller 268. In one embodiment, memory controller 268 includes an internal direct memory access (DMA) controller 269 or may include logic to perform operations that would otherwise be performed by a DMA controller.

[0060] The GPGPU 270 includes a number of cache memories, including an L2 cache 253, an L1 cache 254, an instruction cache 255, and a shared memory 256, some of which may also be partitioned as cache memories. The GPGPU 270 also includes a number of computation units 260A-260N. Each computation unit 260A-260N includes a vector register 261, a scalar register 262, a vector logic unit 263, and a scalar logic unit 264. The computation units 260A-260N may also include a local shared memory 265 and a program counter 266. The computation units 260A-260N may be coupled with a constant cache 267, which may be used to store constant data, which is data that does not change during the execution of a kernel or shader program executed by the GPGPU 270. In one embodiment, the constant cache 267 is a scalar data cache, and cached data may be fetched directly into the scalar register 262.

[0061] During operation, one or more CPUs 246 can write commands to registers or memory in GPGPU 270 that are mapped into the accessible address space. Command processor 257 can read commands from registers or memory and determine how those commands are processed in GPGPU 270. Thread dispatcher 258 can then be used to dispatch threads to computation units 260A-260N to execute those commands. Each computation unit 260A-260N can execute threads independently of the other computation units. Moreover, each computation unit 260A-260N can be independently configured for conditional computation and can conditionally output the results of the computation to memory. Command processor 257 can interrupt one or more CPUs 246 when a submitted command is completed.

[0062] 3A-3C depict block diagrams of further graphics processor and computation accelerator architectures provided by embodiments described herein. Elements of FIGs. 3A-3C having the same reference numbers (or names) as elements of any other figure in the present application can operate or function similarly as described elsewhere herein, but are not limited to such.

[0063] 3A is a block diagram of a graphics processor 300. Graphics processor 300 may be a discrete graphics processing unit or may be a graphics processor integrated with multiple processing cores or other semiconductor devices such as, without limitation, memory devices or network interfaces. In some embodiments, the graphics processor communicates with registers on the graphics processor through a memory-mapped I / O interface and with commands placed in the processor memory. In some embodiments, graphics processor 300 may interface to local memory, one or more internal caches, one or more shared external caches, and / or system memory.

[0064] In some embodiments, the graphics processor 300 also includes a display controller 302 that drives display output data to a display device 318. The display controller 302 includes hardware for one or more overlay planes for the display and compositing of multiple layers of video or user interface elements. The display device 318 can be an internal or external display device. In one embodiment, the display device 318 is a head-mounted display device, such as a virtual reality (VR) display device or an augmented reality (AR) display device. In some embodiments, graphics processor 300 includes a video codec engine 306 to encode, decode, or transcode media to or from one or more media coding formats, including but not limited to Moving Picture Experts Group (MPEG) formats such as MPEG-2, Advanced Video Coding (AVC) formats such as H.264 / MPEG-4 AVC, H.265 / HEVC, Alliance for Open Media (AOMedia) VP8, VP9, ​​and Society of Motion Picture & Television Engineers (SMPTE) 421M / VC1, and Joint Photographic Experts Group (JPEG) formats such as JPEG and Motion JPEG (MJPEG) formats.

[0065] In some embodiments, graphics processor 300 includes a block image transfer (BLIT) engine 304 to perform two-dimensional (2D) rasterizer operations, including, for example, bit-boundary block transfers, although in one embodiment, 2D graphics operations are performed using one or more components of a graphics processing engine (GPE) 310. In some embodiments, GPE 310 is a computation engine for performing graphics operations, including three-dimensional (3D) graphics operations and media operations.

[0066] In some embodiments, GPE 310 includes a 3D pipeline 312 for performing 3D operations such as rendering three-dimensional images and scenes using processing functions that operate on 3D primitive shapes (e.g., rectangles, triangles, etc.). 3D pipeline 312 includes programmable fixed-function elements that spawn execution threads to 3D / media subsystem 315 and / or perform various tasks within the elements. While 3D pipeline 312 may be used to perform multiple operations, embodiments of GPE 310 also include a media pipeline 316 that is used specifically to perform media operations such as video post-processing and image enhancement.

[0067] In some embodiments, the media pipeline 316 includes fixed function or programmable logic units to perform one or more specialized media operations, such as video decoding acceleration, video deinterlacing, and video encoding acceleration, on behalf of or for the video codec engine 306. In some embodiments, the media pipeline 316 further includes a thread generation unit to generate threads for execution in the 3D / media subsystem 315. The generated threads perform computations for the media operations in one or more graphics execution units included in the 3D / media subsystem 315.

[0068] In some embodiments, the 3D / Media subsystem 315 includes logic for executing threads created by the 3D pipeline 312 and the media pipeline 316. In one embodiment, the pipelines send thread execution requests to the 3D / Media subsystem 315. The 3D / Media subsystem 315 includes thread dispatch logic that arbitrates and dispatches the various requests to available thread execution resources. The execution resources include an array of graphics execution units that process the 3D and media threads. In some embodiments, the 3D / Media subsystem 315 includes one or more internal caches for thread instructions and data. In some embodiments, the subsystems also include shared memory, including registers and addressable memory, for providing data between threads and for storing output data.

[0069] FIG. 3B illustrates a graphics processor 320 having a tiled architecture in accordance with an embodiment described herein. In one embodiment, the graphics processor 320 includes a graphics processing engine cluster 322 having multiple instances of the graphics processing engine 310 of FIG. 3 in graphics engine tiles 310A-310D. Each graphics engine tile 310A-310D may be interconnected via a set of tile interconnects 323A-323F. Each graphics engine tile 310A-310D may also be connected to a memory module or memory device 326A-326D via memory interconnects 325A-325D. The memory devices 326A-326D may use any graphics memory technology. For example, the memory devices 326A-326D may be graphics double data rate (GDDR) memory. The memory devices 326A-326D, in one embodiment, are high bandwidth memory (HMB) modules that may be on-die with their respective graphics engine tiles 310A-310D. In one embodiment, the memory devices 326A-326D are stacked memory devices that may be stacked on top of their respective graphics engine tiles 310A-310D. In one embodiment, each graphics engine tile 310A-310D and associated memory 326A-326D is on a separate chiplet that is bonded to a base die or base substrate, as described in further detail in FIGs. 11B-11D.

[0070] The graphics processing engine cluster 322 may be connected to an on-chip or on-package fabric interconnect 324. The fabric interconnect 324 may enable communication between the graphics engine tiles 310A-310D and components such as the video codec engine 306 and one or more copy engines 304. The copy engine 304 may be used to move data to, from, into, and between the memory devices 326A-326D and memory external to the graphics processor 320 (system memory). The fabric interconnect 324 may also be used to interconnect the graphics engine tiles 310A-310D. The graphics processor 320 may optionally include a display controller 302 to enable connection to an external display device 318. The graphics processor may also be configured as a graphics or computation accelerator. In an accelerator configuration, the display controller 302 and the display device 318 may be omitted.

[0071] Graphics processor 320 may be connected to a host system via host interface 328. Host interface 328 may enable communication between graphics processor 320, system memory, and / or other system components. Host interface 328 may be, for example, a PCI Express bus or other type of host system interface.

[0072] FIG. 3C illustrates a computation accelerator 330 according to an embodiment described herein. The computation accelerator 330 may include architectural similarities to the graphics processor 320 of FIG. 3B and is optimized for computation acceleration. The computation engine cluster 332 may include a set of computation engine tiles 340A-340D that include execution logic optimized for parallel or vector-based general-purpose computation operations. In some embodiments, the computation engine tiles 340A-340D do not include fixed-function graphics processing logic, but in one embodiment, one or more of the computation engine tiles 340A-340D may include logic to perform media acceleration. The computation engine tiles 340A-340D may be connected to memories 326A-326D via memory interconnects 325A-325D. The memories 326A-326D and memory interconnects 325A-325D may be of similar technology as found in the graphics processor 320, or may be different. The graphics computation engine tiles 340A-340D may also be interconnected via a set of tile interconnects 323A-323F, which may be connected to and / or interconnected by the fabric interconnect 324. In one embodiment, the computation accelerator 330 includes a large L3 cache 336, which may be configured as a device-wide cache. The computation accelerator 330 may also be connected to a host processor and memory via a host interface 328, similar to the graphics processor 320 of FIG. 3B.

[0073] [Graphics processing engine] FIG. 4 is a block diagram of a graphics processing engine 410 of a graphics processor according to some embodiments. In one embodiment, the graphics processing engine (GPE) 410 is a variation of the GPE 310 shown in FIG. 3A and may represent the graphics engine tiles 310A-310D of FIG. 3B. Elements of FIG. 4 having the same reference numbers (or names) as elements of any other figure in the present application may operate or function similarly as described elsewhere herein, but are not limited to such. For example, the 3D pipeline 312 and media pipeline 316 of FIG. 3A are depicted. The media pipeline 316 is optional in some embodiments of the GPE 410 and may not be explicitly included within the GPE 410. For example, in at least one embodiment, a separate media and / or image processor is coupled to the GPE 410.

[0074] In some embodiments, the GPE 410 is coupled to or includes a command streamer 403. The command streamer 403 provides a command stream to the 3D pipeline 312 and / or the media pipeline 316. In some embodiments, the command streamer 403 is coupled to a memory. The memory can be a system memory or one or more of an internal cache memory and a shared cache memory. In some embodiments, the command streamer 403 receives commands from the memory and sends the commands to the 3D pipeline 312 and / or the media pipeline 316. The commands are instructions fetched from a ring buffer that stores commands for the 3D pipeline 312 and the media pipeline 316. In one embodiment, the ring buffer can further include a batch command buffer that stores a batch of multiple commands. The commands for the 3D pipeline 312 can also include references to data stored in memory, such as, for example, without limitation, vertex and geometry data for the 3D pipeline 312 and / or image data and memory objects for the media pipeline 316. The 3D pipeline 312 and the media pipeline 316 process commands and data by executing operations via logic within the respective pipelines or by dispatching one or more execution threads to the graphics core array 414. In one embodiment, the graphics core array 414 includes one or more blocks of graphics cores (e.g., graphics core 415A, graphics core 415B), each block including one or more graphics cores. Each graphics core includes a set of graphics execution resources including general-purpose and graphics-specific execution logic for performing graphics and computational operations, along with fixed-function texture processing and / or machine learning and artificial intelligence acceleration logic.

[0075] In various embodiments, the 3D pipeline 312 may include fixed function and programmable logic to process one or more shader programs, such as vertex shaders, geometry shaders, pixel shaders, fragment shaders, compute shaders, or other shader programs, by processing instructions and dispatching execution threads to the graphics core array 414. The graphics core array 414 provides a unified block of execution resources used in processing those shader programs. General purpose execution logic (e.g., execution units) within the graphics cores 415A-415B of the graphics core array 414 includes support for a variety of 3D API shader languages ​​and can execute multiple concurrent threads of execution associated with multiple shaders.

[0076] In some embodiments, graphics core array 414 includes execution logic to perform media functions such as video and / or image processing. In one embodiment, the execution unit includes general purpose logic that is programmable to perform parallel general purpose computing operations in addition to graphics processing operations. The general purpose logic can perform processing operations in parallel with or together with the general purpose logic in processor core 107 of FIG. 1 or cores 202A-202N of FIG. 2A.

[0077] Output data generated by threads executing in graphics core array 414 may output data to memory in unified return buffer (URB) 418. URB 418 may store data for multiple threads. In some embodiments, URB 418 may be used to route data between different threads executing in graphics core array 414. In some embodiments, URB 418 may also be used for synchronization between threads in graphics core array 414 and fixed function logic in shared functional logic 420.

[0078] In some embodiments, graphics core array 414 is scalable such that the array includes a variable number of graphics cores, each having a variable number of execution units based on the target power and performance levels of GPE 410. In one embodiment, the execution resources are dynamically scalable such that execution resources can be enabled or disabled as needed.

[0079] The graphics core array 414 couples to shared functionality logic 420, which includes a number of resources shared among the graphics cores in the graphics core array. The shared functionality in the shared functionality logic 420 are hardware logic units that provide specialized auxiliary functions to the graphics core array 414. In various embodiments, the shared functionality logic 420 includes, without limitation, sampler 421, mass 422, and inter-thread communication (ITU) 423 logic. Additionally, some embodiments implement one or more caches 425 within the shared functionality logic 420.

[0080] Shared functionality is implemented at least when the demand for a given specialized functionality is insufficient for the instructions in the graphics core array 414. Instead, a single instantiation of that specialized functionality is implemented as a stand-alone entity in the shared functionality logic 420 and shared among the execution resources in the graphics core array 414. The exact set of functionality shared among and included within the graphics core array 414 varies with embodiment. In some embodiments, certain shared functionality in the shared functionality logic 420 that is used extensively by the graphics core array 414 may be included within the shared functionality logic 416 in the graphics core array 414. In various embodiments, the shared functionality logic 416 in the graphics core array 414 may include some or all of the logic in the shared functionality logic 420. In one embodiment, all of the logic elements in the shared functionality logic 420 may be duplicated in the shared functionality logic 416 of the graphics core array 414. In one embodiment, the shared functionality logic 420 is removed in favor of the shared functionality logic 416 of the graphics core array 414 .

[0081] [Execution unit] 5A-5B depict thread execution logic 500 including an array of processing elements for use in a graphics processor according to embodiments described herein. Elements of FIG. 5A-5B having the same reference numbers (or names) as elements of any other figures of the present application may operate or function similarly as described elsewhere herein, but are not limited to such. FIG. 5A-5B depict an overview of thread execution logic 500 that may represent the hardware logic represented by each of sub-cores 221A-221F of FIG. 2B. FIG. 5A depicts an execution unit in a general-purpose graphics processor, and FIG. 5B depicts an execution unit that may be used in a computational accelerator.

[0082] 5A, in some embodiments, the thread execution logic 500 includes a shader processor 502, a thread dispatcher 504, an instruction cache 506, a scalable execution unit array including multiple execution units 508A-508N, a sampler 510, a shared local memory 511, a data cache 512, and a data port 514. In one embodiment, the scalable execution unit array may be dynamically scaled by enabling or disabling one or more execution units (e.g., any of execution units 508A, 508B, 508C, 508D, through 508N-1 and 508N) based on the computational requirements of the workload. In one embodiment, the included components are interconnected via an interconnect fabric that provides links to each of the components. In some embodiments, the thread execution logic 500 includes one or more connections to memory, such as system memory or cache memory, through an instruction cache 506, a data port 514, a sampler 510, and execution units 508A-508N. In some embodiments, each execution unit (e.g., 508A) is a stand-alone programmable general-purpose computational unit capable of executing multiple simultaneous hardware threads, processing multiple data elements in parallel for each thread. In various embodiments, the array of execution units 508A-508N is scalable to include any number of individual execution units.

[0083] In some embodiments, the execution units 508A-508N are primarily used to execute shader programs. The shader processor 502 processes various shader programs and may dispatch execution threads associated with the shader programs by the thread dispatcher 504. In one embodiment, the thread dispatcher includes logic to arbitrate thread initiation requests from the graphics and media pipelines and instantiate the requested threads on one or more of the execution units 508A-508N. For example, the geometry pipeline may dispatch vertex, tessellation, or geometry shaders to thread execution logic for processing. In some embodiments, the thread dispatcher 504 may also handle run-time thread creation requests from executing shader programs.

[0084] In some embodiments, the execution units 508A-508N support an instruction set that includes native support for many standard 3D graphics shader instructions so that shader programs from the graphics fabric (Direct3D and OpenGL) are executed with minimal translation. The execution units support vertex and geometry processing (e.g., vertex programs, geometry programs, vertex shaders), pixel processing (e.g., pixel shaders, fragment shaders), and general-purpose processing (e.g., compute and media shaders). Each of the execution units 508A-508N is capable of multi-issue single instruction multiple data (SIMD) execution, and multi-threaded operation enables an efficient execution environment in the face of longer latency memory accesses. Each hardware thread within each execution unit has a dedicated high-bandwidth register file and associated independent thread state. Execution is multi-issue per clock for a pipeline capable of integer, single and double precision floating point operations, SIMD branch functions, logical operations, transcendental operations, and a variety of other operations. While waiting for data from memory or one of the shared functions, subordinate logic within the execution units 508A-508N causes the waiting thread to suspend activity until the requested data is returned. While the waiting thread is paused, hardware resources may be dedicated to processing other threads. For example, during a delay associated with a vertex shader operation, the execution unit may execute operations for other types of shader programs, including a pixel shader, a fragment shader, or another vertex shader. Various embodiments may be adapted to use execution through the use of single instruction multiple threads (SIMT) as an alternative to or in addition to the use of SIMD. Reference to a SIMD core or operation may also apply to SIMT or SIMD in addition to SIMT.

[0085] Each execution unit in execution units 508A-508N operates on an array of data elements. The number of data elements is the "execution size," or number of channels for an instruction. An execution channel is a logical unit of execution for data element access, masking, and flow control within an instruction. The number of channels is independent of the number of physical Arithmetic Logic Units (ALUs) or Floating Point Units (FPUs) for a particular graphics processor. In some embodiments, execution units 508A-508N support integer and floating point data types.

[0086] The execution unit instruction set includes SIMD instructions. Various data elements may be stored as packed data types in registers, and the execution unit processes the various elements based on the data size of the elements. For example, when operating on a 256-bit wide vector, 256 bits of the vector are stored in a register, and the execution unit operates on the vector as four separate 54-bit packed data elements (quadword (QW) size data elements), eight separate 32-bit packed data elements (doubleword (DW) size data elements), sixteen separate 16-bit packed data elements (word (W) size data elements), or thirty-two separate 8-bit data elements (byte (B) size data elements). Note that different vector widths and register sizes are possible.

[0087] In one embodiment, one or more execution units may be grouped into a fused execution unit 509A-509N with thread control logic (507A-507N) common to the fused EUs. Multiple EUs may be fused into an EU group. Each EU in a fused EU group may be configured to execute a separate SIMD hardware thread. The number of EUs in a fused EU group may vary according to the embodiment. Additionally, various SIMD widths may be executed per EU, including, without limitation, SIMD8, SIMD16, and SIMD32. Each fused graphics execution unit 509A-509N includes at least two execution units. For example, the fused execution units 509A-509N include a first EU 508A, a second EU 508B, and thread control logic 507A common to the first EU 508A and the second EU 508B. The thread control logic 507A controls the threads executed in the fused graphics execution unit 509A, and enables each EU in the fused execution units 509A-509N to execute using a common instruction pointer register.

[0088] One or more internal instruction caches (e.g., 506) are included in the thread execution logic 500 for caching thread instructions for the execution units. In some embodiments, one or more data caches (e.g., 502) are included for caching thread data during thread execution. Threads executing in the execution logic 500 may also store explicitly managed data in the shared local memory 511. In some embodiments, a sampler 510 is included to provide texture sampling for 3D operations and media sampling for media operations. In some embodiments, the sampler 510 includes specialized texture or media sampling functionality to process texture or media data during the sampling process before providing the sampled data to the execution units.

[0089] During execution, the graphics and media pipeline sends thread start requests to the thread execution logic 500 via thread creation and dispatch logic. Once a group of geometric objects has been processed and rasterized into pixel data, pixel processor logic (e.g., pixel shader logic, fragment shader logic, etc.) within the shader processor 502 is invoked to further compute output information and cause the results to be written to an output surface (e.g., color buffer, depth buffer, stencil buffer, etc.). In some embodiments, the pixel shader or fragment shader computes values ​​for various vertex attributes that are to be interpolated across the rasterized objects. In some embodiments, the pixel processor logic within the shader processor 502 then executes pixel or fragment shader programs provided by an application programming interface (API). To execute the shader programs, the shader processor 502 dispatches threads to the execution units (e.g., 508A) via the thread dispatcher 504. In some embodiments, the shader processor 502 uses texture sampling logic within the sampler 510 to access texture data within texture maps stored in memory. Arithmetic operations on the texture data and the input geometry data compute pixel color data for each geometric fragment or discard one or more pixels from further processing.

[0090] In some embodiments, the data port 514 provides a memory access mechanism for the thread execution logic 500 to output processed data to memory for further processing in the graphics processor output pipeline. In some embodiments, the data port 514 includes or is coupled to one or more cache memories (e.g., data cache 512) that cache data for memory access via the data port.

[0091] In one embodiment, the execution logic 500 may also include a ray tracer 505 that may provide ray tracing acceleration functionality. The ray tracer 505 may support a ray tracing instruction set that includes instructions / functions for ray generation. The ray tracing instruction set may be similar to or different from the ray tracing instruction set supported by the ray tracing core 245 of FIG. 2C.

[0092] 5B illustrates internal details of execution unit 508 according to an embodiment. GSU 508 may include instruction fetch unit 537, general purpose register file array (GRF) 524, architectural register file array (ARF) 526, thread arbiter 522, send unit 530, branch unit 532, a set of SIMD floating point units (FPUs) 534, and in one embodiment, a set of dedicated integer SIMD ALUs 535. GRF 524 and ARF 526 include a set of general purpose register files and architectural register files associated with each concurrent hardware thread that may be active in GSU 508. In one embodiment, per-thread architectural state is maintained in ARF 526, while data used during thread execution is stored in GRF. Execution state of each thread, including an instruction pointer for each thread, may be maintained in thread-specific registers within ARF 526.

[0093] In one embodiment, the graphics execution unit 508 has an architecture that is a combination of simultaneous multithreading (SMT) and fine-grained interleaved multithreading (IMT). The architecture has a modular organization that can be tuned at design time based on the target number of simultaneous threads and the number of registers per execution unit, and the execution unit resources are divided across the logic used to execute multiple simultaneous threads. The number of logical threads that can be executed by the graphics execution unit 508 is not limited to the number of hardware threads, and multiple logical threads can be assigned to each hardware thread.

[0094] In one embodiment, the graphics execution unit 508 can co-issue multiple instructions, each of which can be a different instruction. The thread arbiter 522 of the graphics execution unit thread 508 can dispatch the instruction to one of the send unit 530, the branch unit 532, or the SIMD FPU 534 for execution. Each execution thread can access 128 general purpose registers in the GRF 524, each of which can store 32 bytes accessible as a SIMD 8-element vector of 32-bit data elements. In one embodiment, each execution unit thread has access to 4K bytes in the GRF 524, although embodiments are not so limited and more or less register resources may be provided in other embodiments. In one embodiment, the graphics execution unit 508 is partitioned into seven hardware threads that can independently perform computational operations, although the number of threads per execution unit can also vary according to the embodiment. For example, in one embodiment, up to 16 hardware threads are supported. In an embodiment where seven threads may access 4K bytes, GRF 524 can store a total of 28K bytes. If 16 threads may access 4K bytes, GRF 524 can store a total of 64K bytes. Flexible addressing modes can allow registers to be addressed together to build effectively wider registers or to represent strided rectangular block data structures.

[0095] In one embodiment, memory operations, sampler operations, and other longer latency system communications are dispatched via "send" instructions executed by a message passing send unit 530. In one embodiment, branch instructions are dispatched to a dedicated branch unit 532 to expedite SIMD branching and eventual convergence.

[0096] In one embodiment, the graphics execution unit 508 includes one or more SIMD floating point units (FPUs) 534 to perform floating point operations. In one embodiment, the FPUs 534 also support integer calculations. In one embodiment, the FPUs 534 can SIMD execute up to M 32-bit floating point (or integer) operations or SIMD execute up to 2M 16-bit integer or 16-bit floating point operations. In one embodiment, at least one of the FPUs provides extended math capabilities to support high throughput transcendental math functions and double precision 54-bit floating point. In some embodiments, a set of 8-bit integer SIMD ALUs 535 are also present and may be specifically optimized to perform operations related to machine learning calculations.

[0097] In one embodiment, an array of multiple instances of the graphics execution unit 508 may be instantiated in a graphics sub-core grouping (e.g., a sub-slice). For scalability, product designers may choose the exact number of execution units per sub-core grouping. In one embodiment, the execution unit 508 may execute instructions across multiple execution channels. In a further embodiment, each thread executing in the graphics execution unit 508 may execute on a different channel.

[0098] FIG. 6 depicts a further execution unit 600 according to an embodiment. The execution unit 600 may be, for example, a compute-optimized execution unit used in the compute engine tiles 340A-340D of FIG. 3C, but is not limited as such. Variations of the execution unit 600 may also be used in the graphics engine tiles 310A-310B of FIG. 3B. In one embodiment, the execution unit 600 includes a thread control unit 601, a thread state unit 602, an instruction fetch / prefetch unit 603, and an instruction decode unit 604. The execution unit 600 further includes a register file 606 that stores registers that may be assigned to hardware threads within the execution unit. The execution unit 600 further includes a send unit 607 and a branch unit 608. In one embodiment, the send unit 607 and the branch unit 608 may operate similarly to the send unit 530 and the branch unit 532 of the graphics execution unit 508 of FIG. 5B.

[0099] Execution unit 600 also includes computation unit 610, which includes a variety of different types of functional units. In one embodiment, computation unit 610 includes ALU unit 611, which includes an array of ALUs. ALU unit 611 may be configured to perform 64-bit, 32-bit, and 16-bit integer and floating point operations. The integer and floating point operations may be performed simultaneously. Computation unit 610 may also include systolic array 612 and math unit 613. Systolic array 612 includes a wide W and deep D network of data processing units that may be used to systolicly perform vector or other data parallel operations. In one embodiment, systolic array 612 is configured to perform matrix operations, such as matrix dot product operations. In one embodiment, systolic array 612 supports 16-bit floating point operations along with 8-bit and 4-bit integer operations. In one embodiment, systolic array 612 may be configured to accelerate machine learning operations. In such an embodiment, systolic array 612 may be configured with support for the bfloat 16-bit floating point format. In one embodiment, math unit 613 may be included to perform a particular subset of mathematical operations more efficiently and with less power than ALU unit 611. Math unit 613 may include a variation of math logic found in the shared functional logic of a graphics processing engine provided by other embodiments (e.g., math logic 422 of shared functional logic 420 of FIG. 4). In one embodiment, math logic 613 may be configured to perform 32-bit and 64-bit floating point operations.

[0100] The thread control unit 601 includes logic to control the execution of threads in the execution units. The thread control unit 601 may include thread arbitration logic to start, stop, and preempt the execution of threads in the execution units 600. The thread state unit 602 may be used to store the thread states of threads assigned to execute in the execution units 600. Storing the thread states in the execution units 600 allows for fast preemption of threads if they become blocked or idle. The instruction fetch / prefetch unit 603 may fetch instructions from an instruction cache of higher level execution logic (e.g., instruction cache 506 of FIG. 5A). The instruction fetch / prefetch unit 603 may also issue prefetch instructions for instructions to be loaded into the instruction cache based on analysis of currently executing threads. The instruction decode unit 604 may be used to decode instructions to be executed by the computation units. In one embodiment, the instruction decode unit 604 may be used as a secondary decoder to decode the decoded instructions into constituent micro-operations.

[0101] Execution unit 600 further includes a register file 606 that may be used by the hardware threads executing in execution unit 600. The registers in register file 606 may be divided across logic used to execute multiple simultaneous threads within compute units 610 of execution unit 600. The number of logical threads that may be executed by graphics execution unit 600 is not limited to the number of hardware threads, and multiple logical threads may be assigned to each hardware thread. The size of register file 606 may vary from embodiment to embodiment based on the number of hardware threads supported. In one embodiment, register renaming may be used to dynamically assign registers to hardware threads.

[0102] 7 is a block diagram illustrating a graphics processor instruction format 700 according to some embodiments. In one or more embodiments, a graphics processor execution unit supports an instruction set having instructions in multiple formats. The solid lined boxes represent components that are typically included in execution unit instructions, while the dashed lines include components that are optional or are only included in a subset of the instructions. In some embodiments, the instruction format 700 described and illustrated are macro-instructions in that they are supplied to the execution units as the instructions are processed, as opposed to micro-operations that result from instruction decode.

[0103] In some embodiments, the graphics processor execution units natively support instructions in a 128-bit instruction format 710. A 64-bit compressed instruction format 730 is available for some instructions based on the selected instruction, instruction options, and number of operands. While the native 128-bit instruction format 710 provides access to all instruction options, some options and operations are restricted in the 64-bit format 730. The native instructions available in the 64-bit format 730 vary depending on the embodiment. In some embodiments, the instructions are partially compressed using a set of index values ​​in index field 713. The execution unit hardware looks up a set of compaction tables based on the index values ​​and uses the compaction table output to reconstruct the native instructions in the 128-bit instruction format 710. Instructions of other sizes and formats are available.

[0104] For each format, the instruction opcode 712 defines the operation that the execution unit is to perform. The execution unit executes each instruction in parallel across multiple data elements of each operand. For example, in response to an add instruction, the execution unit performs a simultaneous add operation across each color channel representing a texture or picture element. By default, the execution unit executes each instruction across all data channels of an operand. In some embodiments, the instruction control field 714 enables control over certain execution options, such as channel selection (e.g., predication) and data channel order (e.g., swizzle). For instructions in the 128-bit instruction format 710, the exec-size field 716 limits the number of data channels that are executed in parallel. In some embodiments, the exec-size field 716 is not available for use with the 64-bit compressed instruction format 730.

[0105] Some execution unit instructions have up to three operands, including two source operands SRC0 720, SRC1 722, and one destination 718. In some embodiments, the execution units support dual destination instructions, where one of the destinations is implicit. Data manipulation instructions can have a third source operand (e.g., SRC2 724), and the instruction opcode 712 determines the number of source operands. The last source operand of an instruction can be an intermediate (hard-coded) value sent along with the instruction.

[0106] In some embodiments, the 128-bit instruction format 710 includes an access / address mode field 726 that specifies, for example, whether direct or indirect register addressing mode is used. When direct register addressing mode is used, the register addresses of one or more operands are provided directly by bits in the instruction.

[0107] In some embodiments, the 128-bit instruction format 710 includes an access / address mode field 726 that specifies the address mode and / or access mode of the instruction. In one embodiment, the access mode is used to define the data access alignment of the instruction. Some embodiments support access modes including a 16-byte aligned access mode and a 1-byte aligned access mode, where the byte alignment of the access mode determines the access alignment of the instruction operands. For example, when in a first mode, the instruction may use byte-aligned addressing for source and destination operands, and when in a second mode, the instruction may use 16-byte aligned addressing for all source and destination operands.

[0108] In one embodiment, the address mode portion of the access / address mode field 726 determines whether the instruction should use direct or indirect addressing. If the direct register addressing mode is used, bits in the instruction directly provide the register address of one or more operands. If the indirect register addressing mode is used, the register address of one or more operands may be calculated based on an address register value and an address intermediate field in the instruction.

[0109] In some embodiments, instructions are grouped based on a bit field in the opcode 712 to simplify opcode decoding 740. In the case of an 8-bit opcode, bits 4, 5, and 6 allow the execution unit to determine the type of opcode. The exact opcode grouping shown is only an example. In some embodiments, the move / logical opcode group 742 includes data movement instructions and logical instructions (e.g., move (mov), compare (cmp)). In some embodiments, the move and logical group 742 share the five most significant bits (MSBs), with move (mov) instructions taking the form 0000xxxxb and logical instructions taking the form 0001xxxb. The flow control instruction group 744 (e.g., call, jump (jmp)) taking the form 0010xxxb (e.g., 0x20). The miscellaneous instruction group 746 includes a mix of instructions including synchronization instructions (e.g., wait, send) in the form of 0011xxxb (e.g., 0x30). The parallel math instruction group 748 includes component-wise arithmetic instructions (e.g., add, multiply (mul)) in the form of 0100xxxb (e.g., 0x40). The parallel math instruction group 748 performs arithmetic operations in parallel across data channels. The vector math group 750 includes arithmetic instructions (e.g., dp4) in the form of 0101xxxb (e.g., 0x50). The vector math group 750 performs calculations such as dot product calculations on vector operands. The depicted opcode decode 740, in one embodiment, may be used to determine which portion of the execution unit is used to execute the decoded instruction. For example, some instructions may be designed as systolic instructions executed by a systolic array. Other instructions, such as ray tracing instructions (not shown), may be sent to a ray tracing core or ray tracing logic within a slice or partition of the execution logic.

[0110] [Graphics Pipeline] Figure 8 is a block diagram of another embodiment of a graphics processor 800. Elements of Figure 8 having the same reference numbers (or names) as elements of any other figure in the present application may operate or function similarly as described elsewhere herein, but are not limited to such.

[0111] In some embodiments, graphics processor 800 includes a geometry pipeline 820, a media pipeline 830, a display engine 840, thread execution logic 850, and a render output pipeline 870. In some embodiments, graphics processor 800 is a graphics processor in a multi-core processing system that includes one or more general-purpose processing cores. The graphics processor is controlled by register writes to one or more control registers (not shown) or via commands issued to graphics processor 800 via a ring interconnect 802. In some embodiments, ring interconnect 802 couples graphics processor 800 to other processing components, such as other graphics processors or general-purpose processors. Commands from ring interconnect 802 are interpreted by command streamer 803. Command streamer 803 provides instructions to individual components of geometry pipeline 820 or media pipeline 830.

[0112] In some embodiments, command streamer 803 reads vertex data from memory and directs the operation of vertex fetcher 805, which executes vertex processing commands provided by command streamer 803. In some embodiments, vertex fetcher 805 provides vertex data to vertex shader 807, which performs coordinate space transformations and lighting operations on the extension points. In some embodiments, vertex fetcher 805 and vertex shader 807 execute vertex processing instructions by dispatching execution threads to execution units 852A-852B via thread dispatcher 831.

[0113] In some embodiments, the execution units 852A-852B are arrays of vertex processors with instruction sets for performing graphics and media operations. In some embodiments, the execution units 852A-852B have an attached L1 cache 851 that is local to each array or shared between the arrays. The cache may be configured as a data cache or an instruction cache, or a single cache that is partitioned to contain data and instructions in different partitions.

[0114] In some embodiments, the geometry pipeline 820 includes a tessellation component to perform hardware accelerated tessellation of 3D objects. In some embodiments, a programmable hull shader 817 provides back-end evaluation of the tessellation output. The tessellator 813 operates under the direction of the hull shader 811 and includes special purpose logic that generates a set of detailed geometric objects based on a coarse geometric model that is provided as input to the geometry pipeline 820. In some embodiments, the tessellation components (e.g., the hull shader 811, the tessellator 813, and the domain shader 817) may be bypassed if tessellation is not used.

[0115] In some embodiments, a complete geometric object may be processed by the geometry shader 819 via one or more threads dispatched to execution units 852A-852B, or may proceed directly to the clipper 829. In some embodiments, the geometry shader operates on the entire geometric object, rather than on vertices or patches of vertices as found in earlier stages of the graphics pipeline. The geometry shader 819 receives input from the vertex shader 807 when tessellation is disabled. In some embodiments, the geometry shader 819 is programmable by a geometry shader program to perform geometry tessellation when the tessellation units are disabled.

[0116] Prior to rasterization, a clipper 829 processes the vertex data. The clipper 829 may be a fixed function clipper or a programmable clipper with clipping and geometry shader functionality. In some embodiments, a rasterizer and depth test component 873 in the render output pipeline 870 dispatches pixel shaders to convert geometric objects into per-pixel representations. In some embodiments, pixel shader logic is included in the thread execution logic 850. In some embodiments, an application can bypass the rasterizer and depth test component 873 and access un-rasterized vertex data via the stream output unit 823.

[0117] Graphics processor 800 includes an interconnect bus, interconnect fabric, or other interconnect mechanism that allows data and message passing between the major components of the processor. In some embodiments, execution units 825A-825B and associated logic units (e.g., L1 cache 851, sampler 854, texture cache 858, etc.) interconnect via data port 856 to perform memory accesses and communicate with the processor's render output pipeline components. In some embodiments, sampler 854, caches 851, 858, and execution units 852A-852B each have a separate memory access path. In one embodiment, texture cache 858 may also be configured as a sampler cache.

[0118] In some embodiments, the render output pipeline 870 includes a rasterizer and depth test component 873 that converts vertex-based objects into an associated pixel-based representation. In some embodiments, the rasterizer logic includes a windower / marker unit to perform fixed function triangle and line rasterization. An associated render cache 878 and depth cache 879 are also available in some embodiments. A pixel operations component 877 performs pixel-based operations on the data, although in some instances pixel operations related to 2D operations (e.g., bit-block image movement with blending) are performed by the 2D engine 841 or are replaced at display time by the display controller 843 using an overlay display surface. In some embodiments, a shared L3 cache 875 is available to all graphics components, allowing sharing of data without using main system memory.

[0119] In some embodiments, the graphics processor media pipeline 830 includes a media engine 837 and a video front end 834. In some embodiments, the video front end 834 receives pipeline commands from a command streamer 803. In some embodiments, the media pipeline 830 includes a separate command streamer. In some embodiments, the video front end 834 processes the media commands before sending the commands to the media engine 837. In some embodiments, the media engine 837 includes a thread spawning function to spawn threads for dispatch to the thread execution logic 850 via the thread dispatcher 831.

[0120] In some embodiments, graphics processor 800 includes a display engine 840. In some embodiments, display engine 840 is external to processor 800 and couples to graphics processor 800 via ring interconnect 802 or other interconnect bus or fabric. In some embodiments, display engine 840 includes a 2D engine 841 and a display controller 843. In some embodiments, display engine 840 includes special purpose logic that can operate independently of the 3D pipeline. In some embodiments, display controller 843 couples to a display device (not shown). The display device may be a display device integrated into the system, such as a laptop computer, or an external display device attached via a display device connector.

[0121] In some embodiments, the geometry pipeline 820 and the media pipeline 830 are configurable to perform operations based on multiple graphics and media programming interfaces and are not specific to any one application programming interface (API). In some embodiments, driver software for the graphics processor translates API calls that are specific to a particular graphics or media library into commands that can be processed by the graphics processor. In some embodiments, support is provided for the Open Graphics Library (OpenGL), Open Computing Language (OpenCL), and / or Vulkan graphics and compute APIs, all from the Khronos Group. In some embodiments, support may also be provided for the Direct3D library from Microsoft Corporation. In some embodiments, a combination of these libraries may be supported. Support may also be provided for the Open Source Computer Vision (OpenCV) library. Future APIs with compatible 3D pipelines may also be supported if a mapping can be made from the pipeline of the future API to the pipeline of the graphics processor.

[0122] [Graphics Pipeline Programming] Figure 9A is a block diagram illustrating a graphics processor command format 900 according to some embodiments. Figure 9B is a block diagram illustrating a graphics processor command sequence 910 according to an embodiment. The solid blocks in Figure 9A represent components that are typically included in a graphics command, while the dashed lines include components that are optional or that are only included in a subset of the graphics commands. The example graphics processor command format 900 in Figure 9A includes data fields that identify a client 902, a command operation code (opcode) 904, and data for the command 906. Sub-opcodes 905 and command size 908 are also included in some commands.

[0123] In some embodiments, the client 902 specifies which client units of the graphics device will process the command data. In some embodiments, the graphics processor command parser examines the client field of each command to condition further processing of the command and route the command data to the appropriate client unit. In some embodiments, the graphics processor client units include a memory interface unit, a render unit, a 2D unit, a 3D unit, and a media unit. Each client unit has a corresponding processing pipeline that processes the command. When a command is received by a client unit, the client unit reads the opcode 904 and, if present, the sub-opcode 905 to determine the operation to perform. The client unit executes the command using the information in field 906. For some commands, an explicit command size 908 is expected to specify the size of the command. In some embodiments, the command parser automatically determines the size of at least some of the commands based on the command opcode. In some embodiments, the commands are aligned by multiples of doublewords. Other command formats are possible.

[0124] The flow diagram of Figure 9B depicts an example graphics processor command sequence 910. In some embodiments, the data processing system software or firmware featuring an embodiment of a graphics processor uses variations of the command sequence shown to set up, execute, and terminate a set of graphics operations. The sample command sequence is shown and described merely as an example in that embodiments are not limited to those particular commands or to this command sequence. Furthermore, commands may be issued as a batch of commands in the command sequence such that the graphics processor processes the sequence of commands at least partially concurrently.

[0125] In some embodiments, the graphics processor command sequence 910 may begin with a pipeline flush command 912 to cause any active graphics pipelines to complete any currently pending commands for the pipeline. In some embodiments, the 3D pipeline 922 and the media pipeline 924 do not operate simultaneously. A pipeline flush is performed to cause the active graphics pipelines to complete any pending commands. In response to the pipeline flush, the command parser for the graphics processor pauses command processing until the active drawing engines complete pending operations and the associated read cache is invalidated. Optionally, any data in the render cache that is marked as "dirty" may be flushed to memory. In some embodiments, the pipeline flush command 912 may be used for pipeline synchronization or before placing the graphics processor in a low power state.

[0126] In some embodiments, the pipeline select command 913 is used when a command sequence requests the graphics processor to explicitly switch pipelines. In some embodiments, the pipeline select command 913 is only needed once within an execution context, before issuing a pipeline command, unless the context should issue commands for both pipelines. In some embodiments, a pipeline flush command 912 is needed immediately before a pipeline switch due to the pipeline select command 913.

[0127] In some embodiments, pipeline control commands 914 are used to configure the graphics pipeline for operation and to program the 3D pipeline 922 and the media pipeline 924. In some embodiments, pipeline control commands 914 configure the pipeline state of the active pipeline. In one embodiment, pipeline control commands 914 are used for pipeline synchronization and to clear data from one or more cache memories in the active pipeline before processing a batch of commands.

[0128] In some embodiments, the return buffer state command 916 is used to configure a set of return buffers to which each pipeline writes data. Some pipeline operations require the allocation, selection, or configuration of one or more return buffers to which the operation writes intermediate data during processing. In some embodiments, the graphics processor also uses one or more return buffers to store output data and to perform cross-thread communication. In some embodiments, the return buffer state 916 includes selecting the size and number of return buffers to use for a set of pipeline operations.

[0129] The remaining commands in the command sequence differ based on the active pipeline for the operation. In the pipeline decision 920, the command sequence is tailored for the 3D pipeline 922 to start at the 3D pipeline state 930 or the media pipeline 924 to start at the media pipeline state 940.

[0130] Commands for configuring the 3D pipeline 922 include 3D set commands for vertex buffer states, vertex element states, constant color states, depth buffer states, and other state variables that should be configured before 3D primitive commands are processed. The values ​​of these commands are determined at least in part based on the particular 3D API being used. In some embodiments, the 3D pipeline state 930 commands can also selectively disable or bypass certain pipeline elements if those elements are not used.

[0131] In some embodiments, the 3D primitive 932 command is used to issue a 3D primitive to be processed by the 3D pipeline. The command and associated parameters sent to the graphics processor via the 3D primitive 932 command are forwarded to a vertex fetch function in the graphics pipeline. The vertex fetch function uses the 3D primitive 932 command to generate a vertex data structure. The vertex data structure is stored in one or more return buffers. In some embodiments, the 3D primitive 932 command is used to perform vertex operations on the 3D primitives by vertex shaders. To process the vertex shaders, the 3D pipeline 922 dispatches shader execution threads to the graphics processor execution units.

[0132] In some embodiments, the 3D pipeline 922 is triggered by an execute 934 command or event. In some embodiments, a register write triggers command execution. In some embodiments, execution is triggered by a 'go' or 'kick' command in a command sequence. In one embodiment, command execution is triggered using a pipeline synchronization command to flush the command sequence through the graphics pipeline. The 3D pipeline performs geometry processing for 3D primitives. Once the operations are complete, the resulting geometric objects are rasterized and the pixel engine colors the resulting pixels. Additional commands to control pixel shading and pixel backend operations may also be included in the operations.

[0133] In some embodiments, the graphics processor command sequence 910 follows the media pipeline 924 path when performing media operations. In general, the particular use and manner of programming the media pipeline 924 depends on the media or computational operations being performed. Certain media decoding operations may be offloaded to the media pipeline during media decoding. In some embodiments, the media pipeline may also be bypassed and media decoding is performed in whole or in part using resources provided by one or more general purpose processing cores. In one embodiment, the media pipeline also includes elements for general purpose graphics processor unit (GPGPU) operation, where the graphics processor is used to perform SIMD vector operations using computational shader programs that are not explicitly related to rendering graphics primitives.

[0134] In some embodiments, the media pipeline 924 is configured similarly to the 3D pipeline 922. A set of commands to configure the media pipeline state 940 is dispatched or placed in a command queue before the media object commands 942. In some embodiments, the commands for the media pipeline state 940 contain data to configure the media pipeline elements used to process the media object. This includes data to configure the video decoding and video encoding logic in the media pipeline, such as the encoding or decoding format. In some embodiments, the commands for the media pipeline state 940 also support the use of one or more pointers to "indirect" state elements that contain a batch of state settings.

[0135] In some embodiments, a media object command 942 provides a point to a media object for processing by the media pipeline. A media object includes a memory buffer that contains the video data to be processed. In some embodiments, all media pipeline state must be valid before issuing a media object command 942. Once the pipeline state is configured and the media object command 942 is queued, the media pipeline 924 is triggered by an execute command 944 or equivalent execution event (e.g., a register write). The output from the media pipeline 924 may then be post-processed by operations provided by the 3D pipeline 922 or the media pipeline 924. In some embodiments, GPGPU operations are configured and executed similarly to media operations.

[0136] [Graphics Software Architecture] 10 depicts an example of a graphics software architecture for a data processing system 1000 in accordance with some embodiments. In some embodiments, the software architecture includes a 3D graphics application 1010, an operating system 1020, and at least one processor 1030. In some embodiments, the processor 1030 includes a graphics processor 1032 and one or more general-purpose processor cores 1034. The graphics application 1010 and the operating system 1020 each execute programs in a system memory 1050 of the data processing system.

[0137] In some embodiments, the 3D graphics application 1010 includes one or more shader programs that include shader instructions 1012. The shader language instructions may be in a high-level shader language such as Drect3D's High-Level Shader Language (HLSL), OpenGL Shader Language (GLSL), etc. The application also includes executable instructions 1014 in a machine language suitable for execution by a general-purpose processor core 1034. The application also includes graphics objects 1016 that are defined by vertex data.

[0138] In some embodiments, the operating system 1020 is a Microsoft® Windows® operating system from Microsoft Corporation, a proprietary UNIX-like operating system, or an open source UNIX-like operating system that uses a variant of the Linux® kernel. The operating system 1020 can support a graphics API 1022, such as a Direct3D API, an OpenGL API, or a Vulkan API. If Direct3D is in use, the operating system 1020 uses a front-end shader compiler 1024 to compile any instructions in HLSL 1012 into a lower-level shader language. The compilation can be a just-in-time (JIT) compilation, or the application can perform shader precompilation. In some embodiments, the higher-level shaders are compiled into lower-level shaders during compilation of the 3D graphics application 1010. In some embodiments, the shader instructions 1012 are provided in an intermediate form, such as a variant of the Standard Portable Intermediate Representation (SPIR) used by the Vulkan API.

[0139] In some embodiments, user mode graphics driver 1026 includes a backend shader compiler 1027 to convert shader instructions 1012 into a hardware-specific representation. If the OpenGL API is in use, shader instructions 1012 in the GLSL high-level language are sent to user mode graphics driver 1026 for compilation. In some embodiments, user mode graphics driver 1026 uses operating system kernel mode functions 1028 to communicate with kernel mode graphics driver 1029. In some embodiments, kernel mode graphics driver 1029 communicates with graphics processor 1032 to dispatch commands and instructions.

[0140] [IP core implementation] One or more aspects of at least one embodiment may be implemented by representative code stored on a machine-readable medium that represents and / or defines logic within an integrated circuit, such as a processor. For example, the machine-readable medium may include instructions that represent various logic within a processor. When read by a machine, the instructions cause the machine to assemble the logic to perform the techniques described herein. Such representations, known as "IP cores," are reusable units of logic for an integrated circuit that may be stored in a tangible machine-readable medium as a hardware model that describes the structure of the integrated circuit. The hardware model may be provided to various customers of a manufacturing facility who load the hardware model into a manufacturing machine that produces the integrated circuit. The integrated circuit may be assembled such that the circuit performs the operations described in connection with any of the embodiments described herein.

[0141] FIG. 11A is a block diagram illustrating an IP core development system 1100 that may be used to fabricate an integrated circuit to perform operations according to an embodiment. The IP core development system 1100 may be used to generate modular, reusable designs that may be incorporated into a larger design or used to construct an entire integrated circuit (e.g., a SOC integrated circuit). The design facility 1130 may generate a software simulation 1110 of the IP core design in a high-level programming language (e.g., C / C++). The software simulation 1110 may be used to design, test, and verify the behavior of the IP core using a simulation model 1112. The simulation model 1112 may include functional, behavioral, and / or timing simulations. A register transfer level (RTL) design 1115 may then be created or synthesized from the simulation model 1112. The RTL design 1115 is an abstraction of the behavior of the integrated circuit that models the flow of digital signals between hardware registers with associated logic that is implemented using the modeled digital signals. In addition to the RTL design 1115, sub-designs at the logic or transistor level may also be created, designed, or synthesized. As such, the specific details of the initial design and simulation may vary.

[0142] The RTL design 1115 or equivalent may be further synthesized by a design facility into a hardware model 1120. The hardware model 1120 may be in a hardware description language (HDL) or other representation of the physical design data. The HDL may be further simulated or tested to verify the IP core design. The IP core design may be stored using a non-volatile memory 1140 (e.g., a hard disk, a flash memory, or any non-volatile storage medium) for delivery to a third-party manufacturing facility 1165. Alternatively, the IP core design may be transmitted via a wired connection 1150 or a wireless connection 1160 (e.g., via the Internet). The manufacturing facility 1165 may then manufacture an integrated circuit based at least in part on the IP core design. The manufactured integrated circuit may be configured to perform operations according to at least one embodiment described herein.

[0143] 11B illustrates a cross-sectional side view of an integrated circuit package assembly 1170 according to some embodiments described herein. The integrated circuit package assembly 1170 represents an implementation of one or more processors or accelerators described herein. The package assembly 1170 includes multiple units of hardware logic 1172, 1174 coupled to a substrate 1180. The logic 1172, 1174 may be at least partially implemented in configurable logic or fixed function logic hardware and may include one or more portions of a processor core, graphics processor, or any of the other accelerator devices described herein. Each unit of logic 1172, 1174 may be implemented in a semiconductor die and coupled to the substrate 1180 via an interconnect structure 1173. The interconnect structure 1173 may be configured to route electrical signals between the logic 1172, 1174 and the substrate 1180 and may include, for example, without limitation, an interconnect such as a bump or pillar. In some embodiments, the interconnect structure 1173 may be configured to route electrical signals, such as, for example, input / output (I / O) signals and / or power or ground signals related to the operation of the logic 1172, 1174. In some embodiments, the substrate 1180 is an epoxy-based laminate substrate. The substrate 1180 may include other suitable types of substrates in other embodiments. The package assembly 1170 may be connected to other electrical devices via the package interconnect 1183. The package interconnect 1183 may be coupled to a surface of the substrate 1180 to route electrical signals to other electrical devices, such as a motherboard, another chipset, or a multi-chip module.

[0144] In some embodiments, the units of logic 1172, 1174 are electrically coupled to a bridge 1182 configured to route electrical signals between the logic 1172, 1174. The bridge 1182 may be a dense interconnect structure that provides a path for the electrical signals. The bridge 1182 may include a bridge substrate made of glass or a suitable semiconductor material. Electrical routing features may be formed on the bridge substrate to provide chip interconnects between the logic 1172, 1174.

[0145] Although two units of logic 1172, 1174 and a bridge 1182 are shown, the embodiments described herein may include more or less logic on one or more dies. One or more dies may be connected by zero or more bridges, in that bridge 1182 would be omitted if the logic were contained on a single die. Alternatively, multiple dies or units of logic may be connected by one or more bridges. Additionally, multiple logic, units, dies, and bridges may be connected together in other possible configurations, including three-dimensional configurations.

[0146] FIG. 11C depicts a package assembly 1190 including multiple units of hardware logic chiplets connected to a substrate 1180 (e.g., a base die). Graphics processing units, parallel processors, and / or computational accelerators as described herein can be composed of multiple silicon chiplets that are manufactured separately. In this context, a chiplet is an at least partially packaged integrated circuit that includes separate units of logic that can be assembled with other chiplets into a larger package. Multiple sets of chiplets with different IP core logic can be assembled into a single device. Furthermore, chiplets can be integrated into a base die or base chiplet using active interposer technology. The concepts described herein allow interconnection and communication between different forms of IP within a GPU. IP cores can be manufactured using different process technologies and assembled during manufacturing. This avoids the complexity of assembling multiple IPs in the same manufacturing process, especially on large SoCs that include several types of IP. Enabling the use of multiple process technologies improves time to market and provides a cost-effective way to make multiple product SKUs. Furthermore, non-collective IP is more amenable to being power-gated independently, so that components not being used for a given workload can be powered off, reducing overall power consumption.

[0147] The hardware logic chiplets include special purpose hardware logic chiplets 1172, logic or I / O chiplets 1174, and / or memory chiplets 1175. The hardware logic chiplets 1172 and logic or I / O chiplets 1174 may be implemented at least in part in configurable logic or fixed function logic hardware and may include one or more portions of a processor core, a graphics processor, a parallel processor, or any of the other accelerator devices described herein. The memory chiplets 1175 can be DRAM (e.g., GDDR, HBM) memory or cache (SRAM) memory.

[0148] Each chiplet may be fabricated as a separate semiconductor die and coupled to substrate 1180 via interconnect structures 1173. Interconnect structures 1173 may be configured to route electrical signals between the various chiplets and logic within substrate 1180. Interconnect structures 1173 may include interconnects such as, for example, without limitation, bumps or pillars. In some embodiments, interconnect structures 1173 may provide input / output (I / O) related to logic, I / O, and memory chiplet operation, for example. ) signals and / or electrical signals, such as power or ground signals.

[0149] In some embodiments, the substrate 1180 is an epoxy-based laminate substrate. The substrate 1180 may include other suitable types of substrates in other embodiments. The package assembly 1190 may be connected to other electrical devices via a package interconnect 1183. The package interconnect 1183 may be coupled to a surface of the substrate 1180 to route electrical signals to other electrical devices, such as a motherboard, another chipset, or a multi-chip module.

[0150] In some embodiments, the logic or I / O chiplets 1174 and the memory chiplets 1175 may be electrically coupled through a bridge 1187 configured to route electrical signals between the logic or I / O chiplets 1174 and the memory chiplets 1175. The bridge 1187 may be a dense interconnect structure that provides a path for the electrical signals. The bridge 1187 may include a bridge substrate made of glass or a suitable semiconductor material. Electrical routing mechanisms may be formed on the bridge substrate to provide chip interconnects between the logic or I / O chiplets 1174 and the memory chiplets 1175. The bridge 1187 may also be referred to as a silicon bridge or an interconnect bridge. For example, the bridge 1187 is an Embedded Multi-die Interconnect Bridge (EMIB) in some embodiments. In some embodiments, the bridge 1187 may simply be a direct connection from one chiplet to another.

[0151] Substrate 1180 can include hardware components for I / O 1191, cache memory 1192, and other hardware logic 1193. Fabric 1185 can be embedded in substrate 1180 to enable communication between various logic chiplets and logic 1191, 1193 within substrate 1180. In one embodiment, I / O 1191, fabric 1185, cache, bridges, and other hardware logic 1193 can be incorporated into a base die that is stacked on top of substrate 1180.

[0152] In various embodiments, package assembly 1190 can include more or fewer components and chiplets interconnected by fabric 1185 or one or more bridges 1187. Chiplets in package assembly 1190 can be arranged in a 3D or 2.5D arrangement. In general, bridge structures 1187 can be used to facilitate point-to-point connections between, for example, logic or I / O chiplets and memory chiplets. Fabric 1185 can be used to interconnect various logic and / or I / O chiplets (e.g., chiplets 1172, 1174, 1191, 1193) with other logic and / or I / O chiplets. In one embodiment, cache memory 1192 in the substrate can operate as a global cache for package assembly 1190, as part of a discrete global cache, or as a dedicated cache for fabric 1185.

[0153] 11D illustrates a package assembly 1194 including a replaceable chiplet 1195, according to an embodiment. The replaceable chiplet 1195 may be assembled into a standardized slot on one or more base chiplets 1196, 1198. The base chiplets 1196, 1198 may be coupled through a bridge interconnect 1197. The bridge interconnect 1197 may be similar to other bridge interconnects described herein, for example, an EMIB. Memory chiplets may also be connected to logic or I / O chiplets through a bridge interconnect. The I / O and logic chiplets may communicate through an interconnect fabric. The base chiplets may each support one or more slots in a standardized format for one of logic or I / O or memory / cache.

[0154] In one embodiment, the SRAM and power distribution circuitry may be fabricated in one or more of the base chiplets 1196, 1198. These may be fabricated using a different process technology relative to the replaceable chiplets 1195 that are stacked on top of the base chiplets. For example, the base chiplets 1196, 1198 may be fabricated using a larger scale process technology while the replaceable chiplets may be fabricated using a smaller scale process technology. One or more of the replaceable chiplets 1195 may be memory (e.g., DRAM) chiplets. Different memory densities may be selected for the package assembly 1194 based on the performance and / or power targeted for the product using the package assembly 1194. Furthermore, logic chiplets with a wide variety of functional units may be selected at assembly time based on the performance and / or power targeted for the product. Furthermore, chiplets containing various types of IP logic cores may be inserted into the replaceable chiplet slots, enabling composite processor designs where IP blocks from different technologies can be mixed and matched.

[0155] [Example of a system on a chip] 12-13 depict examples of integrated circuits and associated graphics processors that may be manufactured using one or more IP cores in accordance with various embodiments described herein. In addition to what is depicted, other logic and circuitry may be included, including additional graphics processors / cores, peripheral interface controllers, or general-purpose processor cores.

[0156] 12 is a block diagram illustrating a system on a chip integrated circuit 1200 that may be manufactured using one or more IP cores, according to an embodiment. The example integrated circuit 1200 includes one or more application processors 1205 (e.g., a CPU) and at least one graphics processor 1210, and may further include an image processor 1215 and / or a video processor 1220, any of which may be modular IP cores from the same or multiple different design facilities. The integrated circuit 1200 includes a USB controller 1225, a UART controller 1230, an SPI / SDIO controller 1235, and an I / O controller 1236. 2 S / I 2 The integrated circuit 1200 may further include peripheral or bus logic including a C controller 1240. Additionally, the integrated circuit 1200 may include a display device 1245 coupled to one or more of a High-Definition Multimedia Interface (HDMI®) controller 1250 and a Mobile Industry Processor Interface (MIPI) display interface 1255. Storage may be provided by a flash memory subsystem 1260 including a flash memory and a flash memory controller. Memory interface may be provided via a memory controller 1265 for access to an SDRAM or SRAM memory device. Some integrated circuits further include an embedded security engine 1270.

[0157] 13A-13B are block diagrams illustrating a graphics processor used in an SoC according to embodiments described herein. FIG. 13A illustrates a graphics processor 1310 of a system on a chip integrated circuit that may be fabricated using one or more IP cores according to embodiments. FIG. 13B illustrates a further example of a graphics processor 1340 of a system on a chip integrated circuit that may be fabricated using one or more IP cores according to embodiments. The graphics processor 1310 of FIG. 13A is an example of a low-power graphics processor core. The graphics processor 1340 of FIG. 13B is an example of a higher performance graphics processor core. Each of the graphics processors 1310, 1340 may be a variation of the graphics processor 1210 of FIG. 12.

[0158] As shown in FIG. 13A, the graphics processor 1310 includes a vertex processor 1305 and one or more fragment processors 1315A-1315N (e.g., 1315A, 1315B, 1315C, 1315D, through 1315N-1 and 1315N). The graphics processor 1310 may execute different shader programs through separate logic such that the vertex processor 1305 is optimized to execute the operations of a vertex shader program, while one or more fragment processors 1315A-1315N execute the fragment (e.g., pixel) shading operations of a fragment or pixel shader program. The vertex processor 1305 executes the vertex processing stage of the 3D graphics pipeline and generates primitive and vertex data. The fragment processors 1315A-1315N use the primitive and vertex data generated by the vertex processor 1305 to generate a frame buffer that is displayed on a display device. In one embodiment, the fragment processors 1315A-1315N are optimized to execute fragment shader programs such as those provided in the OpenGL API, which can be used to perform operations similar to pixel shader programs such as those provided in Direct3D.

[0159] The graphics processor 1310 further includes one or more memory management units (MMUs) 1320A-1320B, caches 1325A-1325B, and circuit interconnects 1330A-1330B. The one or more MMUs 1320A-1320B provide virtual to physical address mapping for the graphics processor 1310, including the vertex processor 1305 and / or fragment processors 1315A-1315N. The graphics processor 1310 may reference vertex or image / texture data stored in memory in addition to vertex or image / texture data stored in one or more caches 1325A-1325B. In one embodiment, the one or more MMUs 1320A-1320B may be synchronized with other MMUs in the system, including one or more MMUs associated with one or more application processors 1205, image processor 1215, and / or video processor 1220 of FIG. 12. Each processor 1205-1220 may thereby participate in a shared or unified virtual memory system. One or more circuit interconnects 1330A-1330B allow graphics processor 1310 to interface with other IP cores within the SoC, either via the SoC's internal bus or via direct connections, depending on the embodiment.

[0160] As shown in Figure 13B, graphics processor 1340 includes one or more MMUs 1320A-1320B, caches 1325A-1325B, and circuit interconnects 1330A-1330B of graphics processor 1310 of Figure 13A. Graphics processor 1340 includes one or more shader cores 1355A-1355N (e.g., 1355A, 1355B, 1355C, 1355D, 1355E, 1355F, through 1355N-1, and 1355N) that provide a unified shader core architecture in which a single core, or a single type of core, can execute all types of programmable shader code, including vertex shaders, fragment shaders, and / or compute shaders. The exact number of shader cores present can vary among embodiments and implementations. Additionally, graphics processor 1340 includes an inter-core task manager 1345 that acts as a thread dispatcher to dispatch execution threads to one or more shader cores 1355A-1355N, and a tiling unit 1358 that accelerates tiling operations for tile-based rendering. In tile-based rendering, rendering operations of a scene are subdivided in image space, for example to exploit local spatial coherence within the scene or to optimize internal cache usage.

[0161] Figure 14 illustrates one embodiment of a computing device 1400. The computing device 1400 (e.g., a smart wearable device, a virtual reality (VR) device, a head mounted display (HDM), a mobile computer, an Internet of Things (IoT) device, a laptop computer, a desktop computer, a server computer, etc.) may be the same as the data processing system 100 of Figure 1, and therefore, for the sake of brevity, clarity, and ease of understanding, many of the details described above with reference to Figures 1-13 will not be further discussed or repeated hereinafter.

[0162] Computing device 1400 may include any number and type of communication devices, such as large-scale computer systems, such as server computers, desktop computers, and the like, and may further include set-top boxes (e.g., Internet-based cable television set-top boxes, etc.), global positioning system (GPU) based devices, and the like. Computing device 1400 may include mobile computing devices that function as communication devices, such as mobile phones including smartphones, personal digital assistants (PDAs), tablet computers, laptop computers, e-readers, smart televisions, television platforms, wearable devices (e.g., glasses, watches, bracelets, smart cards, jewelry, clothing accessories, etc.), media players, and the like. For example, in one embodiment, computing device 1400 may include a mobile computing device that uses a computing platform that hosts an integrated circuit ("IC"), such as a system-on-a-chip ("SoC" or "SOC") that incorporates various hardware and / or software components of computing device 1400 on a single chip.

[0163] As depicted, in one embodiment, computing device 1400 may include any number and types of hardware and / or software components, such as (without limitation) a GPU 1414, a graphics driver (also referred to as a “GPU driver”, “graphics driver logic”, “driver logic”, “user mode driver (UMD)”, “UMD”, “user mode driver framework (UMDF)”, “UMDF”, or simply “driver”) 1416, a CPU 1412, memory 1408, network devices, drivers, etc., as well as input / output (I / O) sources 1404, such as a touch screen, touch panel, touch pad, virtual or standard keyboard, virtual or standard mouse, ports, connectors, etc.

[0164] Computing device 1400 may include an operating system (OS) 1406 that serves as an interface between a user and the hardware and / or physical resources of computing device 1400. It is contemplated that CPU 1412 may include one or more processors, while GPU 1414 may include one or more graphics processors.

[0165] It should be noted that terms such as "node," "computer node," "server," "server device," "cloud computer," "cloud server," "cloud server computer," "machine," "host machine," "device," "computer device," "computer," "computer system," etc. may be used synonymously throughout this document. It should also be noted that terms such as "application," "software application," "program," "software program," "package," "software package," etc. may be used synonymously throughout this document. Additionally, terms such as "job," "input," "request," "message," etc. may be used synonymously throughout this document.

[0166] As will be further described with reference to Figures 1-13, it is contemplated that some processes of the above graphics pipeline are implemented in software, while others are implemented in hardware. The graphics pipeline may be implemented in a graphics processor design in which a CPU 1412 is designed to operate with a GPU 1414, which may be included in or co-located with the CPU 1412. In one embodiment, the GPU 1414 may employ any number and type of conventional software and hardware logic to perform conventional functions related to graphics rendering, in addition to novel software and hardware logic to execute any number and type of instructions.

[0167] As described above, memory 1408 may include random access memory (RAM) including an application database with object information. A memory controller hub may access data in the RAM and transfer it to GPU 1414 for graphics pipeline processing. The RAM may include double data rate RAM (DDR RAM), extended data output RAM (EDO RAM), etc. CPU 1412 interacts with the hardware graphics pipeline to share graphics pipeline functions.

[0168] The processed data is stored in buffers within the hardware graphics pipeline, and state information is stored in memory 1408. The resulting image is then transferred to an I / O source 1404, such as a display component, for display of the image. The display device may be of various types, such as a cathode ray tube (CRT), thin film transistor (TFT), liquid crystal display (LCD), organic light emitting diode (OLED) array, etc., to display information to a user.

[0169] Memory 1408 may include a pre-allocated region of a buffer (e.g., a frame buffer), although one skilled in the art will appreciate that embodiments are not so limited and any memory accessible to the lower graphics pipeline may be used. Computing device 1400 may further include one or more I / O sources 1404, such as platform controller hub (PCH) 130, referred to in FIG. 1.

[0170] The CPU 1412 may include one or more processors to execute instructions to perform whatever software routines the computer system implements. Instructions often involve some sort of operation performed on data. Both data and instructions are typically designed to have lower latency than the system memory 1408; for example, a cache may be integrated on the same silicon chip as the processor and / or may be comprised of faster static RAM (SRAM) cells, while the system memory 1408 may be comprised of slower dynamic RAM (DRAM) cells. The tendency to store more frequently used instructions and data in the cache, as opposed to the system memory 1408, improves the overall performance efficiency of the computer device 1400. In some embodiments, it is contemplated that the GPU 1414 may exist as part of the CPU 1412 (e.g., part of the physical CPU package), in which case the memory 1408 may be provided by the CPU 1412 and the GPU 1414, or may remain separate.

[0171] System memory 1408 may be made available to other components within computing device 1400. For example, any data (e.g., input graphics data) received from various interfaces to computing device 1400 (e.g., keyboard and mouse, printer port, local area network (LAN) port, modem port, etc.) or read from an internal storage element (e.g., hard disk drive) of computing device 1400 is often temporarily queued in system memory 1408 before being acted upon by one or more processors in the execution of a software program. Similarly, data that a software program determines should be sent from computing device 1400 to an external entity through one of the computer system interfaces or stored in an internal storage element is often temporarily queued in system memory 1408 before being transmitted or stored.

[0172] Further, for example, the PCH may be used to ensure that such data is properly passed between system memory 1408 and its appropriate corresponding computer system interfaces (and internal storage devices, if the computer system is so designed), and may have bidirectional point-to-point links between itself and the monitored I / O sources / devices 1404. Similarly, the MCH may be used to manage various competing requests for access of system memory 1408 between the CPU 1412 and GPU 1414, interfaces, and internal storage elements that may occur in close time proximity to one another.

[0173] I / O sources 1404 may include one or more I / O devices implemented for transferring data to and / or from computing device 1400 (e.g., a networking adapter) or for large-scale non-volatile storage within computing device 1400 (e.g., a hard disk drive). User input devices including alphanumeric and other keys may be used to send information and command selections to GPU 1414. Other types of user input devices are cursor controls such as a mouse, trackball, touch screen, touch pad, or cursor direction keys to send directional information and command selections to GPU 1414 and to control cursor movement on a display device. Camera and microphone arrays in computing device 1400 may be used to observe gestures, record audio and video, and send and receive visual and voice commands.

[0174] The computing device 1400 may further include a network interface to provide access to a network such as a LAN, a wide area network (WAN), a metropolitan area network (MAN), a personal area network (PAN), Bluetooth, a cloud network, a mobile network (e.g., third generation (3G), fourth generation (4G), etc.), an intranet, the Internet, etc. The network interface may include, for example, a wireless network interface with an antenna, which may correspond to one or more antennas. The network interface may also include a wired network interface, for example, to communicate with a remote device via a network cable, which may be, for example, an Ethernet cable, a coaxial cable, an optical fiber, a serial cable, or a parallel cable.

[0175] The network interface may provide access to a LAN, for example, by conforming to the IEEE 802.11b and / or IEEE 802.11g standards, and / or the wireless network interface may provide access to a personal area network, for example, by conforming to the Bluetooth standard. Other wireless network interfaces and / or protocols, including earlier and later versions of the standard, may also be supported. In addition to or instead of communication according to the wireless LAN standard, the network interface may provide wireless communication using, for example, a time division multiple access (TDMA) protocol, a Global Systems for Mobile communications (GSM) protocol, a code division multiple access (CDMA) protocol, and / or any other type of wireless communication protocol.

[0176] The network interface may include one or more communications interfaces such as a modem, a network interface card, or other well-known interface devices, such as those used to couple Ethernet, token ring, or other types of physical wired or wireless attachments intended to provide a communications link supporting a LAN or WAN. In this manner, the computer system may also be coupled to a number of peripherals, clients, control surfaces, consoles, or servers via conventional network infrastructures including, for example, an intranet or the Internet.

[0177] Of course, a lesser or more equipped system than the above examples may be preferred for a particular implementation. Thus, the configuration of the computing device 1400 may vary from implementation to implementation depending on a number of factors, such as price constraints, performance requirements, technological improvements, or other circumstances. Examples of electronic or computing devices 1400 include (without limitation) a mobile device, a personal digital assistant, a mobile computing device, a smart phone, a mobile phone, a handset, a one-way pager, a two-way pager, a messaging device, a computer, a personal computer (PC), a desktop computer, a laptop computer, a notebook computer, a handheld computer, a tablet computer, a server, a server array or server farm, a web server, a network server, an Internet server, a workstation, a minicomputer, a mainframe computer, a supercomputer, a network appliance, a web appliance, a distributed computing system, a multiprocessor system, a processor-based system, a consumer electronics, a programmable consumer electronics, a television set, a digital television set, a set-top box, a wireless access point, a base station, a subscriber station, a mobile subscriber center, a wireless network controller, a router, a hub, a gateway, a bridge, a switch, a machine, or a combination thereof.

[0178] The embodiments may be implemented as any or a combination of one or more microchips or integrated circuits interconnected using a parent board, hardwired logic, software stored by a memory device and executed by a microprocessor, firmware, application specific integrated circuits (ASICs), and / or field programmable gate arrays (FPGAs). The term "logic" may include, by way of example, software or hardware and / or a combination of software and hardware.

[0179] The embodiments may be provided as a computer program product, which may include one or more machine-readable media having stored thereon machine-executable instructions that, when executed by one or more machines, such as, for example, a computer, a network of computers, or other electronic devices, may cause the one or more machines to perform operations in accordance with the embodiments described herein. Machine-readable media may include, without limitation, floppy diskettes, optical disks, Compact Disc-Read Only Memories (CD-ROMs), and optical magnetic disks, ROM, RAM, Erasable Programmable Read Only Memories (EPROMs), Electrically Erasable Programmable Read Only Memories (EEPROMs), magnetic or optical cards, flash memory, or other types of media / machine-readable media suitable for storing machine-readable instructions.

[0180] Additionally, embodiments may be downloaded as a computer program product, and the program may be transferred from a remote computer (e.g., a server) to a requesting computer (e.g., a client) by one or more data signals carried and / or modulated by a carrier wave or other propagation medium over a communications link (e.g., a modem and / or network connection).

[0181] According to one embodiment, the GPU 1414 is divided into slices, with each slice including multiple slices. In such an embodiment, each slice includes three sub-slices. However, other embodiments may feature a different number of sub-slices. FIG. 15 illustrates one embodiment of a sub-slice 1500. As shown in FIG. 15, the sub-slice 1500 includes an execution unit (EU) 1510 and a shared local memory (SLM) 1520. In an embodiment, the EU 1510 is a combination of simultaneous multi-threading (SMT) and fine-grained interleaved multi-threading (IMT) computation processor driving multiple issue single instruction, multiple data (SIMD) arithmetic logic unit(s) (ALUs) pipelined across multiple threads for high throughput floating point and integer computation. In a further embodiment, the sub-slice 1500 has eight EUs. However, other embodiments may feature a different number of EUs.

[0182] The SLM 1520 is a 128KB to 256KB highly banked data structure accessible to the EUs 1510 in the subslice 1500. Its proximity to the EUs results in lower latency and higher efficiency since SLM traffic does not interfere with other memory accesses (e.g., cache / main memory accesses). In one embodiment, the SLM 1520 is banked to byte granularity allowing a high degree of access flexibility from the EUs 1510.

[0183] According to one embodiment, the SLM 1520 is implemented to provide an additional cache way when some (or all) of the memory capacity of the SLM 1520 is not in use. In such an embodiment, the SLM 1520 is partitioned to be used as a cache for the EU 1510 and as a shared memory. Thus, the SLM 1520 may be implemented as an additional cache for workloads in that the use of the SLM 1520 is typically sparse across various applications. In a further embodiment, the driver 1416 controls the allocation size of the SLM 1520 on a per-walker (or per-kernel) granularity. In this embodiment, the size (or SLM size) of the portion of the SLM 1520 to be used as the shared local memory (or SLM space) is programmed via a command (or compute walker command) included in the interface descriptor data.

[0184] In one embodiment, the command may indicate that the SLM 1520 should be used as a full cache or non-cache. In another embodiment, the SLM 1520 is partitioned such that the SLM size may be configured as a set of discrete steps (e.g., 0 KB, 16 KB, 32 KB, 64 KB, 96 KB, 128 KB, 256 KB). In such an embodiment, the driver 1416 sets a default value for the SLM configuration, and the default is set such that the entire SLM 1520 is implemented as a cache. However, in other embodiments, the default may be changed based on application or workload requirements.

[0185] In further embodiments, the SLM size may be programmed on a per-dispatch basis based on the kernel's SLM space requirements and / or based on cache configuration hints. In one embodiment, an application may provide cache configuration hints (e.g., on a per-device basis) that allow the application to bias the SLM 1520 toward additional SLMs (e.g., for kernels where it is beneficial to have more parallelism resulting from running more workgroups) or toward additional conventional cache (e.g., when additional cache is beneficial). In yet other embodiments, the driver 1416 may select the hint. In other embodiments, hardware may monitor SLM utilization and turn unused SLM blocks into additional cache ways.

[0186] In one embodiment, each subslice (e.g., subslice 1500) partitions the SLM 1520 based on the amount (or number) of active threads in the subslice. In a further embodiment, partitioning in the subslice 1500 is controlled in a thread spawner logic (TSL) 1560 that manages the assignment of the SLM 1520 to thread groups in the EU 1510. FIG. 16 illustrates one embodiment of the TSL 1560, which includes SLM partitioning logic 1620. According to one embodiment, the SLM partitioning logic 1620 starts the SLM 1520 by setting (or resetting) the SLM 1520 to be mapped as a cache. Thus, the SLM 1520 is initially set to operate as a cache. A thread dispatch is then received at the SLM partitioning logic 1620 from the driver 1416, which includes a command to allocate SLM space (e.g., an SLM_request command). In one embodiment, the command includes a header that indicates the SLM size of the SLM space to be allocated.

[0187] Upon receiving the command, the SLM partitioning logic 1620 obtains the space indicated in the dispatch as an SLM block. In one embodiment, the SLM partitioning logic 1620 determines whether there is existing SLM space allocated to satisfy the thread dispatch. Upon a determination that the allocated SLM space is not sufficient for the dispatch, the SLM partitioning logic 1620 makes a request (e.g., to the load store & cache (LSC) unit 1550 of FIG. 15) to allocate more SLM space. In one embodiment, the dispatch is stalled until adequate SLM space is allocated. Once SLM is available, the thread dispatch unit 1610 performs thread group dispatching for operations on the obtained SLM block. In a further embodiment, the SLM partitioning logic 1620 maintains a bit vector (e.g., in_use[127:0]) to track the usage of SLM blocks by thread groups in 1 KB increments.

[0188] Upon a determination that the allocated SLM space is sufficient (e.g., the SLM size is less than the currently allocated SLM size), the portion of the SLM 1520 that was allocated as SLM space may be released (or freed). In this embodiment, all thread groups previously assigned to the freed SLM space are evicted. When a dispatch queue has been idle for a defined interval (e.g., a programmed number of periods), the SLM_request for that queue is cleared. This occurs when there are no new dispatches for a given queue associated with the subslice 1500. Once the dispatches are cleared, the SLM 1520 is remapped to operate as a cache.

[0189] According to one embodiment, the SLM partitioning logic 1620 assigns a value (e.g., Preferred_SLM_Size) for each pipelined state load, where Preferred_SLM_Size indicates the estimated amount of total SLM space that each subslice 1500 needs to execute its associated walker. In one embodiment, Preferred_SLM_Size is calculated by the driver 1416. However, in other embodiments, Preferred_SLM_Size is set by the application.

[0190] In one embodiment, Preferred_SLM_Size=fn(SLM_size_per_working group, number of threads per working group, number of physical threads in subslice 1500), and Per_TG_SLM≦Preferred_SLM_Size≦128KB. In a further embodiment, the SLM partitioning logic 1620 maintains the following states: current_SLM_size = currently reserved SLM size; new_SLM_size = desired SLM size based on last walker request; in_use = a bit vector that tracks which blocks of the SLM are currently in use by dispatched TGs; and Per queue hysteresis_ctr = Queue idle "timeout".

[0191] If no dispatching has been active for a given queue for a programmed number of periods, the Preferred_SLM_Size for that queue is set to 0. In one embodiment, SLM space is allocated from the bottom (0KB) and is contiguous (e.g., grows upwards in a stack). In a further embodiment, SLM space is deallocated from the top of the current stack. In yet another embodiment, the SLM allocation steps have multiples of one cache way size (e.g., in a typical GPU cache configuration, one way size=16KB). Allocating / deallocating SLM space as a stack removes any sparsity in the allocated SLM space. Figure 17A illustrates one embodiment of the SLM stack described above.

[0192] According to one embodiment, LSC 1550 includes a flash manager 1555 to repartition SLM 1520. In this embodiment, flash manager 1555 broadcasts SLM allocation or free requests to all SLM banks. As a result, each bank blocks its pipeline and then walks through all sets to allocate or free SLM space. In a further embodiment, the bottom four ways of each set are mapped to SLM RAM. Additionally, there is a bit in the tag that indicates whether the way is currently mapped as SLM (e.g., 1 if mapped, 0 if not). Once a determination is made that a way is mapped as SLM, that way is ignored for cache operations (lookup, allocation, flush, etc.).

[0193] The above embodiment describes a process where pipelined walkers are required to have different SLM sizes. In a further embodiment, new pipelined state can be loaded while threads from a previous walker are still executing in the EU 1510. This causes new_SLM_size to differ from current_SLM_size. In this embodiment, the SLM partitioning logic 1620 waits until the top of the allocated SLM is freed (e.g., in_use is 0) before sending a free() command to the LSC 1550. In a further embodiment, the new SLM allocation for the thread group from the latest walker does not exceed min(new_SLM_size, current_SLM_size). Figure 17B illustrates an embodiment of the SLM stack where the new SLM size is reduced from the current allocation.

[0194] 18 is a flow diagram illustrating one embodiment of a process for partitioning shared local memory. At processing block 1810, the SLM 1520 is initially mapped to the cache. At processing block 1820, a thread dispatch is received. As described above, the dispatch includes a command (e.g., an SLM_request command) to allocate SLM space in the SLM 1520 and a header indicating the SLM size to allocate. At processing block 1830, the SLM space is allocated according to the size indicated in the header.

[0195] At decision block 1840, a determination is made as to whether the SLM space is sufficient. If so, the new thread group is dispatched at processing block 1860. If not, additional SLM space is allocated at processing block 1850 before dispatching the thread group to the dispatch queue. At decision block 1870, a determination is made as to whether the dispatch queue has been idle (the thread group is no longer active) for a defined interval. If so, control is returned to processing block 1810 and SLM 1520 is reset by remapping to the cache. Otherwise, control is returned to decision block 1870 to determine whether the dispatch queue has been idle for a defined interval. Figure 19 illustrates one embodiment of pseudocode for partitioning a shared local memory.

[0196] The following sections and / or examples relate to further embodiments or examples. Details in the examples may be used anywhere in one or more embodiments. Various features of different embodiments or examples may be combined in various ways, such as including some features and excluding others, to suit a wide variety of applications. Examples may include objects such as a method, a means for performing the operations of the method, at least one machine-readable medium including instructions that, when executed by a machine, cause the machine to perform the operations of the method, or an apparatus or system that facilitates hybrid communication in accordance with the embodiments and examples described herein.

[0197] Some embodiments relate to Example 1, which includes an apparatus for facilitating partitioning of local memory, the apparatus having a plurality of execution units executing a plurality of execution threads, a memory coupled to share access between the plurality of execution units, and partitioning hardware that partitions the memory to be used as a cache and as a shared local memory (SLM), the partitioning hardware partitioning the memory based on a number of the plurality of execution threads executing on the execution units that are active.

[0198] Example 2 includes the subject matter of example 1, wherein the partitioning hardware activates the memory to act as the cache.

[0199] Example 3 includes the subject matter of examples 1 and 2, wherein the partitioning hardware receives a thread dispatch that includes a command to allocate a portion of the memory as an SLM block.

[0200] Example 4 includes the subject matter of Examples 1 to 3, and the command includes a header indicating the size of the SLM block.

[0201] Example 5 includes the subject matter of examples 1-4, wherein the partitioning hardware obtains the SLM blocks.

[0202] Example 6 includes the subject matter of examples 1-5, further comprising dispatch hardware for dispatching a group of the plurality of execution threads to act on the SLM block.

[0203] Example 7 includes the subject matter of examples 1-6, wherein the dispatch hardware stalls the thread dispatch until the partitioning hardware acquires the portion of the memory.

[0204] Example 8 includes the subject matter of examples 1-7, wherein the partitioning hardware tracks usage of the SLM blocks by the multiple threads of execution.

[0205] Example 9 includes the subject matter of examples 1-8, wherein the partitioning hardware resets the memory to act as the cache upon a determination that the multiple execution threads are no longer active.

[0206] Some embodiments relate to Example 10, which includes a method for facilitating partitioning of local memory, comprising partitioning memory for use as a cache and as a shared local memory (SLM), the partitioning being performed based on a number of execution threads executing on a number of execution units that are active.

[0207] Example 11 includes the subject matter of example 10, wherein partitioning the memory includes initiating the memory to operate as the cache, receiving a thread dispatch including a command to allocate a portion of the memory as an SLM block, allocating the SLM block, and obtaining the SLM block.

[0208] Example 12 includes the subject matter of examples 10 and 11, wherein the command includes a header indicating the size of the SLM block.

[0209] Example 13 includes the subject matter of examples 10-12, further comprising dispatching a group of the plurality of execution threads to act on the SLM block.

[0210] Example 14 includes the subject matter of examples 10-13, further comprising tracking usage of the SLM blocks by the multiple threads of execution.

[0211] Example 15 includes the subject matter of examples 10-14, further comprising resetting the memory to act as the cache upon a determination that the plurality of execution threads are no longer active.

[0212] Some embodiments relate to Example 16, which includes a graphics processing unit (GPU) having multiple slices, each slice having multiple sub-slices including multiple execution units executing multiple execution threads, a memory coupled to share access between the multiple execution units, and partitioning hardware that partitions the memory to be used as a cache and as a shared local memory (SLM), the partitioning hardware partitioning the memory based on the number of the multiple execution threads executing on the execution units that are active.

[0213] Example 17 includes the subject matter of example 16, wherein the partitioning hardware activates the memory to act as the cache.

[0214] Example 18 includes the subject matter of examples 16 and 17, wherein the partitioning hardware receives a thread dispatch including a command to allocate a portion of the memory as an SLM block, the command including a header indicating a size of the SLM block.

[0215] Example 19 includes the subject matter of examples 16-18, further comprising dispatch hardware for dispatching a group of the plurality of execution threads to act on the SLM block.

[0216] Example 20 includes the subject matter of examples 16-19, wherein the partitioning hardware tracks usage of the SLM blocks by the multiple execution threads and resets the memory to act as the cache upon a determination that the multiple execution threads are no longer active.

[0217] The present invention has been described above with reference to specific embodiments. However, those skilled in the art will appreciate that various modifications and changes may be made to those embodiments without departing from the broader spirit and scope of the invention as set forth in the appended claims. The foregoing description and drawings should therefore be regarded in an illustrative and not a limiting sense.

[0218] [Copyright Notice] A portion of the disclosure of this patent document contains material that is subject to copyright protection. The copyright owner has no objection to the copying by anyone of this document or this disclosure, as set forth in the Patent and Trademark Office patent application or records, but otherwise reserves all copyright rights whatsoever. [Explanation of symbols]

[0219] 100 Processing Systems 102,200 processors 104 Cache Memory 106 Register File 107,202 processor cores 108,208 Graphics Processor 109 Instruction Set 110 Interface Bus 111 Display Device 112 Accelerator 116,214 Integrated Memory Controller 120 Memory Devices 121 Command 122 Data 130 Platform Controller Hub 204 Internal Cache Unit 206 Shared Cache Unit 219 Graphics Processor Core 330 Computational Accelerator 500 Execution logic 508,600 execution units 612 Systolic Array 1400 Computer Devices 1406 Operating System (OS) 1408 Memory 1412 Central Processing Unit (CPU) 1414 Graphics Processing Unit (GPU) 1416 Graphics Driver 1500 subslices 1510 Execution Unit 1520 Shared Local Memory (SLM) 1560 Thread Spawner Logic (TSL) 1610 Thread Dispatch Unit 1620 SLM Partitioning Logic

Claims

1. 1. An apparatus that facilitates partitioning of a local memory, comprising: a plurality of execution units executing a plurality of threads of execution; a memory coupled to share access among the execution units; partitioning hardware that partitions the memory to be used as a cache and as a shared local memory (SLM), the partitioning hardware partitioning the memory based on the number of execution threads executing on the execution units that are active; and having the partitioning hardware receives a thread dispatch including a command to allocate a portion of the memory as an SLM block; Device.

2. the partitioning hardware activates the memory to act as the cache; 2. The apparatus of claim 1.

3. the command includes a header indicating the size of the SLM block; 3. Apparatus according to claim 1 or 2.

4. The partitioning hardware obtains the SLM block.

4. Apparatus according to any one of claims 1 to 3.

5. and dispatch hardware for dispatching groups of the plurality of execution threads to act on the SLM block.

5. Apparatus according to any one of claims 1 to 4.

6. the dispatch hardware stalls the thread dispatch until the partitioning hardware acquires the portion of the memory.

6. The apparatus of claim 5.

7. the partitioning hardware tracks usage of the SLM blocks by the multiple execution threads.

7. Apparatus according to any one of claims 1 to 6.

8. the partitioning hardware resets the memory to act as the cache upon a determination that the plurality of execution threads are no longer active.

8. Apparatus according to any one of claims 1 to 7.

9. 1. A method for facilitating partitioning of a local memory, comprising: partitioning the memory for use as a cache and as a shared local memory (SLM); The partitioning is performed based on a number of execution threads executing on a number of execution units that are active; the partitioning includes receiving a thread dispatch including a command to allocate a portion of the memory as an SLM block. method.

10. Partitioning the memory includes: activating the memory to operate as the cache; allocating the SLM block; obtaining said SLM block; having 10. The method of claim 9.

11. the command includes a header indicating the size of the SLM block; 11. The method according to claim 9 or 10.

12. and dispatching the group of execution threads to act on the SLM block.

12. The method according to any one of claims 9 to 11.

13. and tracking usage of the SLM block by the multiple execution threads.

13. A method according to any one of claims 9 to 12.

14. resetting the memory to act as the cache upon a determination that the plurality of execution threads are no longer active.

14. The method according to any one of claims 9 to 13.

15. a plurality of execution units executing a plurality of threads of execution; a memory coupled to share access among the execution units; partitioning hardware that partitions the memory to be used as a cache and as a shared local memory (SLM), the partitioning hardware partitioning the memory based on the number of execution threads executing on the execution units that are active; and a plurality of slices each having a plurality of sub-slices including the partitioning hardware receives a thread dispatch including a command to allocate a portion of the memory as an SLM block; Graphics Processing Unit (GPU).

16. the partitioning hardware activates the memory to act as the cache; 16. The GPU of claim 15.

17. The command includes a header indicating a size of the SLM block.

17. A GPU as claimed in claim 15 or 16.

18. and dispatch hardware for dispatching groups of the plurality of execution threads to act on the SLM block.

18. A GPU according to any one of claims 15 to 17.

19. the partitioning hardware tracks usage of the SLM blocks by the multiple execution threads and resets the memory to act as the cache upon a determination that the multiple execution threads are no longer active.

19. A GPU according to any one of claims 15 to 18.

Images