Page table mapping mechanism

Abstract

To provide an apparatus and a method for facilitating page conversion.SOLUTION: In a processing system, a device is configured to include a page table having: a frame buffer to data of a plurality of pages; and a plurality of display page tables (DPT) and a plurality of page table entries (PTE) for storing conversion in a page of the data in the frame buffer from a virtual address to a physical address. A method maps each PTE to one of a plurality of display page tables. The page table is searched to capture an address associated with the DPT and page conversion is processed 1940. Access is made by using the address captured to obtain the conversion to the DPT 1950. Thereafter, the page conversion is returned 1960.SELECTED DRAWING: Figure 19

Description

[Background technology]

[0001] Graphics processing units (GPUs) typically implement page tables containing arrays of page translation entries (PTEs) used in mapping logical graphics memory addresses to physical memory addresses. Page tables typically create a single system-wide 4 gigabyte (GB) virtual address space to provide scatter-gather access to system or local memory for integrated or discrete hardware. Page tables are often implemented as single-level, physically contiguous tables to avoid the latency of multi-level tables as well as requests from a virtual machine monitor or operating system to physically allocate contiguous pages.

[0002] However, the 4GB limit for page tables is disadvantageous because high display resolutions combined with the increasing use of display hardware for window compositing create exponential demands on display memory addressability. Furthermore, operating systems and applications are limited by installed memory and therefore do not expect a fixed limit on representable allocations. Increasing the size of page tables is not a scalable solution and would impact current display hardware implementations. [Brief explanation of the drawings]

[0003] So that the above-described features of the invention may be understood in detail, a more particular description of the invention briefly summarized above will be obtained by reference to embodiments, some of which are illustrated in the accompanying drawings. It should be noted, however, that the accompanying drawings illustrate only typical embodiments of the invention, and therefore the invention should not be considered as limiting its scope, since other equally effective embodiments may be recognized.

[0004] [Figure 1]FIG. 1 is a block diagram of a processing system according to one embodiment.

[0005] [Figure 2A] 1 illustrates a computing system and graphics processor provided by embodiments described herein. [Figure 2B] 1 illustrates a computing system and graphics processor provided by embodiments described herein. [Figure 2C] 1 illustrates a computing system and graphics processor provided by embodiments described herein. [Figure 2D] 1 illustrates a computing system and graphics processor provided by embodiments described herein.

[0006] [Figure 3A] 1 illustrates a block diagram of an additional graphics processor and compute accelerator architecture provided by an embodiment. [Figure 3B] 1 illustrates a block diagram of an additional graphics processor and compute accelerator architecture provided by an embodiment. [Figure 3C] 1 illustrates a block diagram of an additional graphics processor and compute accelerator architecture provided by an embodiment.

[0007] [Figure 4] FIG. 1 is a block diagram of a graphics processing engine of a graphics processor according to some embodiments.

[0008] [Figure 5A] 5 illustrates thread execution logic 500 including an array of processing elements employed in a graphics processor core according to an embodiment. [Figure 5B] 5 illustrates thread execution logic 500 including an array of processing elements employed in a graphics processor core according to an embodiment.

[0009] [Figure 6] 6 illustrates an additional execution unit 600 according to one embodiment.

[0010] [Figure 7] FIG. 1 is a block diagram illustrating an instruction format for a graphics processor according to some embodiments.

[0011] [Figure 8] FIG. 10 is a block diagram of a graphics processor according to another embodiment.

[0012] [Figure 9A] 1 illustrates a graphics processor command format and command sequence according to some embodiments. [Figure 9B] 1 illustrates a graphics processor command format and command sequence according to some embodiments.

[0013] [Figure 10] 1 illustrates an exemplary graphics software architecture for a data processing system according to some embodiments.

[0014] [Figure 11A] 1 illustrates an integrated circuit package assembly according to one embodiment. [Figure 11B] 1 illustrates an integrated circuit package assembly according to one embodiment. [Figure 11C] 1 illustrates an integrated circuit package assembly according to one embodiment. [Figure 11D] 1 illustrates an integrated circuit package assembly according to one embodiment.

[0015] [Figure 12] 1 is a block diagram illustrating an exemplary system-on-chip integrated circuit according to one embodiment.

[0016] [Figure 13A] FIG. 1 is a block diagram illustrating an embodiment of a further exemplary graphics processor. [Figure 13B] FIG. 1 is a block diagram illustrating an embodiment of a further exemplary graphics processor.

[0017] [Figure 14] 1 illustrates an embodiment of a computing device that employs a page table mapping mechanism.

[0018] [Figure 15] 1 illustrates a conventional frame buffer mapping.

[0019] [Figure 16] 1 illustrates one embodiment of a graphics processing unit.

[0020] [Figure 17] 1 illustrates one embodiment of a mechanism for performing frame buffer mapping.

[0021] [Figure 18A] 10 illustrates another embodiment of a mechanism for performing frame buffer mapping.

[0022] [Figure 18B] 1 illustrates one embodiment of a frame buffer.

[0023] [Figure 19] 1 is a flow diagram illustrating one embodiment of a process for performing virtual to physical address translation via frame buffer mapping.

[0024] DETAILED DESCRIPTION OF THE INVENTION

[0025] In the following description, numerous specific details are set forth to provide a more thorough understanding of the present invention. However, it will be apparent to one skilled in the art that the present invention may be practiced without one or more of these specific details. In other instances, well-known features are not described to avoid obscuring the invention.

[0026] In an embodiment, a page mapping mechanism is implemented that performs frame buffer mapping from the page table, providing a two-level page table walk where each page table entry maps to a display page table (DPT) page and the second level walk of the DPT points to a physical frame buffer page. [System Overview]

[0027] 1 is a block diagram of one embodiment of a processing system 100. System 100 may be used in a single-processor desktop system, a multi-processor workstation system, or a server system having multiple processors 102 or processor cores 107. In one embodiment, system 100 is a processing platform integrated into a system-on-chip (SoC) integrated circuit for use in a mobile, handheld, or embedded device, such as an "Internet of Things (IoT)" device with wired or wireless connectivity to a local or wide area network.

[0028] In one embodiment, system 100 can include, be coupled to, or be integrated with a server-based gaming platform, a game console, including a game and media console, a mobile gaming console, a handheld game console, or an online game console. In some embodiments, system 100 is part of an internet-connected mobile device, such as a mobile phone, a smartphone, a tablet computing device, or a laptop with low-capacity internal storage. Processing system 100 can also include, be coupled to, or be integrated with a wearable device, such as a smartwatch-type wearable device, smart eyewear or smart clothing enhanced with augmented reality (AR) or virtual reality (VR) capabilities that provide visual, audio, or tactile output to supplement real-world visual, audio, or tactile experiences or otherwise provide text, audio, graphics, video, holographic images or video, or tactile feedback, other augmented reality (AR) devices, or other virtual reality (VR) devices. In some embodiments, processing system 100 includes or is part of a television or set-top box-type device. In one embodiment, system 100 may include, be coupled to, or be integrated into an autonomous vehicle, such as a bus, a tractor trailer, a passenger car, a motorcycle or electric bicycle, an airplane, or a glider (or any combination thereof), which may use system 100 to process the sensed environment around the vehicle.

[0029] In some embodiments, the one or more processors 102 each include one or more processor cores 107 that process instructions that, when executed, perform operations for system or user software. In some embodiments, at least one of the one or more processor cores 107 is configured to process a particular instruction set 109. In some embodiments, the instruction set 109 may facilitate computing via complex instruction set computing (CISC), reduced instruction set computing (RISC), or very long instruction word (VLIW). One or more processor cores 107 may process different instruction sets 109, which may include instructions that facilitate emulation of other instruction sets. The processor cores 107 may also include other processing devices, such as digital signal processors (DSPs).

[0030] In some embodiments, processor 102 includes cache memory 104. Depending on the architecture, processor 102 may have a single internal cache or multiple levels of internal cache. In some embodiments, cache memory is shared among various components of processor 102. In some embodiments, processor 102 also uses an external cache (e.g., a level 3 (L3) cache or a last level cache (LLC)) (not shown), which may be shared among multiple processor cores 107 using known cache coherency techniques. A register file 106 may additionally be included in processor 102 and may include different types of registers (e.g., integer registers, floating-point registers, status registers, and instruction pointer registers) that store different types of data. Some registers may be general-purpose registers, while other registers may be specific to the design of processor 102.

[0031] In some embodiments, one or more processors 102 are coupled to one or more interface buses 110 that carry communication signals, such as address, data, or control signals, between the processors 102 and other components of the system 100. In one embodiment, the interface bus 110 may be a processor bus, such as a version of a Direct Media Interface (DMI) bus. However, the processor bus is not limited to a DMI bus and may include one or more Peripheral Component Interconnect buses (e.g., PCI, PCI Express), memory buses, or other types of interface buses. In one embodiment, the processor 102 includes an integrated memory controller 116 and a platform controller hub 130. The memory controller 116 facilitates communication between memory devices and other components of the system 100, while the platform controller hub (PCH) 130 provides connectivity to I / O devices via a local I / O bus.

[0032] Memory device 120 may be a dynamic random access memory (DRAM) device, a static random access memory (SRAM) device, a flash memory device, a phase change memory device, or any other memory device with performance suitable for serving as process memory. In one embodiment, memory device 120 acts as system memory for system 100 and can store data 122 and instructions 121 used by one or more processors 102 when executing applications or processes. Memory controller 116 also couples to an optional external graphics processor 118, which may communicate with one or more graphics processors 108 of processor 102 to perform graphics and media operations. In some embodiments, graphics, media, and / or compute operations may be assisted by accelerator 112, which is a co-processor that can be configured to perform a dedicated set of graphics, media, or compute operations. For example, in one embodiment, accelerator 112 is a matrix multiplication accelerator used to optimize machine learning or compute operations. In one embodiment, accelerator 112 is a ray tracing accelerator that may be used to perform ray tracing operations in conjunction with graphics processor 108. In one embodiment, an external accelerator 119 may be used in place of or in conjunction with accelerator 112.

[0033] In some embodiments, a display device 111 may be connected to the processor 102. The display device 111 may be one or more of an internal display device such as found in a mobile electronic device or laptop device, or an external display device attached via a display interface (e.g., DisplayPort, etc.). In one embodiment, the display device 111 may be a head-mounted display (HMD), such as a stereoscopic display device for use in virtual reality (VR) or augmented reality (AR) applications.

[0034] In some embodiments, the platform controller hub 130 enables peripherals to connect to the memory device 120 and the processor 102 via a high-speed I / O bus. The I / O peripherals include, but are not limited to, an audio controller 146, a network controller 134, a firmware interface 128, a wireless transceiver 126, a touch sensor 125, and a data storage device 124 (e.g., non-volatile memory, volatile memory, a hard disk drive, flash memory, NAND, 3D NAND, 3D XPoint, etc.). The data storage device 124 can be connected via a storage interface (e.g., SATA) or via a peripheral bus such as a Peripheral Component Interconnect bus (e.g., PCI, PCI Express). The touch sensor 125 may include a touchscreen sensor, a pressure sensor, or a fingerprint sensor. The wireless transceiver 126 may be a Wi-Fi® transceiver, a Bluetooth® transceiver, or a mobile network transceiver such as a transceiver for 3G, 4G, 5G, or Long Term Evolution (LTE). Firmware interface 128 enables communication with system firmware and may be, for example, a Unified Extensible Firmware Interface (UEFI). Network controller 134 may enable network connectivity to a wired network. In some embodiments, a high-performance network controller (not shown) couples to interface bus 110. Audio controller 146, in one embodiment, is a multi-channel high-definition audio controller. In one embodiment, system 100 includes an optional legacy I / O controller 140 that couples legacy (e.g., Personal System 2 (PS / 2)) devices to the system. Platform controller hub 130 may also connect to one or more universal serial bus (USB) controllers 142 to connect input devices such as a keyboard and mouse combination 143, a camera 144, or other USB input devices.

[0035] It will be understood that the illustrated system 100 is exemplary and not limiting, as other types of data processing systems configured differently may be used. For example, an example of memory controller 116 and platform controller hub 130 may be integrated into a separate external graphics processor, such as external graphics processor 118. In one embodiment, platform controller hub 130 and / or memory controller 116 may be external to one or more processors 102. For example, system 100 may include external memory controller 116 and platform controller hub 130, which may be configured as a memory controller hub and a peripheral controller hub within a system chipset that communicates with processor 102.

[0036] For example, a circuit board ("sled") may be used in which components such as CPUs, memory, and other components are arranged and designed to enhance thermal performance. In some examples, processing components such as processors are located on the front side of the sled, while nearby memory, such as DIMMs, is located on the back side of the sled. As a result of the improved airflow provided by this design, these components may operate at higher frequencies and power levels than in typical systems, thereby improving performance. Furthermore, the sled is configured to allow for unobtrusive connection to power and data communication cables in the rack, thereby enhancing the ability to quickly remove, upgrade, reinstall, and / or replace the sled. Similarly, the individual components located on the sled, such as the processors, accelerators, memory, and data storage drives, are configured to be easily upgraded due to the increased spacing between them. In the illustrated embodiment, these components further include hardware authentication features to prove their authenticity.

[0037] Data centers can utilize a single network architecture ("fabric") that supports multiple other network architectures, including Ethernet and Omni-Path. Threads can be coupled to switches via optical fiber, which offers higher bandwidth and lower latency than typical twisted-pair cabling (e.g., Category 5, Category 5e, Category 6, etc.). The high-bandwidth, low-latency interconnect and network architecture allows data centers to pool resources, such as memory, accelerators (e.g., GPUs, graphics accelerators, FPGAs, ASICs, neural network and / or artificial intelligence accelerators, etc.), and physically separate data storage drives, and provide compute resources (e.g., processors) to these resources as needed, allowing the compute resources to access the pooled resources as if they were local.

[0038] A power supply or power source can provide voltage and / or current to system 100 or any component or system described herein. In one example, the power supply includes an AC to DC (alternating current to direct current) adapter that connects to a wall outlet. Such an AC power source can be a renewable energy (e.g., solar) power source. In one example, the power source includes a DC power source, such as an external AC / DC converter. In one example, the power source or power supply includes wireless charging hardware that charges by proximity to a charging field. In one example, the power source can include an internal battery, an AC supply, a motion-based power supply, a solar power supply, or a fuel cell power source.

[0039] 2A-2D illustrate a computing system and graphics processor provided by embodiments described herein. Elements in Figures 2A-2D having the same reference number (or name) as elements in any other figure herein may operate or function in any manner similar to that described elsewhere herein, but are not limited to such.

[0040] FIG. 2A is a block diagram of an embodiment of a processor 200 having one or more processor cores 202A-202N, an integrated memory controller 214, and an integrated graphics processor 208. The processor 200 may include additional cores up to and including the additional core 202N, represented by the dashed box. Each of the processor cores 202A-202N includes one or more internal cache units 204A-204N. In some embodiments, each processor core also has access to one or more shared cache units 206. The internal cache units 204A-204N and the shared cache unit 206 represent a cache memory hierarchy within the processor 200. The cache memory hierarchy may include at least one level of instruction and data cache within each processor core and one or more levels of shared mid-level cache (e.g., level 2 (L2), level 3 (L3), level 4 (L4), or other level cache), with the highest level of cache before external memory classified as an LLC. In some embodiments, cache coherency logic maintains coherency between the various cache units 206 and 204A-204N.

[0041] In some embodiments, processor 200 may also include a set of one or more bus controller units 216 and a system agent core 210. The one or more bus controller units 216 manage a set of peripheral buses, such as one or more PCI or PCI Express buses. System agent core 210 provides management functions for various processor components. In some embodiments, system agent core 210 includes one or more integrated memory controllers 214 that manage access to various external memory devices (not shown).

[0042] In some embodiments, one or more of processor cores 202A-202N include support for simultaneous multithreading. In such embodiments, system agent core 210 includes components that coordinate the operation of cores 202A-202N during multithreaded processing. System agent core 210 may also include a power control unit (PCU), which includes logic and components that coordinate the power state of processor cores 202A-202N and graphics processor 208.

[0043] In some embodiments, processor 200 further includes a graphics processor 208 that performs graphics processing operations. In some embodiments, graphics processor 208 couples to a set of shared cache units 206 and to a system agent core 210, which includes one or more integrated memory controllers 214. In some embodiments, system agent core 210 also includes a display controller 211 that drives the graphics processor to output to one or more coupled displays. In some embodiments, display controller 211 may also be a separate module coupled to the graphics processor via at least one interconnect, or may be integrated into graphics processor 208.

[0044] In some embodiments, a ring-based interconnect unit 212 is used to connect the internal components of processor 200. However, alternative interconnect units may be used, including point-to-point interconnects, switched interconnects, or other technologies well known in the art. In some embodiments, graphics processor 208 connects to ring interconnect 212 via I / O link 213.

[0045] Exemplary I / O link 213 represents at least one of a number of different I / O interconnects, including on-package I / O interconnects that facilitate communication between various processor components and a high-performance embedded memory module 218, such as an eDRAM module. In some embodiments, each of processor cores 202A-202N and graphics processor 208 may use embedded memory module 218 as a shared last-level cache.

[0046] In some embodiments, processor cores 202A-202N are homogeneous cores that execute the same instruction set architecture. In another embodiment, processor cores 202A-202N are heterogeneous in terms of instruction set architecture (ISA), where one or more of processor cores 202A-202N execute a first instruction set and at least one of the other cores executes a subset of the first instruction set or a different instruction set. In one embodiment, processor cores 202A-202N are heterogeneous in terms of microarchitecture, where one or more cores with relatively high power consumption are coupled with one or more cores with relatively low power consumption. In one embodiment, processor cores 202A-202N are heterogeneous in terms of computational capability. Furthermore, processor 200 may be implemented on one or more chips or as an SoC integrated circuit having the depicted components in addition to other components.

[0047] FIG. 2B is a block diagram of hardware logic for a graphics processor core 219 according to some embodiments described herein. Elements of FIG. 2B having the same reference number (or name) as elements of any other figure herein can operate or function in any manner similar to that described elsewhere herein, but are not limited to such. The graphics processor core 219 may be referred to as a core slice and may be one or more graphics cores within a modular graphics processor. The graphics processor core 219 is an exemplary graphics core slice; the graphics processors described herein may include multiple graphics core slices based on target power and performance ranges. Each graphics processor core 219 may include a fixed function block 230 coupled with multiple sub-cores 221A-221F, also referred to as sub-slices, that include modular blocks of general-purpose and fixed-function logic.

[0048] In some embodiments, fixed function block 230 includes a geometry / fixed function pipeline 231 that may be shared by all sub-cores of graphics processor core 219, for example, in a lower-performance and / or lower-power graphics processor implementation. In various embodiments, geometry / fixed function pipeline 231 includes a 3D fixed function pipeline (e.g., 3D pipeline 312 as seen in Figures 3A and 4, described below), a video front end unit, a thread generator and thread dispatcher, and a unified return buffer manager that manages a unified return buffer (e.g., unified return buffer 418 in Figure 4, described below).

[0049] In one embodiment, fixed function block 230 also includes graphics SoC interface 232, graphics microcontroller 233, and media pipeline 234. Graphics SoC interface 232 provides an interface between graphics processor core 219 and other processor cores within the system-on-chip integrated circuit. Graphics microcontroller 233 is a programmable sub-processor that can be configured to manage various functions of graphics processor core 219, including thread dispatch, scheduling, and preemption. Media pipeline 234 (e.g., media pipeline 316 in Figures 3A, 3B, 3C, and 4) includes logic that facilitates decoding, encoding, pre-processing, and / or post-processing of multimedia data, including image and video data. Media pipeline 234 implements media operations via requests to compute or sampling logic within sub-cores 221-221F.

[0050] In one embodiment, SoC interface 232 enables graphics processor core 219 to communicate with a general-purpose application processor core (e.g., a CPU) and / or other components within the SoC (including memory hierarchy elements such as shared last-level cache memory, system RAM, and / or embedded on-chip or on-package DRAM). SoC interface 232 may also enable communication with fixed-function devices within the SoC, such as a camera imaging pipeline, and may enable the use and / or implementation of global memory atomics that may be shared between graphics processor core 219 and CPUs within the SoC. SoC interface 232 may also implement power management controls for graphics processor core 219 and enable interfacing between the clock domain of graphics core 219 and other clock domains within the SoC. In one embodiment, SoC interface 232 enables receipt of command buffers from a command streamer and global thread dispatcher configured to provide commands and instructions to each of one or more graphics cores within the graphics processor. Commands and instructions can be dispatched to the media pipeline 234 when media operations are performed, or to the geometry and fixed function pipelines (e.g., geometry and fixed function pipeline 231, geometry and fixed function pipeline 237) when graphics processing operations are performed.

[0051] Graphics microcontroller 233 may be configured to perform various scheduling and management tasks for graphics processor core 219. In one embodiment, graphics microcontroller 233 can perform graphics and / or compute workload scheduling for various graphics parallel engines in execution unit (EU) arrays 222A-222F, 224A-224F within sub-cores 221A-221F. In this scheduling model, host software executing on a CPU core of an SoC that includes graphics processor core 219 can send a workload to one of multiple graphics processor doorbells, which can then invoke scheduling operations on the appropriate graphics engine. Scheduling operations include determining which workload to run next, sending the workload to a command streamer, preempting existing workloads running on an engine, monitoring the progress of workloads, and notifying host software when a workload is complete. In one embodiment, graphics microcontroller 233 can also facilitate low power or idle states for graphics processor core 219 and can provide graphics processor core 219 with the ability to save and restore registers within graphics processor core 219 upon transition to a low power state independent of the operating system and / or graphics driver software on the system.

[0052] Graphics processor core 219 may have more or fewer sub-cores than the shown sub-cores 221A-221F, and may have up to N modular sub-cores. For each set of N sub-cores, graphics processor core 219 may also include shared function logic 235, shared and / or cache memory 236, geometry / fixed function pipeline 237, and additional fixed function logic 238 that accelerates various graphics and compute processing operations. Shared function logic 235 may include logical units associated with shared function logic 420 of FIG. 4 (e.g., samplers, math, and / or inter-thread communication logic), which may be shared by each of the N sub-cores in graphics processor core 219. Shared and / or cache memory 236 may be a last-level cache for the set of N sub-cores 221A-221F in graphics processor core 219 and may also serve as a shared memory accessible by multiple sub-cores. Geometry / fixed function pipeline 237 may be included in place of geometry / fixed function pipeline 231 in fixed function block 230 and may include the same or similar logical units.

[0053] In one embodiment, graphics processor core 219 includes additional fixed-function logic 238, which can include various fixed-function acceleration logic for use by graphics processor core 219. In one embodiment, additional fixed-function logic 238 includes an additional geometry pipeline for use with position-only shading. In position-only shading, there are two geometry pipelines: a full geometry pipeline within geometry / fixed-function pipeline 238, 231, and an additional geometry pipeline, a thinning pipeline, which can be included in additional fixed-function logic 238. In one embodiment, the thinning pipeline is a limited-function version of the full geometry pipeline. The full pipeline and the thinning pipeline can run different instances of the same application, each with a separate context. Position-only shading can hide lengthy duplicating of discarded triangles, allowing shading to complete faster in some instances. For example, in one embodiment, the thinning pipeline logic within additional fixed-function logic 238 can execute position shaders in parallel with the main application and generally produce significant results faster than the full pipeline. The reason is that the decimation pipeline only fetches and shades vertex position attributes, and does not perform pixel rasterization and rendering to the frame buffer. The decimation pipeline can use the generated nontrivial results to calculate visibility information for all triangles, regardless of whether those triangles are decimated or not. The full pipeline (sometimes called a replay pipeline in this example) can skip over decimated triangles and consume the visibility information to shade only the visible triangles that are ultimately passed to the rasterization stage.

[0054] In one embodiment, additional fixed function logic 238 may also include machine learning acceleration logic, such as fixed function matrix multiplication logic, for implementations involving machine learning training or inference optimization.

[0055] Each graphics subcore 221A-221F includes a set of execution resources that can be used to perform graphics operations, media operations, and compute operations as requested by a graphics pipeline, a media pipeline, or a shader program. The graphics subcores 221A-221F include a plurality of EU arrays 222A-222F, 224A-224F, thread dispatch and inter-thread communication (TD / IC) logic 223A-223F, 3D (e.g., texture) samplers 225A-225F, media samplers 206A-206F, shader processors 227A-227F, and shared local memories (SLMs) 228A-228F. Each of the EU arrays 222A-222F, 224A-224F includes multiple execution units, which are general-purpose graphics processing units capable of performing floating-point and integer / fixed-point logic operations, including graphics, media, or compute shader programs, in service of graphics, media, or compute operations. The TD / IC logic 223A-223F performs local thread dispatch and thread control operations for the execution units within a sub-core and facilitates communication between threads executing on the sub-core's execution units. The 3D samplers 225A-225F can read textures or other 3D graphics-related data into memory. The 3D samplers can read different texture data based on the configured sample state and the texture format associated with a given texture. The media samplers 206A-206F can perform similar read operations based on the type and format associated with the media data. In one embodiment, each of the graphics sub-cores 221A-221F can alternately include an integrated 3D and media sampler.Threads executing in execution units within each of the sub-cores 221A-221F can utilize shared local memory 228A-228F within each sub-core, allowing threads executing within a thread group to execute using a common pool of on-chip memory.

[0056] 2C illustrates a graphics processing unit (GPU) 239 that includes a dedicated set of graphics processing resources arranged into multi-core groups 240A-240N. While details of only a single multi-core group 240A are provided, it will be understood that the other multi-core groups 240B-240N may also include the same or similar sets of graphics processing resources.

[0057] As shown, multi-core group 240A may include a set of graphics cores 243, a set of tensor cores 244, and a set of ray tracing cores 245. A scheduler / dispatcher 241 schedules and dispatches graphics threads for execution on the various cores 243, 244, 245. A set of register files 242 stores operand values ​​used by cores 243, 244, 245 when executing the graphics threads. These may include, for example, integer registers for storing integer values, floating-point registers for storing floating-point values, vector registers for storing packed data elements (integer and / or floating-point data elements), and tile registers for storing tensor / matrix values. In one embodiment, the tile registers are implemented as a combined set of vector registers.

[0058] One or more level 1 (L1) caches in combination with shared memory units 247 store graphics data, such as texture data, vertex data, pixel data, ray data, and bounding volume data, locally to each multicore group 240A. One or more texture units 247 may also be used to perform texturing operations, such as texture mapping and sampling. A level 2 (L2) cache 253, shared by all or a subset of the multicore groups 240A-240N, stores graphics data and / or instructions for multiple concurrent graphics threads. As shown, the L2 cache 253 may be shared across multiple multicore groups 240A-240N. One or more memory controllers 248 couple the GPU 239 to memory 249, which may be system memory (e.g., DRAM) and / or dedicated graphics memory (e.g., GDDR6 memory).

[0059] Input / output (I / O) circuitry 250 couples GPU 239 to one or more I / O devices 252, such as a digital signal processor (DSP), a network controller, or a user input device. An on-chip interconnect may be used to couple I / O device 252 to GPU 239 and memory 249. One or more I / O memory management units (IOMMUs) 251 of I / O circuitry 250 couple I / O device 252 directly to system memory 249. In one embodiment, IOMMU 251 manages multiple sets of page tables for mapping virtual addresses to physical addresses in system memory 249. In this embodiment, I / O device 252, CPU 246, and GPU 239 may share the same virtual address space.

[0060] In one implementation, IOMMU 251 supports virtualization. In this case, IOMMU 251 may manage a first set of page tables for mapping guest / graphics virtual addresses to guest / graphics physical addresses and a second set of page tables for mapping guest / graphics physical addresses to system / host physical addresses (e.g., in system memory 249). The base addresses of each of the first and second sets of page tables may be stored in control registers and swapped out during a context switch (e.g., so that a new context has access to the relevant set of page tables). Although not shown in FIG. 2C , each of cores 243, 244, 245 and / or multicore groups 240A-240N may include a translation lookaside buffer (TLB) for caching guest-virtual-to-guest-physical, guest-physical-to-host-physical, and guest-virtual-to-host-physical translations.

[0061] In one embodiment, CPU 246, GPU 239, and I / O devices 252 are integrated into a single semiconductor chip and / or chip package. The illustrated memory 249 may be integrated into the same chip or may be coupled to memory controller 248 via an off-chip interface. In one implementation, memory 249 includes GDDR6 memory that shares the same virtual address space as other physical system-level memory, although the underlying principles of the present invention are not limited to this particular implementation.

[0062] In one embodiment, tensor cores 244 include multiple execution units specifically designed to perform matrix operations, which are fundamental compute operations used to perform deep learning operations. For example, simultaneous matrix multiplication operations may be used for neural network training and inference. Tensor cores 244 may perform matrix processing using a variety of operand precisions, including single-precision floating-point (e.g., 32 bits), half-precision floating-point (e.g., 16 bits), integer words (16 bits), bytes (8 bits), and half-bytes (4 bits). In one embodiment, a neural network implementation extracts feature points from each rendered scene, potentially combining details from multiple frames to construct a high-quality final image.

[0063] In a deep learning implementation, parallel matrix multiplication operations may be scheduled for execution on tensor cores 244. Neural network training, in particular, requires a significant number of matrix dot-product operations. To process the inner product formulation of an N×N×N matrix multiplication, tensor cores 244 may include at least N dot-product processing elements. Before starting the matrix multiplication, an entire matrix is ​​loaded into a tile register, and at least one column of a second matrix is ​​loaded in each of N cycles. In each cycle, there are N dot products processed.

[0064] Matrix elements may be stored in different precisions, including 16-bit words, 8-bit bytes (e.g., INT8), and 4-bit half-bytes (e.g., INT4), depending on the particular implementation. Different precision modes may be specified for tensor cores 244 to ensure that the most efficient precision is used for different workloads (e.g., inference workloads that can tolerate quantization to bytes and half-bytes).

[0065] In one embodiment, ray tracing core 245 accelerates ray tracing operations for both real-time and non-real-time ray tracing implementations. Specifically, ray tracing core 245 includes ray traversal / intersection circuitry for performing ray traversal using a bounding volume hierarchy (BVH) and identifying intersections between rays and primitives enclosed within the BVH volume. Ray tracing core 245 may also include circuitry for performing depth testing and decimation (e.g., using a Z-buffer or similar mechanism). In one implementation, ray tracing core 245 performs traversal and intersection operations in conjunction with the image denoising techniques described herein, at least some of which may be performed by tensor core 244. For example, in one embodiment, tensor core 244 implements a deep learning neural network to perform denoising of frames produced by ray tracing core 245. However, the CPU 246, the graphics core 243, and / or the ray tracing core 245 may also implement all or part of the denoising algorithm and / or the deep learning algorithm.

[0066] Additionally, as mentioned above, a distributed approach to denoising may be utilized in which the GPU 239 resides in a computing device that is coupled to other computing devices via a network or high-speed interconnect. In this embodiment, the interconnected computing devices share neural network learning / training data, improving the speed at which the overall system learns to perform denoising for different types of image frames and / or different graphics applications.

[0067] In one embodiment, the ray tracing core 245 handles all BVH traversal and ray primitive intersections, preventing the graphics core 243 from becoming overloaded with thousands of instructions per ray. In one embodiment, each ray tracing core 245 includes a first set of dedicated circuitry for performing bounding box tests (e.g., for traversal operations) and a second set of dedicated circuitry for performing ray-triangle intersection tests (e.g., intersecting traversed rays). Thus, in one embodiment, the multi-core group 240A only needs to initiate ray probes, and the ray tracing cores 245 independently perform ray traversal and intersection and return hit data (e.g., hit, no hit, multiple hits, etc.) to the thread context. The ray tracing core 245 performs the traversal and intersection operations while the other cores 243, 244 are freed to perform other graphics or compute work.

[0068] In one embodiment, each ray tracing core 245 includes a traversal unit for performing BVH test operations and an intersection unit for performing ray primitive intersection tests. The intersection unit generates a "hit," "no hit," or "multiple hit" response and provides it to the appropriate thread. During traversal and intersection operations, the execution resources of other cores (e.g., graphics core 243 and tensor core 244) are freed up to perform other forms of graphics work.

[0069] In one particular embodiment described below, a hybrid rasterization / ray tracing approach is used, in which work is distributed between the graphics core 243 and the ray tracing core 245.

[0070] In one embodiment, ray tracing core 245 (and / or other cores 243, 244) includes hardware support for a ray tracing instruction set, such as Microsoft® DirectX Raytracing (DXR). DXR includes the DispatchRays command, as well as ray-generation, closest-hit, any-hit, and miss shaders, which allow each object to be assigned a unique set of shaders and textures. Another ray tracing platform that may be supported by ray tracing core 245, graphics core 243, and tensor core 244 is Vulkan 1.1.85. However, it should be noted that the principles underlying the present invention are not limited to any particular ray tracing ISA.

[0071] Generally, the various cores 245, 244, 243 may support a ray tracing instruction set that includes instructions / functions for Ray Generation, Closest Hit, Any Hit, Ray-primitive Intersection, Per-primitive and hierarchical Bounding box Construction, Miss, Visit, and Exceptions. More specifically, one embodiment includes ray tracing instructions for performing the following functions:

[0072] Ray Generation: Ray Generation instructions may be executed for each pixel, each sample, or each allocation of other user-defined operations.

[0073] Closest Hit: The Closest Hit instruction may be executed to find the closest intersection point between a ray and a primitive in the scene.

[0074] Any Hit: The Any Hit instruction identifies multiple intersections between rays and primitives in the scene and potentially identifies a new closest intersection point.

[0075] Intersection: The Intersection instruction performs a ray primitive intersection test and outputs the result.

[0076] Per-primitive Bounding box Construction: This instruction constructs a bounding box around a given primitive or group of primitives (e.g., when building a new BVH or other acceleration data structure).

[0077] Miss: The ray did not hit any geometry in the scene or in a specified region of the scene.

[0078] Visit: Indicates the child volume that the ray will traverse.

[0079] Exceptions: Contains various kinds of exception handlers (e.g., invoked for various error conditions).

[0080] 2D is a block diagram of a general-purpose graphics processing unit (GPGPU) 270, which may be configured as a graphics processor and / or compute accelerator, according to embodiments described herein. GPGPU 270 may interconnect with a host processor (e.g., one or more CPUs 246) and memories 271, 272 via one or more system buses and / or memory buses. In one embodiment, memory 271 is system memory that may be shared with one or more CPUs 246, and memory 272 is device memory dedicated to GPGPU 270. In one embodiment, each component in GPGPU 270 and device memory 272 may be mapped to memory addresses accessible by one or more CPUs 246. Access to memories 271 and 272 may be facilitated via memory controller 268. In one embodiment, memory controller 268 may include an integrated direct memory access (DMA) controller 269 or may include logic to perform operations that a DMA controller would otherwise perform.

[0081] The GPGPU 270 includes multiple cache memories, including an L2 cache 253, an L1 cache 254, an instruction cache 255, and a shared memory 256, at least a portion of which may be partitioned as a cache memory. The GPGPU 270 also includes multiple compute units 260A-260N. Each compute unit 260A-260N includes a set of vector registers 261, scalar registers 262, a vector logic unit 263, and a scalar logic unit 264. The compute units 260A-260N may also include a local shared memory 265 and a program counter 266. The compute units 260A-260N may be coupled to a constant cache 267, which may be used to store constant data, which is data that does not change during execution of a kernel or shader program executing on the GPGPU 270. In one embodiment, constant cache 267 is a scalar data cache, and data stored in the cache may be fetched directly into scalar registers 262 .

[0082] During operation, one or more CPUs 246 can write commands to registers or memory that is mapped to an accessible address space within GPGPU 270. Command processor 257 can read commands from registers or memory and determine how these commands will be processed within GPGPU 270. Thread dispatcher 258 can then be used to dispatch threads to compute units 260A-260N to execute these commands. Each compute unit 260A-260N can execute threads independently of the other compute units. Furthermore, each compute unit 260A-260N can be independently configured for conditional computation and can conditionally output the results of the computation to memory. Command processor 257 can interrupt one or more CPUs 246 when a sent command is completed.

[0083] 3A-3C show block diagrams of additional graphics processor and compute accelerator architectures provided by embodiments described herein. Elements in Figures 3A-3C having the same reference numbers (or names) as elements in any other figure herein can operate or function in any manner similar to that described elsewhere herein, but are not limited to such.

[0084] 3A is a block diagram of a graphics processor 300. Graphics processor 300 may be a discrete graphics processing unit, multiple processing cores, or a graphics processor integrated with other semiconductor devices, such as, but not limited to, memory devices or network interfaces. In some embodiments, the graphics processor communicates using commands placed in processor memory via a memory-mapped I / O interface to registers on the graphics processor. In some embodiments, graphics processor 300 includes a memory interface 314 to access memory. Memory interface 314 may be an interface to local memory, one or more internal caches, one or more shared external caches, and / or system memory.

[0085] In some embodiments, graphics processor 300 also includes a display controller 302 that drives display output data to a display device 318. Display controller 302 includes hardware for one or more overlay planes for displaying and compositing multiple layers of video or for user interface elements. Display device 318 may be an internal or external display device. In one embodiment, display device 318 is a head-mounted display device, such as a virtual reality (VR) display device or an augmented reality (AR) display device. In some embodiments, graphics processor 300 includes a video codec engine 306 for encoding media to, decoding media from, or transcoding media between one or more media encoding formats. These media encoding formats include, but are not limited to, Moving Picture Experts Group (MPEG) formats such as MPEG-2, Advanced Video Coding (AVC) formats such as H.264 / MPEG-4 AVC and H.265 / HEVC, Alliance for Open Media (AOMedia) VP8 and VP9, ​​and Society of Motion Picture and Television Engineers (SMPTE) 421M / VC-1, and Joint Photographic Experts Group (JPEG) formats such as JPEG, and Motion JPEG (MJPEG) formats.

[0086] In some embodiments, graphics processor 300 includes a block image transfer (BLIT) engine 304 for performing two-dimensional (2D) rasterizer operations, including, for example, bit-boundary block transfers. However, in one embodiment, 2D graphics operations are performed using one or more components of a graphics processing engine (GPE) 310. In some embodiments, GPE 310 is a compute engine for performing graphics operations, including three-dimensional (3D) graphics operations and media operations.

[0087] In some embodiments, the GPE 310 includes a 3D pipeline 312 for performing 3D operations, such as rendering three-dimensional images and scenes using processing functions that operate on 3D primitive shapes (e.g., rectangles, triangles, etc.). The 3D pipeline 312 includes programmable and fixed-function elements that perform various tasks within the element and / or spawn execution threads in the 3D / media subsystem 315. While the 3D pipeline 312 may be used to perform media operations, one embodiment of the GPE 310 also includes a media pipeline 316 that is specifically used to perform media operations, such as video post-processing and image enhancement.

[0088] In some embodiments, the media pipeline 316 includes fixed function or programmable logic units for performing one or more specialized media operations, such as video decode acceleration, video deinterlacing, and video encode acceleration, in place of or on behalf of the video codec engine 306. In some embodiments, the media pipeline 316 also includes a thread generation unit that generates threads for execution in the 3D / media subsystem 315. The generated threads perform computations for the media operations on one or more graphics execution units included in the 3D / media subsystem 315.

[0089] In some embodiments, the 3D / media subsystem 315 includes logic for executing threads created by the 3D pipeline 312 and the media pipeline 316. In one embodiment, these pipelines send thread execution requests to the 3D / media subsystem 315. The 3D / media subsystem 315 includes thread dispatch logic for coordinating and dispatching various requests to available thread execution resources. These execution resources include an array of graphics execution units for processing the 3D threads and the media threads. In some embodiments, the 3D / media subsystem 315 includes one or more built-in caches for thread instructions and data. In some embodiments, the subsystem also includes shared memory, including registers and addressable memory, for sharing data between threads and storing output data.

[0090] FIG. 3B illustrates a graphics processor 320 having a tile architecture according to an embodiment described herein. In one embodiment, the graphics processor 320 includes a graphics processing engine cluster 322 having multiple instances of the graphics processing engine 310 of FIG. 3A in graphics engine tiles 310A-310D. Each graphics engine tile 310A-310D may be interconnected via a set of tile interconnects 323A-323F. Each graphics engine tile 310A-310D may also be connected to a memory module or memory device 326A-326D via memory interconnects 325A-325D. The memory devices 326A-326D may use any graphics memory technology. For example, the memory devices 326A-326D may be graphics double data rate (GDDR) memory. In one embodiment, the memory devices 326A-326D are high-bandwidth memory (HBM) modules that may be on-die with the respective graphics engine tiles 310A-310D. In one embodiment, memory devices 326A-326D are stacked memory devices that may be stacked on top of respective graphics engine tiles 310A-310D. In one embodiment, each graphics engine tile 310A-310D and associated memory 326A-326D resides in a separate chiplet, which is bonded to a base die or base substrate, as described in further detail in Figures 11B-11D.

[0091] The graphics processing engine cluster 322 can be connected to an on-chip or on-package fabric interconnect 324. The fabric interconnect 324 may enable communication between the graphics engine tiles 310A-310D and components such as the video codec 306 and one or more copy engines 304. The copy engine 304 may be used to move data to and from the memory devices 326A-326D and between the memory devices 326A-326D and memory external to the graphics processor 320 (e.g., system memory). The fabric interconnect 324 may also be used to interconnect the graphics engine tiles 310A-310D. The graphics processor 320 may optionally include a display controller 302 to enable connection to an external display device 318. The graphics processor may also be configured as a graphics accelerator or a compute accelerator. In the accelerator configuration, the display controller 302 and the display device 318 may be omitted.

[0092] Graphics processor 320 may connect to a host system via host interface 328. Host interface 328 may enable communication between graphics processor 320, system memory, and / or other system components. Host interface 328 may be, for example, a PCI Express bus or another type of host system interface.

[0093] FIG. 3C illustrates a compute accelerator 330 according to embodiments described herein. The compute accelerator 330 may include architectural similarities to the graphics processor 320 of FIG. 3B and is optimized for compute acceleration. A compute engine cluster 332 may include a set of compute engine tiles 340A-340D that include execution logic optimized for parallel or vector-based general-purpose compute operations. In some embodiments, the compute engine tiles 340A-340D do not include fixed-function graphics processing logic, but in one embodiment, one or more of the compute engine tiles 340A-340D may include logic for performing media acceleration. The compute engine tiles 340A-340D may be connected to memories 326A-326D via memory interconnects 325A-325D. The memories 326A-326D and memory interconnects 325A-325D may be of similar technology as the graphics processor 320, or may be different. Graphics compute engine tiles 340A-340D may be interconnected via a set of tile interconnects 323A-323F, which may be connected to and / or by fabric interconnect 324. In one embodiment, compute accelerator 330 includes a large L3 cache 336, which may be configured as a device-wide cache. Compute accelerator 330 may also be connected to a host processor and memory via host interface 328, in a manner similar to graphics processor 320 of FIG. 3B. [Graphics Processing Engine]

[0094] FIG. 4 is a block diagram of a graphics processing engine 410 of a graphics processor, according to some embodiments. In one embodiment, the graphics processing engine (GPE) 410 is a version of the GPE 310 shown in FIG. 3A and may represent the graphics engine tiles 310A-310D of FIG. 3B. Elements of FIG. 4 having the same reference numbers (or names) as elements of any other figure herein can operate or function in any manner similar to that described elsewhere herein, but are not limited to such. For example, the 3D pipeline 312 and media pipeline 316 of FIG. 3A are shown. The media pipeline 316 is optional in some embodiments of the GPE 410 and may not be explicitly included in the GPE 410. For example, in at least one embodiment, a separate media and / or image processor is coupled to the GPE 410.

[0095] In some embodiments, the GPE 410 couples with or includes a command streamer 403, which provides a command stream to the 3D pipeline 312 and / or the media pipeline 316. In some embodiments, the command streamer 403 couples with memory, which may be system memory or one or more of an internal cache memory and a shared cache memory. In some embodiments, the command streamer 403 receives commands from memory and sends the commands to the 3D pipeline 312 and / or the media pipeline 316. These commands are instructions fetched from a ring buffer, which stores commands for the 3D pipeline 312 and the media pipeline 316. In one embodiment, the ring buffer may further include a batch command buffer that stores batches of commands. Commands for the 3D pipeline 312 may also include references to data stored in memory, such as, but not limited to, vertex and geometry data for the 3D pipeline 312 and / or image data and memory objects for the media pipeline 316. 3D pipeline 312 and media pipeline 316 process commands and data by executing operations through logic within the respective pipelines or by dispatching one or more execution threads to graphics core array 414. In one embodiment, graphics core array 414 includes one or more blocks of graphics cores (e.g., graphics core 415A, graphics core 415B), each block including one or more graphics cores. Each graphics core includes a set of graphics execution resources, including general-purpose and graphics-specific execution logic for performing graphics and compute operations, as well as fixed-function texture processing and / or machine learning artificial intelligence acceleration logic.

[0096] In various embodiments, the 3D pipeline 312 may include fixed-function logic and programmable logic for processing one or more shader programs by processing instructions and dispatching execution threads to the graphics core array 414. The shader programs may be vertex shaders, geometry shaders, pixel shaders, fragment shaders, compute shaders, or other shader programs. The graphics core array 414 provides a unified block of execution resources for use in processing these shader programs. Multipurpose execution logic (e.g., execution units) within the graphics cores 415A-415B of the graphics core array 414 includes support for various 3D API shader languages ​​and can execute multiple concurrent threads of execution associated with multiple shaders.

[0097] In some embodiments, graphics core array 414 includes execution logic for performing media functions such as video processing and / or image processing. In one embodiment, the execution unit includes general-purpose logic that is programmable to perform parallel general-purpose computing operations in addition to graphics processing operations. The general-purpose logic can perform processing operations in parallel or in conjunction with general-purpose logic present in processor core 107 of FIG. 1 or cores 202A-202N as shown in FIG. 2A.

[0098] Output data generated by threads executing in graphics core array 414 can output data to memory in unified return buffer (URB) 418. URB 418 can store data for multiple threads. In some embodiments, URB 418 can be used to send data between different threads executing in graphics core array 414. In some embodiments, URB 418 can also be used for synchronization between the threads of the graphics core array and fixed function logic in shared function logic 420.

[0099] In some embodiments, graphics core array 414 is scalable, such that the array includes a variable number of graphics cores, each with a variable number of execution units based on the target power and performance level of GPE 410. In one embodiment, these execution resources are dynamically scalable, such that execution resources may be enabled and disabled as needed.

[0100] Graphics core array 414 couples with shared function logic 420, which includes multiple resources shared among the graphics cores in the graphics core array. Shared functions within shared function logic 420 are hardware logic units that provide dedicated, complementary functionality to graphics core array 414. In various embodiments, shared function logic 420 includes, but is not limited to, sampler logic 421, math logic 422, and inter-thread communication (ITC) logic 423. Additionally, some embodiments implement one or more caches 425 within shared function logic 420.

[0101] Shared functionality is implemented at least when demand for a given dedicated functionality becomes insufficient to justify inclusion within the graphics core array 414. Instead, a single instantiation of this dedicated functionality is implemented in the shared functionality logic 420 as a standalone entity and shared among the execution resources within the graphics core array 414. The exact set of functionality shared among and included within the graphics core array 414 varies across multiple embodiments. In some embodiments, certain shared functionality within the shared functionality logic 420 that is used extensively by the graphics core array 414 may be included in the shared functionality logic 416 within the graphics core array 414. In various embodiments, the shared functionality logic 416 within the graphics core array 414 may include some or all of the logic within the shared functionality logic 420. In one embodiment, all logic elements within the shared functionality logic 420 may overlap with logic elements within the shared functionality logic 416 of the graphics core array 414. In one embodiment, the shared functionality logic 420 is excluded in favor of the shared functionality logic 416 within the graphics core array 414. [Execution Unit]

[0102] 5A-5B illustrate thread execution logic 500 including an array of processing elements utilized in a graphics processor core, according to embodiments described herein. Elements in FIGS. 5A-5B having the same reference number (or name) as elements in any other figure herein may operate or function in any manner similar to that described elsewhere herein, but are not limited to such. FIGS. 5A-5B illustrate an overview of thread execution logic 500, which may represent hardware logic illustrated with each of sub-cores 221A-221F in FIG. 2B. FIG. 5A represents an execution unit within a general-purpose graphics processor, while FIG. 5B represents an execution unit that may be used within a compute accelerator.

[0103] 5A , in some embodiments, thread execution logic 500 includes a shader processor 502, a thread dispatcher 504, an instruction cache 506, a scalable execution unit array including multiple execution units 508A-508N, a sampler 510, a shared local memory 511, a data cache 512, and a data port 514. In one embodiment, the scalable execution unit array can be dynamically scaled by enabling or disabling one or more execution units (e.g., any of execution units 508A, 508B, 508C, 508D, ..., 508N-1, and 508N) based on the computational requirements of a workload. In one embodiment, the included components are interconnected via an interconnect fabric that connects each of these components. In some embodiments, the thread execution logic 500 includes one or more connections to memory, such as system memory or cache memory, through one or more of an instruction cache 506, a data port 514, a sampler 510, and execution units 508A-508N. In some embodiments, each execution unit (e.g., 508A) is a standalone, programmable, general-purpose computational unit capable of executing multiple simultaneous hardware threads and processing multiple data elements for each thread in parallel. In various embodiments, the array of execution units 508A-508N is scalable to include any number of individual execution units.

[0104] In some embodiments, the execution units 508A-508N are primarily used to execute shader programs. The shader processor 502 processes various shader programs and may dispatch execution threads associated with the shader programs via a thread dispatcher 504. In one embodiment, the thread dispatcher includes logic for coordinating thread initiation requests from the graphics pipeline and media pipeline and instantiating the requested threads on one or more of the execution units 508A-508N. For example, the geometry pipeline may dispatch a vertex shader, tessellation shader, or geometry shader to thread execution logic for processing. In some embodiments, the thread dispatcher 504 may also handle runtime thread creation requests from executing shader programs.

[0105] In some embodiments, the execution units 508A-508N support an instruction set that includes native support for many standard 3D graphics shader instructions, allowing shader programs from graphics libraries (e.g., Direct3D and OpenGL) to execute with minimal translation. The execution units support vertex and geometry processing (e.g., vertex programs, geometry programs, vertex shaders), pixel processing (e.g., pixel shaders, fragment shaders), and general-purpose processing (e.g., compute shaders and media shaders). Each of the execution units 508A-508N is capable of multi-issue single instruction multiple data (SIMD) execution, and multithreaded operation enables an efficient execution environment despite high-latency memory accesses. Each hardware thread within each execution unit has a dedicated high-bandwidth register file and associated independent thread state. Execution is multi-issue per clock for a pipeline capable of integer operations, single- and double-precision floating-point operations, SIMD branching performance, logical operations, transcendental operations, and other miscellaneous operations. While waiting for data from memory or one of multiple shared functions, dependency logic within execution units 508A-508N puts the waiting thread to sleep until the requested data is returned. While the waiting thread is sleeping, hardware resources may be dedicated to processing other threads. For example, during the delay associated with a vertex shader operation, an execution unit may execute a pixel shader, fragment shader, or another type of shader program operation, including a different vertex shader. Various embodiments may be applicable to using execution using single instruction multiple threads (SIMT) as an alternative to or in addition to using SIMD. References to SIMD cores or operations may also apply to SIMT, or to SIMD in combination with SIMT.

[0106] Each execution unit in execution units 508A-508N operates on an array of data elements. The number of data elements is the "execution size" or number of channels for the instruction. An execution channel is a logical unit of execution for data element access, masking, and flow control within the instruction. The number of channels may be independent of the number of physical arithmetic logic units (ALUs) or floating-point units (FPUs) for a particular graphics processor. In some embodiments, execution units 508A-508N support integer and floating-point data types.

[0107] The instruction set of the execution unit includes SIMD instructions. Various data elements may be stored in registers as packed data types, and the execution unit will process the various elements based on the data size of these elements. For example, when processing a 256-bit wide vector, the 256-bit vector is stored in a register, and the execution unit processes the vector as four separate 54-bit packed data elements (quadword (QW) sized data elements), eight separate 32-bit packed data elements (doubleword (DW) sized data elements), sixteen separate 16-bit packed data elements (word (W) sized data elements), or thirty-two separate 8-bit data elements (byte (B) sized data elements). However, different vector widths and register sizes are possible.

[0108] In one embodiment, one or more execution units may be combined into fused execution units 509A-509N with thread control logic (507A-507N) common to the fused EUs. Multiple EUs may be fused into EU groups. Each EU in a fused EU group may be configured to execute a separate SIMD hardware thread. The number of EUs in a fused EU group may vary depending on the embodiment. Furthermore, various SIMD widths may be implemented per EU, including, but not limited to, SIMD8, SIMD16, and SIMD32. Each fused graphics execution unit 509A-509N includes at least two execution units. For example, fused execution unit 509A includes a first EU 508A, a second EU 508B, and thread control logic 507A common to the first EU 508A and the second EU 508B. Thread control logic 507A controls the threads executing in fused graphics execution unit 509A, allowing each EU in fused execution units 509A-509N to execute using a common instruction pointer register.

[0109] One or more built-in instruction caches (e.g., 506) are included in the thread execution logic 500 for caching thread instructions for the execution units. In some embodiments, one or more data caches (e.g., 512) are included for caching thread data during thread execution. Threads executing in the execution logic 500 may also store explicitly managed data in shared local memory 511. In some embodiments, a sampler 510 is included to provide texture sampling for 3D operations and media sampling for media operations. In some embodiments, the sampler 510 includes dedicated texture or media sampling functionality for processing texture or media data during the sampling process before providing the sampled data to the execution units.

[0110] During execution, the graphics pipeline and media pipeline send thread start requests to thread execution logic 500 via thread spawning dispatch logic. As groups of geometric objects are processed and rasterized into pixel data, pixel processor logic (e.g., pixel shader logic, fragment shader logic, etc.) within shader processor 502 is invoked to further compute output information and write the results to an output surface (e.g., a color buffer, a depth buffer, a stencil buffer, etc.). In some embodiments, a pixel shader or fragment shader computes values ​​for various vertex attributes that are interpolated across the rasterized objects. In some embodiments, the pixel processor logic within shader processor 502 then executes pixel shader or fragment shader programs supplied by an application programming interface (API). To execute shader programs, shader processor 502 dispatches threads to execution units (e.g., 508A) via thread dispatcher 504. In some embodiments, shader processor 502 uses texture sampling logic within sampler 510 to access texture data within texture maps stored in memory. Arithmetic operations on the texture data and input geometry data compute pixel color data for each geometric fragment or discard one or more pixels from further processing.

[0111] In some embodiments, data port 514 provides a memory access mechanism to thread execution logic 500 and outputs processed data to memory for further processing in the graphics processor output pipeline. In some embodiments, data port 514 includes or couples to one or more cache memories (e.g., data cache 512) for caching data for memory access via the data port.

[0112] In one embodiment, the execution logic 500 may also include a ray tracer 505 that can provide ray tracing acceleration functionality. The ray tracer 505 may support a ray tracing instruction set that includes instructions / functions for ray generation. The ray tracing instruction set may be similar to or different from the ray tracing instruction set supported by the ray tracing core 245 of FIG. 2C.

[0113] 5B illustrates exemplary internal details of an execution unit 508, according to multiple embodiments. The graphics execution unit 508 may include an instruction fetch unit 537, a general-purpose register file array (GRF) 524, an architectural register file array (ARF) 526, a thread arbiter 522, a send unit 530, a branch unit 532, a set of SIMD floating-point units (FPUs) 534, and, in one embodiment, a set of dedicated integer SIMD ALUs 535. The GRF 524 and ARF 526 include a general-purpose register file and a set of architectural register files associated with each concurrent hardware thread that may be active in the graphics execution unit 508. In one embodiment, per-thread architectural state is maintained in the ARF 526, and data used during thread execution is stored in the GRF 524. The execution state of each thread, including each thread's instruction pointer, may be kept in thread-specific registers in the ARF 526.

[0114] In one embodiment, the graphics execution unit 508 has an architecture that combines simultaneous multithreading (SMT) and fine-grained interleaved multithreading (IMT). This architecture has a modular structure that can be tuned at design time based on the target number of simultaneous threads and the number of registers per execution unit, and execution unit resources are divided across the logic used to execute multiple simultaneous threads. The number of logical threads that can be executed by the graphics execution unit 508 is not limited to the number of hardware threads, and multiple logical threads can be assigned to each hardware thread.

[0115] In one embodiment, the graphics execution unit 508 can simultaneously issue multiple instructions, each of which may be a different instruction. The thread arbiter 522 of a graphics execution unit thread 508 can dispatch instructions for execution to one of the send unit 530, the branch unit 532, or the SIMD FPU 534. Each execution thread can access 128 general-purpose registers in the GRF 524, each capable of storing 32 bytes accessible as a SIMD 8-element vector of 32-bit data elements. In one embodiment, each execution unit thread can access 4K bytes in the GRF 524, although embodiments are not so limited and other embodiments may provide more or less register resources. In one embodiment, the graphics execution unit 508 is partitioned into seven hardware threads, which can independently perform computational operations, although the number of threads per execution unit may vary depending on the embodiment. For example, in one embodiment, up to 16 hardware threads are supported. In an embodiment where seven threads can access 4K bytes, GRF 524 can store a total of 28K bytes. If 16 threads can access 4K bytes, GRF 524 can store a total of 64K bytes. Flexible addressing modes may allow multiple registers to be addressed together, effectively building wide registers or representing sequential rectangular block data structures.

[0116] In one embodiment, memory operations, sampler operations, and other longer latency system communications are dispatched via "send" instructions executed by message passing send unit 530. In one embodiment, branch instructions are dispatched to dedicated branch unit 532 to facilitate SIMD divergence and eventual convergence.

[0117] In one embodiment, the graphics execution unit 508 includes one or more SIMD floating-point units (FPUs) 534 to perform floating-point operations. In one embodiment, the FPUs 534 also support integer calculations. In one embodiment, the FPUs 534 can SIMD execute up to M 32-bit floating-point (or integer) operations, or SIMD execute up to 2M 16-bit integer or 16-bit floating-point operations. In one embodiment, at least one of the FPUs provides extended math functionality to support high-throughput transcendental math functions and double-precision 54-bit floating-point. In some embodiments, a set of 8-bit integer SIMD ALUs 535 is also present and may be specifically optimized to perform operations related to machine learning calculations.

[0118] In one embodiment, an array of multiple instances of graphics execution unit 508 may be instantiated into a graphics sub-core grouping (e.g., a sub-slice). For scalability, product designers can select the exact number of execution units per sub-core grouping. In one embodiment, execution unit 508 can execute instructions across multiple execution channels. In a further embodiment, each thread executing on a graphics execution unit 508 executes on a different channel.

[0119] FIG. 6 illustrates an additional execution unit 600 according to one embodiment. Execution unit 600 may be, for example, a compute-optimized execution unit for use in compute engine tiles 340A-340D as shown in FIG. 3C , but is not limited to such. Variations of execution unit 600 may also be used in graphics engine tiles 310A-310D as shown in FIG. 3B . In one embodiment, execution unit 600 includes a thread control unit 601, a thread state unit 602, an instruction fetch / prefetch unit 603, and an instruction decode unit 604. Execution unit 600 further includes a register file 606 that stores registers that may be allocated to hardware threads within the execution unit. Execution unit 600 also includes a send unit 607 and a branch unit 608. In one embodiment, send unit 607 and branch unit 608 can operate similarly to send unit 530 and branch unit 532 of graphics execution unit 508 of FIG. 5B .

[0120] The execution unit 600 also includes a compute unit 610 that includes multiple different types of functional units. In one embodiment, the compute unit 610 includes an ALU unit 611 that includes an array of arithmetic logic units. The ALU unit 611 can be configured to perform 64-bit, 32-bit, and 16-bit integer and floating-point operations. The integer and floating-point operations may be performed simultaneously. The compute unit 610 can also include a systolic array 612 and a math unit 613. The systolic array 612 includes a wide (W) deep (D) network of data processing units that can be used to perform vector operations or other data-parallel operations in a systolic fashion. In one embodiment, the systolic array 612 can be configured to perform matrix operations, such as matrix dot-product operations. In one embodiment, the systolic array 612 supports 16-bit floating-point operations as well as 8-bit and 4-bit integer operations. In one embodiment, the systolic array 612 can be configured to accelerate machine learning operations. In such an embodiment, systolic array 612 may be configured to support the bfloat 16-bit floating-point format. In one embodiment, math unit 613 may be included to perform a specific subset of mathematical operations efficiently and in a lower power manner than ALU unit 611. Math unit 613 may include variations of math logic that may be found in shared function logic of graphics processing engines provided by other embodiments (e.g., math logic 422 of shared function logic 420 of FIG. 4). In one embodiment, math unit 613 may be configured to perform 32-bit and 64-bit floating-point operations.

[0121] The thread control unit 601 includes logic for controlling the execution of threads within the execution units. The thread control unit 601 may include thread coordination logic for starting, stopping, and preempting the execution of threads within the execution units 600. The thread state unit 602 may be used to store the thread state of threads assigned to execute in the execution units 600. Storing the thread state in the execution units 600 allows for rapid preemption of threads when they become blocked or idle. The instruction fetch / prefetch unit 603 may fetch instructions from an instruction cache of higher-level execution logic (e.g., instruction cache 506 as seen in FIG. 5A). The instruction fetch / prefetch unit 603 may also issue prefetch requests for instructions to be loaded into the instruction cache based on an analysis of the currently executing thread. The instruction decode unit 604 may be used to decode instructions to be executed by the compute units. In one embodiment, the instruction decode unit 604 may be used as a secondary decoder that decodes complex instructions into constituent micro-operations.

[0122] Execution unit 600 further includes a register file 606 that can be used by hardware threads executing in execution unit 600. The registers of register file 606 can be divided throughout the logic used to execute multiple simultaneous threads within the compute units 610 of execution unit 600. The number of logical threads that can be executed by graphics execution unit 600 is not limited to the number of hardware threads, and multiple logical threads can be assigned to each hardware thread. The size of register file 606 can vary between embodiments based on the number of hardware threads supported. In one embodiment, register renaming can be used to dynamically allocate registers to hardware threads.

[0123] Figure 7 is a block diagram illustrating an instruction format 700 of a graphics processor according to some embodiments. In one or more embodiments, a graphics processor execution unit supports an instruction set having instructions in multiple formats. Solid boxes indicate components that are typically included in execution unit instructions, while dashed lines include optional components or components that are included in only a subset of the instructions. In some embodiments, the instruction format 700 described and shown are macro-instructions. These macro-instructions are instructions that are supplied to the execution unit, as opposed to micro-operations that result from decoding an instruction each time an instruction is processed.

[0124] In some embodiments, the graphics processor execution units natively support instructions in 128-bit instruction format 710. A 64-bit compressed instruction format 730 is available for some instructions based on the selected instruction, instruction options, and number of operands. The native 128-bit instruction format 710 provides access to all instruction options, with some options and operations restricted to the 64-bit format 730. The native instructions available in the 64-bit format 730 vary by embodiment. In some embodiments, instructions are partially compressed using a set of index values ​​in index field 713. The execution unit hardware looks up a set of compression tables based on the index values ​​and uses the output of the compression tables to reconstruct the native instruction in the 128-bit instruction format 710. Other sizes and formats of instructions may be used.

[0125] For each format, an instruction opcode 712 defines the operation the execution unit performs. The execution unit executes each instruction in parallel across multiple data elements of each operand. For example, in response to an add instruction, the execution unit performs a simultaneous add operation on each color channel representing a texture or picture element. By default, the execution unit executes each instruction on all data channels of the operand. In some embodiments, an instruction control field 714 allows control over certain execution options, such as channel selection (e.g., predication), data channel order (e.g., swizzle), etc. For instructions in the 128-bit instruction format 710, an execution size field 716 limits the number of data channels that will be executed in parallel. In some embodiments, the execution size field 716 is not available for use with the 64-bit compressed instruction format 730.

[0126] Some execution unit instructions have up to three operands, including two source operands, src0 720 and src1 722, and one destination 718. In some embodiments, the execution unit supports dual-destination instructions, where one of these destinations is implied. Data manipulation instructions can have a third source operand (e.g., SRC2 724), and the instruction opcode 712 determines the number of source operands. The last source operand of an instruction may be a direct (e.g., hard-coded) value passed with the instruction.

[0127] In some embodiments, the 128-bit instruction format 710 includes an access / address mode field 726 that specifies, for example, whether direct or indirect register addressing mode is used. When direct register addressing mode is used, the register addresses of one or more operands are provided directly by bits of the instruction.

[0128] In some embodiments, the 128-bit instruction format 710 includes an access / address mode field 726 that specifies the address mode and / or access mode of the instruction. In one embodiment, the access mode is used to define the data access alignment of the instruction. Some embodiments support access modes including a 16-byte-aligned access mode and a 1-byte-aligned access mode, where the byte alignment of the access mode determines the access alignment of the instruction operands. For example, in a first mode, the instruction may use byte-aligned addressing for source and destination operands, and in a second mode, the instruction may use 16-byte-aligned addressing for all source and destination operands.

[0129] In one embodiment, the address mode portion of the access / address mode field 726 determines whether the instruction uses direct or indirect addressing. When direct register addressing mode is used, bits of the instruction directly provide the register addresses of one or more operands. When indirect register addressing mode is used, the register addresses of one or more operands may be calculated based on the address register value and the address immediate field in the instruction.

[0130] In some embodiments, to simplify opcode decode 740, instructions are grouped based on a bit field in the opcode 712. For 8-bit opcodes, bits 4, 5, and 6 allow the execution unit to determine the type of opcode. The exact opcode grouping shown is merely an example. In some embodiments, the move and logical opcode group 742 includes data movement instructions and logical instructions (e.g., move (mov), compare (cmp)). In some embodiments, the move and logical group 742 share five most significant bits (MSBs), with move (mov) instructions being of the form 0000xxxxb and logical instructions being of the form 0001xxxxb. The flow control instruction group 744 (e.g., call, jump (jmp)) includes instructions in the form 0010xxxxb (e.g., 0x20). The miscellaneous instruction group 746 contains a mix of instructions and includes synchronization instructions (e.g., wait, send) in the form of 0011xxxxb (e.g., 0x30). The parallel math instruction group 748 contains component-wise arithmetic instructions (e.g., add, multiply (mul)) in the form of 0100xxxxb (e.g., 0x40). The parallel math group 748 performs arithmetic operations on data channels in parallel. The vector math group 750 contains arithmetic instructions (e.g., dp4) in the form of 0101xxxxb (e.g., 0x50). The vector math group performs arithmetic, such as dot product calculations, on vector operands. The illustrated opcode decode 740, in one embodiment, may be used to determine which portion of the execution unit will be used to execute the decoded instruction. For example, some instructions may be designated systolic instructions that will be executed by a systolic array. Other instructions, such as ray tracing instructions (not shown), may be routed to ray tracing cores or ray tracing logic within a slice or partition of the execution logic. [Graphics Pipeline]

[0131] Figure 8 is a block diagram of another embodiment of a graphics processor 800. Elements of Figure 8 having the same reference number (or name) as elements of any other figure herein may operate or function in any manner similar to that described elsewhere herein, but are not limited to such.

[0132] In some embodiments, graphics processor 800 includes a geometry pipeline 820, a media pipeline 830, a display engine 840, thread execution logic 850, and a rendering output pipeline 870. In some embodiments, graphics processor 800 is a graphics processor within a multi-core processing system that includes one or more general-purpose processing cores. The graphics processor is controlled by register writes to one or more control registers (not shown) or by commands issued to graphics processor 800 via ring interconnect 802. In some embodiments, ring interconnect 802 couples graphics processor 800 to other processing components, such as other graphics processors or general-purpose processors. Commands from ring interconnect 802 are interpreted by command streamer 803, which provides instructions to individual components of geometry pipeline 820 or media pipeline 830.

[0133] In some embodiments, command streamer 803 directs the operation of vertex fetcher 805, which reads vertex data from memory and executes vertex processing commands provided by command streamer 803. In some embodiments, vertex fetcher 805 provides the vertex data to vertex shader 807, which performs coordinate space transformations and lighting operations for each vertex. In some embodiments, vertex fetcher 805 and vertex shader 807 execute vertex processing instructions by dispatching execution threads to execution units 852A-852B via thread dispatcher 831.

[0134] In some embodiments, the execution units 852A-852B are arrays of vector processors with instruction sets for performing graphics and media operations. In some embodiments, the execution units 852A-852B have an associated L1 cache 851 that is specific to each array or shared between arrays. This cache can be configured as a data cache, an instruction cache, or a single cache that is partitioned to contain data and instructions in different partitions.

[0135] In some embodiments, geometry pipeline 820 includes a tessellation component for performing hardware-accelerated tessellation of 3D objects. In some embodiments, programmable hull shader 811 configures the tessellation operations. Programmable domain shader 817 provides back-end evaluation of the tessellation output. Tessellator 813 operates under the direction of hull shader 811 and includes special-purpose logic for generating a set of detailed geometric objects based on a coarse geometric model provided as input to geometry pipeline 820. In some embodiments, if tessellation is not used, the tessellation components (e.g., hull shader 811, tessellator 813, and domain shader 817) can be bypassed.

[0136] In some embodiments, a complete geometric object may be processed by the geometry shader 819 via one or more threads dispatched to execution units 852A-852B, or may proceed directly to the clipper 829. In some embodiments, the geometry shader processes entire geometric objects, rather than vertices or patches of vertices as found in earlier stages of the graphics pipeline. If tessellation is disabled, the geometry shader 819 receives input from the vertex shader 807. In some embodiments, the geometry shader 819 is programmable in the geometry shader program to perform geometry tessellation if the tessellation units are disabled.

[0137] Before rasterization, a clipper 829 processes the vertex data. The clipper 829 may be a fixed-function clipper or a programmable clipper that has clipping and geometry shader functions. In some embodiments, a rasterizer and depth test component 873 in the rendering output pipeline 870 dispatches pixel shaders to convert geometric objects into per-pixel representations. In some embodiments, pixel shader logic is included in the thread execution logic 850. In some embodiments, an application can bypass the rasterizer and depth test component 873 and access unrasterized vertex data through the stream-out unit 823.

[0138] Graphics processor 800 includes an interconnect bus, interconnect fabric, or some other interconnect mechanism that allows data and message passing between the major components of the processor. In some embodiments, execution units 852A-852B and associated logic units (e.g., L1 cache 851, sampler 854, texture cache 858, etc.) interconnect via data port 856 to perform memory accesses and communicate with the processor's rendering output pipeline components. In some embodiments, sampler 854, caches 851, 858, and execution units 852A-852B each have a separate memory access path. In one embodiment, texture cache 858 may be configured as a sampler cache.

[0139] In some embodiments, the rendering output pipeline 870 includes a rasterizer and depth test component 873 that converts vertex-based objects into associated pixel-based representations. In some embodiments, the rasterizer logic includes a windowing / masking unit that performs fixed-function triangle and line rasterization. An associated rendering cache 878 and depth cache 879 are also available in some embodiments. A pixel operations component 877 performs pixel-based operations on the data, although in some instances pixel calculations related to 2D operations (e.g., bit-block image transfer with blending) are performed by the 2D engine 841 or are replaced at display time by the display controller 843 using overlay display planes. In some embodiments, a shared L3 cache 875 is available to all graphics components, allowing data sharing without using main system memory.

[0140] In some embodiments, graphics processor media pipeline 830 includes a media engine 837 and a video front end 834. In some embodiments, video front end 834 receives pipeline commands from command streamer 803. In some embodiments, media pipeline 830 includes a separate command streamer. In some embodiments, video front end 834 processes media commands before sending the commands to media engine 837. In some embodiments, media engine 837 includes a thread spawning function that spawns threads for dispatch to thread execution logic 850 via thread dispatcher 831.

[0141] In some embodiments, graphics processor 800 includes a display engine 840. In some embodiments, display engine 840 is external to processor 800 and couples to the graphics processor via ring interconnect 802 or some other interconnect bus or fabric. In some embodiments, display engine 840 includes a 2D engine 841 and a display controller 843. In some embodiments, display engine 840 includes special purpose logic that can operate independently of the 3D pipeline. In some embodiments, display controller 843 couples to a display device (not shown). The display device may be a system-integrated display device, such as found in a laptop computer, or an external display device attached via a display device connector.

[0142] In some embodiments, the geometry pipeline 820 and the media pipeline 830 are configurable to perform operations based on multiple graphics and media programming interfaces and are not specific to any one application programming interface (API). In some embodiments, the graphics processor's driver software translates API calls specific to a particular graphics or media library into commands that can be processed by the graphics processor. In some embodiments, support is provided for the Open Graphics Library (OpenGL), Open Computing Language (OpenCL), and / or Vulkan graphics and compute APIs, all from the Khronos Group. In some embodiments, support may also be provided for Microsoft Corporation's Direct3D library. In some embodiments, a combination of these libraries may be supported. Support may also be provided for the open source computer vision library (OpenCV). Future APIs with compatible 3D pipelines may also be supported, provided that a mapping can be made from the future API's pipeline to the graphics processor's pipeline. [Graphics Pipeline Programming]

[0143] Figure 9A is a block diagram illustrating a graphics processor command format 900 according to some embodiments. Figure 9B is a block diagram illustrating a graphics processor command sequence 910 according to one embodiment. The solid boxes in Figure 9A indicate components typically included in a graphics command, while the dashed boxes include optional components or components that are included in only a subset of the graphics commands. The example graphics processor command format 900 of Figure 9A includes data fields identifying a client 902, a command operation code (opcode) 904, and data for the command 906. A sub-opcode 905 and a command size 908 are also included in some commands.

[0144] In some embodiments, client 902 specifies a client unit of the graphics device that will process the command data. In some embodiments, a graphics processor command parser examines the client field of each command to determine further processing of the command and route the command data to the appropriate client unit. In some embodiments, the client units of the graphics processor include a memory interface unit, a rendering unit, a 2D unit, a 3D unit, and a media unit. Each client unit has a corresponding processing pipeline that processes the command. When a command is received by a client unit, the client unit reads the opcode 904 and, if present, the sub-opcode 905 to determine the operation to perform. The client unit executes the command using the information in the data field 906. For some commands, an explicit command size 908 is expected to specify the size of the command. In some embodiments, the command parser automatically determines the size of at least some of the commands based on the command opcode. In some embodiments, commands are aligned by multiples of doublewords. Other command formats may be used.

[0145] The flow diagram of Figure 9B shows an exemplary graphics processor command sequence 910. In some embodiments, software or firmware in a data processing system featuring an embodiment of a graphics processor uses versions of the command sequence shown to set up, execute, and terminate a set of graphics operations. The sample command sequence is shown and described for illustrative purposes only, and embodiments are not limited to these particular commands or this command sequence. Furthermore, these commands may be issued as a batch of commands in a command sequence, resulting in the graphics processor processing a series of commands at least partially concurrently.

[0146] In some embodiments, the graphics processor command sequence 910 may begin with a pipeline flush command 912, causing any active graphics pipelines to complete currently pending pipeline commands. In some embodiments, the 3D pipeline 922 and the media pipeline 924 do not operate simultaneously. A pipeline flush is performed to cause the active graphics pipelines to complete any pending commands. In response to a pipeline flush, the graphics processor's command parser will pause command processing until the active drawing engines complete pending operations and invalidate their associated read caches. Optionally, any data in the rendering cache that is marked as dirty may be flushed to memory. In some embodiments, the pipeline flush command 912 may be used for pipeline synchronization or before placing the graphics processor into a low power state.

[0147] In some embodiments, when a command sequence requires the graphics processor to explicitly switch between pipelines, a pipeline select command 913 is used. In some embodiments, the pipeline select command 913 is requested only once in an execution context before issuing a pipeline command, unless the context issues commands to both pipelines. In some embodiments, a pipeline flush command 912 is requested immediately before a pipeline switch via the pipeline select command 913.

[0148] In some embodiments, pipeline control commands 914 are used to configure the graphics pipeline for operation and to program the 3D pipeline 922 and the media pipeline 924. In some embodiments, pipeline control commands 914 configure the pipeline state for the active pipeline. In one embodiment, pipeline control commands 914 are used for pipeline synchronization and to flush data from one or more cache memories in the active pipeline before processing a batch of commands.

[0149] In some embodiments, the return buffer state command 916 is used to configure a set of return buffers for each pipeline to write data to. Some pipeline operations require the allocation, selection, or configuration of one or more return buffers to which the operation writes intermediate data during processing. In some embodiments, the graphics processor also uses one or more return buffers to store output data and perform cross-thread communication. In some embodiments, the return buffer state 916 includes selecting the size and number of return buffers to use for a set of pipeline operations.

[0150] The remaining commands in the command sequence differ based on the active pipeline for the operation. Based on a pipeline decision 920, the command sequence is either aligned with the 3D pipeline 922, starting at a 3D pipeline state 930, or aligned with the media pipeline 924, starting at a media pipeline state 940.

[0151] The commands that configure the 3D pipeline state 930 include 3D state setting commands for vertex buffer state, vertex element state, constant color state, depth buffer state, and other state variables that are configured before 3D primitive commands are processed. The values ​​of these commands are determined, at least in part, based on the particular 3D API in use. In some embodiments, the 3D pipeline state 930 commands can also selectively disable or bypass certain pipeline elements if those elements will not be used.

[0152] In some embodiments, 3D primitive 932 commands are used to submit 3D primitives to be processed by the 3D pipeline. Commands and associated parameters passed to the graphics processor via 3D primitive 932 commands are forwarded to the graphics pipeline's vertex fetch function. The vertex fetch function uses the 3D primitive 932 command data to generate vertex data structures. The vertex data structures are stored in one or more return buffers. In some embodiments, 3D primitive 932 commands are used to perform vertex operations on 3D primitives via vertex shaders. To process vertex shaders, 3D pipeline 922 dispatches shader execution threads to the graphics processor execution units.

[0153] In some embodiments, the 3D pipeline 922 is triggered via an execute command 934 or an event. In some embodiments, a register write triggers command execution. In some embodiments, execution is triggered via a "go" or "kick" command in a command sequence. In one embodiment, command execution is triggered using a pipeline synchronization command to flush the command sequence through the graphics pipeline. The 3D pipeline will perform geometry operations on 3D primitives. Once the operations are complete, the resulting geometric objects are rasterized and the pixel engine colors the resulting pixels. Additional commands to control pixel shading and pixel back-end operations may also be included for these operations.

[0154] In some embodiments, the graphics processor command sequence 910 follows the path of the media pipeline 924 when performing media operations. In general, the specific use and manner of programming for the media pipeline 924 depends on the media or compute operations being performed. Certain media decode operations may be offloaded to the media pipeline during media decoding. In some embodiments, the media pipeline may also be bypassed, and media decoding may be performed in whole or in part using resources provided by one or more general-purpose processing cores. In one embodiment, the media pipeline also includes elements for general-purpose graphics processor unit (GPGPU) operation, where the graphics processor is used to perform SIMD vector operations using compute shader programs not explicitly related to rendering graphics primitives.

[0155] In some embodiments, the media pipeline 924 is configured in a similar manner to the 3D pipeline 922. A set of commands that configure the media pipeline state 940 is dispatched or placed in a command queue before the media object commands 942. In some embodiments, the commands for the media pipeline state 940 contain data that configures the media pipeline elements that will be used to process the media object. This includes data that configures the video decoding logic and video encoding logic in the media pipeline, such as the encoding format or decoding format. In some embodiments, the commands for the media pipeline state 940 also support the use of one or more pointers to "indirect" state elements that contain a batch of state settings.

[0156] In some embodiments, the media object command 942 provides a pointer to a media object for processing by the media pipeline. The media object includes a memory buffer containing the video data to be processed. In some embodiments, all media pipeline state must be valid before issuing the media object command 942. Once the pipeline state is configured and the media object command 942 is queued, the media pipeline 924 is triggered by an execute command 944 or equivalent execution event (e.g., a register write). The output from the media pipeline 924 may then be post-processed by operations provided by the 3D pipeline 922 or the media pipeline 924. In some embodiments, GPGPU operations are configured and executed in a similar manner to media operations. [Graphics Software Architecture]

[0157] 10 illustrates an exemplary graphics software architecture for a data processing system 1000 according to some embodiments. In some embodiments, the software architecture includes a 3D graphics application 1010, an operating system 1020, and at least one processor 1030. In some embodiments, the processor 1030 includes a graphics processor 1032 and one or more general-purpose processor cores 1034. The graphics application 1010 and the operating system 1020 each execute in system memory 1050 of the data processing system.

[0158] In some embodiments, the 3D graphics application 1010 includes one or more shader programs that include shader instructions 1012. The shader language instructions may be instructions in a high-level shader language, such as Direct3D High-Level Shader Language (HLSL) and OpenGL Shader Language (GLSL). The application also includes executable instructions 1014 in a machine language suitable for execution by the general-purpose processor core 1034. The application also includes graphics objects 1016 that are defined by vertex data.

[0159] In some embodiments, the operating system 1020 is a Microsoft Windows operating system from Microsoft Corporation, a proprietary UNIX-like operating system, or an open-source UNIX-like operating system that uses a variant of the Linux kernel. The operating system 1020 can support a graphics API 1022, such as the Direct3D API, the OpenGL API, or the Vulkan API. If the Direct3D API is used, the operating system 1020 uses a front-end shader compiler 1024 to compile any shader instructions 1012 in HLSL into a low-level shader language. The compilation can be a just-in-time (JIT) compilation, or the application can perform shader pre-compilation. In some embodiments, high-level shaders are compiled into low-level shaders during compilation of the 3D graphics application 1010. In some embodiments, the shader instructions 1012 are provided in an intermediate form, such as a version of the Standard Portable Intermediate Representation (SPIR) used by the Vulkan API.

[0160] In some embodiments, user mode graphics driver 1026 includes a back-end shader compiler 1027 for converting shader instructions 1012 into a hardware-specific representation. When the OpenGL API is used, shader instructions 1012 in the GLSL high-level language are passed to user mode graphics driver 1026 for compilation. In some embodiments, user mode graphics driver 1026 communicates with kernel mode graphics driver 1029 using operating system kernel mode functions 1028. In some embodiments, kernel mode graphics driver 1029 communicates with graphics processor 1032 to dispatch commands and instructions. [IP core implementation]

[0161] One or more aspects of at least one embodiment may be implemented by representative code stored on a machine-readable medium that represents and / or defines logic within an integrated circuit, such as a processor. For example, the machine-readable medium may include instructions that represent various logic within a processor. The instructions, when read by a machine, may cause the machine to manufacture logic that performs the techniques described herein. Such representations, known as "IP cores," are reusable units of logic for an integrated circuit that may be stored on a tangible, machine-readable medium as a hardware model that describes the structure of the integrated circuit. The hardware model may be provided to various customers or manufacturing facilities, which load the hardware model into manufacturing machines that produce the integrated circuit. The integrated circuit may be manufactured such that the circuit performs the operations described in connection with any of the embodiments described herein.

[0162] FIG. 11A is a block diagram illustrating an IP core development system 1100 that can be used to fabricate an integrated circuit that performs operations, according to one embodiment. The IP core development system 1100 can be used to generate modular, reusable designs that can be incorporated into larger designs or to build entire integrated circuits (e.g., SoC integrated circuits). A design facility 1130 can generate a software simulation 1110 of the IP core design in a high-level programming language (e.g., C / C++). The software simulation 1110 can be used to design, test, and verify the behavior of the IP core using a simulation model 1112. The simulation model 1112 may include functional, behavioral, and / or timing simulation. A register transfer level (RTL) design 1115 can then be created or synthesized from the simulation model 1112. The RTL design 1115 is an extraction of the behavior of the integrated circuit that models the flow of digital signals between hardware registers, including associated logic that is executed using the modeled digital signals. In addition to the RTL design 1115, lower level designs at the logic or transistor level may also be created, designed, or synthesized, and therefore the specific details of the initial design and simulation may vary.

[0163] The RTL design 1115 or equivalent may be further synthesized at the design facility into a hardware model 1120, which may be a hardware description language (HDL) or some other representation of the physical design data. The HDL may then be simulated and tested to verify the IP core design. The IP core design may be stored using non-volatile memory 1140 (e.g., a hard disk, flash memory, or any non-volatile storage medium) for delivery to a third-party manufacturing facility 1165. Alternatively, the IP core design may be transmitted over a wired connection 1150 or a wireless connection 1160 (e.g., via the Internet). The manufacturing facility 1165 may then manufacture an integrated circuit based at least in part on the IP core design. The manufactured integrated circuit may be configured to perform operations in accordance with at least one embodiment described herein.

[0164] 11B illustrates a cross-sectional view of an integrated circuit package assembly 1170 according to some embodiments described herein. The integrated circuit package assembly 1170 illustrates an implementation of one or more processor or accelerator devices described herein. The package assembly 1170 includes multiple units of hardware logic 1172, 1174 coupled to a substrate 1180. The logic 1172, 1174 may be at least partially implemented in configurable logic hardware or fixed-function logic hardware and may include one or more portions of a processor core, a graphics processor, or any of the other accelerator devices described herein. Each unit of logic 1172, 1174 may be implemented on a semiconductor die and coupled to the substrate 1180 via an interconnect structure 1173. The interconnect structure 1173 may be configured to route electrical signals between the logic 1172, 1174 and the substrate 1180 and may include interconnects such as, but not limited to, bumps or pillars. In some embodiments, the interconnect structure 1173 may be configured to route electrical signals, such as input / output (I / O) signals and / or power or ground signals, associated with the operation of the logic 1172, 1174. In some embodiments, the substrate 1180 is an epoxy-based laminate substrate. In other embodiments, the substrate 1180 may include other suitable types of substrates. The package assembly 1170 may be connected to other electrical devices via the package interconnect 1183. The package interconnect 1183 may be coupled to a surface of the substrate 1180 to route electrical signals to other electrical devices, such as a motherboard, another chipset, or a multi-chip module.

[0165] In some embodiments, the units of logic 1172, 1174 are electrically coupled with a bridge 1182 configured to route electrical signals between the logic 1172 and 1174. The bridge 1182 may be a high-density interconnect structure that provides a path for the electrical signals. The bridge 1182 may include a bridge substrate composed of glass or a suitable semiconductor material. Electrical routing mechanisms may be formed in the bridge substrate to provide chip-to-chip connections between the logic 1172 and 1174.

[0166] Although two units of logic 1172, 1174 and a bridge 1182 are shown, embodiments described herein may include more or fewer logic units on one or more dies. If logic is included on a single die, bridge 1182 may be omitted, and one or more dies may be connected with zero or more bridges. Alternatively, multiple dies or multiple logic units may be connected with one or more bridges. Furthermore, multiple logic units, dies, and bridges may be connected together in other possible configurations, including three-dimensional configurations.

[0167] FIG. 11C illustrates a package assembly 1190 including multiple units of hardware logic chiplets connected to a substrate 1180 (e.g., a base die). The graphics processing units, parallel processors, and / or compute accelerators described herein may be composed of various silicon chiplets fabricated separately. In this context, a chiplet is an at least partially packaged integrated circuit containing separate units of logic that may be assembled with other chiplets into a larger package. A set of various chiplets with different IP core logic may be assembled into a single device. Furthermore, these chiplets may be integrated into a base die or base chiplet using active interposer technology. The concepts described herein enable interconnection and communication between different forms of IP within a GPU. IP cores may be fabricated using different process technologies and configured during fabrication. This avoids the complexity of concentrating multiple IPs in the same fabrication process, especially for large SoCs with several types of IP. Enabling the use of multiple process technologies improves time to market and provides a cost-effective way to create multiple product SKUs. Additionally, separate IPs lend themselves to independent power gating, allowing components not used for a given workload to be powered off, reducing overall power consumption.

[0168] The hardware logic chiplets may include special purpose hardware logic chiplets 1172, logic chiplets or I / O chiplets 1174, and / or memory chiplets 1175. The hardware logic chiplets 1172, logic chiplets or I / O chiplets 1174 may be at least partially implemented in configurable logic hardware or fixed function logic hardware and may include one or more portions of any of the processor cores, graphics processors, parallel processors, or other accelerator devices described herein. The memory chiplets 1175 may be DRAM (e.g., GDDR, HBM) memory or cache (SRAM) memory.

[0169] Each chiplet may be fabricated as a separate semiconductor die and coupled to substrate 1180 via interconnect structures 1173. Interconnect structures 1173 may be configured to route electrical signals between the various chiplets and logic within substrate 1180. Interconnect structures 1173 may include interconnects such as, but not limited to, bumps or pillars. In some embodiments, interconnect structures 1173 may be configured to route electrical signals, such as, for example, input / output (I / O) signals and / or power or ground signals associated with the operation of logic chiplets, I / O chiplets, and memory chiplets.

[0170] In some embodiments, substrate 1180 is an epoxy-based laminate substrate. In other embodiments, substrate 1180 may comprise other suitable types of substrate. Package assembly 1190 may be connected to other electrical devices via package interconnect 1183. Package interconnect 1183 may be coupled to a surface of substrate 1180 to route electrical signals to other electrical devices, such as a motherboard, another chipset, or a multi-chip module.

[0171] In some embodiments, logic or I / O chiplets 1174 and memory chiplets 1175 may be electrically coupled via bridges 1187 configured to route electrical signals between the logic or I / O chiplets 1174 and the memory chiplets 1175. Bridges 1187 may be high-density interconnect structures that provide paths for electrical signals. Bridges 1187 may include a bridge substrate composed of glass or a suitable semiconductor material. Electrical routing mechanisms may be formed in the bridge substrate to provide inter-chip connections between the logic or I / O chiplets 1174 and the memory chiplets 1175. Bridges 1187 may also be referred to as silicon bridges or interconnect bridges. For example, bridges 1187 may be embedded multi-die interconnect bridges (EMIBs) in some embodiments. In some embodiments, bridges 1187 may simply be direct connections from one chiplet to another.

[0172] Substrate 1180 may include hardware components for I / O 1191, cache memory 1192, and other hardware logic 1193. Fabric 1185 may be embedded in substrate 1180 to enable communication between the various logic chiplets and the logic 1191, 1193 within substrate 1180. In one embodiment, I / O 1191, fabric 1185, cache, bridges, and other hardware logic 1193 may be integrated into a base die that is layered on top of substrate 1180.

[0173] In various embodiments, package assembly 1190 may include fewer or more components and chiplets interconnected by fabric 1185 or one or more bridges 1187. Chiplets within package assembly 1190 may be arranged in a 3D or 2.5D configuration. In general, bridge structure 1187 may be used to facilitate point-to-point interconnections between logic or I / O chiplets and memory chiplets, for example. Fabric 1185 may be used to interconnect various logic and / or I / O chiplets (e.g., chiplets 1172, 1174, 1191, 1193) with other logic and / or I / O chiplets. In one embodiment, cache memory 1192 within the substrate may serve as a global cache for package assembly 1190, i.e., part of a distributed global cache or a dedicated cache for fabric 1185.

[0174] FIG. 11D illustrates a package assembly 1194 including compatible chiplets 1195, according to one embodiment. The compatible chiplets 1195 may be assembled as standardized slots of one or more base chiplets 1196, 1198. The base chiplets 1196, 1198 may be coupled via a bridge interconnect 1197. The bridge interconnect 1197 may be similar to other bridge interconnects described herein, such as an EMIB. Memory chiplets may also be connected to logic chiplets or I / O chiplets via the bridge interconnect. The I / O chiplets and logic chiplets can communicate via the interconnect fabric. Each base chiplet can support one or more slots in a standardized format for one of logic, I / O, or memory cache.

[0175] In one embodiment, SRAM and power delivery circuits may be fabricated as one or more of the base chiplets 1196, 1198. These circuits may be fabricated using a different process technology than the compatible chiplets 1195 stacked on top of the base chiplets. For example, the base chiplets 1196, 1198 may be fabricated using a large-scale process technology, while the compatible chiplets may be fabricated using a small-scale process technology. One or more of the compatible chiplets 1195 may be memory (e.g., DRAM chiplets). Different memory densities may be selected for the package assembly 1194 based on power and / or performance targeted for the product using the package assembly 1194. Furthermore, logic chiplets having different numbers and types of functional units may be selected during assembly based on power and / or performance targeted for the product. Furthermore, chiplets containing different types of IP logic cores may be inserted into compatible chiplet slots, enabling hybrid processor designs that can mix and match different technology IP blocks. [Exemplary System-on-Chip Integrated Circuit]

[0176] 12 through 13B illustrate an exemplary integrated circuit and associated graphics processor that may be fabricated using one or more IP cores according to various embodiments described herein. In addition to what is shown, other logic and circuitry may be included, including, for example, additional graphics processors / cores, peripheral interface controllers, or general-purpose processor cores.

[0177] 12 is a block diagram illustrating an exemplary system-on-chip integrated circuit 1200 that may be fabricated using one or more IP cores, according to one embodiment. The exemplary integrated circuit 1200 includes one or more application processors 1205 (e.g., CPUs), at least one graphics processor 1210, and may further include an image processor 1215 and / or a video processor 1220, any of which may be modular IP cores from the same design facility or multiple different design facilities. The integrated circuit 1200 also includes a USB controller 1225, a UART controller 1230, an SPI / SDIO controller 1235, and an I / O controller 1240. 2 S / I 2 The integrated circuit may further include peripheral or bus logic including a C controller 1240. Additionally, the integrated circuit may include a display device 1245 coupled to one or more of a High-Definition Multimedia Interface (HDMI®) controller 1250 and a Mobile Industry Processor Interface (MIPI) display interface 1255. Storage may be provided by a flash memory subsystem 1260 including a flash memory and a flash memory controller. A memory interface may be provided via a memory controller 1265 for accessing an SDRAM memory device or an SRAM memory device. Some integrated circuits further include an embedded security engine 1270.

[0178] 13A-13B are block diagrams illustrating exemplary graphics processors for use in an SoC according to embodiments described herein. FIG. 13A illustrates an exemplary graphics processor 1310 of a system-on-chip integrated circuit that may be fabricated using one or more IP cores according to one embodiment. FIG. 13B illustrates a further exemplary graphics processor 1340 of a system-on-chip integrated circuit that may be fabricated using one or more IP cores according to one embodiment. The graphics processor 1310 of FIG. 13A is an example of a low-power graphics processor core. The graphics processor 1340 of FIG. 13B is an example of a high-performance graphics processor core. Each of the graphics processors 1310, 1340 may be a variation of the graphics processor 1210 of FIG. 12.

[0179] As shown in FIG. 13A , graphics processor 1310 includes vertex processor 1305 and one or more fragment processors 1315A-1315N (e.g., 1315A, 1315B, 1315C, 1315D, ..., 1315N-1, and 1315N). Because graphics processor 1310 can execute different shader programs through separate logic, vertex processor 1305 is optimized to perform operations for vertex shader programs, and one or more fragment processors 1315A-1315N perform fragment (e.g., pixel) shading operations for fragment shader programs or pixel shader programs. Vertex processor 1305 executes the vertex processing stage of the 3D graphics pipeline, generating primitive and vertex data. Fragment processors 1315A-1315N use the primitive and vertex data generated by vertex processor 1305 to generate a frame buffer that is displayed on a display device. In one embodiment, fragment processors 1315A-1315N are optimized to execute fragment shader programs provided in an OpenGL API, which may be used to perform operations similar to pixel shader programs provided in a Direct3D API.

[0180] Graphics processor 1310 further includes one or more memory management units (MMUs) 1320A-1320B, caches 1325A-1325B, and circuit interconnects 1330A-1330B. The one or more MMUs 1320A-1320B provide virtual-to-physical address mapping for graphics processor 1310, including vertex processor 1305 and / or fragment processors 1315A-1315N, and graphics processor 1310 may reference vertex data or image / texture data stored in memory in addition to vertex data or image / texture data stored in one or more caches 1325A-1325B. In one embodiment, one or more MMUs 1320A-1320B may synchronize with other MMUs in the system, including one or more MMUs associated with one or more application processors 1205, image processor 1215, and / or video processor 1220 of Figure 12, so that each processor 1205-1220 can participate in a shared or unified virtual memory system. According to an embodiment, one or more circuit interconnects 1330A-1330B enable graphics processor 1310 to interface with other IP cores in the SoC, either via the SoC's internal bus or via a direct connection.

[0181] As shown in Figure 13B, graphics processor 1340 includes one or more MMUs 1320A-1320B, caches 1325A-1325B, and circuit interconnects 1330A-1330B of graphics processor 1310 of Figure 13A. Graphics processor 1340 includes one or more shader cores 1355A-1355N (e.g., 1355A, 1355B, 1355C, 1355D, 1355E, 1355F, ..., 1355N-1, and 1355N), which provide a unified shader core architecture in which a single core or type or cores can execute all types of programmable shader code, including shader program code implementing vertex shaders, fragment shaders, and / or compute shaders. The exact number of shader cores present may vary between embodiments and implementations. Additionally, graphics processor 1340 includes an inter-core task manager 1345, which acts as a thread dispatcher that dispatches execution threads to one or more shader cores 1355A-1355N and to a tiling unit 1358 that accelerates tiling operations for tile-based rendering, where rendering operations for a scene are subdivided in image space, for example, to exploit local spatial coherence within the scene or to optimize the use of built-in caches.

[0182] Figure 14 illustrates one embodiment of a computing device employing a page table mapping mechanism ("mapping mechanism") 1410. For example, in one embodiment, the mapping mechanism of Figure 14 may be employed or hosted by computing device 1400. As shown, mapping mechanism 1410 may be hosted by, or a portion of, a graphics processing unit ("GPU" or "graphics processor") 1414, according to one embodiment.

[0183] In other embodiments, the mapping mechanism 1410 may be hosted by, or part of, firmware of the central processing unit ("CPU" or "application processor") 1412. In yet other embodiments, the mapping mechanism 1410 may be hosted by, or part of, the graphics driver 1416. For purposes of brevity, clarity, and ease of understanding, throughout the remainder of this specification, the mapping mechanism 1410 may be discussed as part of the GPU 1414. However, embodiments are not so limited.

[0184] In yet another embodiment, mapping mechanism 1410 may be hosted as software or firmware logic by operating system 1406. Furthermore, in a further embodiment, mapping mechanism 1410 may be partially and simultaneously hosted by multiple components of computing device 1400, such as one or more of graphics driver 1416, GPU 1414, GPU firmware, CPU 1412, CPU firmware, operating system 1406, and / or the like. It is contemplated that mapping mechanism 1410 or one or more components thereof may be implemented as hardware, software, and / or firmware.

[0185] Computing device 1400 represents communications and data processing devices including or representing any number and type of smart devices such as (but not limited to) smart command devices or intelligent personal assistants, home / office automation systems, home appliances (e.g., washing machines, television sets, etc.), mobile devices (e.g., smartphones, tablet computers, etc.), gaming devices, handheld devices, wearable devices (e.g., smart watches, smart bracelets, etc.), virtual reality (VR) devices, head-mounted displays (HMDs), Internet of Things (IoT) devices, laptop computers, desktop computers, server computers, set-top boxes (e.g., Internet-based cable television set-top boxes, etc.), global positioning system (GPS)-based devices, etc.

[0186] In some embodiments, computing device 1400 may include (without limitation) an autonomous machine or artificially intelligent agent, such as a mechanical agent or machine, an electronic agent or machine, a virtual agent or machine, an electromechanical agent or machine, etc. Examples of autonomous machines or artificially intelligent agents may include (without limitation) a robot, an autonomous vehicle (e.g., a self-driving car, a self-flying airplane, a self-sailing boat, etc.), an autonomous device (a self-operated construction vehicle, a self-operated medical device, etc.), and / or the like. Furthermore, "autonomous vehicle" is not limited to automobiles but can include any number and type of autonomous machine, such as a robot, an autonomous device, a domestic autonomous device, and / or the like, and any one or more tasks or operations associated with such an autonomous machine may be interchangeably referred to as autonomous driving.

[0187] Further, for example, the computing device 1400 may include a cloud computing platform consisting of multiple server computers, where each server computer employs or hosts a multifunction perceptron mechanism. For example, automatic ISP tuning may be performed using the components, systems, and architectural setups previously described herein. For example, some of the types of devices described above may be used to implement custom training procedures, such as using field programmable gate arrays (FPGAs).

[0188] Additionally, for example, computing device 1400 may include a computer platform that hosts an integrated circuit ("IC"), such as a system-on-chip ("SoC" or "SOC") that integrates various hardware and / or software components of computing device 1400 onto a single chip.

[0189] As shown, in one embodiment, computing device 1400 may include any number and types of hardware and / or software components, such as (but not limited to) a graphics processing unit 1414 (“GPU” or simply “graphics processor”), a graphics driver 1416 (also referred to as a “GPU driver,” “graphics driver logic,” “driver logic,” user-mode driver (UMD), UMD, user-mode driver framework (UMDF), UMDF, or simply “driver”), a central processing unit 1412 (“CPU” or simply “application processor”), memory 1404, network devices, drivers, or the like, or other input / output (I / O) sources 1408, such as a touchscreen, touch panel, touchpad, virtual or conventional keyboard, and virtual or conventional mouse, ports, connectors, etc. Computing device 1400 may include an operating system (OS) that serves as an interface between a user and the hardware and / or physical resources of computing device 1400.

[0190] It should be understood that in particular implementations, a lesser or more equipped system than the above examples may be preferred. Thus, the configuration of computing device 1400 may vary from implementation to implementation depending on numerous factors, such as price constraints, performance requirements, technological improvements, or other circumstances.

[0191] Embodiments may be implemented as any or combination of one or more microchips or integrated circuits interconnected using a motherboard, hardwired logic, software stored by a memory device and executed by a microprocessor, firmware, application-specific integrated circuits (ASICs), and / or field-programmable gate arrays (FPGAs). The terms "logic," "module," "component," "engine," and "mechanism" may include, by way of example, software or hardware, such as firmware, and / or combinations thereof.

[0192] According to one embodiment, computing device 1400 is coupled to one or more client computing devices (or clients) 1440 via one or more networks 1445. Accordingly, server 1400 and client 1440 may further include network interfaces for providing access to networks such as a LAN, a wide area network (WAN), a metropolitan area network (MAN), a personal area network (PAN), Bluetooth, a cloud network, a mobile network (e.g., third generation (3G), fourth generation (4G), etc.), an intranet, the Internet, etc. The network interfaces may include, for example, a wireless network interface having an antenna, which may represent one or more antennas. The network interfaces may also include, for example, a wired network interface for communicating with remote devices via a network cable, which may be, for example, an Ethernet cable, a coaxial cable, an optical fiber cable, a serial cable, or a parallel cable.

[0193] Embodiments may be provided, for example, as a computer program product, which may include one or more machine-readable media having stored thereon machine-executable instructions that, when executed by one or more machines, such as a computer, a network of computers, or other electronic devices, cause the one or more machines to perform operations in accordance with the embodiments described herein. Machine-readable media may include, but are not limited to, floppy diskettes, optical disks, CD-ROMs (compact disk read-only memories) and magneto-optical disks, ROMs, RAMs, EPROMs (erasable programmable read-only memories), EEPROMs (electrically erasable programmable read-only memories), magnetic or optical cards, flash memories, or other types of media / machine-readable media suitable for storing a plurality of machine-executable instructions.

[0194] Furthermore, embodiments may be downloaded as a computer program product, and the program may be transferred from a remote computer (e.g., a server) to a requesting computer (e.g., a client) by one or more data signals embodied in and / or modulated by a carrier wave or other propagation medium via a communications link (e.g., a modem and / or network connection).

[0195] Throughout this specification, the term "user" may be referred to interchangeably as "viewer," "observer," "speaker," "person," "individual," "end user," and / or the like. It should be noted that throughout this specification, terms such as "graphics domain" may be referred to interchangeably as "graphics processing unit," "graphics processor," or simply "GPU," and similarly, "CPU domain" or "host domain" may be referred to interchangeably as "computer processing unit," "application processor," or simply "CPU."

[0196] It should be noted that terms such as "node," "computing node," "server," "server device," "cloud computer," "cloud server," "cloud server computer," "machine," "host machine," "device," "computing device," "computer," "computing system," and the like may be used interchangeably throughout this specification. Furthermore, it should be noted that terms such as "application," "software application," "program," "software program," "package," "software package," and the like may be used interchangeably throughout this specification. Also, terms such as "job," "input," "request," "message," and the like may be used interchangeably throughout this specification.

[0197] As discussed above, page table PTEs map logical graphics memory addresses to physical memory addresses. Figure 15 shows a conventional page table in which a 540 KB PTE maps to a 270 MB page in the frame buffer. However, a 540 KB PTE occupies a significant amount of space in a page table that has at most 8 MB of space to store all translations. As mentioned above, increasing the size of the page table would be impractical because such an increase in size would impact current display hardware implementations.

[0198] According to one embodiment, the mapping mechanism 1410 is implemented to perform frame buffer mapping from page tables. In such an embodiment, the page mapping mechanism 1410 provides a two-level page table walk where each page table entry maps to a display page table (DPT) page and the second-level walk of the DPT points to a physical frame buffer page. In this embodiment, the page table entry contains a pointer to the associated DPT page that contains the actual translation.

[0199] In one embodiment, a frame buffer mapped from a DPT resides at address zero in its own DPT virtual address (VA) space. Thus, the DPT uses the same PTE format as a conventional page table, with each DPT mapping 4KB of frame buffer memory. In a further embodiment, the DPT may be large enough to map into the frame buffer (e.g., including padding bytes) and implement physically non-contiguous pages similar to the frame buffer.

[0200] FIG. 16 illustrates one embodiment of a GPU 1414. As shown in FIG. 16, the GPU 1414 includes a mapping mechanism 1510 and a memory management unit (MMU) 1610. The MMU 1610 includes a TLB 1620. In one embodiment, the TLB 1620 is a set-associative cache that stores recent translations from virtual memory to physical memory. The page table mapping mechanism 1410 first searches the TLB 1620 whenever a virtual address is translated to a physical address. If a match is found (e.g., a TLB hit), the page table mapping mechanism 1410 returns the physical address and the memory access may continue. However, if there is no match (e.g., a TLB miss), the page table mapping mechanism 1410 accesses the page table 1610 to perform the translation.

[0201] In one embodiment, a page table 1610 PTE contains a small component (e.g., 1 KB) of the virtual address that points to the associated DPT. In such an embodiment, each PTE is associated with (or covers) a defined range of the frame buffer. For example, each PTE may cover a 2 MB frame buffer mapping (e.g., an entire tile column for a strided rectangular block structure (or stride) <= 64 KB). In a further embodiment, two PTEs are implemented for strides greater than 64 KB and less than or equal to 128 KB. In yet a further embodiment, each page table 1610 cache line fetch contains eight translation entries. Thus, only one page table 1610 first level cache line fetch is implemented for an entire tile column. For strides greater than 64 KB, the surface base address is 8K aligned to ensure that one cache line GTT fetch is sufficient to retrieve the two first level PTEs required for a tile row.

[0202] FIG. 17 illustrates one embodiment of the page table 1610 and memory 1408, including the DPT 1720 and associated frame buffer 1750. As shown in FIG. 17, a PTE in the page table 1610 maps to a 540 KB DPT 1720, which translates to a 270 MB page of data in the frame buffer 1750. According to one embodiment, a surface starts on a new 4K DPT 1720 at offset 0. In this embodiment, planar YUV format, Y and UV surfaces, and compression control surfaces have separate DPT 1720s starting at offset 0. In a further embodiment, multiple tile row entries may be contained within a single DPT 1720 page. However, entries for a tile row do not span two different DPT 1720 pages, except for strides greater than 64K where two DPT 1720 pages are implemented. This is implemented by restricting the surface stride (e.g., in number of tiles) to a set of power-of-two values, with valid values ​​being 8, 16, 32, 64, 128, 256, 512, 1024.

[0203] FIG. 18A illustrates one embodiment of a page-level view of page table 1610 and memory 1408. As shown in FIG. 18A, each PTE (e.g., DPTP0-DPTP2) maps to a DPT 1720. Furthermore, DPT 1720 includes multiple translations to physical pages within frame buffer 1850. For example, each DPT 1720 includes 512 translations (e.g., 0 through 511) to 512 pages (e.g., page 0 through page 511) within frame buffer 1750. FIG. 18B illustrates one embodiment of a frame buffer including pages 0 through 511.

[0204] According to one embodiment, the mapping mechanism 1510 is configurable to operate according to either traditional page table mapping or mapping via DPT2 level mapping. In this embodiment, DPT2 level mapping is implemented for tile-based (e.g., Tile 4, Tile Y, and Tile X) frame buffer surfaces, while linear frame buffer surfaces implement direct page table mapping with a single level of lookup. In a further embodiment, the mapping mechanism 1410 implements a DPT-mapped frame buffer by specifying a page table VA located in a page table VA associated with a DPT (e.g., because the frame buffer does not have a VA itself).

[0205] Figure 19 is a flow diagram illustrating one embodiment of a process for performing a virtual to physical address translation via mapping mechanism 1510. At processing block 1910, a page translation request for an address is received. At processing block 1920, a TLB lookup 1610 for the page translation is performed. At decision block 1930, it is determined whether there is a miss for the page in TLB 1610. The page translation is returned if it is determined that the page is found in TLB 1610, processing block 1960. Otherwise, the page table is searched to retrieve the address associated with the DPT, processing block 1940. At processing block 1950, the DPT is accessed using the retrieved address to obtain the translation. The page translation is then returned at processing block 1960. At processing block 1970, data is retrieved from the frame at the physical address indicated in the returned translation.

[0206] The above mechanism improves the addressability of display memory to overcome the existing problems / complexities caused by exhausting the page table based display address space and is compatible with existing hardware implementations.

[0207] The following sections and / or examples pertain to further embodiments or examples. Specific details in the examples may be used anywhere in one or more embodiments. Various features of the various embodiments or examples may be combined in various ways, with some features included and others excluded, to suit a variety of different applications. Examples may include subject matter such as a method, a means for performing the operations of the method, at least one machine-readable medium including instructions that, when executed by a machine, cause the machine to perform the operations of the method, or an apparatus or system for facilitating hybrid communication in accordance with the embodiments and examples described herein.

[0208] Some embodiments relate to the subject matter of Example 1, including an apparatus for facilitating page translation, the apparatus comprising: a frame buffer for multiple pages of data; multiple display page tables for storing translations from virtual addresses to physical addresses for the pages of data in the frame buffer; and a page table having multiple page table entries (PTEs), each PTE mapped to one of the multiple display page tables.

[0209] Example 2 includes the subject matter described in Example 1, with each PTE having a pointer to a display page table.

[0210] Example 3 includes the subject matter described in Examples 1 and 2, with a component of the virtual address having a pointer stored in a display page table.

[0211] Example 4 includes the subject matter described in examples 1-3, wherein each PTE is associated with a defined range of frame buffer mappings.

[0212] Example 5 includes the subject matter of Examples 1-4, further comprising a translation lookaside buffer (TLB) including a plurality of entries that store translations from virtual memory addresses to physical memory addresses.

[0213] Example 6 includes the subject matter of Examples 1-5, further comprising mapping hardware receiving a page translation request for the address.

[0214] Example 7 includes the subject matter described in Examples 1 through 6, wherein the mapping hardware performs a TLB lookup for the translation upon receiving a page translation request.

[0215] Example 8 includes the subject matter described in Examples 1 through 7, wherein the mapping hardware, when determining that the TLB does not contain the translation, searches the PTE in the page table to find an address associated with a first of multiple display page tables.

[0216] Example 9 includes the subject matter described in Examples 1 through 8, wherein the mapping hardware accesses the first display page table to obtain the translation.

[0217] Some embodiments relate to the subject matter of Example 10, including a method of facilitating page translation, comprising: receiving a page translation request for a virtual address; searching a first page table to retrieve an address associated with a first of a plurality of display page tables; searching the first display page table for a physical page translation based on the retrieved address; and returning the physical address from the first display page table.

[0218] Example 11 includes the subject matter of example 10, further comprising searching a translation lookaside buffer (TLB) for a physical page translation before searching the first page table.

[0219] Example 12 includes the subject matter of Examples 10 and 11. The first table has a plurality of page table entries (PTEs), each containing a pointer to one of the plurality of display page tables.

[0220] Example 13 includes the subject matter of Examples 1 through 12, with each PTE having a pointer to a display page table.

[0221] Example 14 includes the subject matter of Examples 1 through 13, wherein the pointer has a component of a virtual address stored in a display page table.

[0222] Example 15 includes the subject matter of Examples 1 through 14, where each PTE is associated with a defined range of frame buffer mappings.

[0223] Some embodiments relate to the subject matter of Example 16, including a system for facilitating page translation, comprising: a memory including a frame buffer for multiple pages of data; multiple display page tables that store virtual address to physical address translations for the pages of data in the frame buffer; and a memory management unit (MMU) coupled to the memory, the page table including multiple page table entries (PTEs), each PTE mapped to one of the multiple display page tables.

[0224] Example 17 includes the subject matter of example 16, wherein the MMU further comprises a translation lookaside buffer (TLB) including a plurality of entries that store translations from virtual memory addresses to physical memory addresses.

[0225] Example 18 includes the subject matter described in Examples 16 and 17, wherein the MMU further comprises mapping hardware that receives a page translation request for the address.

[0226] Example 19 includes the subject matter described in Examples 16-18, wherein the mapping hardware performs a TLB lookup for the translation upon receiving a page translation request.

[0227] Example 20 includes the subject matter described in Examples 16 to 19, wherein the mapping hardware, when determining that the TLB does not contain the translation, searches the PTE in the page table to find the address associated with the first of multiple display page tables.

[0228] Example 21 includes the subject matter of Examples 16-20, wherein the mapping hardware accesses the first display page table to obtain the translation.

[0229] The present invention has been described above with reference to particular embodiments. However, those skilled in the art will understand that various modifications and changes may be made thereto without departing from the broader spirit and scope of the invention as set forth in the appended claims. Accordingly, the foregoing description and drawings should be regarded in an illustrative sense, rather than a restrictive sense. [Other possible claims] [Item 1] A frame buffer for multiple pages of data; a plurality of display page tables for storing translations of virtual addresses to physical addresses for said pages of data in said frame buffer; a page table having a plurality of page table entries (PTEs), each PTE being mapped to one of the plurality of display page tables; A device that facilitates page transformation. [Item 2] Each PTE contains a pointer to a display page table. Item 1. The device according to item 1. [Item 3] the pointer having a component of a virtual address stored in the display page table; Item 2. The device according to item 2. [Item 4] Each PTE is associated with a defined range of frame buffer mappings. Item 3. The device according to item 3. [Item 5] a translation lookaside buffer (TLB) including a plurality of entries storing translations from the virtual memory addresses to the physical memory addresses; Item 1. The device according to item 1. [Item 6] further comprising mapping hardware receiving a page translation request for the address; Item 5. The device according to item 5. [Item 7] the mapping hardware, upon receiving the page translation request, performs a TLB lookup for the translation; Item 6. The device according to item 6. [Item 8] if the mapping hardware determines that the TLB does not contain the translation, it searches the PTE in the page table to find the address associated with a first of the plurality of display page tables; Item 7. The device according to item 7. [Item 9] the mapping hardware accessing the first display page table to obtain the translation; Item 9. The device according to item 8. [Item 10] receiving a page translation request for a virtual address; searching a first page table to retrieve an address associated with a first of the plurality of display page tables; searching the first display page table for a physical page translation based on the captured address; returning a physical address from the first display page table. How to boost page conversions. [Item 11] and searching a translation lookaside buffer (TLB) for the physical page translation before searching the first page table. Item 11. The method according to item 10. [Item 12] the first table having a plurality of page table entries (PTEs), each containing a pointer to one of the plurality of display page tables; Item 12. The method according to item 11. [Item 13] Each PTE contains a pointer to a display page table. Item 13. The method according to item 12. [Item 14] the pointer having a component of a virtual address stored in the display page table; Item 14. The method according to item 13. [Item 15] Each PTE is associated with a defined range of frame buffer mappings. Item 15. The method according to item 14. [Item 16] A frame buffer for multiple pages of data; a plurality of display page tables storing translations of virtual addresses to physical addresses for said pages of data in said frame buffer; a memory management unit (MMU) coupled to the memory, the memory management unit (MMU) including a page table having a plurality of page table entries (PTEs), each PTE being mapped to one of a plurality of display page tables; A system that accelerates page conversion. [Item 17] the MMU further includes a translation lookaside buffer (TLB) including a plurality of entries storing translations from the virtual memory addresses to physical memory addresses; Item 17. The system according to item 16. [Item 18] the MMU further includes mapping hardware for receiving a page translation request for an address; Item 18. The system according to item 17. [Item 19] the mapping hardware performs a lookup of the TLB for a translation upon receiving the page translation request; Item 19. The system according to item 18. [Item 20] if the mapping hardware determines that the TLB does not contain the translation, it searches the PTE in the page table to find the address associated with a first of the plurality of display page tables; Item 19. The system of item 19. [Item 21] the mapping hardware accessing the first display page table to obtain the translation; Item 19. The system of item 19.

Claims

1. A frame buffer for multiple pages of data; a plurality of display page tables storing translations from virtual memory addresses to physical memory addresses for the plurality of pages of data in the frame buffer; a page table having a plurality of page table entries (PTEs), each PTE being mapped to one of the plurality of display page tables; A device that facilitates page transformation.

2. Each PTE contains a pointer to a display page table.

2. The apparatus of claim 1.

3. The pointer has a component of a virtual address that is stored in the display page table.

3. Apparatus according to claim 1 or 2.

4. Each PTE is associated with a defined range of frame buffer mappings.

4. Apparatus according to any one of claims 1 to 3.

5. a translation lookaside buffer (TLB) including a plurality of entries for storing translations of the virtual memory addresses to physical memory addresses; 5. Apparatus according to any one of claims 1 to 4.

6. and mapping hardware for receiving a page translation request for the address.

6. Apparatus according to any one of claims 1 to 5.

7. When the mapping hardware receives a page translation request, it performs a TLB lookup for the translation.

7. Apparatus according to any one of claims 1 to 6.

8. if the mapping hardware determines that the TLB does not contain the translation, it searches the PTE in the page table to find an address associated with a first of the plurality of display page tables; 8. Apparatus according to any one of claims 1 to 7.

9. the mapping hardware accessing a first display page table to obtain the translation; 9. Apparatus according to any one of claims 1 to 8.

10. 1. A method for facilitating page conversion, comprising: receiving a page translation request for a virtual address by mapping hardware; said mapping hardware searching a first page table to retrieve an address associated with a first display page table of a plurality of display page tables; said mapping hardware searching said first display page table for a physical page translation based on said captured address; the mapping hardware returning a physical address from the first display page table; the plurality of display page tables storing translations from virtual memory addresses to physical memory addresses for a plurality of pages of data within a frame buffer; method.

11. and prior to searching the first page table, the mapping hardware searching a translation lookaside buffer (TLB) for the physical page translation. The method of claim 10.

12. the first page table having a plurality of page table entries (PTEs), each PTE containing a pointer to one of the plurality of display page tables; 12. The method according to claim 10 or 11.

13. Each PTE contains a pointer to a display page table.

13. The method according to any one of claims 10 to 12.

14. The pointer has a component of a virtual address stored in the display page table.

14. The method according to any one of claims 10 to 13.

15. Each PTE is associated with a defined range of frame buffer mappings.

15. The method according to any one of claims 10 to 14.

16. A frame buffer for multiple pages of data; a plurality of display page tables storing translations from virtual memory addresses to physical memory addresses for the plurality of pages of data in the frame buffer; a memory management unit (MMU) coupled to the memory including a page table having a plurality of page table entries (PTEs), each PTE being mapped to one of the plurality of display page tables; A system that accelerates page conversion.

17. the MMU further comprising a translation lookaside buffer (TLB) including a plurality of entries for storing translations from the virtual memory addresses to physical memory addresses; 17. The system of claim 16.

18. the MMU further comprises mapping hardware for receiving a page translation request for an address; 18. A system according to claim 16 or 17.

19. When the mapping hardware receives a page translation request, it performs a TLB lookup for the translation.

19. A system according to any one of claims 16 to 18.

20. if the mapping hardware determines that the TLB does not contain the translation, it searches the PTE in the page table to find an address associated with a first of the plurality of display page tables; 20. A system according to any one of claims 16 to 19.

21. the mapping hardware accessing a first display page table to obtain the translation; 21. A system according to any one of claims 16 to 20.

Images