Techniques to multiply memory access bandwidth using multiple links

Abstract

Examples include techniques for multiplying memory access bandwidth using multiple links. Example techniques may include generation and use of forwarding tables individually maintained at devices coupled via a high speed internal connection link (HSIL). The forwarding table enables the first device to route a memory request received by the first device to access a memory address of the memory at the second device. A memory request is received by a first device from a host computing device via a link between the first device and the host computing device, the memory request to be forwarded to a second device via an HSIL coupled between the first device and the second device.

Description

Technical Field

[0001] Examples described herein involve accessing memory maintained at a device coupled to a processor. Background Art

[0002] The rapid growth in the use of artificial intelligence (AI) and high performance computing (HPC) has increased the importance of the roles that accelerators such as graphics processing units (GPUs) serve in AI and HPC. In particular, as GPU performance has increased more than 10 times over the past decade, GPUs have received relatively high attention. However, as GPU performance continues to increase, these higher performance GPUs have been somewhat lacking in data due to the relatively slow links coupling these higher performance GPUs to central processing units (CPUs). For example, according to research such as that provided by Links to published specifications or standard operations include the PCI Express Specification (Rev.6.0, Ver.1.0) released in January 2022 ("PCIe Specification").

[0003] Efforts are underway to speed up or increase the bandwidth of links between a group of GPUs in the same system so that the GPUs in the group can exchange data directly at high speed. NVLink has been developed TM NVLink can have a per-GPU bandwidth of 500 gigabytes per second (GB / s) or more. Other GPU manufacturers, such as The company is working on a multi-die fabric that includes a link with a bandwidth similar to that of NVLink. NVLink included in a multi-die fabric or other types of similar links that interconnect GPUs and / or accelerators included in a multi-die fabric may be referred to as "high-speed internal link" (HSIL). In some examples, these types of HSILs that interconnect GPUs are designed to facilitate GPU-GPU or accelerator-accelerator communications. BRIEF DESCRIPTION OF THE DRAWINGS

[0004] Figure 1 An example system is shown.

[0005] Figure 2 An example memory forwarding table is shown.

[0006] Figure 3 An example scenario is shown.

[0007] Figure 4 An example DVSEC range structure is shown.

[0008] Figure 5 An example DVSEC capability register is shown.

[0009] Figure 6 An example CXL Early Detection Table (CEDT) is shown.

[0010] Figure 7 An example CBHS table is shown.

[0011] Figure 8 An example CXL Binding HDM Structure (CBHS) table is shown.

[0012] Fig. 9 An example first logic flow is shown.

[0013] Fig.10 An example second logic flow is shown.

[0014] Fig.11 An example third logic flow is shown.

[0015] Fig.12 An example process is shown.

[0016] Fig.13 is a block diagram of an example processing system.

[0017] Fig.14A is a block diagram of an example of a processor having one or more processor cores, an integrated memory controller, and an integrated graphics processor.

[0018] Fig. 14B is a block diagram of example hardware logic for a graphics processor core block.

[0019] Fig. 14C An example graphics processing unit (GPU) is shown that includes a collection of specialized graphics processing resources arranged into multi-core groups.

[0020] Fig.14D is a block diagram of an example general purpose graphics processing unit (GPGPU) that may be configured as a graphics processor and / or a computing accelerator.

[0021] Fig.15A is a block diagram of an example graphics processor, which may be a discrete graphics processing unit, or may be a graphics processor integrated with multiple processing cores or other semiconductor devices such as, but not limited to, a memory device or a network interface.

[0022] Fig. 15B An example graphics processor having a tiled architecture is shown.

[0023] Fig. 15C An example computing accelerator is shown.

[0024] Fig.16 is a block diagram of an example graphics processing engine for a graphics processor.

[0025] Fig.17A An example graphics core cluster is shown.

[0026] Fig. 17B An example vector engine of a graphics core is shown.

[0027] Fig. 17C An example matrix engine of a graphics core is shown.

[0028] Fig.18 An example tile of a multi-tile processor is shown.

[0029] Fig.19 is a block diagram illustrating a graphics processor instruction format according to some examples.

[0030] Fig. 20 is a block diagram of another example of a graphics processor.

[0031] Fig.21A is a block diagram illustrating a graphics processor command format that may be used to program a graphics processing pipeline.

[0032] Fig. 21B is a block diagram illustrating an example graphics processor command sequence.

[0033] Fig. 22 An example graphics software architecture for a data processing system is shown.

[0034] Fig.23A is a block diagram illustrating an example IP core development system that may be used to fabricate an integrated circuit to perform operations.

[0035] Fig. 23B A cross-sectional side view of an example integrated circuit package assembly 1170 is shown.

[0036] Fig.23C An example package assembly including multiple hardware logic chiplet units connected to a substrate is shown.

[0037] Fig.23D An example package assembly including interchangeable chiplets is shown.

[0038] Fig.24 is a block diagram illustrating an example system-on-chip integrated circuit that may be fabricated using one or more IP cores.

[0039] Fig.25A An example graphics processor of a system-on-chip integrated circuit that can be fabricated using one or more IP cores is shown.

[0040] Fig.25BAdditional examples of graphics processors for system-on-chip integrated circuits that can be fabricated using one or more IP cores are shown. DETAILED DESCRIPTION

[0041] In some example computing systems today, an accelerator or GPU is coupled or connected to a CPU via a PCIe link. A PCIe link operating in accordance with the PCIe specification (e.g., Rev. 6.0) may have a bandwidth or throughput of approximately 128 GB / sec. In contrast, a HSIL interconnecting a group of GPUs or accelerators may have a data bandwidth or throughput of 500 GB / sec or greater, and may therefore have a data bandwidth at least 5 times greater than these types of PCIe links. As a result of this at least 5-fold difference in data bandwidth or throughput, in a system that includes a GPU that is interconnected via a fast HSIL but is separately connected to the CPU via a substantially slower PCIe link, the CPU-to-GPU link becomes a bottleneck for processing large amounts of data.

[0042] The new technical specification of the Compute Express Link (CXL) Consortium is the Compute Express Link Specification (Rev.2.0, Ver.1.0) released on October 26, 2020, hereinafter referred to as the "CXL Specification". The CXL specification introduces online and offline memory at a device (e.g., a GPU device or an accelerator device) attached to a host computing device through separate links, each of which is configured to operate according to the CXL specification and is referred to as a "CXL link" below. The memory at a device that is online or offline according to the CXL specification is called host-managed device memory (HDM). Typically, each GPU or accelerator in a GPU or accelerator system is connected to the CPU via a separate CXL link. Since the CXL link also operates according to the PCIe specification, the throughput or bandwidth of the CXL link may be limited to the 128GB / sec mentioned above for the PCIe link. Thus, in an example where a GPU system has a GPU-GPU HSIL interconnect with a bandwidth of at least 5 times that of 128 GB / sec, the CXL link between the CPU and GPU of the system may be excessively busy (e.g., at maximum bandwidth), while other CXL links to other GPUs may be idle or underutilized. With regard to these challenges, there is a need for the examples described herein.

[0043] Figure 1 An example system 100 is shown. In some examples, such as Figure 1As shown, system 100 includes a host computing device 105 having a root complex 120 to couple or connect to devices 130-A, 130-B, and 130-C via respective host links 140-1, 140-2, and 140-3 through respective ports 121-1, 121-2, and 121-N, where "N" is any positive integer greater than 2. Figure 1 As shown, the host computing device 105 is also coupled to a host system memory 110 via one or more memory channels 101. For these examples, the host computing device 105 includes a host operating system (OS) 102 for executing or supporting one or more device drivers 104, a host basic input / output system (BIOS) 106, one or more host applications 108, and a host central processing unit (CPU) 107 for supporting computing operations of the host computing device 105.

[0044] In some examples, although Figure 1 107, but in other examples, the root complex 120 may be integrated with the host CPU 107. For either example, the root complex 120 may be arranged to function as a type of PCIe root complex for the host CPU 107 and / or other elements of the host computing device 105 to communicate with devices (such as device 130) via the use of PCIe-based communication protocols and communication links. The root complex 120 may also be configured to operate in accordance with the CXL specification, and as Figure 1 As shown, home agent 124 is included to facilitate communication with device 130 via host link 140. For these examples, host link 140 may be configured to operate in accordance with the CXL specification. Figure 1As shown, the root complex 120 includes an HDM decoder 126, which can be programmed to facilitate mapping of host to device physical addresses for accessing memory at or attached to the device 130. A memory controller (MC) 122 at the root complex 120 can control / manage access to the host system memory 110 through the memory channel 101. The host system memory 110 can include volatile and / or non-volatile types of memory. In some examples, the host system memory 110 can include one or more dual in-line memory modules (DIMMs), which can include any combination of volatile or non-volatile memory. For these examples, the memory channel 101 and the host system memory 110 may operate in accordance with a variety of memory technologies described in various standards or specifications, such as DDR3 (DDR version 3) originally released by JEDEC (Joint Electron Device Engineering Council) on June 27, 2007, DDR4 (DDR version 4) originally released in September 2012, DDR5 (DDR version 5) originally released in July 2020, LPDDR3 (Low Power DDR version 3) originally released in August 2013, JESD209-3B, LPDDR4 (LPDDR version 4) originally released in August 2014, JESD209-3B, DDR3 (DDR version 3) originally released in June 2015, DDR4 (DDR version 4) originally released in September 2016, DDR5 (DDR version 5) originally released in July 2020, LPDDR3 (Low Power DDR version 3) originally released in August 2013, JESD209-3B, LPDDR4 (LPDDR version 4) originally released in August 2016, JESD209-3B ESD209-4, LPDDR5 (LPDDR Revision 5, JESD209-5A, originally released in January 2020), WIO2 (Wide Input / Output Revision 2) originally released in August 2014, JESD229-2, HBM (High Bandwidth Memory) originally released in October 2013, JESD235, HBM2 (HBM Revision 2) originally released in January 2020, JESD235C, or HBM3 (HBM Revision 3) originally released in January 2022, JESD238, or other memory technologies or combinations of memory technologies, and derivative or extended technologies based on such specifications. JEDEC standards or specifications may be obtained from www.jedec.org.

[0045] According to some examples, such as Figure 1As shown, devices 130-A, 130-B, and 130-C may be interconnected via respective HSILs 152-1, 152-2, and 152-3 included in multi-die fabric 150. For these examples, each link included in HSIL 152 may be capable of having a substantially higher data bandwidth or throughput (e.g., >500 GB / sec) than each link included in host link 140 (e.g., approximately 100 GB / sec). As described in more detail below, consistent memory access capabilities between devices 130-A, 130-B, and 130-C and host computing device 105 via respective host links 140-1, 140-2, and 140-3 are possible when elements of system 100 operate in accordance with the CXL specification. The consistent memory access capability enables increasing data bandwidth between host computing device 105 and devices among devices 130 by transferring data via host link 140 in parallel utilizing adjacent host links so data bandwidth can be multiplexed or multiplied to access memory at device 130 .

[0046] In some examples, such as Figure 1 As shown, the devices 130 each include a port 131, a coherence agent 132 (e.g., a CXL device coherence agent (DCOH)), a memory router (MR) circuitry 133, a host managed device memory (HDM) 134, a memory controller circuitry 135, a computing circuitry 136, a HSIL interface (I / F) 137, or a register 138. As described more below, circuitry or logic at the device 130 (such as a coherence agent 132) may separately receive a CXL.mem address from a host computing device 105 via a host link 140, and then utilize a separately maintained forwarding table included in registers 138 to enable the MC circuitry 135 to access a memory address of a target HDM 134 for a device among the devices 130. The accessed memory address may enable a device that does not include a target HDM memory to read or write data to the target HDM memory via a path routed through a first link included in the host link 140 and a second link included in the HSIL 152. At the same time, the device including the target HDM memory can access the memory address via a path routed through a single link in the host link 140. Since the HSIL 152 has a substantially higher bandwidth than the host link 140, adding these HSIL links to access the memory address in the target device HDM memory at another device memory adds relatively small latency. Therefore, this method or scheme of parallel access using the memory forwarding table can increase the available data bandwidth between the host computing device 105 and the target device by nearly three times.

[0047] According to some examples, registers 138 included in device 130 may be configured (e.g., programmed) to maintain a forwarding table and a PCIe-specified vendor-specific capability (DVSEC) structure. As briefly mentioned above and described in more detail below, the device may use a forwarding table to determine where to access an HDM memory address located at a target HDM memory. Moreover, as described in more detail below, the forwarding table maintained in registers 138 may be set or programmed based at least in part on information included in various DVSEC structures also maintained in registers 138. For example, the various DVSEC structures may be set or programmed by a manufacturer or vendor of device 130, and then used by a BIOS (e.g., host BIOS 106) to set or program at least a portion of registers 138 to include a forwarding table.

[0048] According to some examples, HDM 134 may include volatile and / or non-volatile types of memory for use by computing circuit system 136 to execute, for example, workloads. Volatile and / or non-volatile types of memory may reside on the same die or package as device 130 (e.g., stacked HBM die or stacked DRAM die), or may be an attached memory device (e.g., DIMM). In some examples, computing circuit system 136 may be a GPU, and the workload may be a workload related to graphics processing, or may be related to an AI workload that utilizes GPU processing capabilities that are offloaded to computing circuit system 136 by host CPU 107. In other examples, computing circuit system 136 may be at least part of an FPGA, ASIC, or CPU used as an accelerator, and the workload may be offloaded from host computing device 105 for execution by these types of computing circuits including FPGA, ASIC, or CPU. Moreover, consistency agent 132, MR circuit system 133, or MC circuit system 135 may be at least part of an FPGA, ASIC, or CPU for offloading workloads from host computing device 105.

[0049] As described above, the host system memory 110 and the HDM 134 may include volatile or non-volatile types of memory. The volatile type of memory may include, but is not limited to, random access memory (RAM), dynamic RAM (DRAM), DDR synchronous dynamic RAM (DDR SDRAM), GDDR, HBM, static random access memory (SRAM), thyristor RAM (T-RAM), or zero capacitor RAM (Z-RAM). The non-volatile memory may include a byte or block addressable type of non-volatile memory having a 3-dimensional (3-D) cross point memory structure, including but not limited to a chalcogenide phase change material (e.g., chalcogenide glass), hereinafter referred to as a "3-D cross point memory". Non-volatile types of memory may also include other types of byte or block addressable non-volatile memory, such as (but not limited to) multi-threshold level NAND flash memory, NOR flash memory, single-level or multi-level phase change memory (PCM), resistive memory, nanowire memory, ferroelectric transistor random access memory (FeTRAM), antiferroelectric memory, resistive memory including metal oxide substrate, oxygen vacancy substrate and conductive bridge random access memory (CB-RAM), spin electronic magnetic junction memory, magnetic tunneling junction (MTJ) memory, domain wall (DW) and spin-orbit transfer (SOT) memory, thyristor-based memory, magnetoresistive random access memory (MRAM) incorporating memristor technology, spin transfer torque MRAM (STT-MRAM) or any combination of the above.

[0050] Figure 2 An example forwarding table 200 is shown. In some examples, such as Figure 2 As shown, forwarding table 200 includes forwarding table 210, forwarding table 220, and forwarding table 230. For these examples, forwarding table 210 may be maintained in register 138-A at device 130-A, forwarding table 220 may be maintained in register 138-B at device 130-B, and forwarding table 230 may be maintained in register 138-C at device 130-C.

[0051] As described in more detail below, upon initialization of device 130-A, a BIOS (e.g., host BIOS 106) may program at least a portion of register 138-A to set a first value indicating a local address, which is used as a virtual HDM (vHDM) base address that can be mapped to a host physical address (HPA) system (e.g., maintained by host computing device 105) for memory addresses of HDM 134-A. The BIOS may also set respective second and third values ​​indicating respective destination addresses for devices 130-B and 130-C. In some examples, the respective destination addresses may be used as base addresses for HDM 134-B and HDM 134-C that may also be mapped to the HPA system, and indicate explicit respective HPAs for HDM 134-B and HDM 134-C. Furthermore, the size or memory capacity associated with each base address indicated for the local address may be the same size associated with the respective destination addresses for HDM 134-B and HDM 134-C. A similar process may occur when devices 130-B and 130-C are initialized to program respective at least portions of registers 138-B and 138-C to construct the Figure 2 vHDM and HDM mappings for forwarding tables 220 and 230 are shown.

[0052] Figure 3 An example scheme 300 is shown. According to some examples, scheme 300 shows an example of how data can be routed through a host link that is separately connected between a host computing device and different devices to write or read data to the HDM of a device among the different devices using a forwarding table programmed into registers of each device. For these examples, computing device 105 and devices 130-A, 130-B, and 130-C can be configured to operate host links 140-1 to 140-N according to the CXL specification, and the CXL.mem protocol can be used to route data via these host links. Figure 3 As shown, elements of the host computing device 105, such as the host OS 102, the host system memory 110, the root complex 120, and the ports 121-1 to 121-N, are described below as participating in the scheme 300. Also, elements of the devices 130-A, 130-B, or 130-C interconnected via the HSILs 152-1 to 152-3 of the multi-die structure 150 are described below as participating in the scheme 300. The scheme 300 is not limited to these elements of the devices 130-A, 130-B, 130-C or the computing device 105 as participating in the scheme 300.

[0053] According to some examples, such as Figure 3The HPA / system address space 301 shown in FIG. 301 may be an example of how host managed device memory (HDM) 134-A may be mapped to host system memory 110. As described in more detail below, binding 310 of HPA / system address space 301 may be based on a forwarding table programmed into registers 138-A, 138-B, and 138-C (e.g., by host BIOS 106) that binds virtual HDMs (vHDMs) of devices 130-B and 130-C to HDM 134-A of device 130-A. For example, as shown in FIG. Figure 3 As shown, vHDM_B2A included in forwarding table 220 of device 130-B and vHDM_C2A included in forwarding table 230 of device 130-B may be bound to HDM 134-A according to binding 310. Each bound address may be the same size or represent a memory address or memory address range to access the same amount of memory capacity.

[0054] In some examples, logic and / or features of host OS 102 may decide to split data associated with access to memory addresses of HDM 134-A. For these examples, the splitting of data includes a decision to multiplex host links 140-1 through 140-N to increase data bandwidth for accessing HDM 134-A, which also utilizes the higher bandwidth capabilities of HSILs 152-1 through 152-3 of multi-die fabric 150 to facilitate access to HDM 134-A. Figure 3 Data 1, Data 2, Data 3 shown represent splits of data to be multiplexed on host links 140-1 to 140-N. Routing legend 305 indicates that Data 1 with a solid black line is routed to device 130-A via port 121-1 and host link 140-1, Data 2 with a dashed black line is routed to device 130-B via port 121-2 and host link 140-2, and Data 3 with a dashed black / dotted line is routed via port 121-N and host link 140-3.

[0055] According to some examples, as described in more detail below, host OS 102 may utilize CXL.mem request messages sent via host links 140-1 to 140-N to access HDM 134-A via multiple routes through these host links. Each device among devices 130-A, 130-B, and 130-C may receive a separate CXL.mem request message indicating which portion of HDM 134-A to access. For these examples, circuitry of device 130-A (such as MR circuitry 133-A) may receive a CXL.mem request message associated with a memory address of HDM 134-A to be accessed indicating data 1. Since HDM 134-A is local to device 130-A, MC circuitry 135-A accesses HDM 134-A to complete access to the memory address associated with data 1. Device 130-A may then send a response message back to host OS 102 via host link 140-1. The circuit system of device 130-B (such as MR circuit system 133-B) may receive a CXL.mem request message indicating that Data 2 is associated with the memory address of HDM 134-A to be accessed. Since HDM 134-A is not local to device 130-B, MR circuit system 133-B may refer to forwarding table 220 to determine that vHDM_B2A indicates that the memory request for Data 2 is routed to device 130-A via HSIL 152-1. The circuit system of device 130-C (such as MR circuit system 133-C) may receive a CXL.mem request message indicating that Data 3 is associated with the memory address of HDM 134-A to be accessed. Since HDM 134-A is not local to device 130-C, MR circuit system 133-C may refer to forwarding table 220 to determine that vHDM_C2A indicates that the memory request for Data 3 is routed to device 130-A via HSIL 152-3. Upon receiving separate memory requests for data 2 and data 3, MC 135-A may access HDM 134-A according to the memory addresses indicated in the separate memory requests, and then send a response indicating the status of the memory access to HDM 134-A back to device 130-B via HSIL 152-1, and send a response indicating the status of the memory access to HDM 134-A back to device 130-C via HSIL 152-3. Devices 130-B and 130-C may then send separate responses indicating the respective status of the memory access to HDM 134-A back to host OS 102 via respective host links 140-2 and 140N. In some examples, if the CXL.mem request is to read data from HDM 134-A, the respective responses sent by devices 130-A, 130-B, and 130-C may include the data read from HDM 134-A.In other examples, if the CXL.mem request is to write data to HDM 134-A, the corresponding responses sent by devices 130-A, 130-B, and 130-C may indicate that the data has been successfully written to HDM 134-A.

[0056] Figure 4 An example DVSEC range structure 400 is shown. In some examples, the DVSEC range structure 400 may be maintained in register 138 of device 130 of system 100 and may represent an example of a type of PCIe DVSEC structure that uses bits 7:5 of the DVSEC range structure 400 to indicate a virtual HDM class. For these examples, Figure 4 As shown, a bit pattern of 100 in bits 7:5 in the DVSEC scope structure 400 indicates that the device supports vHDM to implement multiplexing of host links, as mentioned above for scheme 300 and described more below. A vendor or manufacturer of device 130 may set the bit pattern in bits 7:5 to 100 to indicate support for the use of vHDMs. The DVSEC scope structure 400 may be similar to the DVSEC CXL scope 2 size low structure described in the CXL specification, but with a bit pattern of 100 added in bits 7:5 to indicate support for the vHDM memory class.

[0057] Figure 5 An example DVSEC capability structure 500 is shown. According to some examples, the DVSEC capability may be maintained in register 138 of device 130 of system 100, and may represent an example of a type of PCIe DVSEC structure that uses bits 0:3 of the DVSEC capability structure 500 to indicate the number of vHDM ranges implemented by the device. For these examples, a value of 0 indicates that no vHDM ranges are supported, and values ​​1-15 indicate that 1 to 15 vHDM ranges are supported. Examples are not limited to 15 vHDM ranges or a DVSEC capability register having 4 bits to indicate bit values ​​of 0 to 15.

[0058] In some examples, a vendor or manufacturer of device 130 may set a bit value of DVSEC capability structure 500 to indicate the number of vHDM ranges supported by device 130. For example, devices 130-A, 130-B, and 130-C may have a DVSEC capability structure with a bit value of 2 to indicate support for 2 vHDM ranges. Support for 2 vHDM ranges indicates that device 130 may route memory requests to up to 2 neighboring devices, as described above for scenario 300.

[0059] Figure 6An example CEDT 600 is shown. According to some examples, the CEDT 600 may be linked to the Advanced Configuration and Power Interface (ver. 6.4) (hereinafter referred to as the "ACPI specification") released by the U.S. Extensible Firmware Interface (UEFI) Forum in January 2021. A BIOS (such as host BIOS 106) may expose device vHDM support information to an OS (such as host OS 102) via indications in CEDT 600 of a CXL Bound HDM Structure (CBHS) table. For example, a value of 2 in CEDT 600 indicates that a CBHS including device vHDM support information has been built. For these examples, the BIOS may collect information from the device's registers to build a CBHS table, and also build a Bound vHDM Structure table (both described more below) to expose the OS to the vHDM support information collected during initialization of the device to enable the OS to map HDM and vHDM to HPA / system address space, such as Figure 3 HPA / system address space 301 shown in and described above.

[0060] Figure 7 An example CBHS table 700 is shown. As briefly mentioned above, the CBHS table 700 can be constructed by the BIOS based on vHDM support information collected during device initialization, for example, from a DVSEC structure maintained in a device register (such as register 138). In some examples, Figure 7 As shown, the CBHS table 700 includes a type field, and a value of 2 in the type field indicates that a CBHS entry for the HDM is included in the CHBHS table 700. The HDM_Base field indicates the base address of the HPA (e.g., the system memory HPA) of the HDM. The HDM_Size field indicates the size or memory capacity of the HDM. The binding mode field indicates whether the binding mode is supported or not. The Num_vHDM field indicates the number of bound vHDM structures. The binding_vHDM[n] field indicates each bound vHDM structure that is fully mapped to the HDM, where each vHDM comes from an adjacent device that can forward memory requests to the HDM. The "[n]" in the binding_vHDM[n] field can indicate that the field can contain up to "n" different entries, where "n" represents any positive integer greater than 0.

[0061] Figure 8An example binding vHDM structure table 800 is shown. The binding vHDM structure table 800 can be constructed by an OS (e.g., host OS 102) based on vHDM support information collected during device initialization (e.g., collected by a BIOS). In some examples, the information included in the binding vHDM structure table 800 can be based on standard memory attributes provided by a vendor or manufacturer of a device having host managed device memory, and a separate binding vHDM structure table can be constructed by the OS for each vHDM that the device indicates as supported. For example, the OS can construct a separate binding vHDM structure table based on information included in a CBHS table (such as the example CBHS table 700) and based on information provided in an ACPI heterogeneous memory attribute table (HMAT). For these examples, as shown in FIG. Figure 8 As shown, the vHDM_Base field indicates the base address of the HPA (e.g., system memory HPA) of the vHDM. The vHDM_Size field indicates the size or memory capacity of the vHDM. The vHDM_Read_Latency field indicates the read latency of the vHDM address. The vHDM_Write_Latency field indicates the write latency of the vHDM address. The vHDM_Read_Bandwidth field indicates the read bandwidth of the vHDM address. The vHDM_Write_Bandwidth field indicates the write bandwidth of the vHDM address.

[0062] Fig. 9 An example logic flow 900 is shown. In some examples, the logic flow 900 can be implemented by a BIOS for a host computing device coupled to multiple devices (e.g., a GPU device and / or an accelerator device with host-managed device memory). For these examples, Figure 1 or Figure 3 140 and 150. The host BIOS 106 of the host computing device 105 coupled to the device 130 via the host link 140 shown for the system 100 in FIG. 140 may implement at least a portion of the logic flow 900. The host computing device and the device connected via the host link may be arranged to operate according to the CXL specification. The device may include registers having a DVEC structure that include information collected by the BIOS during initialization or booting of a system including the host computing device and the device connected via a separate host link. For example, the registers included in the DVEC structure may include information collected by the BIOS during initialization or booting of a system including the host computing device and the device connected via a separate host link. Figures 4 to 5 The information in the example DVSEC range structure 400 or the example DVSEC capability structure 500 shown is maintained in registers 138 of device 130, which can be collected by host BIOS 106. In addition, host BIOS 106 can cause a forwarding table (such as forwarding table 200) to be maintained in a register 138 of device 130. Figure 2Device 130 is shown as part of implementing logic flow 900. In addition, host BIOS 106 may populate ACPI related tables, such as Figure 6 to Figure 7 The CEDT table 600 , CBHS table 700 , or bound vHDM structure table 800 is shown as part of implementing the logic flow 900 .

[0063] Logic flow 900 begins at block 905 where host BIOS 106 enters the flow. In some examples, host BIOS 106 enters logic flow 900 in response to starting of system 100 and / or initialization of devices 130 included in system 100.

[0064] Moving from block 905 to decision block 910, the host BIOS 106 completes any necessary training of the host link 140 coupled between the host computing device 105 and the device 130 to ensure that the host link and the device can operate in accordance with the CXL specification and to confirm whether the device 130 indicates HDM / vHDM support. According to some examples, the host BIOS 106 collects information from a DVSEC range structure (such as the example DVSEC range structure 400 maintained in register 138 of the device 130) to determine whether a 100-bit pattern is set in bits 7:5 of the DVSEC range structure 400. If the 100-bit pattern is set in bits 7:5, indicating vHDM support, the logic flow 900 moves to block 920. Otherwise, the logic flow 900 moves to block 915.

[0065] Moving from decision block 910 to block 915 , host BIOS 106 completes other processes to complete its portion of initialization of system 100 .

[0066] Moving from decision block 910 to block 920, host BIOS 106 collects HDM and vHDM information from device 130. According to some examples, BIOS 106 collects HDM and vHDM information from a DVSEC capabilities structure, such as example DVSEC capabilities structure 500 maintained in registers 138 of device 130.

[0067] Moving from block 920 to block 925 , host BIOS 106 maps the system address space. In some examples, the system address space that is mapped may be HPA / system address space 301 .

[0068] Moving from block 925 to block 930, the host BIOS 106 builds vHDM and HDM mappings for each device. According to some examples, as part of building the mappings for each device, sub-block 925-A is implemented to bind vHDM-B2A / vHDM-C2A to HDM_A to communicate with Figure 2 The forwarding table 210 and Figure 3Various base addresses are set in the HPA / system address space 301 in a manner similar to that shown in the binding 310 shown in FIG. 9 , and sub-block 925-B is implemented to bind vHDM_A2B / vHDM_C2B to HDM_B to communicate with Figure 2 Various base addresses are set in a manner similar to that shown in the forwarding table 220 of FIG. 1 , and sub-block 925-C is implemented to bind vHDM-A2C / vHDM_B2C to HDM_C to communicate with Figure 2 The various base addresses are set in a similar manner as shown in the forwarding table 230 of the host BIOS 106. For these examples, the host BIOS 106 can cause a portion of the registers 138 of the devices 130-A, 130-B, and 130-C to be individually programmed to maintain the corresponding forwarding tables 210, 220, and 230 based on the binding mentioned above.

[0069] Moving from block 925 to block 930, the host BIOS 106 populates the ACPI table for the vHDMs bound to the HDMs. In some examples, an example ACPI table to be populated may be a CEDT table (such as Figure 6 CEDT table 600 shown in FIG. 100 ) to indicate that the host BIOS 106 has also populated the CBHS table (such as Figure 7 CBHS table 700 shown in FIG.

[0070] Moving from block 915 or from block 930 to block 935, host BIOS 106 completes its portion of initialization of system 100 to allow subsequent operations to switch to the OS. According to some examples, an OS, such as host OS 102, may implement the subsequent operations.

[0071] Fig.10 An example logic flow 1000 is shown. In some examples, the logic flow 1000 can be executed by an OS for a host computing device, such as Figure 1 or Figure 3 1000 is implemented by a host OS 102 for a host computing device 105 as shown. For these examples, the host OS 102 can utilize various ACPI tables populated by a BIOS (e.g., the host BIOS 106 mentioned above for logic flow 900) to initialize and bind HDM mapping tables in the host OS 102 for subsequent use in multiplexing host links 140-1 through 140-N to increase data bandwidth to access an HDM at one of the devices 130-A, 130-B, or 130-C. The host OS 102 can implement the logic flow 1000 for each HDM of the devices 130-A, 130-B, and 130-C.

[0072] Logic flow 1000 begins at block 1010, where host OS 102 enters logic flow 1000. According to some examples, OS 102 may enter logic flow 1000 when a kernel within the OS (eg, executed by host CPU 107) boots up.

[0073] Moving from block 1010 to block 1020, host OS 102 may cause BootMem_Init to boot an HDM at a device coupled via a host link to host computing system 105. For example, host OS 102 may cause HBM 134-A at device 130-A coupled via host link 140-1 to boot.

[0074] Moving from block 1020 to block 1030, host OS 102 may parse the binding of the HDM of the device based on the ACPI table including information associated with the HBM to obtain HDM information for the HDM. In some examples, the ACPI table includes a CBHS table (such as CBHS table 700) that includes information for parsing the binding of the HDM of the device. For example, the CBHS table for HBM 134-A at device 130-A may include information for parsing the binding of HBM 134-A to vWDM-B2A / vWDM-C2A.

[0075] Moving from block 1030 to block 1040, the host OS 102 builds a mapping table for the HBM. According to some examples, the host OS 102 may build a mapping table by first populating a standard HPA table that may be maintained in the host system memory 110 with HDM information for the HDM. For example, the standard HPA table may be used by the HDM decoder 126 and / or logic / features of the host OS 102 to determine what HPA will be mapped to a device physical address (DPA) at a device that includes the HDM. The host OS 102 may then obtain binding address information for the vHDM bound to the HDM and populate the standard HPA table based on the binding address information. The binding address information for each vHDM may be based on information obtained from a binding vHDM structure table, such as the example binding vHDM structure table 800.

[0076] Moving from block 1040 to block 1050, the host OS 102 may record or store the mapping table. According to some examples, as described more below, the mapping table may be stored to the host system memory 110 for later use in determining how to route data accesses to the HDM via multiple host links to increase data bandwidth for data accesses compared to using a single host link to complete data accesses to the HDM.

[0077] Moving from block 1040 to decision block 1060, host OS 102 determines whether to start any additional HDMs and whether to build and store associated mapping tables. If there are more HDMs, logic flow 1000 moves to block 1020. If there are no more HDMs, logic flow 1000 ends.

[0078] Fig.11 An example logic flow 1100 is shown. In some examples, the logic flow 1100 can be executed by an OS for a host computing device, such as Figure 1 or Figure 3 1000 ). For these examples, the host OS 102 may have completed building a mapping table for multiple HDMs of devices coupled to the host computing device 105, as described above with respect to the logic flow 1000.

[0079] Logic flow 1100 begins at block 1110, where host OS 102 accesses an address in the HDM of a device having a Length. For example, the access may be based on a data request from an application among host applications 108 at host computing device 105 (e.g., associated with an AI or HPC workload). According to some examples, Length may be associated with a data capacity (e.g., 1 GB) and a range of physical memory addresses in the HDM required to store the data capacity. For these examples, the HDM to be accessed may be HDM 134-A of device 130.

[0080] Moving from block 1110 to block 1120, host OS 102 obtains all vHDMs of the HDM to be accessed from the recorded mapping table. In some examples, the recorded mapping table may be stored to host system memory 110. For example, the vHDMs may be vHDM-B2A at device 130-B and vHDM-C2A at device 130-C that has been bound to HDM 134-A.

[0081] Moving from box 1120 to decision box 1130, host OS 102 determines whether to select multiplexing mode. According to some examples, multiplexing mode can include routing access requests via host links 140-1 to 140-N and through devices 130-B and 130-C to increase the data bandwidth for accessing addresses in HDM 134-A with Length. For these examples, the determination can be based on meeting latency and / or data bandwidth requirements associated with the system that executes or supports the execution of the workload. The requirement can be based on, but is not limited to, meeting a service level agreement or a quality of service policy. If multiplexing mode is selected, logic flow 1100 moves to box 1150. If multiplexing mode is not selected, logic flow 1100 moves to box 1140.

[0082] Moving from decision block 1130 to block 1140, host OS 102 uses traditional access of addresses in HDM with full length. In some examples, this will mean that host OS 102 will only access addresses of HDM 134-A with full length via host link 141-1.

[0083] Moving from decision block 1130 to block 1150, host OS 102 splits Length into N+1 parts, obtains the HDM address based on the mapping table, and obtains N+1 sub-addresses for the vHDM based on the mapping table. According to some examples, "N" may represent the number of vHDMs bound to the HDM. For example, N for HDM 134-A would be 2. An example equation for determining the sub-address of the vHDM may be Sub_Addr_m=Addr_in_HDM+N*(Length / N+1), where n=1, ..., N and m=0, ..., N-1.

[0084] Moving from block 1150 to block 1160, host OS 102 accesses the address range of the HDM of size = Length / (N+1), and accesses each subaddress of size = Length / (N+1). In some examples, host OS 102 accesses HDM 134-A by sending a first CXL.mem request to device 130-A via host link 140-1, a second CXL.mem request to device 130-B via host link 140-2, and a third CXL.mem request to device 130-C via host link 140-N. The first CXL.mem request includes the determined HDM address for HDM 134-A, and the second and third CXL.mem requests include the determined subaddresses for the respective vHDMs B2A and C2A.

[0085] Figures 9 to 11 The logical flow set shown in can represent the example method for executing the novel aspects described in the present disclosure. Although for the purpose of simplifying the description, one or more methods shown in this article are shown and described as a series of actions, it will be understood and understood by those skilled in the art that the method is not limited by the order of actions. According to this, some actions can occur in different orders and / or occur simultaneously with other actions shown and described in this article. For example, it will be understood and understood by those skilled in the art that the method can be alternatively represented as a series of interrelated states or events, such as in a state diagram. In addition, for novel implementations, all actions shown in the method may not be required.

[0086] The logic flow may be implemented in software, firmware, and / or hardware. In software and firmware embodiments, the logic flow may be implemented by computer executable instructions stored on at least one non-transitory computer readable medium or machine readable medium, such as optical, magnetic, or semiconductor memory. The embodiments are not limited in this context.

[0087] Fig.12 An example process 1200 is shown. According to some examples, Figure 1 and Figure 3 Elements of system 100 are shown for describing process 1200. Elements of system 100 may include, but are not limited to, host 105, host link 140, device 130, or HSIL 152. Also, elements of device 130 are shown for describing process 1200, and elements of device 130 may include, but are not limited to, consistency agent 132, MR circuitry 133, MC circuitry 135, or HDM 134. For these examples, process 1200 may be similar to that described above and in Figure 3 100 and 1100, but additionally focuses on the actions taken by device 130-B upon receiving a CXL.mem request message from host 105 (e.g., from host OS 102) to perform a CXL.mem request based on the mapping tables constructed and described above with respect to logic flows 1000 and 1100 and using methods such as those described above with respect to Figure 2 The forwarding table of forwarding table 220 described above is used to access HDM 134-A.

[0088] Process 1200 begins at process 12.1, where host 105 sends a CXL.mem request message to device 130-B. According to some examples, host OS 102 of host 105 may have determined to multiplex multiple CXL.mem requests via host links 140-1 through 140-N to access addresses in HDM 134-A. For these examples, the CXL.mem request includes a translated address (e.g., Addr_vHDM_1) in an address range associated with vHDM-B2A that has been bound to HDM 134-A. Data 2 indicates that device 130-B is processing a CXL.mem request for a portion of the Length associated with the entire memory access.

[0089] At process 12.2, consistency agent 132 receives the CXL.mem request and forwards the translated address in the address range associated with vMDM-B2A.

[0090] At process 12.3, MR circuitry 133-B uses forwarding table 220 maintained in register 138-B to translate Addr_vHDM_1 to the sub-address of HDM 134-A.

[0091] At process 12.4, MR circuitry 133-B forwards a memory request to access the translated sub-address of HDM 134-A associated with the CXL.mem request to device 130-A via HSIL 152-1.

[0092] At process 12.5, the MR circuit system 133-A of device 130-A receives the memory request forwarded from device 130-B, and confirms that the conversion subaddress of HDM 134-A is the address for HDM 134-A, and forwards the memory request to the MC circuit system 135-A for accessing HDM 134-A at the conversion subaddress.

[0093] At process 12.6, MR circuit system 133-A obtains a response from MC circuit system 135-A. In some examples, if the memory request is for writing data 2 to HDM 134-A, the response is an indication of whether data 2 was successfully written to the conversion sub-address of HDM 134-A. If the memory request is for reading data 2 from HDM 134-A, the response may include data 2 read from HDM 134-A.

[0094] At process 12.7, MR circuitry 133-A forwards the response from MC circuitry 135-A to MR circuitry 133-B at device 130-B via HSIL 152-1. MR circuitry 133-B may then translate the sub-address of HDM 134-A back to Addr_vHDM_1 indicated in the CXL.mem request received at device 130-B.

[0095] At process 12.8, MR circuitry 133-A then forwards the response of Addr_vHDM_1 to the CXL.mem request to coherence agent 132-B.

[0096] At process 12.9, coherence agent 132-B sends a response to the CXL.mem request to host 105 via host link 140-2.

[0097] According to some examples, a separate CXL.mem request including a request to access HDM 134-A for portions Data 1 and Data 3 may be sent to device 130-A and device 130-C in parallel with sending a CXL.mem request to access HDM 134-A for portion Data 2, as described above for processes 12.1 through 12.9. Device 130-C will follow a process similar to that described for device 130-B to access its requested portion of Data 3 at HDM 134-a. Device 130-a will directly access HDM 134-A and provide a response directly to host 105 without using HSIL.

[0098] System Overview

[0099] Fig.13 13 is a block diagram of a processing system 1300 according to an embodiment. The processing system 1300 may be used in a single processor desktop system, a multi-processor workstation system, or a server system having a large number of processors 1302 or processor cores 1307. In one embodiment, the processing system 1300 is a processing platform integrated within a system-on-chip (SoC) integrated circuit for use in a mobile, handheld, or embedded device, such as in an Internet of Things (IoT) device with a wired or wireless connection to a local area network or a wide area network.

[0100] In one embodiment, the processing system 1300 may include, be coupled to, or be integrated into: a server-based gaming platform; a gaming console including a gaming and media console; a mobile gaming console, a handheld gaming console, or an online gaming console. In some embodiments, the processing system 1300 is part of a mobile phone, a smart phone, a tablet computing device, or a mobile Internet-connected device such as a laptop computer with low internal storage capacity. The processing system 1300 may also include, be coupled to, or be integrated into: a wearable device such as a smart watch wearable device; smart glasses or clothing enhanced with augmented reality (AR) or virtual reality (VR) features to provide visual, audio, or tactile output to supplement the real-world visual, audio, or tactile experience or otherwise provide text, audio, graphics, video, holographic images, or video or tactile feedback; other augmented reality (AR) devices; or other virtual reality (VR) devices. In some embodiments, the processing system 1300 includes or is part of a television or set-top box device. In one embodiment, processing system 1300 may include, be coupled to, or be integrated within an autonomous vehicle (e.g., a bus, a tractor trailer, an automobile, an electric motor or electric power cycle, an aircraft, or a skid steer (or any combination thereof)). The autonomous vehicle may use processing system 1300 to process the environment sensed around the vehicle.

[0101] In some embodiments, one or more processors 1302 each include one or more processor cores 1307 to process instructions, which when executed perform operations for system or user software. In some embodiments, at least one of the one or more processor cores 1307 is configured to process a specific instruction set 1309. In some embodiments, the instruction set 1309 can facilitate complex instruction set computing (CISC), reduced instruction set computing (RISC) or computing via very long instruction words (VLIW). One or more processor cores 1307 can process different instruction sets 1309, and the instruction set 1309 may include instructions for facilitating the simulation of other instruction sets. The processor core 1307 may also include other processing devices, such as a digital signal processor (DSP).

[0102] In some embodiments, the processor 1302 includes a cache memory 1304. Depending on the architecture, the processor 1302 may have a single internal cache or multiple levels of internal cache. In some embodiments, the cache memory is shared between various components of the processor 1302. In some embodiments, the processor 1302 also uses an external cache (e.g., a level 3 (L3) cache or a last level cache (LLC)) (not shown), which may be shared between the processor cores 1307 using known cache coherence techniques. A register file 1306 may additionally be included in the processor 1302 and may include different types of registers (e.g., integer registers, floating point registers, status registers, and instruction pointer registers) for storing different types of data. Some registers may be general purpose registers, while other registers may be specific to the design of the processor 1302.

[0103] In some embodiments, one or more processors 1302 are coupled to one or more interface buses 1310 to send communication signals, such as address, data, or control signals, between the processor 1302 and other components in the processing system 1300. In one embodiment, the interface bus 1310 may be a processor bus, such as a version of a direct media interface (DMI) bus. However, the processor bus is not limited to a DMI bus, and may include one or more peripheral component interconnect buses (e.g., PCI, PCI Express), memory buses, or other types of interface buses. In one embodiment, the processor 1302 includes a memory controller 1316 and a platform controller hub 1330. The memory controller 1316 facilitates communication between memory devices and other components of the processing system 1300, while the platform controller hub (PCH) 1330 provides connections to I / O devices via a local I / O bus.

[0104] The memory device 1320 may be a dynamic random access memory (DRAM) device, a static random access memory (SRAM) device, a flash memory device, a phase change memory device, or some other memory device with suitable performance to act as a process memory. In one embodiment, the memory device 1320 may operate as a system memory of the processing system 1300 to store data 1322 and instructions 1321 for use when one or more processors 1302 execute an application or process. The memory controller 1316 is also coupled to an optional external graphics processor 1318, which may communicate with one or more graphics processors 1308 in the processor 1302 to perform graphics and media operations. In some embodiments, graphics, media, and / or computing operations may be assisted by an accelerator 1312, which is a coprocessor that may be configured to perform a dedicated set of graphics, media, or computing operations. For example, in one embodiment, the accelerator 1312 is a matrix multiplication accelerator for optimizing machine learning or computing operations. In one embodiment, accelerator 1312 is a ray tracing accelerator that can be used to perform ray tracing operations in conjunction with graphics processor 1308. In one embodiment, external accelerator 1319 can be used in place of accelerator 1312 or in conjunction with accelerator 1312.

[0105] In some embodiments, the display device 1311 may be connected to the processor 1302. The display device 1311 may be one or more display devices such as an internal display device in a mobile electronic device or a laptop device or an external display device attached via a display interface (e.g., DisplayPort, etc.). In one embodiment, the display device 1311 may be a head mounted display (HMD), such as a stereoscopic display device for use in virtual reality (VR) applications or augmented reality (AR) applications.

[0106] In some embodiments, the platform controller hub 1330 enables peripheral devices to be connected to the memory device 1320 and the processor 1302 via a high-speed I / O bus. The I / O peripherals include, but are not limited to, an audio controller 1346, a network controller 1334, a firmware interface 1328, a wireless transceiver 1326, a touch sensor 1325, a data storage device 1324 (e.g., non-volatile memory, volatile memory, hard drive, flash memory, NAND, 3D NAND, 3D XPoint, etc.). The data storage device 1324 can be connected via a storage interface (e.g., SATA) or via a peripheral bus (e.g., PCI, PCI Express). The touch sensor 1325 can include a touch screen sensor, a pressure sensor, or a fingerprint sensor. The wireless transceiver 1326 can be a Wi-Fi transceiver, a Bluetooth transceiver, or a mobile network transceiver, such as a 3G, 4G, 5G, or long term evolution (LTE) transceiver. The firmware interface 1328 enables communication with the system firmware and can be, for example, a unified extensible firmware interface (UEFI). The network controller 1334 can enable network connection to a wired network. In some embodiments, a high-performance network controller (not shown) is coupled to the interface bus 1310. In one embodiment, the audio controller 1346 is a multi-channel high-definition audio controller. In one embodiment, the processing system 1300 includes an optional traditional I / O controller 1340 for coupling traditional (e.g., personal system 2 (PS / 2)) devices to the system. The platform controller hub 1330 can also be connected to one or more universal serial bus (USB) controllers 1342 to connect input devices such as a keyboard and mouse 1343 combination, a camera 1344, or other USB input devices.

[0107] It should be understood that the processing system 1300 shown is exemplary and not limiting, as other types of data processing systems in different configurations may also be used. For example, instances of memory controller 1316 and platform controller hub 1330 may be integrated into a discrete external graphics processor, such as external graphics processor 1318. In one embodiment, platform controller hub 1330 and / or memory controller 1316 may be external to one or more processors 1302 and reside in a system chipset that communicates with processor 1302.

[0108] For example, a circuit board ("sled") on which components such as a CPU, memory, and other components are placed can be used, designed to improve thermal performance. In some examples, processing components such as processors are located on the top side of the sled, while near memory such as DIMMs are located on the bottom side of the sled. Due to the enhanced airflow provided by the design, the components can operate at higher frequencies and power levels than typical systems, thereby improving performance. In addition, the sled is configured to blindly mate with power and data communication cables in the rack, thereby enhancing its ability to be quickly removed, upgraded, reinstalled, and / or replaced. Similarly, the various components located on the sled (such as processors, accelerators, memory, and data storage drives) are configured to be easily upgraded due to their increased spacing from each other. In an illustrative embodiment, the component additionally includes a hardware authentication feature to prove its authenticity.

[0109] The data center can utilize a single network architecture ("fabric") that supports multiple other network architectures including Ethernet and omnipath. The sleds can be coupled to the switches via optical fiber, which provides higher bandwidth and lower latency than typical twisted pair cables (e.g., Category 5, Category 5e, Category 6, etc.). Due to the high-bandwidth, low-latency interconnect and network architecture, the data center can pool resources in use, such as memory, accelerators (e.g., GPUs, graphics accelerators, FPGAs, ASICs, neural network and / or artificial intelligence accelerators, etc.), and physically disaggregated data storage drives, and provide them to computing resources (e.g., processors) as needed, thereby enabling the computing resources to access the pooled resources as if they were local.

[0110] A power supply or source can provide voltage and / or current to the processing system 1300 or any component or system described herein. In one example, the power supply includes an AC to DC (alternating current to direct current) adapter to plug into a wall outlet. Such AC power can be a renewable energy (e.g., solar) power source. In one example, the power supply includes a DC power source, such as an external AC to DC converter. In one example, the power supply or power supply includes wireless charging hardware to charge via proximity to a charging field. In one example, the power supply can include an internal battery, an AC power source, a motion-based power supply, a solar power supply, or a fuel cell source.

[0111] Figures 14A-14D 1 shows a computing system and a graphics processor provided by the embodiments described herein. Elements having the same reference numerals (or names) as elements of any other figure herein Figures 14A-14D The elements of may operate or function in any manner similar to that described elsewhere herein, but are not limited thereto.

[0112] Fig.14A1400 . The processor 1400 may include additional cores up to and including the additional core 1402N represented by the dashed box. Each of the processor cores 1402A-1402N includes one or more internal cache units 1404A-204N. In some embodiments, each processor core may also have access to one or more shared cache units 1406. The internal cache units 1404A-1404N and the shared cache units 1406 represent a cache memory hierarchy within the processor 1400. The cache memory hierarchy may include at least one level of instruction and data cache within each processor core and one or more levels of shared mid-level cache, such as level 2 (L2), level 3 (L3), level 4 (L4), or other levels of cache, where the highest level of cache before external memory is classified as LLC. In some embodiments, cache coherency logic maintains coherency between the various cache units 1406 and 1404A-1404N.

[0113] In some embodiments, the processor 1400 may also include a set of one or more bus controller units 1416 and a system agent core 1410. The one or more bus controller units 1416 manage a set of peripheral buses, such as one or more PCI or PCI Express buses. The system agent core 1410 provides management functions for various processor components. In some embodiments, the system agent core 1410 includes one or more integrated memory controllers 1414 to manage access to various external memory devices (not shown).

[0114] In some embodiments, one or more of the processor cores 1402A-1402N include support for simultaneous multithreading. In such embodiments, the system agent core 1410 includes components for coordinating and operating the cores 1402A-1402N during multithreaded processing. The system agent core 1410 may additionally include a power control unit (PCU) that includes logic and components for regulating the power state of the processor cores 1402A-1402N and the graphics processor 1408.

[0115] In some embodiments, the processor 1400 additionally includes a graphics processor 1408 to perform graphics processing operations. In some embodiments, the graphics processor 1408 is coupled to a set of shared cache units 1406 and a system agent core 1410 including one or more integrated memory controllers 1414. In some embodiments, the system agent core 1410 also includes a display controller 1411 to drive the graphics processor output to one or more coupled displays. In some embodiments, the display controller 1411 can also be a separate module coupled to the graphics processor via at least one interconnect, or can be integrated within the graphics processor 1408.

[0116] In some embodiments, a ring-based interconnect 1412 is used to couple the internal components of the processor 1400. However, alternative interconnect units may be used, such as point-to-point interconnects, switched interconnects, mesh interconnects, or other technologies, including those well known in the art. In some embodiments, the graphics processor 1408 is coupled to the ring-based interconnect 1412 via an I / O link 1413.

[0117] Exemplary I / O links 1413 represent at least one of a variety of I / O interconnects, including an on-package I / O interconnect that facilitates communication between various processor components and high-performance embedded memory modules 1418, such as eDRAM modules or high-bandwidth memory (HBM) modules. In some embodiments, each of processor cores 1402A-1402N and graphics processor 1408 can use embedded memory modules 1418 as a shared last-level cache.

[0118] In some embodiments, the processor cores 1402A-1402N are homogeneous cores that execute the same instruction set architecture. In another embodiment, the processor cores 1402A-1402N are heterogeneous in terms of instruction set architecture (ISA), wherein one or more of the processor cores 1402A-1402N execute a first instruction set, and at least one of the other cores executes a subset of the first instruction set or a different instruction set. In one embodiment, the processor cores 1402A-1402N are heterogeneous in terms of microarchitecture, wherein one or more cores with relatively high power consumption are coupled with one or more power cores with lower power consumption. In one embodiment, the processor cores 1402A-1402N are heterogeneous in terms of computing power. Additionally, in addition to other components, the processor 1400 can also be implemented on one or more chips or as a SoC integrated circuit with the components shown.

[0119] Figure 2B is a block diagram of the hardware logic of the graphics processor core block 1419 according to some embodiments described herein. In some embodiments, elements having the same reference numerals (or names) as elements of any other figure herein Fig. 14B The elements of may operate or function in a manner similar to that described elsewhere herein. Graphics processor core block 1419 is an example of a partition of a graphics processor. Graphics processor core block 1419 may be included in Fig.14A The graphics processor 1408 or discrete graphics processor, parallel processor and / or computing accelerator of the integrated graphics processor 1408 may include a plurality of graphics core blocks based on a target power and performance envelope. Each graphics processor core block 1419 may include a functional block 1430 coupled to a plurality of graphics cores 1421A-1421F, and the graphics cores 1421A-1421F include modular blocks of fixed function logic and general programmable logic. The graphics processor core block 1419 also includes a shared / cache memory 1436 accessible by all graphics cores 1421A-1421F, rasterizer logic 1437, and additional fixed function logic 1438.

[0120] In some embodiments, functional block 1430 includes a geometry / fixed function pipeline 1431 that can be shared by all graphics cores in graphics processor core block 1419. In various embodiments, geometry / fixed function pipeline 1431 includes a 3D geometry pipeline, a video front end unit, a thread generator and a global thread dispatcher, and a unified return buffer manager that manages a unified return buffer. In one embodiment, functional block 1430 also includes a graphics SoC interface 1432, a graphics microcontroller 1433, and a media pipeline 1434. Graphics SoC interface 1432 provides an interface between graphics processor core block 1419 and other core blocks within a graphics processor or computing accelerator SoC. Graphics microcontroller 1433 is a programmable subprocessor that can be configured to manage various functions of graphics processor core block 1419, including thread dispatching, scheduling, and preemption. Media pipeline 1434 includes logic for facilitating decoding, encoding, preprocessing, and / or post-processing of multimedia data including image and video data. The media pipeline 1434 implements media operations via requests to computational or sampling logic within the graphics cores 1421-1421F. One or more pixel backends 1435 may also be included within the functional block 1430. The pixel backend 1435 includes a cache memory to store pixel color values ​​and may perform blending operations and lossless color compression on rendered pixel data.

[0121] In one embodiment, the graphics SoC interface 1432 enables the graphics processor core block 1419 to communicate with a general-purpose application processor core (e.g., CPU) and / or other components within the SoC or a system host CPU coupled to the SoC via a peripheral interface. The graphics SoC interface 1432 also enables communication with off-chip memory hierarchy elements (such as shared last-level cache memory, system RAM, and / or embedded on-chip or on-package DRAM). The SoC interface 1432 may also enable communication with fixed-function devices within the SoC (e.g., camera imaging pipelines) and enable the use and / or implementation of global memory atoms that can be shared between the graphics processor core block 1419 and the CPU within the SoC. The graphics SoC interface 1432 may also implement power management controls for the graphics processor core block 1419 and implement an interface between the clock domain of the graphics processor core block 1419 and other clock domains within the SoC. In one embodiment, graphics SoC interface 1432 implements reception of command buffers from a command stream converter and a global thread dispatcher, which are configured to provide commands and instructions to each of one or more graphics cores within a graphics processor. When media operations are to be performed, commands and instructions may be dispatched to media pipeline 1434, and when graphics processing operations are to be performed, commands and instructions may be dispatched to geometry and fixed function pipeline 1431. When compute operations are to be performed, compute dispatch logic may dispatch commands to graphics cores 1421A-1421F, bypassing the geometry and media pipelines.

[0122] The graphics microcontroller 1433 may be configured to perform various scheduling and management tasks for the graphics processor core block 1419. In one embodiment, the graphics microcontroller 1433 may perform graphics and / or computational workload scheduling for the various vector engines 1422A-1422F, 1424A-1424F and matrix engines 1423A-1423F, 1425A-1425F within the graphics cores 1421A-1421F. In this scheduling model, host software executing on a CPU core of a SoC including the graphics processor core block 1419 may submit a workload to one of a plurality of graphics processor doorbells, which invokes scheduling operations on the appropriate graphics engine. Scheduling operations include determining which workload to run next, submitting a workload to a command stream converter, preempting an existing workload running on an engine, monitoring the progress of a workload, and notifying the host software when a workload is completed. In one embodiment, the graphics microcontroller 1433 may also facilitate a low power or idle state for the graphics processor core block 1419, thereby providing the graphics processor core block 1419 with the ability to save and restore registers within the graphics processor core block 1419 across low power state transitions independent of the operating system and / or graphics driver software on the system.

[0123] The graphics processor core block 1419 may have more or fewer graphics cores 1421A-1421F than shown, up to N modular graphics cores. For each group of N graphics cores, the graphics processor core block 1419 may also include a shared / cache memory 1436 (which may be configured as a shared memory or a cache memory), a rasterizer logic 1437, and additional fixed function logic 1438 to accelerate various graphics and compute processing operations.

[0124] Within each graphics core 1421A-1421F is a collection of execution resources that can be used to perform graphics, media, and compute operations in response to requests from a graphics pipeline, a media pipeline, or a shader program. Graphics cores 1421A-1421F include multiple vector engines 1422A-1422F, 1424A-1424F, matrix acceleration units 1423A-1423F, 1425A-1425D, cache / shared local memory (SLM), samplers 1426A-1426F, and ray tracing units 1427A-1427F.

[0125] The vector engines 1422A-1422F, 1424A-1424F are general purpose graphics processing units capable of performing floating point and integer / fixed point logic operations in the service of graphics, media, or compute operations, including graphics, media, or compute / GPGPU programs. The vector engines 1422A-1422F, 1424A-1424F can operate with variable vector widths using SIMD, SIMT, or SIMT+SIMD execution modes. The matrix acceleration units 1423A-1423F, 1425A-1425D include matrix-matrix and matrix-vector acceleration logic that improves the performance of matrix operations, particularly low-precision and mixed-precision (e.g., INT8, FP16, BF16) matrix operations for machine learning. In one embodiment, each of the matrix acceleration units 1423A-1423F, 1425A-1425D includes one or more systolic arrays of processing elements that can perform concurrent matrix multiplication or dot product operations on matrix elements.

[0126] Samplers 1426A-1426F can read media or texture data into memory and can sample data differently based on the configured sampler state and the texture / media format being read. Threads executing on vector engines 1422A-1422F, 1424A-1424F or matrix acceleration units 1423A-1423F, 1425A-1425D can utilize caches / SLMs 1428A-1428F within each execution core. Caches / SLMs 1428A-1428F can be configured as a cache memory or a shared memory pool local to each graphics core in the corresponding graphics cores 1421A-1421F. Ray tracing units 1427A-1427F within graphics cores 1421A-1421F include ray traversal / intersection circuitry for performing ray traversals using a bounding volume hierarchy (BVH) and identifying intersections between rays and primitives bounded within the BVH. In one embodiment, ray tracing units 1427A-1427F include circuitry for performing depth testing and culling (e.g., using a depth buffer or similar arrangement). In one implementation, ray tracing units 1427A-1427F perform traversal and intersection operations in conjunction with image denoising, at least a portion of which may be performed using associated matrix acceleration units 1423A-1423F, 1425A-1425D.

[0127] Fig. 14CGraphics processing unit (GPU) 1439 is shown, which includes a set of dedicated graphics processing resources arranged into multi-core groups 1440A-1440N. Details of multi-core group 1440A are shown. Multi-core groups 1440B-1440N may be equipped with the same or similar set of graphics processing resources.

[0128] As shown, multi-core group 1440A may include a set of graphics cores 1443, a set of tensor cores 1444, and a set of ray tracing cores 1445. Scheduler / dispatcher 1441 schedules and dispatches graphics threads to execute on the various cores 1443, 1444, 1445. In one embodiment, tensor core 1444 is a sparse tensor core with hardware to enable bypassing of multiplication operations with zero-valued inputs. Fig. 14C GPU 1439 graphics core 1443 relative to Fig. 14B The graphics cores 1421A-1421F differ in the hierarchical abstraction level, which is similar to Fig. 14C Multi-core group 1440A-1440N. Fig. 14C The graphics core 1443, tensor core 1444 and ray tracing core 1445 are similar to Fig. 14B Vector engines 1422A-1422F, 1424A-1424F, matrix engines 1423A-1423F, 1425A-1425F and ray tracing units 1427A-1427F.

[0129] A set of register files 1442 may store operand values ​​used by cores 1443, 1444, 1445 when executing graphics threads. These may include, for example, integer registers for storing integer values, floating point registers for storing floating point values, vector registers for storing packed data elements (integer and / or floating point data elements), and tile registers for storing tensor / matrix values. In one embodiment, the tile registers are implemented as a combined set of vector registers.

[0130] One or more combined level 1 (L1) caches and shared memory units 1447 store graphics data locally within each multi-core group 1440A, such as texture data, vertex data, pixel data, light data, bounding volume data, etc. One or more texture units 1447 can also be used to perform texturing operations, such as texture mapping and sampling. A level 2 (L2) cache 1453 shared by all or a subset of the multi-core groups 1440A-1440N stores graphics data and / or instructions for multiple concurrent graphics threads. As shown, the L2 cache 1453 can be shared across multiple multi-core groups 1440A-1440N. One or more memory controllers 1448 couple the GPU 1439 to a memory 1449, which can be system memory (e.g., DRAM) and / or dedicated graphics memory (e.g., GDDR6 memory).

[0131] Input / output (I / O) circuitry 1450 couples GPU 1439 to one or more I / O devices 1452, such as a digital signal processor (DSP), a network controller, or a user input device. On-chip interconnects may be used to couple I / O devices 1452 to GPU 1439 and memory 1449. One or more I / O memory management units (IOMMUs) 1451 of I / O circuitry 1450 couple I / O devices 1452 directly to memory 1449. In one embodiment, IOMMU 1451 manages multiple sets of page tables to map virtual addresses to physical addresses in memory 1449. In this embodiment, I / O devices 1452, CPU 1446, and GPU 1439 may share the same virtual address space.

[0132] In one implementation, IOMMU 1451 supports virtualization. In this case, it may manage a first set of page tables to map guest / graphics virtual addresses to guest / graphics physical addresses, and a second set of page tables to map guest / graphics physical addresses to system / host physical addresses (e.g., within memory 1449). The base address of each of the first set of page tables and the second set of page tables may be stored in a control register and swapped out on a context switch (e.g., to provide access to the relevant set of page tables for a new context). Although in Fig. 14C Not shown, but each of the cores 1443, 1444, 1445 and / or multi-core groups 1440A-1440N may include a translation lookaside buffer (TLB) to cache guest virtual to guest physical translations, guest physical to host physical translations, and guest virtual to host physical translations.

[0133] In one embodiment, CPU 1446, GPU 1439, and I / O devices 1452 are integrated on a single semiconductor chip and / or chip package. Memory 1449 may be integrated on the same chip, or may be coupled to memory controller 1448 via an off-chip interface. In one implementation, memory 1449 includes GDDR6 memory that shares the same virtual address space with other physical system-level memory, although the underlying principles of the embodiments described herein are not limited to this particular implementation.

[0134] In one embodiment, the tensor core 1444 includes multiple functional units specifically designed to perform matrix operations, which are basic computational operations used to perform deep learning operations. For example, simultaneous matrix multiplication operations can be used for neural network training and reasoning. The tensor core 1444 can perform matrix processing using a variety of operand precisions, including single-precision floating point (e.g., 32 bits), half-precision floating point (e.g., 16 bits), integer words (16 bits), bytes (8 bits), and half bytes (4 bits). In one embodiment, a neural network implementation extracts features of each rendered scene, potentially combining details from multiple frames to construct a high-quality final image.

[0135] In a deep learning implementation, parallel matrix multiplication work may be scheduled to execute on the tensor core 1444. In particular, the training of neural networks requires a large number of matrix dot product operations. To process the inner product formula of the N×N×N matrix multiplication, the tensor core 1444 may include at least N dot product processing elements. Before the matrix multiplication begins, each of the N cycles, a full matrix is ​​loaded into the tile register and at least one column of the second matrix is ​​loaded. Each cycle, there are N dot products being processed.

[0136] Matrix elements may be stored at different precisions, including 16-bit words, 8-bit bytes (e.g., INT8), and 4-bit nibbles (e.g., INT4), depending on the specific implementation. Different precision modes may be specified for the tensor cores 1444 to ensure the most efficient precision for different workloads (e.g., such as inference workloads that can tolerate quantization to bytes and nibbles).

[0137] In one embodiment, the ray tracing core 1445 accelerates ray tracing operations for both real-time ray tracing and non-real-time ray tracing implementations. Specifically, the ray tracing core 1445 includes a ray traversal / intersection circuit system for performing ray traversals using a bounding volume hierarchy (BVH) and identifying intersections between rays and primitives enclosed within the BVH. The ray tracing core 1445 may also include circuit systems for performing depth testing and culling (e.g., using a Z buffer or similar arrangement). In one implementation, the ray tracing core 1445 performs traversal and intersection operations in conjunction with the image denoising techniques described herein, at least a portion of which may be executed on the tensor core 1444. For example, in one embodiment, the tensor core 1444 implements a deep learning neural network to perform denoising on frames generated by the ray tracing core 1445. However, the CPU 1446, the graphics core 1443, and / or the ray tracing core 1445 may also implement all or part of the denoising and / or deep learning algorithms.

[0138] Additionally, as described above, a distributed denoising approach may be employed, wherein the GPU 1439 is in a computing device coupled to other computing devices via a network or high-speed interconnect. In this embodiment, the interconnected computing devices share neural network learning / training data to improve the speed at which the entire system learns to perform denoising for different types of image frames and / or different graphics applications.

[0139] In one embodiment, ray tracing core 1445 handles all BVH traversals and ray-primitive intersections so that graphics core 1443 is not overloaded with thousands of instructions per ray. In one embodiment, each ray tracing core 1445 includes a first set of dedicated circuit systems for performing bounding box tests (e.g., for traversal operations) and a second set of dedicated circuit systems for performing ray-triangle intersection tests (e.g., intersecting rays that have been traversed). Thus, in one embodiment, multi-core group 1440A can simply launch ray probes, and ray tracing core 1445 independently performs ray traversals and intersections and returns hit data (e.g., hits, no hits, multiple hits, etc.) to the thread context. The other cores 1443, 1444 are freed up to perform other graphics or compute work while ray tracing core 1445 performs traversal and intersection operations.

[0140] In one embodiment, each ray tracing core 1445 includes a traversal unit for performing BVH test operations and an intersection unit for performing ray-primitive intersection tests. The intersection unit generates a "hit", "no hit", or "multiple hits" response, which it provides to the appropriate thread. During traversal and intersection operations, execution resources of other cores (e.g., graphics core 1443 and tensor core 1444) are freed up to perform other forms of graphics work.

[0141] In one particular embodiment described below, a hybrid rasterization / ray tracing approach is used where the work is distributed between graphics core 1443 and ray tracing core 1445 .

[0142] In one embodiment, the ray tracing core 1445 (and / or other cores 1443, 1444) includes hardware support for a ray tracing instruction set (such as Microsoft's DirectX Ray Tracing (DXR)), which includes the DispatchRays command and ray generation, nearest hit, any hit, and miss shaders, which enables a unique set of shaders and textures to be assigned to each object. Another ray tracing platform that can be supported by the ray tracing core 1445, graphics core 1443, and tensor core 1444 is Vulkan 1.1.85. However, note that the underlying principles of the embodiments described herein are not limited to any particular ray tracing ISA.

[0143] In general, the various cores 1445, 1444, 1443 may support a ray tracing instruction set including instructions / functions for ray generation, nearest hit, any hit, ray-primitive intersection, per-primitive and hierarchical bounding box construction, misses, accesses, and exceptions. More specifically, one embodiment includes ray tracing instructions for performing the following functions:

[0144] Ray Generation - Ray generation instructions can be executed per pixel, sample, or other user-defined work assignment.

[0145] Closest Hit - A closest hit instruction may be executed to locate the closest intersection point of a ray with a primitive within the scene.

[0146] Any Hit - The Any Hit instruction identifies multiple intersections between a ray and primitives within the scene, potentially identifying a new closest intersection point.

[0147] The Intersect-Intersect instructions perform ray-primitive intersection tests and output the results.

[0148] Per-primitive bounding box construction - This instruction builds a bounding box around a given primitive or group of primitives (e.g., when building a new BVH or other acceleration data structure).

[0149] Miss - Indicates that the ray missed the scene or all geometry within a specified area of ​​the scene.

[0150] Visits - Indicates the sub-volumes that the ray will traverse.

[0151] Exceptions - includes various types of exception handlers (eg, called for various error conditions).

[0152] In one embodiment, the ray tracing core 1445 can be adapted to accelerate general computing operations, which can be accelerated using computing techniques similar to ray intersection tests. A computing framework can be provided that enables shader programs to be compiled into low-level instructions and / or primitives that perform general computing operations via the ray tracing core. Exemplary computing problems that can benefit from computing operations performed on the ray tracing core 1445 include calculations involving beams, waves, light or particle propagation in a coordinate space. Interactions associated with the propagation can be calculated relative to geometric shapes or grids in the coordinate space. For example, calculations associated with the propagation of electromagnetic signals through the environment can be accelerated using instructions or primitives executed via the ray tracing core. The diffraction and reflection of the signal by objects in the environment can be calculated as a direct ray tracing simulation.

[0153] The ray tracing core 1445 can also be used to perform calculations that are not directly similar to ray tracing. For example, the ray tracing core 1445 can be used to accelerate mesh projection, mesh refinement, and volume sampling calculations. General coordinate space calculations, such as nearest neighbor calculations, can also be performed. For example, a set of points near a given point can be found by defining a bounding box around the given point in the coordinate space. Then, the BVH and ray detection logic within the ray tracing core 1445 can be used to determine the intersection set of points within the bounding box. The intersection constitutes the origin and the nearest neighbor point of the origin. The calculations performed using the ray tracing core 1445 can be performed in parallel with the calculations performed on the graphics core 1443 and the tensor core 1444. The shader compiler can be configured to compile a compute shader or other general graphics processing program into a low-level primitive that can be parallelized across the graphics core 1443, the tensor core 1444, and the ray tracing core 1445.

[0154] Fig.14D1470 is a block diagram of a general purpose graphics processing unit (GPGPU) 1470 that can be configured as a graphics processor and / or a computing accelerator according to embodiments described herein. The GPGPU 1470 can be interconnected with a host processor (e.g., one or more CPUs 1446) and memories 1471, 1472 via one or more system and / or memory buses. In one embodiment, memory 1471 is system memory that can be shared with one or more CPUs 1446, while memory 1472 is device memory dedicated to the GPGPU 1470. In one embodiment, components within the GPGPU 1470 and memory 1472 can be mapped to memory addresses accessible to one or more CPUs 1446. Access to memories 1471 and 1472 can be facilitated via a memory controller 1468. In one embodiment, the memory controller 1468 includes an internal direct memory access (DMA) controller 1469 or can include logic to perform operations that would otherwise be performed by a DMA controller.

[0155] GPGPU 1470 includes a plurality of cache memories, including L2 cache 1453, L1 cache 1454, instruction cache 1455, and shared memory 1456, at least a portion of which may also be divided into cache memories. GPGPU 1470 also includes a plurality of computing units 1460A-1460N, which represent similar Fig. 14B Graphics core 1421A-1421F and Fig. 14C The hierarchical abstraction level of the multi-core group 1440A-1440N of the GPGPU 1470 is shown. Each compute unit 1460A-1460N includes a set of vector registers 1461, scalar registers 1462, vector logic units 1463, and scalar logic units 1464. The compute units 1460A-1460N may also include a local shared memory 1465 and a program counter 1466. The compute units 1460A-1460N may be coupled with a constant cache 1467, which may be used to store constant data, which is data that does not change during the execution of a kernel or shader program executed on the GPGPU 1470. In one embodiment, the constant cache 1467 is a scalar data cache, and the cached data may be directly fetched into the scalar registers 1462.

[0156] During operation, one or more CPUs 1446 may write commands to registers or memories in GPGPU 1470 that have been mapped into an accessible address space. Command processor 1457 may read commands from registers or memories and determine how to process those commands within GPGPU 1470. Thread dispatcher 1458 may then be used to dispatch threads to computing units 1460A-1460N to execute those commands. Each computing unit 1460A-1460N may execute threads independently of other computing units. Additionally, each computing unit 1460A-1460N may be independently configured for conditional calculations, and may conditionally output calculation results to memory. Command processor 1457 may interrupt one or more CPUs 1446 when the submitted commands are completed.

[0157] Figure 15A-Figure 15C A block diagram of an additional graphics processor and computing accelerator architecture provided by embodiments described herein is shown. Figure 15A-Figure 15C The elements of may operate or function in any manner similar to that described elsewhere herein, but are not limited thereto.

[0158] Fig.15A 1 is a block diagram of a graphics processor 1500, which may be a discrete graphics processing unit, or may be a graphics processor integrated with multiple processing cores or other semiconductor devices (such as, but not limited to, memory devices or network interfaces). In some embodiments, the graphics processor communicates with registers on the graphics processor via a memory-mapped I / O interface and communicates with commands placed in processor memory. In some embodiments, the graphics processor 1500 includes a memory interface 1514 for accessing memory. The memory interface 1514 may be an interface to local memory, one or more internal caches, one or more shared external caches, and / or to system memory.

[0159] In some embodiments, the graphics processor 1500 also includes a display controller 1502 to drive the display output data to a display device 1518. The display controller 1502 includes hardware for displaying and composing one or more overlay planes of multi-layer video or user interface elements. The display device 1518 can be an internal or external display device. In one embodiment, the display device 1518 is a head-mounted display device, such as a virtual reality (VR) display device or an augmented reality (AR) display device. In some embodiments, the graphics processor 1500 includes a video codec engine 1506 for encoding media into one or more media encoding formats, decoding from one or more media encoding formats, or transcoding between one or more media encoding formats, including but not limited to a moving picture experts group (MPEG) format (such as MPEG-2), an advanced video coding (AVC) format (such as H.264 / MPEG-4AVC, H.265 / HEVC), an open media alliance (AOMedia) VP8, VP9, ​​and a society of film and television engineers (SMPTE) 421M / VC-1, and a joint picture experts group (JPEG) format (such as JPEG and motion JPEG (MJPEG) format).

[0160] In some embodiments, graphics processor 1500 includes a block image transfer (BLIT) engine 1504 to perform two-dimensional (2D) rasterizer operations, including, for example, bit-boundary block transfers. However, in one embodiment, 2D graphics operations are performed using one or more components of a graphics processing engine (GPE) 1510. In some embodiments, GPE 1510 is a compute engine for performing graphics operations, including three-dimensional (3D) graphics operations and media operations.

[0161] In some embodiments, GPE 1510 includes a 3D pipeline 1512 for performing 3D operations, such as rendering three-dimensional images and scenes using processing functions that act on 3D primitive shapes (e.g., rectangles, triangles, etc.). 3D pipeline 1512 includes programmable and fixed function elements that perform various tasks within the element and / or spawn execution threads to 3D / media subsystem 1515. While 3D pipeline 1512 can be used to perform media operations, embodiments of GPE 1510 also include a media pipeline 1516 specifically for performing media operations, such as video post-processing and image enhancement.

[0162] In some embodiments, the media pipeline 1516 includes fixed-function or programmable logic units to perform one or more specialized media operations, such as video decoding acceleration, video deinterlacing, and video encoding acceleration, instead of or on behalf of the video codec engine 1506. In some embodiments, the media pipeline 1516 additionally includes a thread generation unit to generate threads for execution on the 3D / media subsystem 1515. The generated threads perform calculations for media operations on one or more graphics cores included in the 3D / media subsystem 1515.

[0163] In some embodiments, 3D / media subsystem 1515 includes logic for executing threads generated by 3D pipeline 1512 and media pipeline 1516. In one embodiment, the pipeline sends thread execution requests to 3D / media subsystem 1515, which includes thread dispatch logic for arbitrating various requests and dispatching various requests to available thread execution resources. The execution resources include an array of graphics cores for processing 3D and media threads. In some embodiments, 3D / media subsystem 1515 includes one or more internal caches for thread instructions and data. In some embodiments, the subsystem also includes shared memory, including registers and addressable memory, to share data between threads and store output data.

[0164] Fig. 15B A graphics processor 1520 having a tiled architecture is shown in accordance with embodiments described herein. In one embodiment, the graphics processor 1520 includes a graphics processing engine cluster 1522 having graphics engine tiles 1510A-1510D. Fig.15A1523A-1523F. Each graphics engine tile 1510A-1510D may also be connected to a memory module or memory device 1526A-1526D via a memory interconnect 1525A-1525D. The memory devices 1526A-1526D may use any graphics memory technology. For example, the memory devices 1526A-1526D may be graphics double data rate (GDDR) memory. In one embodiment, the memory devices 1526A-1526D are HBM modules that may be on a die with their corresponding graphics engine tiles 1510A-1510D. In one embodiment, the memory devices 1526A-1526D are stacked memory devices that may be stacked on top of their corresponding graphics engine tiles 1510A-1510D. In one embodiment, each graphics engine tile 1510A-1510D and associated memory 1526A-1526D resides on a separate chiplet that is bonded to a base die or base substrate, such as in Figure 23B-Figure 23D As described in further detail in .

[0165] The graphics processor 1520 may be configured with a non-uniform memory access (NUMA) system in which memory devices 1526A-1526D are coupled to associated graphics engine tiles 1510A-1510D. A given memory device may be accessed by graphics engine tiles other than the tile to which it is directly connected. However, when accessing the local tile, access latency to the memory devices 1526A-1526D may be minimal. In one embodiment, a cache-consistent NUMA (ccNUMA) system is enabled, and the system uses tile interconnects 1523A-1523F to enable communication between cache controllers within the graphics engine tiles 1510A-1510D to maintain a consistent memory image when more than one cache stores the same memory location.

[0166] The graphics processing engine cluster 1522 may be connected to an on-chip or on-package fabric interconnect 1524. In one embodiment, the fabric interconnect 1524 includes a network processor, a network on chip (NoC), or another switch processor to enable the fabric interconnect 1524 to function as a packet-switched fabric interconnect for exchanging data packets between components of the graphics processor 1520. The fabric interconnect 1524 may enable communication between the graphics engine tiles 1510A-1510D and components such as the video codec engine 1506 and one or more replication engines 1504. The replication engines 1504 may be used to move data to, from, and between the memory devices 1526A-1526D and memory external to the graphics processor 1520 (e.g., system memory). The fabric interconnect 1524 may also be coupled to one or more of the tile interconnects 1523A-1523F to facilitate or enhance interconnection between the graphics engine tiles 1510A-1510D. The fabric interconnect 1524 may also be configured to interconnect multiple instances of the graphics processor 1520 (e.g., via the host interface 1528) to enable tile-to-tile communication between the graphics engine tiles 1510A-1510D of multiple GPUs. In one embodiment, the graphics engine tiles 1510A-1510D of multiple GPUs may be presented to the host system as a single logical device.

[0167] Graphics processor 1520 may optionally include display controller 1502 to enable connection to display device 1518. Graphics processor may also be configured as a graphics or computing accelerator. In an accelerator configuration, display controller 1502 and display device 1518 may be omitted.

[0168] Graphics processor 1520 may be connected to a host system via host interface 1528. Host interface 1528 may enable communication between graphics processor 1520, system memory, and / or other system components. Host interface 1528 may be, for example, a PCI Express bus or another type of host system interface. For example, host interface 1528 may be an NVLink or NVSwitch interface. Host interface 1528 and fabric interconnect 1524 may collaborate to enable multiple instances of graphics processor 1520 to function as a single logical device. The collaboration between host interface 1528 and fabric interconnect 1524 may also enable individual graphics engine tiles 1510A-1510D to be presented to the host system as different logical graphics devices.

[0169] Fig. 15C FIG. 1 shows a computing accelerator 1530 according to an embodiment described herein. The computing accelerator 1530 may include Fig. 15BThe computing engine cluster 1532 may include a set of computing engine tiles 1540A-1540D that include execution logic optimized for parallel or vector-based general-purpose computing operations. In some embodiments, the computing engine tiles 1540A-1540D do not include fixed-function graphics processing logic, although in one embodiment, one or more of the computing engine tiles 1540A-1540D may include logic for performing media acceleration. The computing engine tiles 1540A-1540D may be connected to memory 1526A-1526D via memory interconnects 1525A-1525D. The memories 1526A-1526D and the memory interconnects 1525A-1525D may be similar technologies as in the graphics processor 1520, or may be different technologies. The graphics compute engine tiles 1540A-1540D may also be interconnected via a set of tile interconnects 1523A-1523F and may be connected to and / or interconnected through a fabric interconnect 1524. Cross-tile communications may be facilitated via the fabric interconnect 1524. The fabric interconnect 1524 may also facilitate communications between the compute engine tiles 1540A-1540D of multiple instances of a compute accelerator 1530 (e.g., via a host interface 1528). In one embodiment, the compute accelerator 1530 includes a large L3 cache 1536 that may be configured as a device-wide cache. The compute accelerator 1530 may also communicate with the host computer 1528 to communicate with the host computer 1528. Fig. 15B Graphics processor 1520 is similarly connected to a host processor and memory via host interface 1528 .

[0170] The computing accelerator 1530 may also include an integrated network interface 1542. In one embodiment, the network interface 1542 includes a network processor and controller logic that enables the computing engine cluster 1532 to communicate via the physical layer interconnect 1544 without requiring the data to pass through the memory of the host system. In one embodiment, one of the computing engine tiles 1540A-1540D is replaced by the network processor logic, and the data to be sent or received via the physical layer interconnect 1544 can be directly sent to or from the memory 1526A-1526D. Multiple instances of the computing accelerator 1530 can be joined into a single logical device via the physical layer interconnect 1544. Alternatively, the various computing engine tiles 1540A-1540D can be presented as different network accessible computing accelerator devices.

[0171] Graphics processing engine

[0172] Fig.16is a block diagram of a graphics processing engine 1610 of a graphics processor according to some embodiments. In one embodiment, the graphics processing engine (GPE) 1610 is Fig.15A The version of the GPE 1510 shown in the figure can also represent Fig. 15B Graphics engine tiles 1510A-1510D of FIG. Elements having the same reference numerals (or names) as elements of any other drawings herein Fig.16 The elements of can operate or function in any manner similar to that described elsewhere herein, but are not limited thereto. For example, Fig.15A 1610. The 3D pipeline 1512 and the media pipeline 1516 of the GPE 1610. The media pipeline 1516 is optional in some embodiments of the GPE 1610 and may not be explicitly included in the GPE 1610. For example and in at least one embodiment, a separate media and / or image processor is coupled to the GPE 1610.

[0173] In some embodiments, GPE 1610 is coupled to or includes a command stream converter 1603, which provides a command stream to 3D pipeline 1512 and / or media pipeline 1516. Alternatively or additionally, command stream converter 1603 may be directly coupled to unified return buffer 1618. Unified return buffer 1618 may be communicatively coupled to graphics core cluster 1614. In some embodiments, command stream converter 1603 is coupled to a memory, which may be system memory, or one or more of an internal cache memory and a shared cache memory. In some embodiments, command stream converter 1603 receives commands from memory and sends the commands to 3D pipeline 1512 and / or media pipeline 1516. The commands are instructions obtained from a ring buffer storing commands for 3D pipeline 1512 and media pipeline 1516. In one embodiment, the ring buffer may additionally include a batch command buffer storing multiple batches of multiple commands. Commands for 3D pipeline 1512 may also include references to data stored in memory, such as, but not limited to, vertex and geometry data for 3D pipeline 1512 and / or image data and memory objects for media pipeline 1516. 3D pipeline 1512 and media pipeline 1516 process commands and data by performing operations via logic within the respective pipelines or by dispatching one or more execution threads to graphics core cluster 1614. In one embodiment, graphics core cluster 1614 includes one or more graphics core blocks (e.g., graphics core block 1615A, graphics core block 1615B), each block including one or more graphics cores. Each graphics core includes a set of graphics execution resources, which includes general and graphics-specific execution logic for performing graphics and compute operations, as well as fixed-function texture processing and / or machine learning and artificial intelligence acceleration logic, such as matrix or AI acceleration logic.

[0174] In various embodiments, the 3D pipeline 1512 may include fixed function and programmable logic to process one or more shader programs, such as vertex shaders, geometry shaders, pixel shaders, fragment shaders, compute shaders, or other shaders and / or GPGPU programs, by processing instructions and dispatching execution threads to the graphics core cluster 1614. The graphics core cluster 1614 provides a unified execution resource block for processing these shader programs. The multi-purpose execution logic within the graphics core blocks 1615A-1615B of the graphics core cluster 1614 includes support for various 3D API shader languages ​​and can execute multiple simultaneous execution threads associated with multiple shaders.

[0175] In some embodiments, graphics core cluster 1614 includes execution logic to perform media functions, such as video and / or image processing. In one embodiment, the graphics core includes general purpose logic that can be programmed to perform parallel general purpose computing operations in addition to graphics processing operations. The general purpose logic can be executed in parallel or in combination with Fig.13 Processor core 1307 or Fig.14A The processing operations are performed by the general logic within the cores 1402A-1402N.

[0176] Output data generated by threads executing on graphics core cluster 1614 can output data to memory in unified return buffer (URB) 1618. URB 1618 can store data for multiple threads. In some embodiments, URB 1618 can be used to send data between different threads executing on graphics core cluster 1614. In some embodiments, URB 1618 can additionally be used for synchronization between threads on the graphics core array and fixed function logic within shared function logic 1620.

[0177] In some embodiments, graphics core cluster 1614 is scalable such that a cluster includes a variable number of graphics cores, each having a variable number of graphics cores based on the target power and performance level of GPE 1610. In one embodiment, execution resources are dynamically scalable such that execution resources can be enabled or disabled as needed.

[0178] Graphics core cluster 1614 is coupled to shared function logic 1620, which includes a plurality of resources shared between graphics cores in the graphics core array. The shared functions within shared function logic 1620 are hardware logic units that provide dedicated supplemental functions to graphics core cluster 1614. In various embodiments, shared function logic 1620 may include, but is not limited to, sampler 1621, math 1622, and inter-thread communication (ITC) 1623 logic. Additionally, some embodiments implement one or more caches 1625 within shared function logic 1620. Shared function logic 1620 may implement the same or similar functionality as additional fixed function logic 11638 of FIG. 116B.

[0179] Shared functionality is implemented at least where the demand for a given dedicated functionality is insufficient to include within graphics core cluster 1614. Instead, a single instantiation of the dedicated functionality is implemented as an independent entity in shared functionality logic 1620 and shared between execution resources within graphics core cluster 1614. The exact set of functionality shared between graphics core clusters 1614 and included within graphics core cluster 1614 varies across embodiments. In some embodiments, specific shared functionality within shared functionality logic 1620 that is widely used by graphics core cluster 1614 may be included within shared functionality logic 1616 within graphics core cluster 1614. In various embodiments, shared functionality logic 1616 within graphics core cluster 1614 may include some or all of the logic within shared functionality logic 1620. In one embodiment, all logic elements within shared functionality logic 1620 may be replicated within shared functionality logic 1616 of graphics core cluster 1614. In one embodiment, shared functionality logic 1620 is excluded in favor of shared functionality logic 1616 within graphics core cluster 1614.

[0180] Graphics processing resources

[0181] Figure 17A-Figure 17C Execution logic including an array of processing elements employed in a graphics processor is shown according to embodiments described herein. Fig.17A A graphics core cluster is shown according to an embodiment. Fig. 17B A vector engine of a graphics core is shown according to an embodiment. Fig. 17C FIG. 1 shows a matrix engine of a graphics core according to an embodiment. FIG. 10 has the same reference numerals as elements of any other figure herein. Figure 17A-Figure 17C The elements of may operate or function in any manner similar to, but not limited to, any manner described elsewhere herein. Fig. 14B The graphics processor core block 1419 and / or Fig.16 Considered in the context of graphics core blocks 1615A-1615B Figure 17A-Figure 17C In one embodiment, Figure 17A-Figure 17C The components have Fig.14A Graphics processor 1408, Fig. 14C GPU 1439 or Fig.14D The GPGPU 1470 is an equivalent component with similar functionality.

[0182] like Fig.17A As shown, in one embodiment, the graphics core cluster 1614 includes a graphics core block 1615, which may be Fig.16Graphics core block 1615 may include any number of graphics cores (e.g., graphics core 1715A, graphics core 1715B, through graphics core 1715N). Multiple instances of graphics core block 1615 may be included. In one embodiment, the components of graphics cores 1715A-1715N have the same Fig. 14B In such an embodiment, graphics cores 1715A-1715N each include circuitry including, but not limited to, vector engines 1702A-1702N, matrix engines 1703A-1703N, memory load / store units 1704A-1704N, instruction caches 1705A-1705N, data caches / shared local memory 1706A-1706N, ray tracing units 1708A-1708N, and samplers 1710A-1710N. The circuitry of graphics cores 1715A-1715N may additionally include fixed function logic 1712A-1712N. The number of vector engines 1702A-1702N and matrix engines 1703A-1703N within a design's graphics cores 1715A-1715N may vary based on the design's workload, performance, and power goals.

[0183] With reference to graphics core 1715A, vector engine 1702A and matrix engine 1703A can be configured to perform parallel computing operations on data of various integer and floating point data formats based on instructions associated with shader programs. Each vector engine 1702A and matrix engine 1703A can be used as a programmable general-purpose computing unit capable of executing multiple simultaneous hardware threads while processing multiple data elements in parallel for each thread. Vector engine 1702A and matrix engine 1703A support the processing of variable width vectors of various SIMD widths, including but not limited to SIMD8, SIMD16 and SIMD32. Input data elements can be stored in registers as compact data types, and vector engine 1702A and matrix engine 1703A can process various elements based on the data size of the elements. For example, when operating on a 256-bit wide vector, the 256 bits of the vector are stored in registers, and the vector is processed as four separate 64-bit packed data elements (quadword (QW) sized data elements), eight separate 32-bit packed data elements (doubleword (DW) sized data elements), sixteen separate 16-bit packed data elements (word (W) sized data elements), or thirty-two separate 8-bit data elements (byte (B) sized data elements). However, different vector widths and register sizes are possible. In one embodiment, the vector engine 1702A and the matrix engine 1703A can also be configured to perform SIMT operations on thread warps or thread groups of various sizes (e.g., 8, 16, or 32 threads).

[0184] Continuing with the graphics core 1715A, the memory load / store unit 1704A services memory access requests issued by the vector engine 1702A, the matrix engine 1703A, and / or other components of the graphics core 1715A that have access to memory. The memory access request may be processed by the memory load / store unit 1704A to load the requested data into a cache or memory, or to store from a cache or memory into a register file associated with the vector engine 1702A and / or the matrix engine 1703A. The memory load / store unit 1704A may also perform prefetch operations. In one embodiment, the memory load / store unit 1704A is configured to provide SIMT scatter / gather prefetches or block prefetches for data stored in memory 1810 from memory that is local to other tiles via the tile interconnect 1808 or from system memory. Prefetches may be performed on a specific L1 cache (e.g., data cache / shared local memory 1706A), L2 cache 1804, or L3 cache 1806. In one embodiment, a prefetch to L3 cache 1806 automatically causes the data to be stored in L2 cache 1804 .

[0185] Instruction cache 1705A stores instructions to be executed by graphics core 1715A. In one embodiment, graphics core 1715A also includes instruction acquisition and pre-fetch circuitry that acquires or pre-fetches instructions into instruction cache 1705A. Graphics core 1715A also includes instruction decoding logic for decoding instructions within instruction cache 1705A. Data cache / shared local memory 1706A can be configured as a data cache managed by a cache controller that implements a cache replacement policy, and / or configured as an explicitly managed shared memory. Ray tracing unit 1708A includes circuitry for accelerating ray tracing operations. Sampler 1710A provides texture sampling for 3D operations and media sampling for media operations. Fixed function logic 1712A includes fixed function circuitry shared between various examples of vector engine 1702A and matrix engine 1703A. Graphics cores 1715B-1715N can operate in a similar manner to graphics core 1715A.

[0186] The functions of the instruction caches 1705A-1705N, data cache / shared local memory 1706A-1706N, ray tracing units 1708A-1708N, samplers 1710A-1710N, and fixed function logic 1712A-1712N correspond to the equivalent functions in the graphics processor architecture described herein. For example, the instruction caches 1705A-1705N may be in the same Fig.14D The data cache / shared local memory 1706A-1706N, ray tracing units 1708A-1708N, and samplers 1710A-1710N may operate in a manner similar to the instruction cache 1455 of FIG. Fig. 14B The cache / SLM 1428A-1428F, ray tracing units 1427A-1427F, and samplers 1426A-1426F of the fixed function logic 1712A-1712N may include Fig. 14B In one embodiment, ray tracing units 1708A-1708N include components for performing the operations performed by Fig. 14C The ray tracing core 1445 is a circuit system that performs ray tracing acceleration operations.

[0187] like Fig. 17BAs shown, in one embodiment, the vector engine 1702 includes an instruction fetch unit 1737, a general register file array (GRF) 1724, an architectural register file array (ARF) 1726, a thread arbiter 1722, an issue unit 1730, a branch unit 1732, a set of SIMD floating point units (FPUs) 1734, and in one embodiment, a set of integer SIMD ALUs 1735. The GRF 1724 and ARF 1726 include a set of general register files and architectural register files associated with each hardware thread that can be active in the vector engine 1702. In one embodiment, the per-thread architectural state is maintained in the ARF 1726, while the data used during thread execution is stored in the GRF 1724. The execution state of each thread (including the instruction pointer for each thread) can be saved in thread-specific registers in the ARF 1726.

[0188] In one embodiment, the vector engine 1702 has an architecture that is a combination of simultaneous multithreading (SMT) and fine-grained interleaved multithreading (IMT). The architecture has a modular configuration that can be fine-tuned at design time based on the target number of simultaneous threads and the number of registers per graphics core, where graphics core resources are divided across the logic used to execute multiple simultaneous threads. The number of logical threads that can be executed by the vector engine 1702 is not limited to the number of hardware threads, and multiple logical threads can be assigned to each hardware thread.

[0189] In one embodiment, the vector engine 1702 can issue multiple instructions together, and the multiple instructions can each be different instructions. The thread arbiter 1722 can dispatch instructions to one of the send unit 1730, the branch unit 1732 or the SIMD FPU1734 for execution. Each execution thread can access 128 general registers in the GRF 1724, each of which can store 32 bytes and can be accessed as a variable width vector of 32-bit data elements. In one embodiment, each thread can access 4K bytes in the GRF 1724, although the embodiment is not limited to this, and more or less register resources can be provided in other embodiments. In one embodiment, the vector engine 1702 is divided into seven hardware threads that can independently perform computing operations, although the number of threads per vector engine 1702 can also vary according to the embodiment. For example, in one embodiment, up to 16 hardware threads are supported. In an embodiment in which seven threads can access 4K bytes, the GRF 1724 can store a total of 28K bytes. With 16 threads having access to 4K bytes, a total of 64K bytes can be stored by GRF 1724. Flexible addressing modes can allow registers to be addressed together to efficiently build wider registers or represent strided rectangular block data structures.

[0190] In one embodiment, memory operations, sampler operations, and other longer latency system communications are dispatched via "send" instructions executed by message passing send unit 1730. In one embodiment, branch instructions are dispatched to a dedicated branch unit 1732 to facilitate SIMD divergence and eventual convergence.

[0191] In one embodiment, the vector engine 1702 includes one or more SIMD floating point units (FPUs) 1734 to perform floating point operations. In one embodiment, the FPU 1734 also supports integer calculations. In one embodiment, the FPU 1734 can perform up to M 32-bit floating point (or integer) operations, or perform up to 2M 16-bit integers or 16-bit floating point operations. In one embodiment, at least one of the FPUs provides extended mathematical capabilities to support high-throughput transcendental mathematical functions and double-precision 64-bit floating points. In some embodiments, there is also a group of 8-bit integer SIMD ALUs 1735, and they can be specifically optimized for performing operations associated with machine learning calculations. In one embodiment, the SIMD ALU is replaced by an additional group of SIMD FPUs 1734 that can be configured to perform integer and floating point operations. In one embodiment, the SIMD FPU 1734 and the SIMD ALU 1735 can be configured to execute SIMT programs. In one embodiment, combined SIMD+SIMT operations are supported.

[0192] In one embodiment, an array of multiple instances of vector engine 1702 can be instantiated in the graphics core. For scalability, product architects can select the exact number of vector engines per graphics core grouping. In one embodiment, vector engine 1702 can execute instructions across multiple execution lanes. In another embodiment, each thread executed on vector engine 1702 executes on a different lane.

[0193] like Fig. 17CAs shown, in one embodiment, the matrix engine 1703 includes a processing element array, which is configured to perform tensor operations including vector / matrix and matrix / matrix operations, such as but not limited to matrix multiplication and / or dot product operations. The matrix engine 1703 is configured with M rows and N columns of processing elements (PE 1752AA-PE 1752MN), which include multipliers and adder circuits organized in a pipeline manner. In one embodiment, the processing elements 1752AA-PE1752MN constitute the physical pipeline level of the N-wide and M-deep systolic arrays, which can be used to perform vector / matrix or matrix / matrix operations in a data parallel manner, including matrix multiplication, fused multiplication and addition, dot product or other general matrix-matrix multiplication (GEMM) operations. In one embodiment, the matrix engine 1703 supports 16-bit floating point operations and 8-bit, 4-bit, 2-bit and binary integer operations. The matrix engine 1703 can also be configured to accelerate specific machine learning operations. In such an embodiment, the matrix engine 1703 can be configured to support a bfloat (brain floating point) 16-bit floating point format or a tensor float 32-bit floating point format (TF32) having a different number of mantissa and exponent bits relative to the Institute of Electrical and Electronics Engineers (IEEE) 754 format.

[0194] In one embodiment, during each cycle, each stage may add the result of the operation performed at that stage to the output of the previous stage. In other embodiments, after a set of computation cycles, the data movement pattern between the processing elements 1752AA-1752MN may vary based on the instruction or macro operation being executed. For example, in one embodiment, partial and loopbacks are enabled, and the processing element may instead add the output of the current cycle to the output generated in the previous cycle. In one embodiment, the final stage of the systolic array may be configured with a loopback to the initial stage of the systolic array. In such an embodiment, the number of physical pipeline stages may be decoupled from the number of logical pipeline stages supported by the matrix engine 1703. For example, in the case where the processing elements 1752AA-1752MN are configured as a systolic array of M physical stages, a loopback from stage M to the initial pipeline stage may enable the processing elements 1752AA-PE1752MN to operate as a systolic array of, for example, 2M, 3M, 4M, etc. logical pipeline stages.

[0195] In one embodiment, matrix engine 1703 includes memory 1741A-1741N, 1742A-1742M for storing input data in the form of row and column data of input matrix. Memory 1742A-1742M can be configured to store row elements (A0-Am) of first input matrix, and memory 1741A-1741N can be configured to store column elements (B0-Bn) of second input matrix. Row and column elements are provided to processing element 1752AA-1752MN as input for processing. In one embodiment, row and column elements of input matrix can be provided to memory 1741A-1741N, 1742A-1742M before these elements are stored in systolic register file 1740 in matrix engine 1703. In one embodiment, systolic register file 1740 is excluded, and memory 1741A-1741N, 1742A-1742M is removed from associated vector engine (e.g., Fig. 17B 1724) of the vector engine 1702 or other memory of the graphics core including the matrix engine 1703 (e.g., for Fig.17A The results generated by processing elements 1752AA-1752MN are then output to output buffers and / or written to register files (e.g., systolic register file 1740, GRF 1724, data cache / shared local memory 1706A-1706N) for further processing by other functional units of the graphics processor or for output to memory.

[0196] In some embodiments, the matrix engine 1703 is configured to support input sparsity, where multiplication operations for sparse regions of input data can be bypassed by skipping multiplication operations with zero-valued operands. In one embodiment, processing elements 1752AA-1752MN are configured to skip the execution of specific operations with zero-valued inputs. In one embodiment, sparsity within the input matrix can be detected, and operations with known zero output values ​​can be bypassed before being submitted to processing elements 1752AA-1752MN. Zero-valued operands can be bypassed from loading into the processing element, and processing elements 1752AA-1752MN can be configured to perform multiplication on non-zero-valued input elements. The matrix engine 1703 can also be configured to support output sparsity, so that operations with results predetermined to be zero are bypassed. For input sparsity and / or output sparsity, in one embodiment, metadata is provided to processing elements 1752AA-1752MN to indicate which processing elements and / or data channels will be active during the cycle for a processing cycle.

[0197] In one embodiment, matrix engine 1703 includes hardware for implementing operations on sparse data, and sparse data has a compressed representation of a sparse matrix storing non-zero values ​​and metadata of the position of the non-zero values ​​in the identification matrix. Exemplary compressed representations include but are not limited to compressed tensor representations, such as compressed sparse rows (CSR), compressed sparse columns (CSC), and compressed sparse fibers (CSF). Support for compressed representation enables operations to be performed on inputs with compressed tensor formats without requiring decompression or decoding of compressed representations. In such embodiments, operations can be performed only on non-zero input values, and the resulting non-zero output values ​​can be mapped to the output matrix. In some embodiments, hardware support is also provided for the machine-specific lossless data compression format used when transmitting data in hardware or across system buses. Such data can be retained in a compressed format for sparse input data, and matrix engine 1703 can use compressed metadata for compressed data to enable operations to be performed only on non-zero values, or to enable blocks of zero data input to be bypassed for multiplication operations.

[0198] In various embodiments, the input data may be provided by the programmer in a compressed tensor representation, or the codec may compress the input data into a compressed tensor representation or another sparse data encoding. In addition to supporting compressed tensor representations, streaming compression of sparse input data may be performed before providing the data to the processing elements 1752AA-1752MN. In one embodiment, compression is performed on data written to a cache memory associated with the graphics core cluster 414, wherein the compression is performed using an encoding supported by the matrix engine 1703. In one embodiment, the matrix engine 1703 includes support for inputs with structured sparsity, wherein a predetermined level or pattern of sparsity is applied to the input data. The data may be compressed to a known compression ratio, wherein the compressed data is processed by the processing elements 1752AA-1752MN according to metadata associated with the compressed data.

[0199] Fig.18 A block diagram 1800 of a multi-block processor according to an embodiment is shown. In one embodiment, the block diagram 1800 represents Fig. 15B Graphics engine tiles 1510A-1510D or Fig. 15C 1540A-1540D of the compute engine tiles 1540A-1540D of the multi-tile graphics processor. Tile 1800 of the multi-tile graphics processor includes an array of graphics core clusters (e.g., graphics core cluster 414A, graphics core cluster 414B through graphics core cluster 414N), where each graphics core cluster has an array of graphics cores 1715A-1715N. Tile 1800 also includes a global dispatcher 1802 to dispatch threads to processing resources of tile 1800.

[0200] Tile 1800 may include or be coupled with an L3 cache 1806 and a memory 1810. In various embodiments, L3 cache 1806 may be excluded or tile 1800 may include additional levels of cache, such as an L4 cache. In one embodiment, each instance of tile 1800 in a multi-tile graphics processor has an associated memory 1810, such as in Fig. 15B and Figure 5 C. In one embodiment, the multi-tile processor may be configured as a multi-chip module where the L3 cache 1806 and / or memory 1810 reside on a chiplet that is separate from the graphics core clusters 414A-414N. In this context, a chiplet is an integrated circuit that is at least partially packaged and includes different logic units that can be assembled with other chiplets into a larger package. For example, the L3 cache 1806 may be included in a dedicated cache chiplet, or may reside on the same chiplet as the graphics core clusters 414A-414N. In one embodiment, the L3 cache 1806 may be included in an active base die or an active interposer, such as Fig.11 As shown in C.

[0201] Memory structure 1803 enables communication between graphics core cluster 414A-414N, L3 cache 1806, and memory 1810. L2 cache 1804 is coupled to memory structure 1803 and can be configured to cache transactions executed via memory structure 1803. Tile interconnect 1808 enables communication with other tiles on the graphics processor and can be Figure 3 B and Figure 3 1800. In an embodiment where L3 cache 1806 is excluded from tile 1800, L2 cache 1804 may be configured as a combined L2 / L3 cache. In a specific implementation, memory structure 1803 may be configured to route data to L3 cache 1806 or a memory controller associated with memory 1810 based on the presence or absence of L3 cache 1806. L3 cache 1806 may be configured as a per-tile cache dedicated to the processing resources of tile 1800, or may be a partition of a GPU-wide L3 cache.

[0202] Fig.191 is a block diagram illustrating a graphics processor instruction format 1900 according to some embodiments. In one or more embodiments, the graphics processor core supports an instruction set having instructions in multiple formats. Solid line boxes illustrate components that are typically included in the graphics core instructions, while dashed lines include components that are optional or included only in a subset of the instructions. In some embodiments, the graphics processor instruction format 1900 described and illustrated are macroinstructions because they are instructions provided to the graphics core, rather than micro-operations that are generated by instruction decoding once the instructions are processed. Therefore, a single instruction can cause the hardware to perform multiple micro-operations.

[0203] In some embodiments, the graphics processor natively supports instructions in 128-bit instruction format 1910. Based on the selected instruction, instruction options, and the number of operands, a 64-bit compressed instruction format 1930 may be used for some instructions. The native 128-bit format instruction format 1910 provides access to all instruction options, while some options and operations are limited to the 64-bit format 1930. The native instructions available in the 64-bit format 1930 vary depending on the embodiment. In some embodiments, instructions are compressed in part using a set of index values ​​in the index field 1913. The graphics core hardware references a set of compression tables based on the index values ​​and uses the compression table output to reconstruct the native instructions of the 128-bit instruction format 1910. Instructions of other sizes and formats may be used.

[0204] For each format, the instruction opcode 1912 defines the operation to be performed by the graphics core. The graphics core executes each instruction in parallel across multiple data elements of each operand. For example, in response to an addition instruction, the graphics core performs a simultaneous addition operation across each color channel representing a texture element or a picture element. By default, the graphics core executes each instruction across all data channels of the operand. In some embodiments, the instruction control field 1914 implements control of specific execution options, such as channel selection (e.g., prediction) and data channel order (e.g., blending). For instructions in the 128-bit instruction format 1910, the execution size field 1916 limits the number of data channels that will be executed in parallel. In some embodiments, the execution size field 1916 is not available in the 64-bit compact instruction format 1930.

[0205] Some graphics core instructions have up to three operands, including two source operands src0 1920, src1 1922, and one destination 1918. In some embodiments, the graphics core supports dual destination instructions, where one of the destinations is implied. Data manipulation instructions may have a third source operand (e.g., SRC2 1924), where the instruction opcode 1912 determines the number of source operands. The last source operand of an instruction may be an immediate (e.g., hard-coded) value passed with the instruction.

[0206] In some embodiments, the 128-bit instruction format 1910 includes, for example, an access / address mode field 1926 that specifies whether direct register addressing mode or indirect register addressing mode is used. When direct register addressing mode is used, the register addresses of one or more operands are provided directly by bits in the instruction.

[0207] In some embodiments, the 128-bit instruction format 1910 includes an access / address mode field 1926 that specifies an address mode and / or access mode for the instruction. In one embodiment, the access mode is used to define the data access alignment for the instruction. Some embodiments support access modes including a 16-byte aligned access mode and a 1-byte aligned access mode, wherein the byte alignment of the access mode determines the access alignment of the instruction operands. For example, when in a first mode, the instruction may use byte-aligned addressing for source and destination operands, and when in a second mode, the instruction may use 16-byte aligned addressing for all source and destination operands.

[0208] In one embodiment, the address mode portion of the access / address mode field 1926 determines whether the instruction uses direct addressing or indirect addressing. When direct register addressing mode is used, the bits in the instruction directly provide the register address of one or more operands. When indirect register addressing mode is used, the register address of one or more operands can be calculated based on the address register value and the address immediate field in the instruction.

[0209] In some embodiments, instructions are grouped based on the opcode 1912 bit field to simplify opcode decoding 1940. For 8-bit opcodes, bits 4, 5, and 6 allow the graphics core to determine the type of opcode. The precise opcode grouping shown is only an example. In some embodiments, the move and logic opcode group 1942 includes data movement and logic instructions (e.g., move (mov), compare (cmp)). In some embodiments, the move and logic group 1942 shares five most significant bits (MSBs), where the move (mov) instruction is in the form of 0000xxxxb and the logic instruction is in the form of 0001xxxxb. The flow control instruction group 1944 (e.g., call, jump (jmp)) includes instructions in the form of 0010xxxxb (e.g., 0x20). The miscellaneous instruction group 1946 includes a mixture of instructions, including synchronization instructions (e.g., wait, send) in the form of 0011xxxxb (e.g., 0x30). The parallel math instruction group 1948 includes component-wise arithmetic instructions (e.g., addition, multiplication (mul)) of the form 0100xxxxb (e.g., 0x40). The parallel math instruction group 1948 performs arithmetic operations in parallel across data channels. The vector math group 1950 includes arithmetic instructions (e.g., dp4) of the form 0101xxxxb (e.g., 0x50). The vector math group performs arithmetic on vector operands, such as dot product calculations. In one embodiment, the opcode decode 1940 shown can be used to determine which portion of the graphics core will be used to execute the decoded instructions. For example, some instructions can be designated as systolic instructions to be executed by a systolic array. Other instructions, such as ray tracing instructions (not shown), can be routed to ray tracing logic within a slice or partition of a ray tracing core or execution logic.

[0210] Graphics Pipeline

[0211] Fig. 20 2 is a block diagram of another embodiment of a graphics processor 2000. Elements having the same reference numerals (or names) as elements of any other figure herein Fig. 20 The elements of may operate or function in any manner similar to that described elsewhere herein, but are not limited thereto.

[0212] In some embodiments, graphics processor 2000 includes geometry pipeline 2020, media pipeline 2030, display engine 2040, thread execution logic 2050, and render output pipeline 2070. In some embodiments, graphics processor 2000 is a graphics processor within a multi-core processing system that includes one or more general-purpose processing cores. The graphics processor is controlled by register writes to one or more control registers (not shown) or via commands issued to graphics processor 2000 through ring interconnect 2002. In some embodiments, ring interconnect 2002 couples graphics processor 2000 to other processing components, such as other graphics processors or general-purpose processors. Commands from ring interconnect 2002 are interpreted by command stream converter 2003, which provides instructions to various components of geometry pipeline 2020 or media pipeline 2030.

[0213] In some embodiments, command stream converter 2003 directs vertex fetcher 2005 to read vertex data from memory and perform operations of vertex processing commands provided by command stream converter 2003. In some embodiments, vertex fetcher 2005 provides vertex data to vertex shader 2007, which performs coordinate space transformation and lighting operations on each vertex. In some embodiments, vertex fetcher 2005 and vertex shader 2007 execute vertex processing instructions by dispatching execution threads to graphics cores 2052A-2052B via thread dispatcher 2031.

[0214] In some embodiments, graphics cores 2052A-2052B are vector processor arrays with instruction sets for performing graphics and media operations. In some embodiments, graphics cores 2052A-2052B have an attached L1 cache 2051 specific to each array or shared between arrays. The cache can be configured as a data cache, an instruction cache, or a single cache that is partitioned to contain data and instructions in different partitions.

[0215] In some embodiments, the geometry pipeline 2020 includes a tessellation component to perform hardware accelerated tessellation of 3D objects. In some embodiments, the programmable hull shader 2011 configures the tessellation operation. The programmable domain shader 2017 provides back-end evaluation of the tessellation output. The tessellation 2013 operates at the direction of the hull shader 2011 and contains dedicated logic to generate a set of detailed geometric objects based on the rough geometric model provided as input to the geometry pipeline 2020. In some embodiments, if tessellation is not used, the tessellation component (e.g., hull shader 2011, tessellation 2013, and domain shader 2017) can be bypassed. The tessellation component can operate based on data received from the vertex shader 2007.

[0216] In some embodiments, the complete geometric object may be processed by the geometry shader 2019 via one or more threads dispatched to the graphics cores 2052A-2052B, or may proceed directly to the clipper 2029. In some embodiments, the geometry shader operates on the entire geometric object rather than on vertices or vertex blocks as in previous stages of the graphics pipeline. If tessellation is disabled, the geometry shader 2019 receives input from the vertex shader 2007. In some embodiments, if the tessellation unit is disabled, the geometry shader 2019 may be programmed by a geometry shader program to perform geometry tessellation.

[0217] Before rasterization, the clipper 2029 processes the vertex data. The clipper 2029 can be a fixed function clipper or a programmable clipper with clipping and geometry shader functions. In some embodiments, the rasterizer and depth test component 2073 in the render output pipeline 2070 dispatches a pixel shader to convert the geometric object into a per-pixel representation. In some embodiments, the pixel shader logic is included in the thread execution logic 2050. In some embodiments, the application can bypass the rasterizer and depth test component 2073 and access the unrasterized vertex data via the stream output unit 2023.

[0218] The graphics processor 2000 has an interconnect bus, interconnect structure or some other interconnect mechanism that allows data and messages to be passed between the main components of the processor. In some embodiments, the graphics core 2052A-2052B and associated logic units (e.g., L1 cache 2051, sampler 2054, texture cache 2058, etc.) are interconnected via data port 2056 to perform memory access and communicate with the rendering output pipeline components of the processor. In some embodiments, the sampler 2054, cache 2051, 2058 and graphics core 2052A-2052B each have a separate memory access path. In one embodiment, the texture cache 2058 can also be configured as a sampler cache.

[0219] In some embodiments, the render output pipeline 2070 includes a rasterizer and depth test component 2073 that converts vertex-based objects into associated pixel-based representations. In some embodiments, the rasterizer logic includes a windower / masker unit for performing fixed-function triangle and line rasterization. In some embodiments, an associated render cache 2078 and depth cache 2079 are also available. A pixel operation component 2077 performs pixel-based operations on data, although in some examples, pixel operations associated with 2D operations (e.g., bit-block image transfers with blending) are performed by the 2D engine 2041, or replaced by a display controller 2043 using an overlay display plane at display time. In some embodiments, a shared L3 cache 2075 is available for all graphics components, allowing data to be shared without using main system memory.

[0220] In some embodiments, the media pipeline 2030 includes a media engine 2037 and a video front end 2034. In some embodiments, the video front end 2034 receives pipeline commands from the command stream converter 2003. In some embodiments, the media pipeline 2030 includes a separate command stream converter. In some embodiments, the video front end 2034 processes the media commands before sending them to the media engine 2037. In some embodiments, the media engine 2037 includes a thread generation function to generate threads for dispatching to the thread execution logic 2050 via the thread dispatcher 2031.

[0221] In some embodiments, the graphics processor 2000 includes a display engine 2040. In some embodiments, the display engine 2040 is external to the processor 2000 and is coupled to the graphics processor via a ring interconnect 2002 or some other interconnect bus or structure. In some embodiments, the display engine 2040 includes a 2D engine 2041 and a display controller 2043. In some embodiments, the display engine 2040 contains dedicated logic that can operate independently of the 3D pipeline. In some embodiments, the display controller 2043 is coupled to a display device (not shown), which can be a system integrated display device such as in a laptop computer, or an external display device attached via a display device connector.

[0222] In some embodiments, geometry pipeline 2020 and media pipeline 2030 can be configured to perform operations based on multiple graphics and media programming interfaces, and are not specific to any one application programming interface (API). In some embodiments, driver software for a graphics processor converts API calls specific to a specific graphics or media library into commands that can be processed by a graphics processor. In some embodiments, support is provided for all open graphics libraries (OpenGL), open computing languages ​​(OpenCL) and / or Vulkan graphics and computing APIs from the Khronos group. In some embodiments, support can also be provided for the Direct3D library from Microsoft. In some embodiments, a combination of these libraries can be supported. Support can also be provided for an open source computer vision library (OpenCV). If it is possible to map from the pipeline of a future API to the pipeline of a graphics processor, future APIs with compatible 3D pipelines will also be supported.

[0223] Graphics pipeline programming

[0224] Fig.21A is a block diagram illustrating a graphics processor command format 2100 that may be used to program a graphics processing pipeline in accordance with some embodiments. Fig. 21B is a block diagram illustrating a graphics processor command sequence 2110 according to an embodiment. Fig.21A The solid-line boxes in show components that are typically included in the graphics commands, while the dashed lines include components that are optional or included only in a subset of the graphics commands. Fig.21A The exemplary graphics processor command format 2100 includes a data field for identifying the client 2102, a command operation code (opcode) 2104, and a data field for the command 2106. In some commands, a sub-opcode 2105 and a command size 2108 are also included.

[0225] In some embodiments, client 2102 specifies the client unit of the graphics device that processes the command data. In some embodiments, the graphics processor command parser checks the client field of each command to adjust the further processing of the command and routes the command data to the appropriate client unit. In some embodiments, the graphics processor client unit includes a memory interface unit, a rendering unit, a 2D unit, a 3D unit, and a media unit. Each client unit has a corresponding processing pipeline for processing commands. Once the client unit receives the command, the client unit reads the opcode 2104 and (if present) the sub-opcode 2105 to determine the operation to be performed. The client unit uses the information in the data field 2106 to execute the command. For some commands, it is expected that the explicit command size 2108 is to specify the size of the command. In some embodiments, the command parser automatically determines the size of at least some commands based on the command opcode. In some embodiments, the command is aligned via a multiple of a double word. Other command formats can be used.

[0226] Fig. 21B An exemplary graphics processor command sequence 2110 is shown in the flowchart of FIG. In some embodiments, software or firmware of a data processing system featuring an embodiment of a graphics processor uses a version of the command sequence shown to set up, execute, and terminate a set of graphics operations. The sample command sequence is shown and described for purposes of example only, as embodiments are not limited to these particular commands or to the command sequence. In addition, commands may be issued as a batch of commands in a command sequence so that the graphics processor will process the command sequence at least partially simultaneously.

[0227] In some embodiments, the graphics processor command sequence 2110 may begin with a pipeline refresh command 2112 to cause any active graphics pipeline to complete currently pending commands for the pipeline. In some embodiments, the 3D pipeline 2122 and the media pipeline 2124 do not operate simultaneously. A pipeline refresh is performed to cause the active graphics pipeline to complete any pending commands. In response to a pipeline refresh, the command parser of the graphics processor will suspend command processing until the active drawing engine completes pending operations and the associated read cache is invalidated. Optionally, any data marked as "dirty" in the render cache may be flushed to memory. In some embodiments, the pipeline refresh command 2112 may be used for pipeline synchronization or before placing the graphics processor in a low power state.

[0228] In some embodiments, when a command sequence requires the graphics processor to explicitly switch between pipelines, pipeline select command 2113 is used. In some embodiments, pipeline select command 2113 is only required once within an execution context before issuing a pipeline command, unless the context is issuing commands for both pipelines. In some embodiments, pipeline flush command 2112 is required immediately before a pipeline switch is made via pipeline select command 2113.

[0229] In some embodiments, pipeline control commands 2114 configure the graphics pipeline for operation and are used to program 3D pipeline 2122 and media pipeline 2124. In some embodiments, pipeline control commands 2114 configure pipeline states for active pipelines. In one embodiment, pipeline control commands 2114 are used for pipeline synchronization and clearing data from one or more cache memories within active pipelines before processing a batch of commands.

[0230] In some embodiments, commands associated with return buffer state 2116 are used to configure a set of return buffers for a corresponding pipeline to write data. Some pipeline operations require the allocation, selection, or configuration of one or more return buffers into which the operation writes intermediate data during processing. In some embodiments, the graphics processor also uses one or more return buffers to store output data and perform cross-thread communication. In some embodiments, return buffer state 2116 includes selecting the size and number of return buffers for a set of pipeline operations.

[0231] The remaining commands in the command sequence differ based on the active pipeline for the operation. Based on the pipeline determination 2120 , the command sequence is tailored to either the 3D pipeline 2122 starting from the 3D pipeline state 2130 or the media pipeline 2124 starting from the media pipeline state 2140 .

[0232] The commands to configure the 3D pipeline state 2130 include 3D state setting commands for vertex buffer state, vertex element state, constant color state, depth buffer state, and other state variables to be configured before processing 3D primitive commands. The values ​​of these commands are determined at least in part based on the specific 3D API in use. In some embodiments, the 3D pipeline state 2130 commands can also selectively disable or bypass specific pipeline elements if they will not be used.

[0233] In some embodiments, the 3D primitive 2132 command is used to submit a 3D primitive to be processed by the 3D pipeline. The command and associated parameters passed to the graphics processor via the 3D primitive 2132 command are forwarded to the vertex acquisition function in the graphics pipeline. The vertex acquisition function uses the 3D primitive 2132 command data to generate a vertex data structure. The vertex data structure is stored in one or more return buffers. In some embodiments, the 3D primitive 2132 command is used to perform vertex operations on the 3D primitive via a vertex shader. In order to process the vertex shader, the 3D pipeline 2122 dispatches the shader program to the graphics core.

[0234] In some embodiments, the 3D pipeline 2122 is triggered via an execute 2134 command or event. In some embodiments, a register write triggers the command execution. In some embodiments, the execution is triggered via a "go" or "kick" command in a command sequence. In one embodiment, the command execution is triggered using a pipeline synchronization command to refresh the command sequence through the graphics pipeline. The 3D pipeline will perform geometry processing on the 3D primitives. Once the operation is completed, the resulting geometric object is rasterized and the pixel engine shades the resulting pixels. Additional commands that control pixel shading and pixel backend operations may also be included for those operations.

[0235] In some embodiments, the graphics processor command sequence 2110 follows the media pipeline 2124 path when performing media operations. Typically, the specific use and manner of programming for the media pipeline 2124 depends on the media or computing operations to be performed. Specific media decoding operations can be offloaded to the media pipeline during media decoding. In some embodiments, the media pipeline can also be bypassed, and resources provided by one or more general processing cores can be used to perform media decoding in whole or in part. In one embodiment, the media pipeline also includes elements for general purpose graphics processor unit (GPGPU) operations, wherein the graphics processor is used to perform SIMD vector operations using compute shader programs that are not explicitly related to the rendering of graphics primitives.

[0236] In some embodiments, the media pipeline 2124 is configured in a similar manner to the 3D pipeline 2122. A set of commands for configuring the media pipeline state 2140 are dispatched or placed into a command queue before the media object commands 2142. In some embodiments, the commands for the media pipeline state 2140 include data for configuring the media pipeline elements that will be used to process the media objects. This includes data for configuring the video decoding and video encoding logic within the media pipeline, such as encoding or decoding formats. In some embodiments, the commands for the media pipeline state 2140 also support the use of one or more pointers to "indirect" state elements that contain a batch of state settings.

[0237] In some embodiments, media object command 2142 provides a pointer to a media object for the media pipeline to process. The media object includes a memory buffer for containing video data to be processed. In some embodiments, all media pipeline states must be valid before issuing media object command 2142. Once the pipeline state is configured and the media object command 2142 is queued, the media pipeline 2124 is triggered via an execution command 2144 or an equivalent execution event (e.g., a register write). The output from the media pipeline 2124 can then be post-processed by the 3D pipeline 2122 or the operation provided by the media pipeline 2124. In some embodiments, GPGPU operations are configured and executed in a manner similar to media operations.

[0238] Graphics software architecture

[0239] Fig. 22 An exemplary graphics software architecture for a data processing system 2200 is shown in accordance with some embodiments. In some embodiments, the software architecture includes a 3D graphics application 2210, an operating system 2220, and at least one processor 2230. In some embodiments, the processor 2230 includes a graphics processor 2232 and one or more general purpose processor cores 2234. The graphics application 2210 and the operating system 2220 are each executed in a system memory 2250 of the data processing system.

[0240] In some embodiments, the 3D graphics application 2210 includes one or more shader programs, which include shader instructions 2212. The shader language instructions can be a high-level shader language, such as Direct3D's High-Level Shader Language (HLSL), OpenGL Shader Language (GLSL), etc. The application also includes executable instructions 2214 written in a machine language suitable for execution by the general-purpose processor core 2234. The application also includes graphics objects 2216 defined by vertex data.

[0241] In some embodiments, operating system 2220 is an operating system from Microsoft Corporation The operating system 2220 may be a proprietary UNIX-like operating system, or an open source UNIX-like operating system using a variant of the Linux kernel. The operating system 2220 may support a graphics API 2222, such as a Direct3D API, an OpenGL API, or a Vulkan API. When the Direct3D API is in use, the operating system 2220 uses a front-end shader compiler 2224 to compile any shader instruction 2212 in the format of HLSL into a lower-level shader language. The compilation may be a just-in-time (JIT) compilation, or the application may perform shader precompilation. In some embodiments, during the compilation of the 3D graphics application 2210, a high-level shader is compiled into a low-level shader. In some embodiments, the shader instruction 2212 is provided in an intermediate form, such as a version of the standard portable intermediate representation (SPIR) used by the Vulkan API.

[0242] In some embodiments, the user mode graphics driver 2226 includes a backend shader compiler 2227 to convert shader instructions 2212 into hardware specific representations. When the OpenGL API is in use, the shader instructions 2212 in the format of the GLSL high-level language are passed to the user mode graphics driver 2226 for compilation. In some embodiments, the user mode graphics driver 2226 uses the operating system internal kernel mode function 2228 to communicate with the internal kernel mode graphics driver 2229. In some embodiments, the kernel mode graphics driver 2229 communicates with the graphics processor 2232 to dispatch commands and instructions.

[0243] IP core implementation

[0244] One or more aspects of at least one embodiment may be implemented by representative code stored on a machine-readable medium, which represents and / or defines logic within an integrated circuit such as a processor. For example, a machine-readable medium may include instructions representing various logic within a processor. When read by a machine, the instructions may cause the machine to fabricate logic to perform the techniques described herein. Such representations, referred to as "IP cores," are reusable logic units of an integrated circuit that may be stored on a tangible machine-readable medium as a hardware model describing the structure of the integrated circuit. The hardware model may be provided to various customers or manufacturing facilities, which load the hardware model onto a manufacturing machine that manufactures the integrated circuit. The integrated circuit may be manufactured so that the circuit performs the operations described in association with any of the embodiments described herein.

[0245] Fig.23A23 is a block diagram showing an IP core development system 2300 that can be used to manufacture an integrated circuit to perform operations according to an embodiment. The IP core development system 2300 can be used to generate modular, reusable designs that can be incorporated into a larger design or used to construct an entire integrated circuit (e.g., a SOC integrated circuit). The design facility 2330 can generate a software simulation 2310 of the IP core design in a high-level programming language (e.g., C / C++). The software simulation 2310 can be used to design, test, and verify the behavior of the IP core using a simulation model 2312. The simulation model 2312 can include functional, behavioral, and / or timing simulations. A register transfer level (RTL) design 2315 can then be created or synthesized based on the simulation model 2312. The RTL design 2315 is an abstraction of the behavior of an integrated circuit that models the flow of digital signals between hardware registers, including associated logic executed using the modeled digital signals. In addition to the RTL design 2315, a lower-level design at the logic level or transistor level can also be created, designed, or synthesized. Therefore, the specific details of the initial design and simulation can vary.

[0246] The RTL design 2315 or equivalent can be further synthesized by the design facility into a hardware model 2320, which can have a hardware description language (HDL) or some other representation of physical design data. The HDL can be further simulated or tested to verify the IP core design. The IP core design can be stored using a non-volatile memory 2340 (e.g., a hard disk, flash memory, or any non-volatile storage medium) for delivery to a third-party manufacturing facility 2365. Alternatively, the IP core design can be transmitted (e.g., via the Internet) via a wired connection 2350 or a wireless connection 2360. Then, the manufacturing facility 2365 can manufacture an integrated circuit based at least in part on the IP core design. The manufactured integrated circuit can be configured to perform operations according to at least one embodiment described herein.

[0247] Fig. 23BA cross-sectional side view of an integrated circuit package assembly 2370 according to some embodiments described herein is shown. The integrated circuit package assembly 2370 shows an implementation of one or more processors or accelerator devices as described herein. The package assembly 2370 includes a plurality of hardware logic units 2372, 2374 connected to a substrate 2380. The logic 2372, 2374 may be implemented at least in part in configurable logic or fixed-function logic hardware, and may include one or more portions of any processor core, graphics processor, or other accelerator device described herein. Each unit of the logic 2372, 2374 may be implemented within a semiconductor die and coupled to the substrate 2380 via an interconnect structure 2373. The interconnect structure 2373 may be configured to route electrical signals between the logic 2372, 2374 and the substrate 2380, and may include interconnects such as, but not limited to, bumps or pillars. In some embodiments, the interconnect structure 2373 may be configured to route electrical signals, such as input / output (I / O) signals and / or power or ground signals associated with the operation of the logic 2372, 2374. In some embodiments, substrate 2380 is an epoxy-based laminate substrate. In other embodiments, substrate 2380 may include other suitable types of substrates. Package assembly 2370 may be connected to other electrical devices via package interconnect 2383. Package interconnect 2383 may be coupled to the surface of substrate 2380 to route electrical signals to other electrical devices, such as a motherboard, other chipsets, or multi-chip modules.

[0248] In some embodiments, the logic units 2372, 2374 are electrically coupled to a bridge 2382, which is configured to route electrical signals between the logic units 2372, 2374. The bridge 2382 can be a dense interconnect structure that provides routing for electrical signals. The bridge 2382 can include a bridge substrate composed of glass or a suitable semiconductor material. Electrical routing features can be formed on the bridge substrate to provide chip-to-chip connections between the logic units 2372, 2374.

[0249] Although two logic cells 2372, 2374 and bridge 2382 are shown, the embodiments described herein may include more or fewer logic cells on one or more dies. One or more dies may be connected by zero or more bridges, as bridge 2382 may be excluded when logic is included on a single die. Alternatively, multiple dies or logic cells may be connected by one or more bridges. Additionally, multiple logic cells, dies, and bridges may be connected together in other possible configurations, including three-dimensional configurations.

[0250] Fig.23CA package assembly 2390 including a plurality of hardware logic chiplet units connected to a substrate 2380 is shown. A graphics processing unit, parallel processor, and / or computing accelerator as described herein may be composed of various silicon chiplets manufactured separately. Different groups of chiplets with different IP core logics may be assembled into a single device. Additionally, active interposer technology may be used to integrate chiplets into a base die or base chiplet. The concepts described herein implement interconnection and communication between different forms of IP within a GPU. IP cores may be manufactured using different process technologies and assembled during manufacturing, which avoids the complexity of converging multiple IPs (especially on large SoCs with several styles of IP) to the same manufacturing process. Enabling the use of multiple process technologies improves time to market and provides a cost-effective way to create multiple product SKUs. Additionally, decomposed IPs are more suitable for independent power gating, and components that are not used on a given workload can be powered off, thereby reducing overall power consumption.

[0251] In various embodiments, the package assembly 2390 may include components and chiplets interconnected by structures 2385 and / or one or more bridges 2387. The chiplets within the package assembly 2390 may have a 2.5D arrangement using chip-on-wafer stacking on a substrate, where multiple dies are stacked side-by-side on a silicon interposer 2389 that couples the chiplets to the substrate 2380. The substrate 2380 includes electrical connections to the package interconnects 2383. In one embodiment, the silicon interposer 2389 is a passive interposer that includes through silicon vias (TSVs) to electrically couple the chiplets within the package assembly 2390 to the substrate 2380. In one embodiment, the silicon interposer 2389 is an active interposer that includes embedded logic in addition to the TSVs. In such an embodiment, the chiplets within the package assembly 2390 are arranged using 3D face-to-face die stacking on top of the active interposer 2389. In addition to the interconnect fabric 2385 and the silicon bridge 2387, the active interposer 2389 may also include hardware logic for I / O 2391, cache memory 2392, and other hardware logic 2393. The fabric 2385 enables communication between the various logic chiplets 2372, 2374 and the logic 2391, 2393 within the active interposer 2389. The fabric 2385 may be a NoC interconnect or another form of packet switching fabric that exchanges data packets between components of the package assembly. For complex assemblies, the fabric 2385 may be a dedicated chiplet that enables communication between the various hardware logic of the package assembly 2390.

[0252] The bridge structure 2387 within the active interposer 2389 can be used to facilitate, for example, point-to-point interconnections between a logic or I / O chiplet 2374 and a memory chiplet 2375. In some implementations, the bridge structure 2387 can also be embedded within the substrate 2380. The hardware logic chiplet can include a dedicated hardware logic chiplet 2372, a logic or I / O chiplet 2374, and / or a memory chiplet 2375. The hardware logic chiplet 2372 and the logic or I / O chiplet 2374 can be implemented at least in part in configurable logic or fixed-function logic hardware, and can include one or more portions of any processor core, graphics processor, parallel processor, or other accelerator device described herein. The memory chiplet 2375 can be DRAM (e.g., GDDR, HBM) memory or cache (SRAM) memory. The cache memory 2392 within the active interposer 2389 (or substrate 2380 ) can be used as a global cache for the package assembly 2390 , part of a distributed global cache, or as a dedicated cache for the structure 2385 .

[0253] Each chiplet can be fabricated as a separate semiconductor die and coupled to a base die embedded within or coupled to the substrate 2380. The coupling to the substrate 2380 can be performed via an interconnect structure 2373. The interconnect structure 2373 can be configured to route electrical signals between the various chiplets and logic within the substrate 2380. The interconnect structure 2373 can include interconnects such as, but not limited to, bumps or pillars. In some embodiments, the interconnect structure 2373 can be configured to route electrical signals, such as input / output (I / O) signals and / or power or ground signals associated with the operation of the logic, I / O, and memory chiplets. In one embodiment, an additional interconnect structure couples the active interposer 2389 to the substrate 2380.

[0254] In some embodiments, substrate 2380 is an epoxy-based laminate substrate. In other embodiments, substrate 2380 may include other suitable types of substrates. Package assembly 2390 may be connected to other electrical devices via package interconnect 2383. Package interconnect 2383 may be coupled to the surface of substrate 2380 to route electrical signals to other electrical devices, such as a motherboard, other chipsets, or multi-chip modules.

[0255] In some embodiments, the logic or I / O chiplet 2374 and the memory chiplet 2375 can be electrically coupled via a bridge 2387, which is configured to route electrical signals between the logic or I / O chiplet 2374 and the memory chiplet 2375. Bridge 2387 can be a dense interconnect structure that provides a route for electrical signals. Bridge 2387 can include a bridge substrate composed of glass or a suitable semiconductor material. Electrical wiring features can be formed on the bridge substrate to provide chip-to-chip connections between the logic or I / O chiplet 2374 and the memory chiplet 2375. Bridge 2387 can also be referred to as a silicon bridge or an interconnect bridge. For example, in some embodiments, bridge 2387 is an embedded multi-die interconnect bridge (EMIB). In some embodiments, bridge 2387 can simply be a direct connection from one chiplet to another chiplet.

[0256] Fig.23D A package assembly 2394 including an interchangeable chiplet 2395 is shown according to an embodiment. The interchangeable chiplet 2395 can be assembled into standardized slots on one or more base chiplets 2396, 2398. The base chiplets 2396, 2398 can be coupled via a bridge interconnect 2397, which can be similar to other bridge interconnects described herein and can be, for example, EMIB. The memory chiplets can also be connected to the logic or I / O chiplets via the bridge interconnect. The I / O and logic chiplets can communicate via the interconnect structure. The base chiplets can each support one or more slots with a standardized format for one of logic or I / O or memory / cache.

[0257] In one embodiment, the SRAM and power delivery circuits can be manufactured into one or more base chiplets 2396, 2398, which can be manufactured using a different process technology relative to the interchangeable chiplets 2395 stacked on top of the base chiplets. For example, the base chiplets 2396, 2398 can be manufactured using a larger process technology, while the interchangeable chiplets can be manufactured using a smaller process technology. One or more of the interchangeable chiplets 2395 can be memory (e.g., DRAM) chiplets. Different memory densities can be selected for the package assembly 2394 based on power and / or performance goals for the product using the package assembly 2394. Additionally, logic chiplets with different numbers of types of functional units can be selected at assembly time based on power and / or performance goals for the product. Additionally, chiplets containing different types of IP logic cores can be inserted into interchangeable chiplet slots, thereby enabling a hybrid processor design that can mix and match different technology IP blocks.

[0258] Exemplary System-on-Chip Integrated Circuit

[0259] Figure 24-25BAn exemplary integrated circuit and associated graphics processor that can be manufactured using one or more IP cores according to various embodiments described herein are shown. In addition to what is shown, other logic and circuits may also be included, including additional graphics processors / cores, peripheral interface controllers, or general purpose processor cores.

[0260] Fig.24 2400, which may be manufactured using one or more IP cores in accordance with an embodiment. The exemplary integrated circuit 2400 includes one or more application processors 2405 (e.g., CPUs), at least one graphics processor 2410, and may additionally include an image processor 2415 and / or a video processor 2420, any of which may be modular IP cores from the same or multiple different design facilities. The integrated circuit 2400 includes peripheral or bus logic, including a USB controller 2425, a UART controller 2430, an SPI / SDIO controller 2435, and an I 2 S / I 2 The integrated circuit may include a display device 2445 coupled to one or more of a high-definition multimedia interface (HDMI) controller 2450 and a mobile industry processor interface (MIPI) display interface 2455. Storage may be provided by a flash memory subsystem 2460 including flash memory and a flash memory controller. A memory interface may be provided via a memory controller 2465 for accessing SDRAM or SRAM memory devices. Some integrated circuits additionally include an embedded security engine 2470.

[0261] Figure 25A-25B is a block diagram illustrating an exemplary graphics processor for use within a SoC according to embodiments described herein. Fig.25A An exemplary graphics processor 2510 of a system-on-chip integrated circuit that may be fabricated using one or more IP cores in accordance with an embodiment is shown. Fig.25B An additional exemplary graphics processor 2540 of a system-on-chip integrated circuit that may be fabricated using one or more IP cores in accordance with an embodiment is shown. Fig.25A Graphics processor 2510 is an example of a low-power graphics processor core. Fig.25B The graphics processor 2540 of FIG. 25 is an example of a higher performance graphics processor core. Each of the graphics processor 2510 and the graphics processor 2540 may be Fig.24 A variation of graphics processor 2410.

[0262] like Fig.25AAs shown, the graphics processor 2510 includes a vertex processor 2505 and one or more fragment processors 2515A-2515N (e.g., 2515A, 2515B, 2515C, 2515D to 2515N-1 and 2515N). The graphics processor 2510 can execute different shader programs via separate logic, so that the vertex processor 2505 is optimized to perform operations for the vertex shader program, while one or more fragment processors 2515A-2515N perform fragment (e.g., pixel) shading operations for fragment or pixel shader programs. The vertex processor 2505 performs the vertex processing stage of the 3D graphics pipeline and generates primitives and vertex data. The fragment processors 2515A-2515N use the primitives and vertex data generated by the vertex processor 2505 to generate a frame buffer displayed on a display device. In one embodiment, the fragment processors 2515A-2515N are optimized to execute fragment shader programs as provided in the OpenGL API, which can be used to perform similar operations as pixel shader programs as provided in the Direct 3D API.

[0263] The graphics processor 2510 additionally includes one or more memory management units (MMUs) 2520A-2520B, caches 2525A-2525B, and circuit interconnects 2530A-2530B. The one or more MMUs 2520A-2520B provide virtual to physical address mappings for the graphics processor 2510 (including for the vertex processor 2505 and / or the fragment processors 2515A-2515N), and the graphics processor 2510 may reference vertex or image / texture data stored in memory in addition to vertex or image / texture data stored in the one or more caches 2525A-2525B. In one embodiment, the one or more MMUs 2520A-2520B may be synchronized with other MMUs within the system, including with Fig.24 One or more MMUs associated with one or more application processors 2405, image processor 2415, and / or video processor 2420 enable each processor 2405-2420 to participate in a shared or unified virtual memory system. According to various embodiments, one or more circuit interconnects 2530A-2530B enable the graphics processor 2510 to interface with other IP cores within the SoC via an internal bus of the SoC or via a direct connection.

[0264] like Fig.25B As shown, the graphics processor 2540 includes Fig.25AOne or more MMUs 2520A-2520B, caches 2525A-2525B, and circuit interconnects 2530A-2530B of a graphics processor 2510. The graphics processor 2540 includes one or more shader cores 2555A-2555N (e.g., 2555A, 2555B, 2555C, 2555D, 2555E, 2555F to 2555N-1 and 2555N), which provide a unified shader core architecture in which a single core or type or core can execute all types of programmable shader code, including shader program code for implementing vertex shaders, fragment shaders, and / or compute shaders. The unified shader core architecture can also be configured to execute directly compiled high-level GPGPU programs (e.g., CUDA). The exact number of shader cores present can vary between embodiments and implementations. Additionally, the graphics processor 2540 includes an inter-core task manager 2545 that functions as a thread dispatcher to dispatch execution threads to one or more shader cores 2555A-2555N, and a tile unit 2558 that accelerates tile operations for tile-based rendering, where rendering operations for a scene are subdivided in image space, for example to exploit local spatial consistency within a scene or to optimize use of internal caches.

[0265] The flowcharts shown herein provide examples of sequences of various process actions. The flowcharts may indicate operations to be performed by software or firmware routines as well as physical operations. The flowcharts may illustrate examples of implementations of states of finite state machines (FSMs) that may be implemented in hardware and / or software. Although shown in a particular order or sequence, the order of the actions may be modified unless otherwise specified. Therefore, the figures shown should be understood only as examples, and the processes may be performed in different orders, and some actions may be performed in parallel. Additionally, one or more actions may be omitted; therefore, not all implementations will perform all actions.

[0266] With respect to the various operations or functions described herein, they may be described or defined as software code, instructions, configurations, and / or data. The content may be directly executable ("object" or "executable" form), source code, or differential code ("incremental" or "patch" code). The software content of the content described herein may be provided via a manufactured product having content stored thereon or via a method of operating a communication interface to send data via a communication interface. A machine-readable storage medium may enable a machine to perform the functions or operations described, and includes any mechanism for storing information in a form accessible to a machine (e.g., a computing device, an electronic system, etc.), such as a recordable / unrecordable medium (e.g., a read-only memory (ROM), a random access memory (RAM), a disk storage medium, an optical storage medium, a flash memory device, etc.). The communication interface includes any mechanism for engaging with any of the hardwired, wireless, optical, etc. media to communicate with another device (such as a memory bus interface, a processor bus interface, an Internet connection, a disk controller, etc.). The communication interface may be configured by providing configuration parameters and / or sending signals so that the communication interface is ready to provide a data signal describing the software content. The communication interface may be accessed via one or more commands or signals sent to the communication interface.

[0267] The various components described herein may be units for performing the described operations or functions. Each component described herein includes software, hardware, or a combination of these. The components may be implemented as software modules, hardware modules, dedicated hardware (e.g., application-specific hardware, application-specific integrated circuits (ASICs), digital signal processors (DSPs), etc.), embedded controllers, hard-wired circuit systems, etc.

[0268] In addition to what is described herein, various modifications may be made to the disclosed content and implementation of the present invention without departing from the scope of the present invention. Therefore, the illustrations and examples herein should be interpreted in an illustrative and non-restrictive sense. The scope of the present invention should be measured only by reference to the appended claims. Although not described, any system may include and use a power supply, such as but not limited to a battery, an AC-DC converter, to at least receive alternating current and supply direct current, renewable energy (e.g., solar energy or power based on motion), etc.

[0269] One or more aspects of at least one example may be implemented by representative instructions stored on at least one machine-readable medium, which represent various logic within a processor, processor circuit, ASIC, or FPGA that, when read by a machine, computing device, or system, causes the machine, computing device, or system to fabricate logic to perform the techniques described herein. Such representations may be stored on tangible machine-readable media and supplied to various customers or manufacturing facilities to be loaded into a manufacturing machine that actually fabricates a processor, processor circuit, ASIC, or FPGA.

[0270] According to some examples, a computer-readable medium may include a non-transitory storage medium for storing or maintaining instructions, which, when executed by a machine, a computing device, or a system, causes the machine, the computing device, or the system to perform methods and / or operations according to the described examples. Instructions may include any suitable type of code, such as source code, compiled code, interpreted code, executable code, static code, dynamic code, etc. Instructions may be implemented according to a predefined computer language, manner, or syntax to instruct a machine, a computing device, or a system to perform a specific function. Instructions may be implemented using any suitable high-level, low-level, object-oriented, visual, compiled, and / or interpreted programming language.

[0271] Some examples may be described using the expression "in one example" or "example" and its derivatives. These terms mean that a particular feature, structure, or characteristic described in connection with the example is included in at least one example. The appearance of the phrase "in one example" in various places in the specification does not necessarily refer to the same example.

[0272] The expressions "coupled" and "connected" and their derivatives may be used to describe some examples. These terms are not necessarily intended to be synonyms for each other. For example, descriptions using the terms "connected" and / or "coupled" may indicate that two or more elements are in direct physical or electrical contact with each other. However, the term "coupled" may also mean that two or more elements are not in direct contact with each other, but still cooperate or interact with each other.

[0273] The following examples relate to additional examples of the techniques disclosed herein.

[0274] Example 1. An example apparatus may include a circuit system at a first device coupled to a host computing device. The circuit system may determine whether a memory request received via a first link coupled to the host computing device includes a memory address for accessing a memory at the first device or includes a memory address for accessing a memory at a second device, the second device being coupled to the first device via a second link included in a multi-die structure. The circuit system may also cause a memory access to a memory address of a memory at the first device or the second device based on the determination. The circuit system may also send a response to the host computing device via the first link to indicate a status of the memory access to the memory address.

[0275] Example 2. The apparatus according to Example 1 may also include a circuit system for determining, based on a forwarding table maintained at the first device, whether the memory request includes a memory address for accessing a memory at the first device or a memory address for accessing a memory at the second device. For this example, the forwarding table may include an entry indicating whether the memory address included in the memory request is for the memory at the first device or for the memory at the second device.

[0276] Example 3. The apparatus according to Example 1 may also include a circuit system for determining that the memory request includes a memory address for accessing the memory at the second device. The circuit system may also forward the memory request to the second device via a second link coupled to the second device. The circuit system may also receive a response to the memory request from the second device via the second link. The circuit system may also include a response from the second device in the response sent to the host computing device to indicate the status of the access to the memory address of the memory at the second device.

[0277] Example 4. According to the apparatus of Example 3, the memory address of the memory at the second device may include a first portion of a memory address range of the memory at the second device. For this example, the second portion of the memory address range may be included in a second memory address of the memory at the second device. The second device may receive a second memory request from the host computing device via a second link coupled between the second device and the host computing device.

[0278] Example 5. The apparatus of Example 1, wherein the first link comprises a CXL link, and the second link included in the multi-die structure is a high-speed internal connection link having a data bandwidth at least 5 times that of the CXL link.

[0279] Example 6. According to the apparatus described in Example 5, the memory at the first device can be a first HDM, and the memory at the second device can be a second HDM.

[0280] Example 7: The apparatus according to Example 1 may further include a computing circuit system, wherein the computing circuit system includes a graphics processing unit.

[0281] Example 8. An example method may include: determining, by a circuit system of a first device, whether a memory request received via a first link coupled to a host computing device includes a memory address for accessing a memory at the first device or includes a memory address for accessing a memory at a second device, the second device being coupled to the first device via a second link included in a multi-die structure. The method may also include: causing a memory access to a memory address of a memory at the first device or the second device based on the determination. The method may also include: sending a response to the host computing device via the first link to indicate a status of the memory access to the memory address.

[0282] Example 9. According to the method described in Example 8, determining whether a memory request includes a memory address for accessing a memory at a first device or includes a memory address for accessing a memory at a second device may also include: determining based on a routing table maintained at the first device, the routing table including an entry indicating whether the memory address included in the memory request is for the memory at the first device or for the memory at the second device.

[0283] Example 10. The method according to Example 8 may also include: determining that the memory request includes a memory address for accessing the memory at the second device. The method may also include: forwarding the memory request to the second device via a second link coupled to the second device. The method may also include: receiving a response to the memory request from the second device via the second link, and including the response from the second device in a response sent to the host computing device to indicate a status of access to the memory address of the memory at the second device.

[0284] Example 11. According to the method of Example 10, the memory address of the memory at the second device can be a first portion of a memory address range of the memory at the second device. For this example, the second portion of the memory address range can be included in a second memory address of the memory at the second device. The second device can receive a second memory request from the host computing device via a second link coupled between the second device and the host computing device. The second memory request can include a request to access the second memory address.

[0285] Example 12. The method of Example 8, wherein the first link may be a CXL link, and the second link included in the multi-die structure may be a high-speed internal connection link having a data bandwidth at least 5 times that of the CXL link.

[0286] Example 13. According to the method of Example 12, the memory at the first device may be a first HDM, and the memory at the second device may be a second HDM.

[0287] Example 14. Example: At least one machine-readable medium may include multiple instructions, which in response to being executed by a system can cause the system to perform a method according to any one of Examples 8 to 13.

[0288] Example 15. Example apparatus may include a unit for performing the method according to any one of Examples 8 to 13.

[0289] Example 16. Example At least one non-transitory computer-readable storage medium may include multiple instructions that, when executed, may cause a circuit system at a device coupled to a host device to determine whether a memory request received via a first link coupled to the host computing device includes a memory address for accessing a memory at the device or includes a memory address for accessing a memory at a second device, the second device being coupled to the device via a second link included in the multi-die structure. The instructions may also cause the circuit system to make a memory access to a memory address of a memory at the device or the second device based on the determination. The instructions may also cause the circuit system to send a response to the host computing device via the first link to indicate a status of the memory access to the memory address.

[0290] Example 17. According to the at least one non-transitory computer-readable storage medium of Example 16, the instructions may further cause the circuit system to determine whether the memory request includes a memory address for accessing a memory at the device or includes a memory address for accessing a memory at a second device based on a forwarding table maintained at the device. For this example, the forwarding table may include an entry indicating whether the memory address included in the memory request is for the memory at the device or for the memory at the second device.

[0291] Example 18. According to at least one non-transitory computer-readable storage medium described in Example 16, the instructions may also cause the circuit system to determine that the memory request includes a memory address for accessing the memory at the second device. The instructions may also cause the circuit system to forward the memory request to the second device via a second link coupled to the second device. The instructions may also cause the circuit system to receive a response to the memory request from the second device via the second link. The instructions may also cause the circuit system to include the response from the second device in a response sent to the host computing device to indicate the status of the access to the memory address of the memory at the second device.

[0292] Example 19. According to the at least one non-transitory computer-readable storage medium of Example 18, the memory address of the memory at the second device may be a first portion of a memory address range of the memory at the second device. For this example, the second portion of the memory address range may be included in a second memory address of the memory at the second device, the second device receiving a second memory request from the host computing device via a second link coupled between the second device and the host computing device, the second memory request including a request to access the second memory address.

[0293] Example 20. According to the at least one non-transitory computer-readable storage medium of Example 16, the first link may be a CXL link, and the second link included in the multi-die structure may be a high-speed internal connection link having a data bandwidth at least 5 times that of the CXL link.

[0294] Example 21. According to the at least one non-transitory computer-readable storage medium of Example 20, the memory at the device may be a first HDM, and the memory at the second device may be a second HDM.

[0295] Example 22. According to the at least one non-transitory computer-readable storage medium of Example 16, the device may include a graphics processing unit to serve as a computing circuit system of the device.

[0296] Example 23. Example At least one non-transitory computer-readable storage medium may include a plurality of instructions that, when executed by a system at a host computing device, may cause the system to initialize a plurality of devices coupled to the host computing device via a separate host link. The instructions may also cause the system to access registers at each device via the host link to collect information about the device's ability to route memory requests received via the host link to other devices in the plurality of devices. The memory request may be routed via one of a plurality of HSILs that couple the plurality of devices together. The instructions may also cause the system to build a system memory address map based on the collected information. The system memory address map may be used by the host computing device to access the memory address of the memory at the corresponding device among the plurality of devices. The instructions may also cause the system to maintain a separate forwarding table at each device among the plurality of devices, the separate forwarding table indicating the device's ability to route memory requests to access the memory address of the memory at another device among the plurality of devices. The memory request may be received via a host link coupled to the host computing device and forwarded to another device via the HSIL based on the device forwarding table.

[0297] Example 24. According to the at least one non-transitory computer-readable storage medium of Example 23, the separate forwarding table to be maintained at each device may include a separate forwarding table maintained in a corresponding register at each device.

[0298] Example 25. According to the at least one non-transitory computer-readable storage medium of Example 23, the host link can be a CXL link, and the HSIL coupling the multiple devices together can be included in the multi-die structure. Each HSIL can have a data bandwidth at least 5 times that of a single CXL link.

[0299] Example 26. According to the at least one non-transitory computer-readable storage medium described in Example 25, the memory at each corresponding device can be an HDM.

[0300] Example 27. According to the at least one non-transitory computer-readable storage medium of Example 26, the system may be a basic input / output system (BIOS) for a host computing device. For these examples, the instructions may also cause the BIOS to populate the CBHS based on the information collected from the device registers for use by the OS of the host computing device to build a mapping table for each HDM at each corresponding device among the plurality of devices. The mapping table may facilitate OS access to memory addresses of the HDM via the plurality of CXL links. One of the plurality of CXL links may be coupled to a device having the HDM, and the remaining CXL links of the plurality of CXL links may be coupled to other devices among the plurality of devices.

[0301] Example 28. According to the at least one non-transitory computer-readable storage medium of Example 23, a plurality of devices may each include a graphics processing unit to function as a computing circuit system.

[0302] Example 29. Example At least one non-transitory computer-readable storage medium may include multiple instructions that, when executed by a system at a host computing device, may cause the system to receive a request for accessing a memory address of a memory at a first device among multiple devices coupled to the host computing device via a separate host link. The instructions may also cause the system to obtain memory address mapping information to determine how to split the memory address into multiple parts including at least a first part and a second part. The instructions may also cause the system to send the first part of the memory address to the first device in a first memory request message via the first host link to access the first part of the memory address of the memory at the first device. The instructions may also cause the system to send the second part of the memory address to the second device among the multiple devices in a second memory request via the second host link. The second memory request may be forwarded to the first device via the first HSIL to access the second part of the memory address of the memory at the first device.

[0303] Example 30. The at least one non-transitory computer-readable storage medium of Example 29 may further include instructions for causing the system to split the memory address into a plurality of portions including the third portion. For these examples, the system may send the third portion of the memory address in a third memory request to a third device among the plurality of devices via a third host link. The third memory request may be forwarded to the first device via a second HSIL to access the third portion of the memory address of the memory at the first device.

[0304] Example 31. According to at least one non-transitory computer-readable storage medium described in Example 30, the instructions may also cause the system to receive a first response to a first memory request from a first device via a first host link, the first response indicating a status of access to a first portion of a memory address of a memory at the first device. The instructions may also cause the system to receive a second response to a second memory request from a second device via a second host link, the second response indicating a status of access to a second portion of a memory address of the memory at the first device. The instructions may also cause the system to receive a third response to a third memory request from a third device via a third host link, the third response indicating a status of access to a third portion of a memory address of the memory at the first device.

[0305] Example 32. According to the at least one non-transitory computer-readable storage medium of Example 29, the separate host link may be a CXL link, and the first HSIL may be included in a multi-die structure, the multi-die structure including a plurality of HSILs coupling the plurality of devices to each other. Each HSIL of the plurality of HSILs may have a data bandwidth at least 5 times that of a single CXL link.

[0306] Example 33. According to the at least one non-transitory computer-readable storage medium of Example 32, the memory at the first device may be an HDM.

[0307] Example 34. According to the at least one non-transitory computer-readable storage medium of Example 29, a plurality of devices may each include a graphics processing unit to function as a computing circuit system.

[0308] It is emphasized that the abstract of the present disclosure is provided to comply with 37 CFR Section 1.72 (b), which requires an abstract that allows the reader to quickly determine the nature of the technical disclosure. It should be understood that the abstract will not be used to interpret or limit the scope or meaning of the claims. In addition, in the aforementioned detailed description, it can be seen that various features are grouped together in a single example for the purpose of simplifying the present disclosure. The method of the present disclosure should not be interpreted as reflecting the intention that the claimed examples require more features than the features explicitly recited in each claim. Instead, as reflected in the following claims, the subject matter of the invention lies in less than all the features of a single disclosed example. Therefore, the attached claims are incorporated into the detailed description here, wherein each claim is independently used as a separate example. In the attached claims, the terms "including" and "in which" are used as the plain English equivalents of the corresponding terms "comprising" and "wherein", respectively. In addition, the terms "first", "second", "third", etc. are used only as labels and are not intended to impose numerical requirements on their objects.

[0309] Although the subject matter has been described in language specific to structural features and / or methodological acts, it is to be understood that the subject matter defined in the appended claims is not necessarily limited to the specific features or acts described above. Rather, the specific features and acts described above are disclosed as example forms of implementing the claims.

Claims

1. A device, include: Circuitry at a first device coupled to a host computing device, the circuitry performing the following operations: determining whether a memory request received via a first link coupled to the host computing device includes a memory address for accessing a memory at the first device or includes a memory address for accessing a memory at a second device, the second device coupled to the first device via a second link included in the multi-die structure; causing a memory access to a memory address of a memory at the first device or the second device based on the determination; as well as A response is sent to the host computing device via the first link to indicate a status of the memory access to the memory address.

2. The apparatus according to claim 1 further includes the circuit system determining whether the memory request includes a memory address for accessing a memory at the first device or includes a memory address for accessing a memory at the second device based on a forwarding table maintained at the first device, the forwarding table including an entry indicating whether the memory address included in the memory request is for the memory at the first device or for the memory at the second device.

3. The apparatus of claim 1 , further comprising the circuit system performing the following operations: determining that the memory request includes a memory address for accessing memory at the second device; forwarding the memory request to the second device via a second link coupled to the second device; receiving a response to the memory request from the second device via the second link; as well as A response from the second device is included in a response sent to the host computing device to indicate a status of an access to a memory address of a memory at the second device.

4. The apparatus of claim 3 , the memory address of the memory at the second device comprising a first portion of a memory address range of the memory at the second device, in, A second portion of the memory address range is included in a second memory address of a memory at the second device, the second device receiving a second memory request from the host computing device via a second link coupled between the second device and the host computing device, the second memory request including a request to access the second memory address.

5. The device according to claim 1, in, The first link comprises a Compute Express Link (CXL) link, and the second link included in the multi-die structure is a high-speed internal connection link having a data bandwidth at least 5 times that of the CXL link.

6. The device according to claim 5, in, The memory at the first device includes a first host managed device memory (HDM), and the memory at the second device includes a second HDM.

7. The device according to claim 1, further comprising: include: Computing circuitry including a graphics processing unit.

8. A method, include: determining, by circuitry of a first device, whether a memory request received via a first link coupled to a host computing device includes a memory address for accessing a memory at the first device or includes a memory address for accessing a memory at a second device, the second device coupled to the first device via a second link included in the multi-die structure; causing a memory access to a memory address of a memory at the first device or the second device based on the determination; as well as A response is sent to the host computing device via the first link to indicate a status of the memory access to the memory address.

9. The method according to claim 8, in, Determining whether the memory request includes a memory address for accessing a memory at the first device or includes a memory address for accessing a memory at the second device also includes: making a determination based on a routing table maintained at the first device, the routing table including an entry indicating whether the memory address included in the memory request is for the memory at the first device or for the memory at the second device.

10. The method according to claim 8, further comprising: include: determining that the memory request includes a memory address for accessing memory at the second device; forwarding the memory request to the second device via a second link coupled to the second device; receiving a response to the memory request from the second device via the second link; as well as A response from the second device is included in a response sent to the host computing device to indicate a status of the access to a memory address of a memory at the second device.

11. The method of claim 10, the memory address of the memory at the second device comprising a first portion of a memory address range of the memory at the second device, in, A second portion of the memory address range is included in a second memory address of a memory at the second device, the second device receiving a second memory request from the host computing device via a second link coupled between the second device and the host computing device, the second memory request including a request to access the second memory address.

12. The method according to claim 8, in, The first link comprises a Compute Express Link (CXL) link, and the second link included in the multi-die structure is a high-speed internal connection link having a data bandwidth at least 5 times that of the CXL link.

13. The method according to claim 12, in, The memory at the first device includes a first host managed device memory (HDM), and the memory at the second device includes a second HDM.

14. At least one machine-readable medium comprising a plurality of instructions which, in response to being executed by a system, cause the system to perform the method according to any one of claims 8 to 13.

15. An apparatus comprising means for performing the method according to any one of claims 8 to 13.

16. At least one non-transitory computer-readable storage medium comprising a plurality of instructions that, when executed by a system at a host computing device, cause the system to: initializing a plurality of devices coupled to the host computing device via separate host links; accessing registers at each device via the host link to gather information about the device's ability to route memory requests received via the host link to other devices of the plurality of devices, the memory requests to be routed via one of a plurality of high speed internal link (HSIL)s coupling the plurality of devices together; constructing a system memory address map based on the collected information, the system memory address map for use by the host computing device to access memory addresses for memory at a corresponding device among the plurality of devices; as well as A separate forwarding table is maintained at each device among the multiple devices, the separate forwarding table indicating the device's ability to route memory requests for accessing a memory address at another device among the multiple devices, the memory request being received via a host link coupled to the host computing device and forwarded to the other device via the HSIL based on the device's forwarding table.

17. At least one non-transitory computer-readable storage medium according to claim 16, in, The separate forwarding table to be maintained at each device comprises the separate forwarding table maintained in a corresponding register at each device.

18. The at least one non-transitory computer-readable storage medium of claim 16, in, The host link comprises a Compute Express Link (CXL) link, and the HSILs coupling the plurality of devices together are included in a multi-die structure, each HSIL having a data bandwidth at least 5 times greater than a data bandwidth of a single CXL link.

19. The at least one non-transitory computer-readable storage medium of claim 18, in, The memory at each respective device includes host managed device memory (HDM).

20. The at least one non-transitory computer-readable storage medium of claim 19, in, The system includes a basic input / output system (BIOS) for the host computing device, and the instructions further cause the BIOS to: Based on the information collected from the device registers, a CXL bound virtual HDM (vHDM) structure (CBHS) is populated for use by an operating system (OS) of the host computing device to build a mapping table for each HDM at each corresponding device of the plurality of devices, the mapping table facilitating access by the OS to memory addresses of the HDM via a plurality of CXL links, wherein one of the plurality of CXL links is coupled to a device having the HDM and the remaining CXL links of the plurality of CXL links are coupled to other devices among the plurality of devices.

21. The at least one non-transitory computer-readable storage medium of claim 16, in, The plurality of devices each include a graphics processing unit that functions as computing circuitry.

22. At least one non-transitory computer-readable storage medium comprising a plurality of instructions that, when executed by a system at a host computing device, cause the system to: receiving a request to access a memory address of a memory at a first device among a plurality of devices coupled to the host computing device via a separate host link; Acquire memory address mapping information to determine how to divide the memory address into a plurality of parts including at least a first part and a second part; sending the first portion of the memory address in a first memory request message to the first device via a first host link to access the first portion of the memory address of a memory at the first device; as well as The second portion of the memory address is sent in a second memory request to a second device among the multiple devices via a second host link, and the second memory request is to be forwarded to the first device via a first high-speed internal link (HSIL) to access the second portion of the memory address of the memory at the first device.

23. The at least one non-transitory computer-readable storage medium of claim 22, further comprising the instructions causing the system to divide the memory address into a plurality of portions including a third portion, in, The system performs the following operations: The third portion of the memory address is sent in a third memory request to a third device among the plurality of devices via a third host link, the third memory request being forwarded to the first device via a second HSIL to access the third portion of the memory address of the memory at the first device.

24. The at least one non-transitory computer-readable storage medium of claim 23, further comprising the instructions causing the system to: receiving, from the first device via the first host link, a first response to the first memory request, the first response indicating a status of access to the first portion of the memory address of the memory at the first device; receiving, from the second device via the second host link, a second response to the second memory request, the second response indicating a status of access to the second portion of the memory address of the memory at the first device; as well as A third response to the third memory request is received from the third device via the third host link, the third response indicating a status of access to the third portion of the memory address of the memory at the first device.

25. The at least one non-transitory computer-readable storage medium of claim 22, in, The separate host link comprises a compute express link (CXL) link, and the first HSIL is included in a multi-die structure including a plurality of HSILs coupling the plurality of devices to each other, each HSIL of the plurality of HSILs having a data bandwidth at least 5 times that of a single CXL link.

26. At least one non-transitory computer-readable storage medium according to claim 25, in, The memory at the first device includes host managed device memory (HDM).

27. The at least one non-transitory computer-readable storage medium of claim 22, in, The plurality of devices each include a graphics processing unit that functions as computing circuitry.

Images