Active bridge chiplet with integrated cache

Abstract

To provide a system and method for improving system performance by employing active bridge chiplets for coupling GPU chiplets.SOLUTION: A processing system 100 employing active bridge chiplets for coupling GPU chiplets includes a central processing unit 102 communicably coupled to a first GPU chiplet 106-1 of a GPU chiplet array 104. The multiple GPU chiplet arrays include the first GPU chiplet communicably coupled to the CPU via a bus 108 and a second GPU chiplet 106-2 communicably coupled to the first GPU chiplet via an active bridge chiplet 118. The active bridge chiplet is an active silicon die that bridges GPU chiplets and allows partitioning of systems-on-a-chip (SoC) functionality into smaller functional chiplet groupings.SELECTED DRAWING: Figure 1

Description

Background Art

[0001] In computing devices such as mobile phones, personal digital assistants (PDAs), digital cameras, portable players, gaming, and other devices, there is a requirement to integrate more performance and features into a smaller space. As a result, the density of processor dies and the number of dies integrated within a single integrated circuit (IC) package are increasing. Some conventional multi-chip modules include those with two or more semiconductor chips arranged and mounted on a carrier substrate, and in some cases, those mounted on an interposer (so-called "2.5D") mounted on the carrier substrate.

[0002] The present disclosure can be better understood by referring to the accompanying drawings, and many of its features and advantages will become apparent to those skilled in the art. When the same reference numerals are used in different drawings, they indicate similar or identical items.

Brief Description of the Drawings

[0003] [Figure 1] FIG. 1 is a block diagram showing a processing system employing an active bridge chiplet for coupling GPU chiplets according to some embodiments. [Figure 2] FIG. 2 is a block diagram showing a cache hierarchy of GPU chiplets coupled by an active bridge chiplet according to some embodiments. [Figure 3] FIG. 3 is a block diagram showing a cross-sectional view of a GPU chiplet and an active bridge chiplet according to some embodiments. [Figure 4] FIG. 4 is a block diagram showing another cross-sectional view of a GPU chiplet and an active bridge chiplet according to some embodiments. [Figure 5] FIG. 5 is a block diagram showing a processing system utilizing a three-chiplet configuration according to some embodiments. [Figure 6]This flowchart shows a method for performing inter-chiplet communication according to several embodiments. [Modes for carrying out the invention]

[0004] Traditional monolithic die designs are becoming increasingly expensive to manufacture. In CPU architecture, dividing CPU cores into separate units that require less interoperability is more suitable for the heterogeneous computational nature of CPUs, and chiplets have been successfully used to reduce manufacturing costs and improve yield. In contrast, GPU work inherently involves parallel work. However, the geometry processed by a GPU includes not only parts of fully parallel work but also parts that require synchronous ordering between different parts. Therefore, GPU programming models that distribute parts of the work across multiple GPUs tend to be inefficient because it is computationally expensive and difficult to synchronize the memory content of shared resources across the entire system and provide applications with a coherent view of memory. Furthermore, from a logical standpoint, applications are written assuming that the system has only a single GPU. That is, even when a traditional GPU contains many GPU cores, applications are programmed to address a single device. For at least these reasons, bringing chiplet design techniques to GPU architecture has historically been difficult.

[0005] To improve system performance using GPU chiplets without altering a relatively simple programming model, Figures 1-6 illustrate systems and methods that utilize active bridge chiplets to couple GPU chiplets. In various embodiments, the active bridge chiplet is an active silicon die for inter-chiplet communication. In various embodiments, the system includes a central processing unit (CPU) communicatively coupled to a first GPU chiplet of a graphics processing unit (GPU) chiplet array. The GPU chiplet array includes a first GPU chiplet communicatively coupled to the CPU via a bus CPU, and a second GPU chiplet communicatively coupled to the first GPU chiplet via an active bridge chiplet, thereby breaking down the system-on-a-chip (SoC) into smaller functional groups called "chiplets" or "GPU chiplets," where the "chiplets" or "GPU chiplets" perform the functions of various cores of the SoC (e.g., a GPU).

[0006] Currently, various architectures already have at least one level of cache (e.g., L3 or other final-level cache (LLC)) that is coherent across the entire GPU die. Here, chiplet-based GPU architectures place those physical resources (e.g., LLC) on different dies and connect them communicatively so that the LLC level is unified across all GPU chiplets, maintaining cache coherence. Thus, the L3 cache level remains coherent even when operating in a massively parallel environment. During operation, memory address requests from the CPU to the GPU are sent to only a single GPU chiplet, which communicates with the active bridge chiplet to find the requested data. From the CPU's perspective, it appears as if it is addressing a monolithic GPU on a single die. This allows applications to use a large-capacity multi-chiplet GPU as if it were a single device.

[0007] Figure 1 is a block diagram showing a processing system 100 employing an active bridge chiplet for coupling GPU chiplets, according to several embodiments. In the illustrated example, system 100 includes a central processing unit (CPU) 102 for executing instructions and an array 104 of one or more GPU chiplets, such as three exemplary GPU chiplets 106-1, 106-2, and 106-N (collectively, GPU chiplet 106). In various embodiments, the term “chiplet” as used herein refers to any device that includes, but is not limited to, the following characteristics: 1) the chiplet includes an active silicon die containing at least a portion of the computational logic used to solve all problems (i.e., the computational workload is distributed across multiple active silicon dies); 2) the chiplets are packaged together as a monolithic unit on the same substrate; and 3) the programming model preserves the concept that the combination of those individual computational dies (i.e., GPU chiplets) is a single monolithic unit (i.e., each chiplet is not exposed as a separate device to applications that use the chiplet to handle the computational workload).

[0008] In various embodiments, the CPU 102 is connected to system memory 110, such as dynamic random access memory (DRAM), via bus 108. In various embodiments, system memory 110 is implemented using other types of memory, including static random access memory (SRAM) and non-volatile RAM. In the exemplary embodiment, the CPU 102 communicates with system memory 110 and the GPU chiplet 106-1 via bus 108, which is implemented as a peripheral component interconnect (PCI) bus, a PCI-E bus, or another type of bus. However, in some embodiments of system 100, the GPU chiplet 106-1 communicates with the CPU 102 via a direct connection or via other buses, bridges, switches, and routers.

[0009] As illustrated, the CPU 102 includes several processes, such as running one or more applications 112 for generating graphics commands and a user-mode driver 116 (or other drivers such as a kernel-mode driver). In various embodiments, the one or more applications 112 include applications that utilize the functionality of the GPU chiplet 106, such as applications that generate work in the system 100 or operating system (OS). In some embodiments, the application 112 includes one or more graphics instructions that instruct the GPU chiplet 106 to render a graphical user interface (GUI) and / or a graphics scene. For example, in some embodiments, the graphics instructions include instructions that define one or more sets of graphics primitives to be rendered by the GPU chiplet 106.

[0010] In some embodiments, application 112 utilizes a graphics application programming interface (API) 114 to invoke a user-mode driver 116 (or a similar GPU driver). The user-mode driver 116 issues one or more commands to an array 104 of one or more GPU chiplets for rendering one or more graphic primitives into a displayable graphic image. Based on the graphics instructions issued by application 112 to the user-mode driver 116, the user-mode driver 116 formulates one or more graphics commands that specify one or more actions that the GPU chiplets perform to render the graphics. In some embodiments, the user-mode driver 116 is part of application 112 running on CPU 102. For example, in some embodiments, the user-mode driver 116 is part of a gaming application running on CPU 102. Similarly, in some embodiments, a kernel-mode driver (not shown) is part of an operating system running on CPU 102.

[0011] In the embodiment shown in Figure 1, the active bridge chiplet 118 interconnects the GPU chiplets 106 (i.e., GPU chiplets 106-1 to 106-N). Although three GPU chiplets 106 are shown in Figure 1, the number of GPU chiplets in the chiplet array 104 is a matter of design choice and varies in other embodiments, as will be described in more detail below. In various embodiments, as will be described in more detail with respect to Figure 2, the active bridge chiplet 118 includes an active silicon bridge that functions as a high-bandwidth die interconnect between GPU chiplet dies. Furthermore, the active bridge chiplet 118 acts as a memory crossbar with a shared, integrated final-level cache (LLC) to provide inter-chiplet communication and route cross-chiplet synchronization signals. Since the cache is an inherently active component (i.e., requires power to operate), the memory crossbar (e.g., the active bridge chiplet 118) is active to hold its cache memory. Therefore, cache sizing is configurable for different applications following different chiplet configurations, depending on the physical size of the active bridge chiplet 118, and the base chiplet(s) (e.g., GPU chiplet 106) to which the active bridge chiplet 118 (e.g., GPU chiplet 106) is communicably coupled (e.g., GPU chiplet 106) does not incur any costs (e.g., costs related to physical space and power constraints, etc.) for this external cache on the active bridge chiplet 118.

[0012] In general terms, the CPU 102 is communicably coupled to a single GPU chiplet (i.e., GPU chiplet 106-1) via bus 108. Transactions or communications from the CPU to the array 104 of chiplets 106 are received at GPU chiplet 106-1. Subsequently, any inter-chiplet communications are routed via the active bridge chiplet 118 as needed to access memory channels on other GPU chiplets 106. In this way, the GPU chiplet-based system 100 includes a GPU chiplet 106 that can be addressed as a single monolithic GPU (for example, the CPU 102 and any associated applications / drivers are unaware of the chiplet-based architecture), and therefore does not require any chiplet-specific considerations on the part of the programmer or developer.

[0013] Figure 2 is a block diagram showing the cache hierarchy of GPU chiplets coupled by active bridge chiplets in several embodiments. View 200 provides a hierarchical view of the GPU chiplets 106-1, 106-2 and the active bridge chiplet 118 in Figure 1. Each of the GPU chiplets 106-1, 106-2 includes multiple workgroup processors 202 (WGPs) and multiple fixed-function blocks 204 (GFXs) that communicate with L1 cache memory 206 on a given channel. Each GPU chiplet 106 includes multiple individually accessible banks of L2 cache memory 208 and multiple memory PHY 212 channels (represented as GDDR in Figure 2 to indicate connectivity to graphics double data rate (GDDR) memory) mapped to L3 channels. The L2 level cache is coherent within a single chiplet, while the L3 level (L3 cache memory 210 or other final level) cache is integrated and coherent across all of the GPU chiplets 106. In other words, the active bridge chiplet 118 includes an integrated cache (e.g., L3 / LLC in Figure 2) located on a separate die from the GPU chiplet 106, and provides an external integrated memory interface that links two or more GPU chiplets 106 together in a communicative manner. Thus, the GPU chiplet 106, starting from the register transfer level (RTL) perspective, acts as a monolithic silicon die, resulting in fully coherent memory access.

[0014] In various embodiments, the L3 level 210 cache is a memory-attached final level. In conventional cache hierarchies, routing occurs between the L1 level cache and the L2 level cache, and also between the L2 level and memory channels. This routing allows the L2 cache to be coherent within a single GPU core. However, routing introduces synchronization points because if different GPU cores (such as the display engine, multimedia core, or CPU) that have access to GDDR memory want to access data manipulated by the GPU core, the L2 level cache needs to be flushed to GDDR memory so that the other GPU cores can access the latest data. Such operation is computationally expensive and inefficient. In contrast, the memory-attached final level L3 210, located between the memory controller and the GPU chiplet 106, avoids these problems by providing all attached cores with a consistent "view" of the cache and memory.

[0015] The memory-attached final level L3 210 places the L3 level of the cache hierarchy on the active bridge chiplet 118, rather than on the GPU chiplet 106. Therefore, when another client accesses data (e.g., data in DRAM accessed by the CPU), the CPU 102 reads from the L3 level 210 by traversing and connecting through the SDF fabric 216. Furthermore, if the requested data is not cached in the L3 level 210, the L3 level 210 reads from GDDR memory (via the memory PHY 212, though not shown). Thus, the L2 level 208 contains data and is not flushed. In other embodiments, instead of the L3 level 210 being the memory-attached final level, the L3 level cache is located above the SDF fabric 216 in the cache hierarchy. However, in such a configuration, the L3 level (and the memory PHY 212) is local to each GPU chiplet 106 and therefore not part of the integrated cache in the active bridge chiplet 118.

[0016] The graphics data fabric 214 (GDF) of each GPU chiplet 106 connects all of the L1 cache memory 206 to each of the channels of the L2 cache memory 208, thereby enabling each of the workgroup processors 202 and fixed-function blocks 204 to access data stored in any bank of the L2 cache memory 208. Each GPU chiplet 106 also includes a scalable data fabric 216 (SDF) (also known as the SOC memory fabric) that routes to the active bridge chiplet 118 across the graphics core (GC) and system-on-chip (SOC) IP core. The GC includes the CU / WGP, fixed-function graphics blocks, and caches above L3, etc. The portion of the GPU used for conventional graphics and computation (i.e., the GC) is distinguishable from the other portion of the GPU used to handle auxiliary GPU functions such as video decoding, display output, and various system support structures contained on the same die.

[0017] The active bridge chiplet 118 includes multiple L3 cache memory 210 channels that route to all of the GPU chiplets (e.g., GPU chiplets 106-1 and 106-2 in Figure 2). In this way, memory address requests are routed to the appropriate lanes on the active bridge chiplet 118 to access the integrated L3 cache memory 210. Furthermore, because the physical size of the active bridge chiplet 118 is large, such as extending to multiple GPU chiplets 106, those skilled in the art will recognize that a scalable amount of L3 / LLC cache memory and logic (scalable to increase or decrease the amount of memory and logic in different embodiments) is located on the active bridge chiplet 118 in some embodiments. The active bridge chiplet 118 bridges multiple GPU chiplets 106 and is therefore interchangeably referred to as a bridge chiplet, active bridge die, or active silicon bridge.

[0018] Referring to Figure 3, further details of the chiplet-based architecture can be understood, and Figure 3 is a block diagram showing cross-sectional views of active bridge-coupled GPU chiplets according to several embodiments. View 300 provides cross-sectional views of the GPU chiplets 106-1, 106-2 and the active bridge chiplet 118 of Figure 1 taken in section AA. In various embodiments, each GPU chiplet 106 is configured without through-silicon vias (TSVs). As described above, the GPU chiplet 106 is communicably coupled by the active bridge chiplet 118. In various embodiments, the active bridge chiplet 118 is composed of silicon, germanium or other semiconductor material, and in different embodiments, it is an interconnect chip composed of bulk semiconductor, semiconductor on an insulator or other design.

[0019] In different embodiments, the active bridge chiplet 118 includes a plurality of internal conductor traces (not shown) on a single or multiple levels. The traces electrically connect via conductive paths to, for example, the conductor structure of the PHY region of the GPU chiplet 106 (e.g., the memory PHY 212 in Figure 2). In this way, the active bridge chiplet 118 is an active bridge die that communicatively couples and routes communication between the GPU chiplets 106, thereby forming an active routing network.

[0020] As shown in Figure 3, the carrier wafer 302 is coupled to GPU chiplets 106-1 and 106-2. In this embodiment, the TSV 304 passes through the active bridge chiplet to the GPU chiplet 106, but the graphics core die(s) themselves are not made up of any TSV. Instead, to pass signal data, the through-dielectric via (TDV) 306 tunnels through the gap-fill dielectric layer 308. The gap-fill dielectric layer 308 (or other gap-fill material) occupies areas where bridge chiplet dies and graphics core dies(s) are not present (e.g., areas with a vertical misalignment between the GPU chiplet 106 and the active bridge chiplet 118). As shown, the TDV 306 connects the input / output (I / O) power of the GPU chiplet 106 downward to a solder interconnect 310, which in a different embodiment includes solder bumps and microbumps, etc. In this way, the gap-fill dielectric layer 308 aligns the planes of both bumps (e.g., bump 312) of both the GPU chiplet 106 and the active bridge chiplet 118.

[0021] In various embodiments, components as shown in Figure 3 are electrically connected to other electrical structures, such as circuit boards or other structures, via interconnection structures 310, 312 (e.g., solder balls). However, those skilled in the art will recognize that in other embodiments, various types of interconnection structures, such as pins, landlid array structures, and other interconnections, may be used without departing from the scope of the present disclosure.

[0022] Figure 4 is a block diagram showing another cross-sectional view of the GPU chiplet and active bridge chiplet according to several embodiments. View 400 provides a cross-sectional view of the GPU chiplets 106-1, 106-2 and active bridge chiplet 118 of Figure 1 taken in section AA. As described above, the GPU chiplet 106 is communicatively coupled by the active bridge chiplet 118. In various embodiments, the active bridge chiplet 118 is an interconnect chip composed of silicon, germanium or other semiconductor material in different embodiments, and of bulk semiconductor, semiconductor on an insulator or other design in different embodiments.

[0023] In different embodiments, the active bridge chiplet 118 includes a plurality of internal conductors (not shown) on a single level or multiple levels. The traces electrically connect via conductive paths to, for example, the conductor structure of the PHY region of the GPU chiplet 106 (e.g., the memory PHY 212 in Figure 2). In this way, the active bridge chiplet 118 is an active bridge die that communicatively couples and routes communication between the GPU chiplets 106, thereby forming an active routing network.

[0024] As shown in FIG. 4 and in a similar manner to the components of FIG. 3, carrier wafer 402 is coupled to GPU chiplets 106-1, 106-2. However, in contrast to the embodiment of FIG. 3, each GPU chiplet 106 includes through-silicon vias (TSVs) 404. In the configuration of this embodiment, the TSVs 404 penetrate the GPU chiplets 106, but the active bridge chiplet 118 itself is not configured using any TSVs. Further, since the TSVs 404 connect the active bridge chiplet input / output (I / O) power downward to solder interconnects 406, such as solder bumps and micro bumps in different embodiments, the active bridge-coupled GPU chiplets do not include any TDVs. The interconnect structure 408 is electrically coupled to the GPU chiplets 106. In various embodiments, a layer of dummy silicon 410 (or other gap-fill material) occupies areas where the bridge chiplet die and the graphic core die(s) do not exist (e.g., areas with a vertical mismatch between the GPU chiplets 106 and the active bridge chiplet 118). In this way, the layer of dummy silicon 410 brings both the interconnect bumps related to communicably and electrically coupling the GPU chiplets 106 and the active bridge chiplet 118 to the same plane, forming a monolithic chip.

[0025] In various embodiments, components such as those shown in FIG. 4 are electrically coupled to other electrical structures, such as circuit boards, substrates, or other structures, via interconnect structures 406, 408 (e.g., solder balls, etc.). However, those skilled in the art will recognize that in other embodiments, various types of interconnect structures, such as pins, land grid array structures, and other interconnects, may be used.

[0026] As described above with respect to FIGS. 1-4, the active bridge chiplet 118 provides communication between the routing fabrics of two or more dies, and provides a uniform (or nearly uniform) memory access operation for coherent L3 memory access. Those skilled in the art will recognize that the performance of the processing system scales linearly based on the number of GPU chiplets utilized by the nature of physical replication (e.g., as the number of GPU chiplets increases, the number of components such as memory PHY 212 and WGP 202 also increases).

[0027] Referring to FIG. 5, a block diagram of a processing system utilizing a three-chiplet configuration according to some embodiments is shown. Processing system 500 is similar to processing system 100 of FIG. 1, but certain elements are omitted for ease of explanation. As shown, system 500 includes a CPU 102 and three GPU chiplets, such as exemplary GPU chiplets 106-1, 106-2, 106-3. CPU 102 communicates with GPU chiplet 106-1 via bus 108. As an overview of the overall operation, processing system 500 utilizes a master-slave topology, in which a single GPU chiplet that communicates directly with CPU 102 (i.e., GPU chiplet 106-1) is designated as the master chiplet (hereinafter, the primary chiplet or host GPU chiplet). The other GPU chiplets communicate indirectly with CPU 102 via active bridge chiplet 118 and are designated as slave chiplets (hereinafter, secondary GPU chiplets (s)). Thus, primary GPU chiplet 106-1 functions as the single entry point from CPU 102 to the entire GPU chiplet array 104.

[0028] As shown in Figure 5, in one example, the CPU 102 sends access requests (e.g., read requests, write requests, and instructions to perform operations on the GPU chiplet) to the primary GPU chiplet 106-1. As described in more detail with respect to Figure 2, the GPU chiplet 106-1 includes multiple workgroup processors (not shown) and multiple fixed-function blocks (not shown). The primary GPU chiplet controller 502 connects to the final-level cache (LLC) of the GPU chiplet array 104 (e.g., L3 cache memory as described herein) and handles routing between the LLC and the electrically active portion of the logic of the data fabric crossbar (e.g., SDF216 in Figure 2).

[0029] The primary GPU chiplet controller 502 determines whether the data associated with the access request is cached locally in coherent memory only within a single primary GPU chiplet 106-1, or whether the data is cached in an integrated L3 cache memory 210 in the active bridge chiplet 118. Based on the determination that the data associated with the access request is cached locally in coherent memory within a single primary GPU chiplet 106-1, the primary GPU chiplet controller 502 services the access request in the primary GPU chiplet 106-1. However, based on the determination that the data associated with the access request is cached in a commonly shared L3 cache memory 210, the primary GPU chiplet controller 502 routes the access request to the active bridge chiplet 118 for service. The active bridge chiplet 118 returns the result to the primary GPU chiplet 106-1, which returns the requested data to the outgoing requester (i.e., the CPU 102). In this way, the CPU 102 has only a single external view and does not require direct communication to two or more GPU chiplets via the bus 108.

[0030] Those skilled in the art will recognize that while Figure 5 illustrates a specific context of a rectangular active bridge chiplet die 118 traversing the center of three GPU chiplets, various other configurations, die shapes, and geometries are available in different embodiments. For example, in some embodiments, a chiplet includes an active bridge chiplet at one or more corners of a square GPU chiplet, resulting in multiple GPU chiplets being tiled together within a chiplet array. Similarly, in other embodiments, a GPU chiplet includes an active bridge chiplet extending across the entire side of the GPU chiplet, resulting in multiple GPU chiplets being arranged in a long row / column configuration with intervening active bridge chiplets.

[0031] Figure 6 is a flowchart illustrating a method 600 for performing inter-chiplet communication according to several embodiments. In block 602, the primary GPU chiplet of the GPU chiplet array receives a memory access request from the requesting CPU. For example, referring to Figure 5, the primary GPU chiplet 106-1 receives an access request from the CPU 102. In some embodiments, the primary GPU chiplet 106-1 receives the access request in its scalable data fabric 216 via bus 108.

[0032] In block 604, the primary GPU chiplet 106-1 identifies the location where the requested data is cached. That is, the primary GPU chiplet 106-1 determines whether the data is cached in the integrated L3 cache memory 210 in the active bridge chiplet 118. Referring to Figure 5, for example, the primary GPU chiplet controller 502 of the primary GPU chiplet 106-1 determines whether the data associated with the access request is cached locally in coherent memory only within a single primary GPU chiplet 106-1. If the primary GPU chiplet controller 502 determines that the data associated with the access request is cached locally in coherent memory within a single primary GPU chiplet 106-1, then in block 606, the primary GPU chiplet controller 502 services the access request in the primary GPU chiplet 106-1. Subsequently, in block 612, the primary GPU chiplet returns the requested data to the outgoing requester (i.e., the CPU 102) via the bus 108. In some embodiments, returning the requested data to the CPU 102 includes receiving the requested data in the scalable data fabric 216 of the primary GPU chiplet (i.e., GPU chiplet 106-1) and transmitting the requested data to the CPU 102 via the bus 108.

[0033] Returning to block 604, if the primary GPU chiplet controller 502 determines that the data related to the access request is cached in a commonly shared L3 cache memory 210, in block 608, the primary GPU chiplet controller 502 routes the access request to the active bridge chiplet 118 for service. In some embodiments, routing a memory access request includes the scalable data fabric 216 communicating with the active bridge chiplet 118 and the scalable data fabric 216 requesting the data related to the memory access request from the active bridge chiplet 118. Furthermore, if the requested data is not cached in L3 of the active bridge chiplet 118, the memory access request is treated as an L3 miss, and the active bridge chiplet 118 routes the request to a GPU chiplet attached to the GDDR memory that is responsible for servicing the request. The GPU chiplet to which the request has been routed fetches the requested data from the GDDR memory and returns the requested data to the active bridge chiplet.

[0034] In block 610, the active bridge chiplet 118 returns the result to the primary GPU chiplet 106-1. In particular, the return communication is routed through the same signal path of the active bridge chiplet 118 to which the memory access request was routed in block 608. In other embodiments, the request data port and the return data port do not share the same physical path.

[0035] In block 612, the primary GPU chiplet returns the requested data to the outgoing requester (i.e., CPU 102) via bus 108. In some embodiments, returning the requested data to CPU 102 includes receiving the requested data from the active bridge chiplet 118 in the scalable data fabric 216 of the primary GPU chiplet (i.e., GPU chiplet 106-1) and sending the requested data to CPU 102 via bus 108. In this way, CPU 102 has only a single external view and does not require direct communication to two or more GPU chiplets 106 via bus 108.

[0036] Therefore, as described herein, Active Bridge Chiplets deploy monolithic GPU functionality using a set of interconnected GPU chiplets, so that from the programmer / developer's perspective, the GPU chiplet implementation appears as a conventional monolithic GPU. A scalable data fabric of one GPU chiplet can access the lower-level cache(s) on the same chiplet almost simultaneously with accessing the lower-level cache(s) on the Active Bridge Chiplet, thus enabling the GPU chiplet to maintain cache coherency without requiring an additional inter-chiplet coherency protocol. This low-latency inter-chiplet cache coherency allows the chiplet-based system to operate as a monolithic GPU from the software developer's perspective, thus avoiding chiplet-specific considerations on the programmer or developer's side.

[0037] As disclosed herein, in some embodiments, the system includes a central processing unit (CPU) communicatively coupled to a first GPU chiplet of a graphics processing unit (GPU) chiplet array, the GPU chiplet array including a first GPU chiplet communicatively coupled to the CPU via a bus, and a second GPU chiplet communicatively coupled to the first GPU chiplet via an active bridge chiplet, the active bridge chiplet including a level of cache memory shared by the first and second GPU chiplets of the GPU chiplet array. In one embodiment, the level of cache memory includes an integrated cache memory that is coherent across the first and second GPU chiplets of the GPU chiplet array. In another embodiment, the level of cache memory includes a memory-attached final-level cache located between the memory controller of the first GPU chiplet and off-die memory. In yet another embodiment, the active bridge chiplet communicatively couples the GPU chiplets in the GPU chiplet array.

[0038] In one embodiment, the first GPU chiplet further includes a scalable data fabric configured to receive memory access requests from the CPU. In another embodiment, the scalable data fabric is further configured to request data related to the memory access requests from an active bridge chiplet. In yet another embodiment, the active bridge chiplet includes a memory crossbar for chiplet-to-chiplet communication between GPU chiplets in a GPU chiplet array. In yet another embodiment, the system includes a first cache memory hierarchy in the first GPU chiplet, where a first level of the first cache memory hierarchy is coherent within the first GPU chiplet, and a second cache memory hierarchy in the second GPU chiplet, where a first level of the second cache memory hierarchy is coherent within the second GPU chiplet. In another embodiment, the cache memory level in the active bridge chiplet includes an integrated cache memory that includes both the final level of the first cache memory hierarchy and the final level of the second cache memory hierarchy, and the integrated cache memory is coherent across the first and second GPU chiplets of the GPU chiplet array.

[0039] In some embodiments, the method includes: receiving a memory access request from a central processing unit (CPU) in a GPU chiplet of a first GPU chiplet array; determining in the active bridge chiplet controller of the first GPU chiplet that data related to the memory access request is cached in an active bridge chiplet shared by the first GPU chiplet and a second GPU chiplet of the GPU chiplet array; routing the memory access request to an integrated final-level cache in the active bridge chiplet; and returning the data related to the memory access request to the CPU. In one embodiment, routing the memory access request further includes the scalable data fabric requesting the data related to the memory access request from the active bridge chiplet. In another embodiment, the method includes returning the data related to the memory access request to the first GPU chiplet via the scalable data fabric.

[0040] In one embodiment, receiving a memory access request includes the scalable data fabric receiving a memory access request from the CPU. In another embodiment, the method includes receiving data related to the memory access request from an active bridge chiplet via the scalable data fabric. In yet another embodiment, the method includes caching the data in an integrated cache memory of an active bridge chiplet, the integrated cache memory including the final level of a first cache memory hierarchy in a first GPU chiplet, where the first level of the first cache memory hierarchy is coherent within the first GPU chiplet, and the final level of a second cache memory hierarchy in a second GPU chiplet of a GPU chiplet array, where the first level of the second cache memory hierarchy is coherent within the second GPU chiplet.

[0041] In some embodiments, the processor comprises a central processing unit (CPU) and a GPU chiplet array including a first GPU chiplet, the first GPU chiplet including an active bridge chiplet controller, the GPU chiplet array, and an integrated final-level cache. The processor is configured to receive a memory access request from the CPU in the first GPU chiplet, determine in the active bridge chiplet controller of the first GPU chiplet that the data related to the memory access request is cached in an active bridge chiplet shared by the first GPU chiplet and a second GPU chiplet of the GPU chiplet array, route the memory access request to the integrated final-level cache in the active bridge chiplet, and route the data related to the memory access request to the CPU. In one embodiment, the processor is configured to request the data related to the memory access request from the active bridge chiplet via a scalable data fabric. In another embodiment, the processor is configured to return the data related to the memory access request to the first GPU chiplet via a scalable data fabric. In yet another embodiment, the processor is configured to receive memory access requests from the CPU via a scalable data fabric. In yet another embodiment, the first GPU chiplet is configured to receive data related to memory access requests from an active bridge chiplet via a scalable data fabric.

[0042] Computer-readable storage media include any non-temporary storage media or combination of non-temporary storage media that are accessible by a computer system during use to provide instructions and / or data to the computer system. Such storage media may include, but are not limited to, optical media (e.g., compact discs (CDs), digital versatile discs (DVDs), Blu-ray® discs), magnetic media (e.g., floppy disks, magnetic tapes, magnetic hard drives), volatile memory (e.g., random access memory (RAM) or cache), non-volatile memory (e.g., read-only memory (ROM) or flash memory), or microelectromechanical system (MEMS) based storage media. Computer-readable storage media (e.g., system RAM or ROM) may be built into the computing system, computer-readable storage media (e.g., magnetic hard drives) may be permanently mounted to the computing system, computer-readable storage media (e.g., optical disks or Universal Serial Bus (USB) based flash memory) may be detachably mounted to the computing system, and computer-readable storage media (e.g., network-accessible storage (NAS)) may be connected to the computer system via a wired or wireless network.

[0043] In some embodiments, some aspects of the above-described technology may be implemented by one or more processors of a processing system that executes the software. The software includes one or more sets of executable instructions stored in or tangibly embodied on a non-temporary computer-readable storage medium. When executed by one or more processors, the software may include instructions and specific data that operate one or more processors to execute one or more aspects of the above-described technology. The non-temporary computer-readable storage medium may include, for example, magnetic or optical disk storage devices, solid-state storage devices such as flash memory, cache, random access memory (RAM), or one or more other non-volatile memory devices. The executable instructions stored in the non-temporary computer-readable storage medium may be source code, assembly language code, object code, or other instruction formats that can be interpreted or executed by one or more processors.

[0044] In addition to the foregoing, it should be noted that not all activities or elements described in the summary are required, and certain activities or parts of devices may not be required, and one or more additional activities may be performed, and one or more additional elements may be included. Furthermore, the order in which the activities are listed does not necessarily indicate the order in which they are performed. Also, the concepts have been described with reference to specific embodiments. However, those skilled in the art will understand that various modifications and variations can be made without departing from the scope of the invention as described in the claims. Therefore, the specification and drawings should be considered illustrative rather than restrictive, and all of these variations are intended to fall within the scope of the invention.

[0045] Benefits, other advantages, and solutions to problems have been described above with respect to specific embodiments. However, benefits, advantages, solutions to problems, and features that may give rise to or manifest any benefits, advantages, or solutions are not to be construed as essential, necessary, or indispensable features to any or all of the claims. Furthermore, the disclosed invention can be modified and implemented in different but similar ways, in ways that are obvious to those skilled in the art who are interested in the teachings of this specification; therefore, the specific embodiments described above are merely illustrative. There are no limitations to the details of the configuration or design shown herein beyond those described in the appended claims. Accordingly, the specific embodiments described above may be modified or altered, and it is clear that all such modifications are within the scope of the disclosed invention. Accordingly, the protection sought herein is described in the appended claims.

Claims

1. It is a processor, A chiplet array comprising a first chiplet, a second chiplet, and an active bridge chiplet shared by the first and second chiplets, The first chiplet is, Receiving memory access requests, Depending on whether the memory access request can be served by either the second chiplet or the active bridge chiplet, the memory access request is routed to the second chiplet or the active bridge chiplet for service. In response to the memory access request being serviced by the active bridge chiplet or the second chiplet, data related to the memory access request is received from the active bridge chiplet. It is configured to do the following: Processor.

2. The first chiplet is configured to receive the memory access request from another processor. The processor according to claim 1.

3. The first chiplet is configured to return data related to the memory access request to the other processor. The processor according to claim 2.

4. The first chiplet comprises a scalable data fabric configured to receive the memory access requests from the other processors. The processor according to claim 2.

5. The first chiplet is configured to route the memory access request by routing it to the second chiplet via the active bridge chiplet. The processor according to claim 1.

6. The first chiplet is configured to route the memory access request to the second chiplet in response to the fact that the data related to the memory access request is not stored in the memory associated with the active bridge chiplet. The processor according to claim 5.

7. The first chiplet is configured to route the memory access request by routing it to a cache memory level shared by the first and second chiplets in the active bridge chiplet. The processor according to claim 1.

8. The cache memory at the aforementioned level comprises an integrated cache memory that is coherent across the first and second chiplets. The processor according to claim 7.

9. The active bridge chiplet connects the chiplets in the chiplet array in a communicative manner. The processor according to claim 1.

10. It is a system, A processor is communicatively coupled to the first chiplet of the chiplet array. The aforementioned chiplet array is A first chiplet is communicably coupled to the processor via a bus, A second chiplet is communicatively coupled to the first chiplet via an active bridge chiplet, The first chiplet is configured to manage memory access requests related to the chiplet array, which are memory access requests received from the processor. The memory access request includes a memory access request that can be served by the active bridge chiplet or the second chiplet. The first chiplet is configured to receive data related to the memory access request from the active bridge chiplet in response to the memory access request being serviced by the active bridge chiplet or the second chiplet. system.

11. The first chiplet is configured to service the memory access request in such a way that the data related to the memory access request received from the processor is locally cached in memory that is coherent only within the first chiplet. The system according to claim 10.

12. The first chiplet is configured to route the memory access request received from the processor to the active bridge chiplet for service, depending on whether the data related to the memory access request is cached in a cache memory shared by the first and second chiplets. The system according to claim 10.

13. The first chiplet is configured to route the memory access request received from the processor to the second chiplet via the active bridge chiplet in order to service the request, depending on whether the data related to the memory access request is not stored in the memory associated with the active bridge chiplet. The system according to claim 10.

14. The active bridge chiplet includes a level of cache memory shared by the first chiplet and the second chiplet. The system according to claim 10.

15. It is a method, In the first chiplet of the chiplet array, a memory access request is received from the processor, Depending on whether the memory access request can be served by the second chiplet of the chiplet array, or by an active bridge chiplet shared by the first chiplet and the second chiplet, the memory access request is routed to the second chiplet or the active bridge chiplet. In response to the memory access request being serviced by the active bridge chiplet or the second chiplet, the first chiplet receives data related to the memory access request from the active bridge chiplet, This includes sending back data related to the memory access request to the processor, method.

16. Routing the memory access request to the second chiplet is performed depending on whether the data related to the memory access request is not stored in the memory associated with the active bridge chiplet. The method of claim 15.

17. Routing the memory access request includes routing the memory access request to the second chiplet via the active bridge chiplet. The method of claim 15.

18. Routing the memory access request includes routing the memory access request to a level of cache memory shared by the first chiplet and the second chiplet in the active bridge chiplet. The method of claim 15.

19. The aforementioned level of cache memory is the integrated final-level cache of the active bridge chiplet. The method of claim 18.

Images