Optical-based distributed unified memory system

Abstract

Techniques related to computing systems using silicon photonics are disclosed. In some embodiments, a computing system includes a plurality of computing die packages including a processor configured to execute program instructions that operate on data stored in a distributed memory accessible via a unified memory architecture. The computing system also includes a plurality of memory die packages configured to implement a unified memory architecture, such that a given one of the memory die packages includes one or more optical interfaces configured to receive memory requests from the processor and one or more memory controllers configured to access a portion of the distributed memory in response to the received memory requests.

Description

Background Technology

[0001] This disclosure relates generally to computing systems, and more specifically to computing systems using silicon photonics. Technical Field

[0002] Modern mobile devices (such as smartphones) typically include a System-on-a-Chip (SoC), where multiple components of a computing device (e.g., a central processing unit (CPU), graphics processing unit (GPU), peripheral interfaces, memory, etc.) are combined into a single chip. This type of architecture offers several advantages over the more traditional personal computer (PC) architecture, where components reside on separate chips (or even separate cards) inserted into a motherboard. For example, SoCs can reduce the physical size of the device, which can be useful for applications requiring portability. SoCs can improve device performance by reducing communication latency and power consumption between different components. SoCs can also reduce the cost of the device by reducing the number of external components and simplifying the design and manufacturing process.

[0003] One of the beneficial effects of SoC architecture is the ability to achieve a unified memory architecture, where multiple components within the SoC can share a common memory pool. This enables high-bandwidth and low-latency access between components because a CPU, for example, wanting to pass data to the GPU can simply inform the GPU where the data is stored in the memory pool, allowing the GPU to access the data directly. This approach can be more efficient, for example, than moving data back and forth between system memory and dedicated GPU memory. Attached Figure Description

[0004] Figure 1 This is a block diagram illustrating an example of a memory die package in an optically based distributed unified memory system.

[0005] Figure 2 Examples include Figure 1 A block diagram showing an example of a memory tray with multiple memory die packages.

[0006] Figure 3 Examples include Figure 2 A block diagram of an example of a memory rack with multiple memory trays.

[0007] Figure 4 This is a block diagram illustrating an example of a computing die package in an optically based distributed unified memory system.

[0008] Figure 5 Examples include Figure 4 A block diagram showing an example of a computing tray with multiple computing die packages.

[0009] Figure 6 Examples include Figure 5 A block diagram of an example computer rack with multiple computing trays.

[0010] Figure 7 Examples include Figure 3 Memory racks and Figure 5 A block diagram of an example optical-based distributed unified memory system in a computer rack.

[0011] Figure 8 and Figure 9 This is a block diagram illustrating an example of a fiber optic array used by an optically based distributed unified memory system.

[0012] Figure 10 This is a block diagram illustrating an example of a multi-node system comprising multiple optically based distributed unified memory systems.

[0013] Figure 11 This is a flowchart illustrating an example of a method that implements the functionality described in this article.

[0014] Figure 12 This is a block diagram illustrating an example computer-readable medium for storing circuit design information used to implement a device having the functionality described herein. Detailed Implementation

[0015] SoCs are primarily used in small computing systems such as smartphones, tablets, wearables, and laptops. Larger computing systems, such as server computing systems, are subject to power and physical space constraints at a much larger scale, and their design focuses less on using SoCs and more on supporting traditional architectures where components are implemented individually. However, when SoCs are used in larger computing systems, the unified memory architecture implemented by a given SoC is limited to the components within that SoC. Therefore, a CPU within a first SoC can pass data to a first GPU within that SoC via the architecture, but cannot use the same architecture to pass data to a second GPU in a second SoC because the second GPU cannot directly access the internal memory pool of the first SoC. This limitation can severely reduce the benefits of using a unified memory architecture.

[0016] This disclosure describes an implementation scheme in which silicon photonics is used to scale the unified memory architecture for larger computing systems, such as server computing systems, to create a distributed unified memory system. As will be described below in various embodiments, the computing system may include multiple computing die packages including processors configured to execute program instructions that operate on data stored in a distributed memory accessible via the unified memory architecture. These processors may include any suitable type of processor, such as one or more central processing units (CPUs), graphics processing units (GPUs), application-specific integrated circuits (ASICs), or field-programmable gate arrays (FPGAs). The computing system may also include multiple memory die packages configured to implement the unified memory architecture, wherein a given memory die package includes one or more optical interfaces configured to receive memory requests from a processor and one or more memory controllers configured to access a portion of the distributed memory in response to a received memory request. A given computing die package may also include a computing die and an optical die, the computing die including one or more processors, and the optical die coupled to the computing die via die-to-die interconnects and including one or more optical interfaces.

[0017] Implementing a unified memory architecture in this way allows for greater resource utilization because processors within the compute die package can share the use of distributed memory implemented by the memory die package. This architecture also allows for easy scaling of the computing system, which in some implementations may include multiple distributed nodes, each comprising a rack of compute die packages and memory die packages.

[0018] Turn now Figure 1A block diagram of a memory die package 100 at the core of a computing system is depicted. In various embodiments, the memory die package 100 includes one or more co-packaged dies comprising: an optical interface configured to receive memory requests from one or more processors via one or more optical fibers; a memory controller coupled to the memory and configured to service the memory requests; and a memory cache configured to store a portion of the data stored in the memory. In an illustrated embodiment, the memory die package 100 includes a memory die 110 including a memory cache 112, a die-to-die physical interface 114, and a memory physical interface 116. The memory physical interface 116 is memory controller circuitry coupled to one or more co-packaged memories 130. The memory cache 112 is a cache configured to store a subset of the data stored in the memory 130. In the illustrated embodiment, the memory die package 100 also includes a plurality of optical dies 120 coupled to the memory controller die 110 via die-to-die physical interface 124 interconnects and including one or more optical input / output interfaces 122 configured to connect via optical fiber to the following. Figure 4 The other memory die package 100 and computing die package 400 discussed communicate with each other. In some embodiments, the memory die package 100 can be implemented in a different manner than shown. In some embodiments, dies 110 and 120 (together with those described below) Figure 4 The computing die 420 discussed is a small chip that works together to implement a system-on-a-chip architecture.

[0019] In some embodiments, the memory die package 100 may provide 64 LP5X channels, which provide up to 1088 GB / s of memory bandwidth (at 8.5 GT / s) and up to 512 GB of storage capacity (16 Gb density, byte mode, dual in-line). The memory cache 112 may include a 1 GB SRAM-based memory cache. In an illustrated embodiment, the Y dimension of the memory die 110 may determine the number of memory channels that can be supported, and the appropriate cache capacity within the die region. The X dimension of the optical die 120 may determine the amount of bandwidth that can be supported to and from the memory die package 100. In some embodiments, for each optical die in the optical die 120, the supported bandwidth is equal to 2.5 TB / s, thus supporting a total of up to 5 TB / s.

[0020] In some implementations, dies 110 and 120 may be small chips connected via silicon interposers, silicon bridges, or other similar high-density interconnect technologies known as 2.5D integration. In other implementations, dies 110 and 120 may be stacked vertically and connected face-to-face or face-to-back using through-silicon vias on one die, a method known as 3D integration. In these 2.5D or 3D integration implementations, D2D PHY 114 and D2D PHY 124 can communicate over a large number of signal paths, such as 8,000 signals operating at 4GT / s, to provide an interconnect bandwidth of 20Tb / s (2.5TB / s).

[0021] In some embodiments, the optical die 120 may be a monolithic die integrating silicon photonic components (such as silicon waveguides, microring resonators, Mach-Zehnder modulators, and photodetectors) as well as active electronics for the D2D PHY 124 and electrical portions of the optical IO 122 (such as modulator drivers, transimpedance amplifiers, and serialization / deserialization (SerDes) circuitry). In some embodiments, the optical die 120 may consist of a stack of two component dies, one containing electronic components and the other containing photonic components, which are fabricated in different semiconductor manufacturing processes. In some embodiments, an external laser may provide continuous-wave optical input to the optical die 120 via an optical fiber cable.

[0022] In some implementations, the optical die 120 may consist of an electronic die on which a series of directly modulated light sources (such as vertical cavity surface emitting lasers (VCSELs) or light-emitting diodes (LEDs)) and photodetectors (such as PIN photodiodes) are mounted.

[0023] In some embodiments, the optical fiber is connected to the memory die package 100 via a fiber optic attachment unit (FAU) that optically couples the optical fiber to a photonic assembly within the optical die 120. In some embodiments, the FAU may be edge-coupled to optically couple the fiber waveguide to a silicon waveguide on the optical die 120 in a coplanar orientation. In some embodiments, the FAU may be vertically coupled to optically couple the fiber waveguide to devices on the face of the optical die 120, such as photodiodes, VCSELs, and LEDs, or to the silicon waveguide of the optical die 120 via a vertical grating coupler (VGC).

[0024] Memory 130 can be any type of volatile memory, such as Dynamic Random Access Memory (DRAM), Synchronous DRAM (SDRAM), Dual Data Rate (DDR, DDR2, DDR3, etc.) SDRAM (including mobile versions of SDRAM, such as mDDR3, and / or low-power versions of SDRAM, such as LPDDR4), RAMBUS DRAM (RDRAM), Static RAM (SRAM), etc. One or more memory devices can be coupled onto a circuit board to form a memory module, such as a Single In-line Memory Module (SIMM), a Dual In-line Memory Module (DIMM), etc. These devices can be mounted with integrated circuits in a chip stack configuration, a package stack configuration, or a multi-chip module configuration. Memory 130 can also be any type of non-volatile memory, such as NAND flash memory, NOR flash memory, nanometer RAM (NRAM), magnetoresistive RAM (MRAM), phase-change RAM (PRAM), racetrack memory, memristor memory, etc.

[0025] In various implementations, memory die package 100 is configured to implement a unified memory architecture (UMA), wherein distributed memory is implemented across memory 130 (including memory 130 in other memory die packages 100) and is handled by multiple processors (such as combined...). Figure 4The processors in the compute die package 400 discussed share the UMA. In some embodiments, the UMA is implemented such that a given processor in the processor is configured to access any memory address within a unified address space defined by the unified memory architecture. In some embodiments, this unified address space is a virtual address space different from the physical address space provided by memory 130. In some embodiments, the memory die package 100 is also configured to implement the UMA in a manner that makes memory access across the memory die package 100 transparent to software executing on the compute die package 400. In some embodiments, the UMA is implemented such that a given page within the unified address space is distributed among multiple memory die packages in the memory die package 100. In some embodiments, the memory die package 100 is configured to perform address hashing to ensure a uniform distribution of data across memory 130 in the UMA. In some embodiments, to achieve different characteristics for accessing memory 130, the optical interface 122 in memory die package 100 and compute die package 400 is configured to implement an optical interconnect structure comprising at least two networks with heterogeneous operating characteristics. In some embodiments, one or more of these characteristics include: a shorter route for memory requests in the first network compared to the second network; increased bandwidth in the first network compared to the second network; and / or a larger number of optical fibers allocated to the first network compared to the second network. In some embodiments, the first and second networks have heterogeneous interconnect topologies, such as star topology, mesh topology, ring topology, tree topology, fat tree topology, hypercube topology, or a combination of one or more of these topologies. In some embodiments, the at least two networks comprise a coherent network interconnecting CPUs in compute die package 400 and memory die 110, and a relaxed-ordered network interconnecting GPUs in compute die package 400 and memory die 110. In some implementations, a given memory die package in memory die package 100 is coupled to a first network and a second network and is configured to receive memory requests from both the first network and the second network.

[0026] Turn now Figure 2A block diagram of a memory tray 200 is depicted. As shown, the memory tray 200 includes one or more memory die packages 100. In some embodiments, memory addresses are uniformly distributed across the memory die packages 100 in the memory tray 200. The total available memory bandwidth and memory capacity provided by the memory tray 200 depend on the number of memory die packages 100 in the memory tray 200. For simplicity, the memory tray 200 may include a power of two number of die packages 100. In an illustrated embodiment, the memory tray 200 includes eight memory die packages 100, each memory die package 100 being mapped to 1 / 8 of the memory address space. This can be equivalent to 512 LP5X channels, which provide up to 8.704 TB / s of memory bandwidth (at 8.5 GT / s) and up to 4 TB of storage capacity (16 Gb density, byte mode, dual in-line). This can also be equivalent to a tray 200 including a memory cache based on 8 GB of SRAM. For completeness, in some embodiments, since each memory die supports up to 5 TB / s of optical bandwidth, the memory tray 200 can support up to 40 TB / s of optical bandwidth. In some embodiments, the tray 200 can be implemented in a different manner than shown.

[0027] For convenience, one side of the memory tray 200 can be referred to as the north side, and the other side as the south side. Each memory tray 200 has a plurality of ports 210, which can be twice the number of memory die packages 100 inside it. Half of these ports 210 are located on the north side of the memory tray 200, and the other half are located on the south side of the memory tray 200. Within a given memory tray 200, the ports 210 can be numbered from left to right, starting with 1 and followed by N or S to indicate the side of the tray. A given memory tray 200 may also have an associated ID, as indicated by the letters in the lower left part of the figure (in this case, memory tray A). Thus, the ports in this example are identified as A1N…A8N, A1S…A8S.

[0028] To facilitate connectivity between each memory die package 100 and each memory tray port 210, in some embodiments, each memory die package 100 is connected via an optical fiber to all ports in the memory tray 200. In some embodiments, this optical interconnect structure is configured to allow interconnection of a variable number of processor cores, graphics processing units, peripheral devices, or memory controller circuitry.

[0029] In some implementations, optical port 210 may terminate to an optical connector such as an MTP connector to allow fiber optic cable or multi-core fiber to be externally connected to memory tray 200.

[0030] Turn now Figure 3 A block diagram of a memory rack 300 is depicted. As shown, the memory rack 300 includes one or more compute trays 200 inserted adjacent to each other. In some embodiments, memory addresses are uniformly distributed across memory die packages 100 within the memory rack 300. The total available memory bandwidth and memory capacity provided by the memory rack 300 depend on the number of memory die packages 100 in the memory rack 300. For simplicity, the memory rack 300 may include a power of two number of memory trays 200, and each memory tray 200 may include a power of two number of memory die packages 100. In an illustrated embodiment, the memory rack 300 includes eight memory trays 200, wherein each memory tray 200 includes eight memory die packages 100. In this case, each memory die package 100 is mapped to 1 / 64 of the memory address space. This can be equivalent to a rack 300 with 4096 LP5X channels, providing up to 69.632 TB / s of memory bandwidth (at 8.5 GT / s) and up to 32 TB of storage capacity (16 Gb density, byte mode, dual in-line). This can also be equivalent to a rack 300 including a memory cache based on 64 GB of SRAM. For completeness, in some embodiments, the rack 300 can support up to 320 TB / s of optical bandwidth because each memory die package 100 supports up to 5 TB / s of optical bandwidth. In some embodiments, the rack 300 can be implemented in a different manner than shown.

[0031] Within a given memory rack 300, trays 200 are identified from bottom to top starting with A. To uniquely identify a given port in the memory rack 300, the port number can be specified by side in the form of {Memory Tray ID} followed by {Tray Port Number} and N or S. For example, port C4N refers to port 4 (north side) on tray C.

[0032] In the illustrated implementation, the memory rack has a total of 128 ports, 64 ports on each side. The ports in this example are labeled A1N……A8N, A1S……, A8S……H1N……H8N, H1S……H8S.

[0033] Turn now Figure 4A block diagram of a computing die package 400 is depicted. As shown, the computing die package 400 includes one or more computing dies 420 co-packaged with one or more optical dies 120, which are coupled via die-to-die interconnects created by physical interfaces 114 and 124. The computing die 420 can be customized for any suitable use case and can include any suitable processor 412. For example, the computing die 420 can include one or more CPUs, GPUs, neural engines, ASICs, FPGAs, IP hybrids, etc. In some embodiments, the computing die package 400 includes an 18-core CPU and a 40-FSTP GPU. The X-dimensional dimension of the optical die 120 can determine the amount of bandwidth that can be supported to and from the computing die package 400. In some embodiments, the supported bandwidth for the optical die 120 is equal to 2.5 TB / s. In some embodiments, the computing die package 400 can be implemented in a different manner than shown.

[0034] Turn now Figure 5 A block diagram of a compute tray 500 is depicted. As shown, the compute tray 500 includes one or more compute die packages 400. The total available computing power provided by the compute tray 500 depends on the number of compute die packages 400 in the compute tray 500. For simplicity, the compute tray 500 may include a power of two number of compute die packages 400. It should also be noted that the compute die packages 400 in the compute tray 500 do not need to be identical—for example, the selection of each compute die package 400 can be left to the system operator. In the illustrated embodiment, the compute tray includes eight identical compute die packages 400, each of which may include an 18-core CPU and a 40-FSTP GPU. This can be equivalent to a 144-core CPU and a 320-FSTP GPU. For completeness, since each compute die package 400 can support up to 2.5 TB / s of optical bandwidth, the compute tray 500 can support up to 20 TB / s of optical bandwidth. In some implementations, tray 500 can be implemented in a different manner than shown.

[0035] Each compute tray 500 has multiple ports 502, which can be equal to the number of compute die packages 400 inside it. Within a given compute tray 500, the ports 502 can be numbered from 1 from left to right. A given compute tray can also have an associated ID, as indicated by the letters in the lower left part of the attached figure (in this case, compute tray A). Thus, the ports 502 in this example are identified as A1...A8.

[0036] To facilitate connectivity between each compute die package 400 and each port of the compute tray 500, in some embodiments, each compute die package 400 is connected via an optical fiber to all ports in the compute tray 500.

[0037] Turn now Figure 6 A block diagram of a computer rack 600 is depicted. As shown, the computer rack 600 includes one or more compute trays 500 inserted adjacent to each other. The total available computing power provided by the computer rack 600 depends on the number and type of compute die packages 400 in the computer rack 600. For simplicity, the computer rack 600 may include a power of two number of compute trays 500, and each compute tray 500 may include a power of two number of compute die packages 400. It should also be noted that the compute die packages 440 in the computer rack 600 do not need to be identical—for example, the selection of each compute die package 400 can be left to the system operator. In an illustrated embodiment, the computer rack 600 includes eight compute trays 500, each of which includes eight compute die packages 400, and each of the eight compute die packages may include an 18-core CPU and a 40-FSTP GPU. This is equivalent to 1152-core CPUs and 2560-FSTP GPUs. For completeness, since each computing die package 400 can support up to 2.5 TB / s of optical bandwidth, the computing rack 600 can support up to 160 TB / s of optical bandwidth. In some embodiments, the rack 600 can be implemented in a different manner than shown.

[0038] Within a given computer rack 600, trays are identified from bottom to top, starting with A. To uniquely identify a given port within the computer rack 600, the port number can be represented in the form of {computer rack ID} followed by {tray port number}. For example, port C4 refers to port 4 on tray C.

[0039] In the illustrated implementation, the computer rack 600 has a total of 64 ports. The ports in this example are labeled A1...A8, ..., H1...H8.

[0040] Turn now Figure 7A block diagram of an optically based distributed memory system / node 700 is depicted. In the illustrated implementation, node 700 includes a single memory rack 300 and multiple computer racks 600A and 600B, each computer rack positioned on either side of the memory rack 300. In the example described so far (8 die packages per tray and 8 trays per rack), each computer rack 600 may have 64 ports; each memory rack 300 may have 128 ports, with 64 ports facing south and 64 ports facing north. To facilitate connectivity between each compute tray 500 and each memory tray 200, each port in a given compute tray 500 may be connected to a different memory tray 200 via optical fiber. For simplicity, it is assumed that a given port number on compute tray 500 is connected to the same port number on memory tray 200. In other words, compute port X4 may be connected to memory port Y4 (N or S, depending on which side it resides on), where X and Y are tray IDs. Furthermore, for simplicity, it is assumed that port 1 on a given compute tray 500 can be connected to port 1 on an equivalent memory tray 200, and that subsequent ports on compute tray 500 can be connected to subsequent memory trays 200.

[0041] In some implementations, node 700 includes 4096 LP5X channels, which can provide up to 69632 TB / s of memory bandwidth (at 8.5 GT / s) and up to 32 TB of storage capacity (16 Gb density, byte mode, dual in-line). In some implementations, node 700 also includes a memory cache based on 64 GB of SRAM. Assuming each compute die package 400 includes an 18-core CPU and a 40-FSTP GPU, node 700 can include 128 compute die packages 400, which have a total of 2304 CPU cores and 5120-FSTP GPUs. For completeness, since each memory die package 100 can support up to 5 TB / s of optical bandwidth, node 700 can support up to 320 TB / s of optical bandwidth.

[0042] It should also be noted that each compute die package 400 exhibits the same latency and bandwidth characteristics as the main memory. In doing so, the node 700 is essentially the largest UMA machine ever designed.

[0043] In some implementations, the system / node 700 can be implemented in a different manner than shown, such as including more or fewer racks 300 and / or 600. In some implementations, multiple compute nodes 700 may also be interconnected, such as by combining... Figure 10 The subject of discussion.

[0044] Turn now Figure 8 A block diagram of a multi-core fiber array 800 is depicted. In the example shown, each optical die 120 can support a bandwidth of 2.5 TB / s. To support an 8×8×8 node configuration, each compute die package 400 (or memory die package 100) communicates with 64 individual memory die packages 100 (or compute die packages 400) (end-to-end). This can be achieved using 128 optical fibers (2 fibers per connectivity to support both directions), with each fiber supporting up to 20 GB / s. One way to achieve this is by using, for example... Figure 8 The multi-core fiber array 800 is shown. In the illustrated embodiment, this multi-core fiber array is a 61-core fiber array, where each core can have a diameter of 0.05 mm and supports a switching speed of 4 Gbps. 40 cores can be used to transmit / receive data, and the remaining 21 cores can be used for any auxiliary functionality outside of this specification. Electro-optical (E / O) conversion and photoelectric (O / E) conversion are achieved at the optical die 120 using modulators or photodetectors. In some embodiments, the fiber array 800 can be implemented in a different manner than shown. For example, a smaller number of cores can be accommodated if a faster speed can be achieved.

[0045] Turn now Figure 9 A block diagram of a fiber optic array 900 is depicted. In the illustrated embodiment, the fiber optic array 900 is a 128-fiber array. Assuming that the data routing to / from the transceiver has a minimum distance, the total beachfront area can be approximately 19.2 mm. Figure 9 In the diagram, light gray represents Tx (transmit) and dark gray represents Rx (receive). In general, assuming the core capabilities mentioned above are present, this 128-fiber array can support data transmission of 2.5 TB / s. In some implementations, the fiber array 900 can be implemented in a different manner than shown.

[0046] Turn now Figure 10 A block diagram of a multi-node system 1000 is depicted. In some implementations, it may still be beneficial to connect multiple nodes 700 together, each with its own shared memory. One option is to use a networking interface for this. An alternative is to replace any number of compute die packages 400 with network direct memory access (DMA) die packages that directly interface with the memory racks 300 of the nodes 700. This can provide very high bandwidth between multiple nodes 700, but may come at the cost of reduced computation. Figure 10An example of connecting four nodes together is shown, where each node can have 126 compute die packages 400. In some implementations, the system 1000 can be implemented in a different manner than shown.

[0047] Turn now Figure 11 A flowchart of method 1100 is depicted. Method 1100 is an implementation of a method performed by a computing system such as an optically based distributed memory system 700.

[0048] Method 1100 begins at step 1110, which includes a processor (e.g., processor 412) in one of a plurality of computing die packages (e.g., computing die package 400) executing program instructions that operate on data stored in a distributed memory (e.g., memory 130) accessible via a unified memory architecture. In step 1120, one or more optical interfaces (e.g., optical I / O 122) in a given memory die package (e.g., memory die package 100) configured to implement a unified memory architecture receive a memory request from the processor. In step 1130, one or more memory controllers (e.g., memory physical interface 116) access a portion of the distributed memory in response to the received memory request.

[0049] Example computer-readable media Various example circuits have been described in detail above in this disclosure. It is intended that this disclosure cover not only embodiments including such circuits, but also computer-readable storage media that include design information specifying such circuits. Therefore, this disclosure is intended to support claims that cover not only devices including the disclosed circuits, but also storage media specifying such circuits in a format recognized by a computing system configured to generate a simulation model of hardware circuitry, a manufacturing system configured to generate hardware (e.g., integrated circuits) including the disclosed circuits. Claims regarding such storage media are intended to cover entities that, for example, generate circuit designs but do not themselves perform complete operations (such as design simulation, design synthesis, circuit fabrication, etc.).

[0050] Turn now Figure 12 This illustrates a block diagram of an example non-transitory computer-readable storage medium for storing circuit design information. In the illustrated embodiment, computing system 1240 is configured to process the design information. This may include executing instructions included in the design information, interpreting instructions included in the design information, compiling, converting, or otherwise updating the design information. Thus, in some embodiments, the design information (e.g., through programming computing system 1240) controls computing system 1240 to perform the various operations discussed below.

[0051] In the illustrated example, computing system 1240 processes design information to generate both a computer simulation model of hardware circuitry 1260 and lower-level design information 1250. In other embodiments, computing system 1240 may generate only one of these outputs, generate other outputs based on the design information, or both. Regarding the computational simulation, computing system 1240 may execute instructions in a hardware description language, which includes register-transfer-level (RTL) code, behavioral code, structural code, or some combination thereof. The simulation model may perform the functionality specified by the design information, facilitate verification of the functional correctness of the hardware design, generate power consumption estimates, generate timing estimates, etc.

[0052] In the illustrated example, computing system 1240 also processes design information to generate lower-level design information 1250 (e.g., gate-level design information, netlists, etc.). As shown, this may include synthesis operations such as constructing multi-level networks, optimizing the network using technology-independent techniques, technology-dependent techniques, or both, and outputting a gate network (with potential constraints based on a library of available gate pairs, size adjustments, delays, power, etc.). Based on the lower-level design information 1250 (and potentially other inputs), semiconductor manufacturing system 1220 is configured to manufacture integrated circuit 1230 (which may correspond to the functionality of simulation model 1260). Note that computing system 1240 may generate different simulation models based on design information at various descriptive levels (including information 1250, 1215, etc.). Data representing design information 1250 and model 1260 may be stored on medium 1210 or one or more other media.

[0053] In some implementations, lower-level design information 1250 controls (e.g., programs) the semiconductor manufacturing system 1220 to manufacture integrated circuit 1230. Thus, when processed by the manufacturing system, the design information can program the manufacturing system to manufacture circuits including the various circuits disclosed herein.

[0054] The nontransitory computer-readable storage medium 1210 may include any of a variety of suitable types of memory devices or storage devices. The nontransitory computer-readable storage medium 1210 may be an installation medium, such as a CD-ROM, floppy disk, or magnetic tape device; computer system memory or random access memory, such as DRAM, DDR RAM, SRAM, EDO RAM, Rambus RAM, etc.; non-volatile memory, such as flash memory; magnetic media, such as hard disk drives or optical storage devices; registers, or other similar types of memory elements. The nontransitory computer-readable storage medium 1210 may also include other types of nontransitory memory or combinations thereof. Therefore, the nontransitory computer-readable storage medium 1210 may include two or more memory media; such media may reside in different locations—for example, in different computer systems connected via a network.

[0055] Design information 1215 may be specified using any of a variety of suitable computer languages, including hardware description languages ​​such as, but not limited to, VHDL, Verilog, SystemC, SystemVerilog, RHDL, M, MyHDL, etc. The formats of various design information may be recognized by one or more applications executed by computing system 1240, semiconductor manufacturing system 1220, or both. In some embodiments, design information may also include one or more cell libraries specifying the composition, layout, or both of integrated circuit 1230. In some embodiments, design information is specified, wholly or partially, in the form of a netlist specifying cell library elements and their connectivity. Individually, the design information discussed herein may or may not include sufficient information for manufacturing the corresponding integrated circuit. For example, design information may specify circuit elements to be manufactured but not their physical layout. In this case, design information may need to be combined with layout information to actually manufacture the specified circuit.

[0056] In various implementations, integrated circuit 1230 may include one or more custom macrocells, such as memory and analog or mixed-signal circuitry. In this case, design information may include information associated with the included macrocells. Such information may include, but is not limited to, schematic capture databases, mask design data, behavioral models, and device or transistor-level netlists. Mask design data may be formatted according to a Graphical Data System (GDSII) or any other suitable format.

[0057] The semiconductor manufacturing system 1220 may include any of the various suitable elements configured to manufacture integrated circuits. This may include, for example, elements for depositing semiconductor material (e.g., on a wafer that may include a mask), removing material, changing the shape of the deposited material, modifying the material (e.g., by doping the material or by using ultraviolet treatment to modify the dielectric constant), etc. The semiconductor manufacturing system 1220 may also be configured to perform various tests on the manufactured circuits to ensure proper operation.

[0058] In various embodiments, integrated circuit 1230 and model 1260 are configured to operate according to a circuit design specified by design information 1215, which may include performing any of the functionalities described herein. For example, integrated circuit 1230 may include Figures 1 to 10 Any of the various components shown. Furthermore, integrated circuit 1230 can be configured to perform the various functions described herein in conjunction with other components. Additionally, the functionality described herein can be performed by multiple interconnected integrated circuits.

[0059] As used herein, a phrase in the form of “design information specifying a circuit configured to…” does not imply that the circuit in question must be manufactured to satisfy the element. Rather, the phrase indicates that the design information describes a circuit that, when manufactured, will be configured to perform the indicated actions or will include the specified components. Similarly, stating that “instructions of a hardware description programming language” are “executable” to program a computing system to generate a computer simulation model does not mean that the instructions must be executed to satisfy the element, but rather specifies the characteristics of those instructions. In this case, additional features associated with the model (or the circuit represented by the model) may similarly relate to the characteristics of those instructions. Therefore, an entity selling a computer-readable medium having instructions that satisfy the stated characteristics may provide an infringing product even if another entity actually executes those instructions on the medium.

[0060] It should be noted that, at least in the context of digital logic, a given design can be implemented using a variety of different gate arrangements, circuit techniques, etc. However, once a digital logic design is specified, those skilled in the art do not need to perform extensive experiments or studies to determine those specific implementations. Instead, they understand the process for reliably and predictably producing one or more circuit implementations that provide the functionality described by the design information. Different circuit implementations may affect the performance, area, power consumption, etc., of a given design (potentially involving trade-offs between different design goals), but the logical function remains unchanged between different circuit implementations of the same circuit design.

[0061] In some implementations, instructions included in the design information instructions provide RTL information (or other higher-level design information) and can be executed by a computing system to synthesize a gate-level netlist representing the hardware circuitry based on the RTL information as input. Similarly, these instructions provide behavioral information and can be executed by the computing system to synthesize netlists or other lower-level design information. The lower-level design information can be used to program the manufacturing system 1220 to manufacture the integrated circuit 1230.

[0062] This disclosure includes references to “one implementation” or “a group of implementations” (e.g., “some implementations” or “various implementations”). An implementation is a different specific embodiment or instance of the disclosed concepts. References to “implementation,” “one implementation,” and “specific implementation,” etc., do not necessarily refer to the same implementation. A large number of possible implementations are contemplated, including those specifically disclosed, as well as modifications or alternatives that fall within the substance or scope of this disclosure.

[0063] This disclosure may discuss potential advantages that may arise from the disclosed embodiments. Not all specific implementations of all these embodiments will necessarily exhibit any or all of the potential advantages. Whether a particular embodiment achieves an advantage depends on many factors, some of which are outside the scope of this disclosure. In fact, there are many reasons why an embodiment falling within the scope of the claims may not exhibit some or all of any of the disclosed advantages. For example, a particular embodiment may include other circuitry outside the scope of this disclosure, in conjunction with one embodiment of the disclosed embodiments, which negates or diminishes one or more of the disclosed advantages. Furthermore, suboptimal design execution of a particular embodiment (e.g., the implementing technique or tool) may also negate or diminish the disclosed advantages. Even assuming an implementation of the technique, the realization of advantages may still depend on other factors, such as the environmental circumstances in which the implementation is deployed. For example, the inputs provided to a particular embodiment may prevent one or more problems addressed in this disclosure from occurring in a particular context, and as a result, the benefits of its solution may not be realized. In view of the existence of possible factors outside this disclosure, it is hereby expressed that any potential advantages described herein should not be construed as a limitation of the claims that must be satisfied in order to prove infringement. Rather, the identification of such potential advantages is intended to illustrate the types of improvements available to the designer who benefits from this disclosure. Describing such advantages permanently (e.g., stating that a particular advantage "may occur") is not intended to convey a question about whether such advantages can actually be realized, but rather to recognize that the realization of such advantages often depends on the technological reality of additional factors.

[0064] Unless otherwise stated, the embodiments are non-limiting. That is, the disclosed embodiments are not intended to limit the scope of the claims drafted based on this disclosure, even where only a single example is described with respect to a particular feature. The disclosed embodiments are intended to be illustrative and not restrictive, without any statement to the contrary in this disclosure. Therefore, this application is intended to allow for claims covering the disclosed embodiments, as well as such alternatives, modifications, and equivalents, which will be apparent to those skilled in the art to the advantage of this disclosure.

[0065] For example, features in this application can be combined in any suitable manner. Therefore, new claims may be made for any such combination of features during the filing of this application (or an application claiming priority thereto). Specifically, referring to the appended claims, features of dependent claims may be combined with features of other dependent claims, including claims dependent on other independent claims, where appropriate. Similarly, features from the respective independent claims may be combined where appropriate.

[0066] Therefore, while the appended dependent claims may be drafted such that each dependent claim is subordinate to a single other claim, additional dependent relationships are also contemplated. Any combination of dependent features conforming to this disclosure is contemplated, and such combinations may be protected by the claims in this application or another application. In short, the combinations are not limited to those specifically listed in the appended claims.

[0067] Where appropriate, it is also envisioned that claims drafted in one format or statutory type (e.g., apparatus) are intended to support corresponding claims in another format or statutory type (e.g., method).

[0068] As this disclosure is a legal document, various terms and phrases are subject to administrative and judicial interpretation. It is hereby announced that the following paragraphs, as well as the definitions provided throughout this disclosure, will be used to determine how the claims drafted based on this disclosure should be interpreted.

[0069] Unless the context clearly specifies otherwise, references to the singular form of an item (i.e., nouns or noun phrases preceded by "an," "a," or "the") are intended to mean "one or more." Therefore, references to "item" in a claim do not exclude additional instances of that item without context. "A plurality of" items refers to a collection of two or more items within a set.

[0070] The term "can" is used in this document in an allowed sense (i.e., having the potential possibility, being able to), rather than in a mandatory sense (i.e., must).

[0071] The terms "comprising" and "including" and their forms are open-ended and mean "including but not limited to".

[0072] When the term "or" is used in this disclosure with respect to a list of options, unless the context otherwise provides, it will generally be understood to be used in an inclusive sense. Thus, the statement "x or y" is equivalent to "x or y, or both", and thus covers 1) x but not y, 2) y but not x, and 3) both x and y. On the other hand, phrases such as "either x or y, but not both" make it clear that "or" is used in an exclusive sense.

[0073] The statements "w, x, y or z, or any combination thereof" or "at least one of... w, x, y, and z" are intended to cover all possibilities of individual elements up to the total number of elements in the set. For example, given the set [w, x, y, z], these phrases cover any single element in the set (e.g., w but not x, y, or z), any two elements (e.g., w and x, but not y or z), any three elements (e.g., w, x, and y, but not z), and all four elements. The phrase "at least one of... w, x, y, and z" thus refers to at least one element in the set [w, x, y, z], thereby covering all possible combinations in that list of elements. This phrase should not be construed as requiring that there be at least one instance of w, at least one instance of x, at least one instance of y, and at least one instance of z.

[0074] In this disclosure, various "labels" may precede a noun or noun phrase. Unless the context otherwise provides, different labels used for features (e.g., "first circuit", "second circuit", "specific circuit", "given circuit", etc.) refer to different instances of the feature. Additionally, unless otherwise stated, the labels "first", "second", and "third" do not imply any type of ordering (e.g., spatial, temporal, logical, etc.) when applied to features.

[0075] The phrase "based on" or is used to describe one or more factors that influence a determination. The term does not exclude the possibility that additional factors may influence the determination. That is, the determination may be based solely on the specified factors or on the specified factors and other unspecified factors. Consider the phrase "determine A based on B". This phrase specifies that B is a factor used to determine A or that B influences the determination of A. This phrase does not exclude the possibility that the determination of A may also be based on some other factor, such as C. This phrase is also intended to cover embodiments in which A is determined solely based on B. As used herein, the phrase "based on" is synonymous with the phrase "at least partially based on".

[0076] The phrases “responding to” and “responding” describe one or more factors that trigger an effect. This phrase does not exclude the possibility that additional factors may influence or otherwise trigger the effect, whether these factors are used in conjunction with or independently of the specified factor. That is, the effect may respond solely to these factors, or it may respond to the specified factor as well as other unspecified factors. Consider the phrase “responding to B and executing A.” This phrase specifies that B is a factor that triggers the execution of A or triggers a specific result of A. This phrase does not exclude that the execution of A may also respond to certain other factors, such as C. This phrase also does not exclude that the execution of A may be jointly executed in response to B and C. This phrase is also intended to cover implementation schemes where A is executed solely in response to B. As used herein, the phrase “responding” is synonymous with the phrase “at least partially responding to.” Similarly, the phrase “responding to” is synonymous with the phrase “at least partially responding to.”

[0077] Within this disclosure, different entities (which may be referred to differently as “units,” “circuits,” other components, etc.) may be described or protected by the claims as being “configured” to perform one or more tasks or operations. This expression—[entity] configured to [perform one or more tasks]—is used herein to refer to a structure (i.e., a tangible thing). More specifically, this expression is used to indicate that the structure is arranged to perform one or more tasks during operation. A structure may be considered “configured” to perform a task even if the structure is not currently being operated. Thus, an entity described or stated as being “configured” to perform a task refers to tangible things such as devices, circuits, systems with processor units, and memory storing program instructions executable to perform the task. This phrase is not used herein to refer to intangible things.

[0078] In some cases, various units / circuits / components may be described herein as a collection of entities that perform tasks or operations. It should be understood that these entities are "configured" to perform those tasks / operations, even if not specifically stated otherwise.

[0079] The term "configured as" is not intended to mean "able to be configured as." For example, an unprogrammed FPGA is not considered "configured as" to perform a specific function. However, the unprogrammed FPGA can be "able to be configured as" to perform that function. After proper programming, the FPGA can then be considered "configured as" to perform a specific function.

[0080] For the purposes of this U.S. patent application, the statement in the claims that the structure is “configured” to perform one or more tasks is expressly intended not to invoke 35 USC § 112(f) for that claim element. If an applicant wishes to invoke part 112(f) during the filing of a U.S. patent application based on this disclosure, it will use the structure “means for [performing a function]” to state the elements of the claims.

[0081] Different “circuits” may be described in this disclosure. These circuits, or “circuitry”, constitute hardware that includes various types of circuit elements, such as combinational logic, clock storage devices (e.g., flip-flops, registers, latches, etc.), finite state machines, memories (e.g., random access memory, embedded dynamic random access memory), programmable logic arrays, etc. Circuits may be custom-designed or taken from standard libraries. In various specific implementations, circuits may include digital components, analog components, or a combination of both, depending on the context. Certain types of circuits may be commonly referred to as “cells” (e.g., decoding units, arithmetic logic units (ALUs), functional units, memory management units (MMUs), etc.). Such cells are also referred to as circuits or circuitry.

[0082] Therefore, the circuits / units / components and other elements illustrated in the accompanying drawings and described herein include hardware elements, such as those described in the preceding paragraphs. In many cases, the internal arrangement of hardware elements in a particular circuit can be specified by describing the function of that circuit. For example, a particular “decoding unit” can be described as having the function of executing “the opcode of a processing instruction and routing that instruction to one or more of a plurality of functional units,” meaning that the decoding unit is “configured” to perform that function. To those skilled in the art of computers, this functional specification is sufficient to suggest a set of possible structures for the circuit.

[0083] In various implementations, as discussed in the preceding paragraphs, circuits, cells, and other elements can be defined by the functions or operations they are configured to perform. The arrangement of these circuits / cells / components relative to each other and the manner in which they interact form a microarchitecture definition of hardware, which is ultimately manufactured in an integrated circuit or programmed into an FPGA to form a physical implementation of the microarchitecture definition. Therefore, a microarchitecture definition is considered by those skilled in the art to be a structure from which many physical implementations can be derived, all of which fall within the broader structure described by the microarchitecture definition. That is, those skilled in the art, with the microarchitecture definition provided according to this disclosure, can implement this structure without excessive experimentation and using the application of a person of ordinary skill in the art, by encoding the description of the circuits / cells / components in a hardware description language (HDL) such as Verilog or VHDL. The HDL description is often expressed in a way that can be revealed as functional. However, for those skilled in the art, the HDL description is a way of translating the structure of a circuit, cell, or component into the details of the next level of implementation. Such HDL descriptions can take the following forms: behavioral code (which is typically non-synthesizable), Register Transfer Language (RTL) code (which is typically synthesizable compared to behavioral code), or structural code (e.g., a netlist specifying logic gates and their connectivity). HDL descriptions can be sequentially synthesized against a library of cells designed for a given integrated circuit manufacturing technology and can be modified for timing, power, and other reasons to obtain a final design database that is sent to the factory to generate masks and ultimately produce integrated circuits. Some hardware circuitry or portions thereof can also be custom-designed in a schematic editor and captured into the integrated circuit design along with the synthesized circuitry. The integrated circuit can include transistors and other circuit elements (e.g., passive components such as capacitors, resistors, inductors, etc.), as well as interconnects between transistors and circuit elements. Some implementations may implement multiple integrated circuits coupled together to implement the hardware circuitry, and / or discrete components may be used in some implementations. Alternatively, the HDL design can be synthesized into a programmable logic array such as a Field Programmable Gate Array (FPGA) and implemented within the FPGA. This decoupling between the design of a set of circuits and their subsequent low-level implementations often results in a situation where the circuit or logic designer never specifies a particular set of structures for the low-level implementation that goes beyond a description of what the circuit is configured to do, because that process is performed at different stages of the circuit implementation process.

[0084] The fact that a circuit can be implemented to the same specifications using many different low-level combinations of circuit elements results in a large number of equivalent circuit structures. As noted, these low-level circuit implementations can vary depending on the manufacturing technology, the foundry chosen to manufacture the integrated circuit, the cell library provided for a particular project, and so on. In many cases, the choice of different design tools or methods to produce these different implementations can be arbitrary.

[0085] Furthermore, for a given implementation, a single concrete implementation of the circuit's specific functional specifications typically involves a large number of devices (e.g., millions of transistors). Therefore, the shearing volume of this information makes it impractical to provide a complete description of the low-level structure used to implement a single implementation, let alone a large number of equivalent possible implementations. To this end, this disclosure describes the structure of a circuit using functional abbreviations commonly used in industry.

Claims

1. A computing system, the computing system comprising: Multiple computing die packages include a processor configured to execute program instructions that operate on data stored in a distributed memory accessible via a unified memory architecture; A plurality of memory die packages configured to implement the unified memory architecture, wherein a given memory die package includes: One or more optical interfaces, the one or more optical interfaces being configured to receive memory requests from the processor; and One or more memory controllers are configured to access a portion of the distributed memory in response to a received memory request.

2. The computing system of claim 1, wherein the given memory die package comprises: A cache configured to store a subset of the data stored in a portion of the distributed memory, wherein a given memory die package is configured to serve one or more memory requests from memory requests originating from the cache.

3. The computing system of claim 1, wherein the given memory die package comprises: One or more memories, which are coupled to one or more memory controllers and correspond to the portion of the distributed memory.

4. The computing system of claim 1, wherein the given memory die package comprises: A memory controller die, the memory controller die including a cache and the one or more memory controllers; and One or more optical dies, the one or more optical dies being coupled to a memory controller die via one or more die-to-die interconnects and including the one or more optical interfaces.

5. The computing system according to claim 1, further comprising: A memory tray, the memory tray including a plurality of memory die packages in the memory die package.

6. The computing system according to claim 5, further comprising: A memory rack, wherein the memory tray is one of a plurality of memory trays inserted into the memory rack.

7. The computing system of claim 1, wherein the processor comprises one or more central processing units (CPUs).

8. The computing system of claim 7, wherein the processor further comprises one or more graphics processing units (GPUs), application-specific integrated circuits (ASICs), or field-programmable gate arrays (FPGAs).

9. The computing system of claim 1, wherein a given computing die package among the plurality of computing die packages comprises: A computing die, the computing die including one or more processors and die-to-die interconnects; and An optical die, which is coupled to the computing die via a die-to-die interconnect and includes one or more optical interfaces.

10. The computing system according to claim 9, further comprising: A computing tray, the computing tray including the plurality of computing die packages.

11. The computing system according to claim 10, further comprising: A computer rack, wherein the computing tray is one of a plurality of computing trays inserted into the computer rack.

12. The computing system according to claim 11, further comprising: Multiple interconnected computing nodes, wherein a given computing node among the computing nodes includes one or more computer racks coupled to one or more memory racks.

13. The computing system of claim 1, wherein the processor is configured to access any address within a unified address space defined by the unified memory architecture.

14. The computing system of claim 1, wherein a given page within a unified address space defined by the unified memory architecture is distributed among a plurality of memory die packages in a memory die package.

15. The computing system of claim 1, wherein the computing die package and the memory die package are coupled together via an optical interconnect structure, the optical interconnect structure comprising at least two networks having heterogeneous operation characteristics.

16. An integrated circuit, the integrated circuit comprising: A memory die package having one or more commonly packaged dies, wherein the memory die package includes: An optical interface configured to receive memory requests from one or more processors via optical fiber; A memory controller, coupled to the memory and configured to serve the memory requests; and The memory's cache, which is configured to store a portion of the data stored in the memory.

17. The integrated circuit of claim 16, wherein the memory die package is configured to implement a unified memory architecture for distributed memory accessible to the one or more processors.

18. The integrated circuit of claim 17, wherein the memory die package is configured to store a given page within a uniform address space defined by the uniform memory architecture, such that the given page is distributed between the memory die package and one or more additional memory die packages.

19. An integrated circuit, the integrated circuit comprising: A computing die package having one or more commonly packaged die components, wherein the computing die package includes: One or more processors, configured to execute program instructions that operate on data stored in a distributed memory accessible via a unified memory architecture; and An optical interface configured to transmit memory requests via optical fiber to one or more memory die packages configured to implement the unified memory architecture.

20. The integrated circuit of claim 19, wherein the one or more processors include one or more central processing units (CPUs), graphics processing units (GPUs), application-specific integrated circuits (ASICs), or field-programmable gate arrays (FPGAs).

Images