Optical-based distributed integrated memory system

Abstract

Techniques relating to computing systems using silicon photonics are disclosed. In some embodiments, the computing system comprises a plurality of compute die packages, each comprising processors configured to execute program instructions that operate on data stored in distributed memory accessible through an integrated memory architecture. The computing system further comprises a plurality of memory die packages configured to implement an integrated memory architecture, wherein a given memory die package among the memory die packages comprises one or more optical interfaces configured to receive memory requests from processors and one or more memory controllers configured to access a portion of the distributed memory in response to the received memory requests.

Description

Technology Field

[0001] The present disclosure generally relates to computing systems, and more specifically to computing systems using silicon photonics. Background Technology

[0002] Modern mobile devices (e.g., smartphones) typically include a System on a Chip (SoC), in which multiple components of a computing device (e.g., central processing units (CPUs), graphics processing units (GPUs), peripheral interfaces, memory, etc.) are combined into a single chip. This type of architecture can offer several advantages over more conventional personal computer (PC) architectures, where components reside on separate chips (or even separate cards) embedded in the motherboard. For example, an SoC can reduce the physical size of the device, which can be useful for applications requiring portability. An SoC can improve device performance by reducing communication latency and power consumption between different components. An SoC can also lower the cost of the device by reducing the number of external components and simplifying the design and manufacturing process.

[0003] One of the advantages of SoC architecture is the ability to implement a unified memory architecture (UMA), where multiple elements within the SoC can share a common memory pool. This enables high-bandwidth and low-latency access between elements, for example, because a CPU wishing to transfer data to a GPU can simply inform the GPU of where the data is stored in the memory pool and allow the GPU to access the data directly. This approach can be significantly more efficient than, for instance, transferring data between system memory and dedicated GPU memory. Brief explanation of the drawing

[0004] FIG. 1 is a block diagram illustrating an example of a memory die package within an optical-based distributed integrated memory system. FIG. 2 is a block diagram illustrating an example of a memory tray including a plurality of memory die packages shown in FIG. 1. FIG. 3 is a block diagram illustrating an example of a memory rack including a plurality of memory trays of FIG. 2. FIG. 4 is a block diagram illustrating an example of a compute die package within an optical-based distributed integrated memory system. FIG. 5 is a block diagram illustrating an example of a compute tray including a plurality of compute die packages shown in FIG. 4. FIG. 6 is a block diagram illustrating an example of a compute rack including a plurality of compute trays of FIG. 5. FIG. 7 is a block diagram illustrating an example of an optical-based distributed integrated memory system including the memory rack of FIG. 3 and the compute rack of FIG. 5. FIGS. 8 and 9 are block diagrams illustrating examples of optical fiber arrays used by an optical-based distributed integrated memory system. FIG. 10 is a block diagram illustrating an example of a multi-node system including a plurality of optical-based distributed integrated memory systems. FIG. 11 is a flowchart illustrating an example of a method for implementing the functions described in the present specification. FIG. 12 is a block diagram illustrating an exemplary computer-readable medium for storing circuit design information for implementing devices having the functions described in this specification. Specific details for implementing the invention

[0005] SoCs have primarily been used in small computing systems such as smartphones, tablets, wearables, and laptops. Larger computing systems, such as server computing systems, face power and physical space constraints on a much larger scale; therefore, the design of these systems has focused less on the use of SoCs, preferring conventional architectures where components are implemented individually. However, when SoCs are used in larger computing systems, the integrated memory architecture implemented by a given SoC is limited to the elements within that SoC. Consequently, while the CPU within the first SoC can transfer data to the first GPU within that SoC via the architecture, it cannot transfer data to the second GPU within the second SoC using the architecture because the second GPU does not directly access the internal memory pool of the first SoC. This limitation can significantly reduce the benefits of using an integrated memory architecture.

[0006] The present disclosure describes embodiments for creating a distributed integrated memory system by scaling an integrated memory architecture for larger computing systems, such as server computing systems, using silicon photonics. As described in the various embodiments below, the computing system may include a plurality of compute die packages comprising processors configured to execute program instructions that operate on data stored in distributed memory accessible through the integrated memory architecture. These processors may include any suitable type of processor, such as one or more central processing units (CPUs), graphics processing units (GPUs), application-specific integrated circuits (ASICs), or field-programmable gate arrays (FPGAs). The computing system may also include a plurality of memory die packages configured to implement an integrated memory architecture, wherein a given memory die package comprises one or more optical interfaces configured to receive memory requests from processors and one or more memory controllers configured to access a portion of the distributed memory in response to the received memory requests. A given compute die package among compute die packages may also include a compute die comprising one or more processors and an optical die coupled to the compute die through a die-to-die interconnect and comprising one or more optical interfaces.

[0007] Implementing an integrated memory architecture in this manner enables greater resource utilization because processors within compute die packages can share the use of distributed memory implemented by memory die packages. This architecture can also enable easy scaling of the computing system, which, in some embodiments, may include a plurality of distributed nodes, each comprising racks of compute die packages and memory die packages.

[0008] Now, referring to FIG. 1, a block diagram of a memory die package (100), which is the core of a computing system, is depicted. In various embodiments, the memory die package (100), comprising one or more co-packaged dies, includes an optical interface configured to receive memory requests from one or more processors via one or more optical fibers, a memory controller coupled to the memory and configured to service the memory requests, and a memory cache configured to store a portion of the data stored in the memory. In an illustrated embodiment, the memory die package (100) includes a memory die (110) comprising a memory cache (112), die-die physical interfaces (114), and memory physical interfaces (116). The memory physical interfaces (116) are memory controller circuits coupled to one or more co-packaged memories (130). The memory cache (112) is a cache configured to store a subset of the data stored in the memories (130). In the illustrated embodiment, the memory die package (100) further comprises a plurality of optical dies (120), which are coupled to the memory controller die (110) via die-die physical interfaces (124) interconnects and include one or more optical input / output interfaces (122) configured to communicate with other memory die packages (100) and compute die packages (400) discussed below with Fig. 4 via optical fibers. In some embodiments, the memory die packages (100) may be implemented differently from that illustrated. In some embodiments, the dies (110, 120) (along with the compute dies (420) discussed below with Fig. 4) are chiplets that operate collectively to implement a system-on-chip architecture.

[0009] In some embodiments, the memory die package (100) may provide 64 LP5X channels, which provide a memory bandwidth of up to 1088 GB / sec (at 8.5 GT / sec) and a memory capacity of up to 512 GB (16 Gb density, byte-mode, dual-rank). The memory cache (112) may include a 1 GB SRAM-based memory cache. In the illustrated embodiment, the Y-dimension of the memory die (110) may determine the number of memory channels that can be supported, along with the cache capacity that fits within the die area. The X-dimension of the optical die (120) may determine the amount of bandwidth that can be supported by and from the memory die package (100). In some embodiments, the supported bandwidth is equal to 2.5 TB / sec for each of the optical dies (120), thereby supporting a total of up to 5 TB / sec.

[0010] In some embodiments, the dies (110, 120) may be chiplets connected via a silicon interposer, silicon bridge, or other similar high-density interconnector technology referred to as 2.5D integration. In other embodiments, the dies (110, 120) may be vertically stacked and connected face-to-face or face-to-back using silicon through-vias on one of the dies, which are approaches referred to as 3D integration. In these 2.5D or 3D integration embodiments, the D2D PHY (114) and D2D PHY (124) may communicate through a large number of signal paths, for example, 8,000 signals operating at 4 GT / second, to provide an interconnect bandwidth of 20 Tb / second (2.5 TB / second).

[0011] In some embodiments, the optical die (120) may be a monolithic die in which active electronic components for the electrical parts of the D2D PHY (124) and optical IO (122), such as modulator drivers, transimpedance amplifiers, and serialization / deserialization (SerDes) circuits, as well as silicon photonic components such as silicon waveguides, micro-ring resonators, Mach-Zehnder modulators, and photodetectors, are integrated. In some embodiments, the optical die (120) may be composed of a stack of two component dies, one die containing electronic components and the other die containing photonic components, which are manufactured in different semiconductor fabrication processes. In some embodiments, an external laser may provide a continuous wave optical input to the optical dies (120) through fiber optic cables.

[0012] In some embodiments, the optical die (120) may be made of an electric die in which an array of direct modulation light sources, such as VCSELs (vertical-cavity surface emitting lasers) or LEDs (light-emitting diodes), and photodetectors, such as PIN photodiodes, are assembled.

[0013] In some embodiments, optical fibers are connected to a memory die package (100) via fiber-attach-units (FAUs) that couple light from the optical fibers to photonic components of the optical dies (120). In some embodiments, the FAUs may be edge-coupled to optically couple optical waveguides to silicon waveguides in a common plane orientation on the optical dies (120). In some embodiments, the FAUs may be vertically coupled to optically couple optical waveguides to devices on the planes of the optical dies (120), such as photodiodes, VCSELs, and LEDs, or to couple light to silicon waveguides of the optical dies (120) via vertical grating couplers (VGCs).

[0014] The memories (130) may be any type of volatile memory, such as dynamic random access memory (DRAM), synchronous DRAM (SDRAM), double data rate (DDR, DDR2, DDR3, etc.) SDRAM (including mobile versions of SDRAM such as mDDR3, etc. and / or low-power versions of SDRAM such as LPDDR4, etc.), RAMBUS DRAM (RDRAM), static RAM (SRAM), etc. One or more memory devices may be coupled on a circuit board to form memory modules such as single inline memory modules (SIMMs), dual inline memory modules (DIMMs), etc. The devices may be equipped with integrated circuits in a chip-on-chip configuration, a package-on-package configuration, or a multi-chip module configuration. The memories (130) may also be any type of non-volatile memory such as NAND flash memory, NOR flash memory, NRAM (nano RAM), MRAM (magneto-resistive RAM), PRAM (phase change RAM), Racetrack memory, Memristor memory, etc.

[0015] In various embodiments, memory die packages (100) are configured to implement a unified memory architecture (UMA), wherein distributed memory is implemented across memories (130) (including memories (130) within other memory die packages (100)) and shared by a number of processors, such as processors within compute die packages (400) discussed together with FIG. 4. In some embodiments, the UMA is implemented such that a given processor among the processors is configured to access any memory address within a unified address space defined by the unified memory architecture. In some embodiments, this unified address space is a virtual address space distinct from the physical address space provided by the memories (130). In some embodiments, the memory die packages (100) are further configured to implement the UMA in such a way that memory accesses across the memory die packages (100) are transparent to software running on the compute die packages (400). In some embodiments, the UMA is implemented such that a given page in the integrated address space is distributed among a number of memory die packages (100). In some embodiments, the memory die packages (100) are configured to perform address hashing to ensure that data is evenly distributed across the memories (130) in the UMA. In some embodiments, to enable different characteristics for accessing the memories (130), the optical interfaces (122) within the memory die packages (100) and compute die packages (400) are configured to implement an optical interconnection fabric comprising at least two networks having heterogeneous operation characteristics.In some embodiments, one or more of the characteristics include the path to the first network for memory requests being shorter than the path to the second network, characteristics that increase the bandwidth of the first network compared to the second network, and / or allocating a larger number of optical fibers to the first network compared to the second network. In some embodiments, the first and second networks have heterogeneous interconnection topologies such as star topology, mesh topology, ring topology, tree topology, fat tree topology, hypercube topology, or a combination of one or more of the topologies. In some embodiments, at least two networks include a coherent network interconnecting CPUs and memory dies (110) within compute die packages (400) and a relaxed-ordered network interconnecting GPUs and memory dies (110) within compute die packages (400). In some embodiments, a given memory die package (100) is coupled to first and second networks and configured to receive memory requests from both first and second networks.

[0016] Now, referring to FIG. 2, a block diagram of a memory tray (200) is depicted. As illustrated, the memory tray (200) includes one or more memory die packages (100). In some embodiments, memory addresses are hashed evenly across the memory die packages (100) within the memory tray (200). The total available memory bandwidth and memory capacity provided by the memory tray (200) depend on the number of memory die packages (100) within the memory tray (200). For simplicity, the memory tray (200) may include a power-of-two number of die packages (100). In an illustrated embodiment, the memory tray (200) includes eight memory die packages (100), each memory die package (100) is mapped to 1 / 8 of the memory address space. This may correspond to 512 LP5X channels, which provide a memory bandwidth of up to 8.704 TB / sec (at 8.5 GT / sec) and a memory capacity of up to 4 TB (16 Gb density, byte-mode, dual-rank). This may also correspond to a tray (200) containing an 8 GB SRAM-based memory cache. For completeness, in some embodiments, since each memory die supports an optical bandwidth of up to 5 TB / sec, the memory tray (200) may support an optical bandwidth of up to 40 TB / sec. In some embodiments, the tray (200) may be implemented differently from that illustrated.

[0017] For convenience, one side of the memory tray (200) may be referred to as the north side and the other side as the south side. Each memory tray (200) has a number of ports (210) that may be equal to twice the number of memory die packages (100) inside it. Half of these ports (210) are located on the north side of the memory tray (200), while the other half are located on the south side of the memory tray (200). Within a given memory tray (200), the ports (210) may be numbered from left to right starting with 1, followed by N or S to indicate the side of the tray. A given memory tray (200) may also have an associated ID, as illustrated in letters in the lower left part of the drawing (in this case, it is memory tray A). Accordingly, the ports in this example are identified as A1N…A8N, A1S…A8S.

[0018] To facilitate connection between all memory die packages (100) and all memory tray ports (210), in some embodiments, each memory die package (100) is connected to all ports of the memory tray (200) via optical cables. In some embodiments, this optical interconnection fabric is configured to allow interconnection of various numbers of processor cores, graphics processing units, peripheral devices, or memory controller circuits.

[0019] In some embodiments, the optical ports (210) are terminated by optical connectors such as MTP connectors so that optical ribbon cables or multi-core optical fibers can be externally connected to the memory tray (200).

[0020] Now, referring to FIG. 3, a block diagram of a memory rack (300) is depicted. As illustrated, the memory rack (300) includes one or more compute trays (200) inserted adjacent to each other. In some embodiments, memory addresses are hashed evenly across memory die packages (100) within the memory rack (300). The total available memory bandwidth and memory capacity provided by the memory rack (300) depend on the number of memory die packages (100) within the memory rack (300). For simplicity, the memory rack (300) may include a power of two number of memory trays (200), and each of the memory trays (200) may include a power of two number of memory die packages (100). In an illustrated embodiment, the memory rack (300) includes eight memory trays (200), where each memory tray (200) includes eight memory die packages (100). In this case, each memory die package (100) is mapped to 1 / 64 of the memory address space. This may correspond to a rack (300) having 4096 LP5X channels, which provides a memory bandwidth of up to 69.632 TB / sec (at 8.5 GT / sec) and a memory capacity of up to 32 TB (16 Gb density, byte-mode, dual-rank). This may also correspond to a rack (300) including a 64 GB SRAM-based memory cache. For completeness, in some embodiments, since each memory die package (100) supports an optical bandwidth of up to 5 TB / sec, the rack (300) may support an optical bandwidth of up to 320 TB / sec. In some embodiments, the rack (300) may be implemented differently from that illustrated.

[0021] Within a given memory rack (300), the trays (200) are identified from bottom to top, starting with A. To uniquely identify a given port in the memory rack (300), the numbering of the port may take the form of {memory tray ID} followed by {tray port number} and N or S designating the side. For example, port C4N refers to port 4 (north side) on tray C.

[0022] In the illustrated embodiment, the memory rack has a total of 128 ports, namely 64 ports on each side. In this example, the ports are identified as A1N…A8N, A1S…, A8S…H1N…H8N, H1S…H8S.

[0023] Now, referring to FIG. 4, a block diagram of a compute die package (400) is depicted. As illustrated, the compute die package (400) includes one or more optical dies (120) coupled via die-die interconnections created by physical interfaces (114, 124) and one or more compute dies (420) co-packaged. The compute dies (420) can be customized for any suitable use case and may include any suitable processors (412). For example, the compute dies (420) may include one or more CPUs, GPUs, neural engines, ASICs, FPGAs, a mix of IPs, etc. In some embodiments, the compute die package (40) includes an 18-core CPU and a 40-FSTP GPU. The X-dimension of the optical die (120) can determine the amount of bandwidth that can be supported by and from the compute die package (400). In some embodiments, the supported bandwidth is 2.5 TB / sec for the optical dies (120). In some embodiments, the compute die package (400) may be implemented differently from that illustrated.

[0024] Now, referring to FIG. 5, a block diagram of a compute tray (500) is depicted. As illustrated, the compute tray (500) contains one or more compute die packages (400). The total available compute capability provided by the compute tray (500) depends on the number of compute die packages (400) within the compute tray (500). For simplicity, the compute tray (500) may contain a power of two number of compute die packages (400). Also, note that the compute die packages (400) within the compute tray (500) do not need to be identical, for example, that the selection of each compute die package (400) may be left to the system operator. In the illustrated embodiment, the compute tray contains eight identical compute die packages (400), each of which may contain an 18-core CPU and a 40-FSTP GPU. This may correspond to a 144-core CPU and a 320-FSTP GPU. For completeness, since each compute die package (400) may support an optical bandwidth of up to 2.5 TB / sec, the compute tray (500) may support an optical bandwidth of up to 20 TB / sec. In some embodiments, the tray (500) may be implemented differently from that illustrated.

[0025] Each compute tray (500) has a number of ports (502) that may be equal to the number of compute die packages (400) inside it. Within a given compute tray (500), the ports (502) may be numbered from left to right, starting with 1. A given compute tray may also have an associated ID, as illustrated in letters in the lower left part of the drawing (in this case, it is compute tray A). Accordingly, the ports (502) in this example are identified as A1…A8.

[0026] To facilitate connection between all compute die packages (400) and all compute tray (500) ports, in some embodiments, each compute die package (400) is connected to all ports of the compute tray (500) via optical cables.

[0027] Now, referring to FIG. 6, a block diagram of a compute rack (600) is depicted. As illustrated, the compute rack (600) includes one or more compute trays (500) inserted adjacent to each other. The total available compute capacity provided by the compute rack (600) depends on the number and type of compute die packages (400) within the compute rack (600). For simplicity, the compute rack (600) may include a power of two compute trays (500), and each of the compute trays (500) may include a power of two compute die packages (400). Also, note that the compute die packages (400) within the compute rack (600) do not need to be identical, for example, the selection of each compute die package (440) may be left to the system operator. In the illustrated embodiment, the compute rack (600) comprises eight compute trays (500), which comprise eight compute die packages (400), each of which may comprise an 18-core CPU and a 40-FSTP GPU. This corresponds to an 1152-core CPU and a 2560-FSTP GPU. For completeness, since each compute die package (400) may support an optical bandwidth of up to 2.5 TB / sec, the compute rack (600) may support an optical bandwidth of up to 160 TB / sec. In some embodiments, the rack (600) may be implemented differently from that illustrated.

[0028] Within a given compute rack (600), trays are identified from bottom to top, starting with A. To uniquely identify a given port in the compute rack (600), the port numbering may be expressed in the form {compute tray ID} followed by {tray port number}. For example, port C4 refers to port 4 of tray C.

[0029] In the illustrated embodiment, the compute rack (600) has a total of 64 ports. In this example, the ports are identified as A1…A8, H1…H8.

[0030] Now, referring to FIG. 7, a block diagram of an optical-based distributed memory system / node (700) is depicted. In the illustrated embodiment, the node (700) includes a single memory rack (300) and a plurality of compute racks (600A, 600B), each of which is positioned on one side of the memory rack (300). In the example described so far (eight die packages per tray and eight trays per rack), each compute rack (600) may have 64 ports and each memory rack (300) may have 128 ports, where 64 ports may face south and 64 ports may face north. To facilitate connection between all compute trays (500) and all memory trays (200), each port of a given compute tray (500) may be connected to a different memory tray (200) via an optical cable. For simplicity, it is assumed that a given port number of the compute tray (500) is connected to the same port number on the memory tray (200). In other words, compute port X4 can be connected to memory port Y4 (N or S, depending on the side it resides on), where X and Y are tray IDs. Additionally, for simplicity, it is assumed that port 1 of a given compute tray (500) can be connected to port 1 of an equivalent memory tray (200), and subsequent ports of the compute tray (500) can be connected to subsequent memory trays (200).

[0031] In some embodiments, the node (700) includes 4,096 LP5X channels, which can provide a memory bandwidth of up to 69,632 TB / sec (at 8.5 GT / sec) and a memory capacity of up to 32 TB (16 Gb density, byte-mode, dual-rank). In some embodiments, the node (700) also includes a 64 GB SRAM-based memory cache. Assuming that each compute die package (400) includes an 18-core CPU and a 40-FSTP GPU, the node (700) may include 128 compute die packages (400), having a total of 2,304 CPU cores and 5,120-FSTP GPUs. For completeness, since each memory die package (100) may support an optical bandwidth of up to 5 TB / sec, the node (700) may support an optical bandwidth of up to 320 TB / sec.

[0032] Additionally, note that each compute die package (400) can observe the same latency and bandwidth characteristics for main memory. In doing so, the node (700) is essentially the largest UMA machine designed to date.

[0033] In some embodiments, the system / nodes (700) may be implemented differently from what is illustrated, such as including more or fewer racks (300 and / or 600). In some embodiments, a number of compute nodes (700) may also be interconnected as will be discussed together with FIG. 10.

[0034] Now, referring to FIG. 8, a block diagram of a multi-core optical fiber array (800) is depicted. In the illustrated example, each optical die (120) can support a bandwidth of 2.5 TB / second. To support an 8x8x8 node configuration, each compute die package (400) (or memory die package (100)) communicates (end-to-end) with 64 distinct memory die packages (100) (or compute die packages (400)). This can utilize 128 optical fibers (2 optical fibers per connection to support bidirectional operation), where each optical fiber can support up to 20 GB / second. One way to achieve this is to use a multi-core optical fiber array (800) as illustrated in FIG. 8. In the illustrated embodiment, this is a 61-core optical fiber array, where each core has a diameter of 0.05 mm and can support a switching speed of 4 Gbps. Forty cores may be used for transmitting / receiving data, and the remaining 21 cores may be used for any auxiliary functions outside the scope of this specification. Electrical-to-optical (E / O) and optical-to-electrical (O / E) conversions are implemented in optical dies (120) using modulators or photodetectors. In some embodiments, the optical fiber array (800) may be implemented differently from that illustrated. For example, if faster speeds are made possible, fewer cores may be accommodated.

[0035] Referring now to FIG. 9, a block diagram of a fiber optic array (900) is depicted. In the illustrated embodiment, the fiber optic array (900) is a 128-fiber optic array. Assuming a minimum distance for data routing to / from transceivers, the total beachfront can be approximately 19.2 mm. In FIG. 9, light gray designates Tx (transmit) and dark gray designates Rx (receive). Overall, this 128-fiber optic array can support a data transmission of 2.5 TB / second, assuming the core capabilities mentioned above. In some embodiments, the fiber optic array (900) may be implemented differently from that illustrated.

[0036] Now, referring to FIG. 10, a block diagram of a multi-node system (1000) is depicted. In some embodiments, it may still be beneficial to connect multiple nodes (700), each having its own shared memory. One option for this is to use a networking interface. An alternative is to replace any number of compute die packages (400) with network direct memory access (DMA) die packages that directly interface with the memory rack (300) of the node (700). This can provide very high bandwidth between multiple nodes (700), but at the cost of reduced compute. An example of connecting four nodes together is shown in FIG. 10, where each node may have 126 compute die packages (400). In some embodiments, the system (1000) may be implemented differently from that shown.

[0037] Now, referring to FIG. 11, a flowchart of method (1100) is depicted. Method (1100) is one embodiment of a method performed by a computing system such as an optical-based distributed memory system (700).

[0038] The method (1100) begins in step (1110) with processors (e.g., processors (412)) included in a plurality of compute die packages (e.g., compute die packages (400)) executing program instructions that operate on data stored in distributed memory (e.g., memories (130)) accessible through an integrated memory architecture. In step (1120), one or more optical interfaces (e.g., optical IOs (122)) within a given memory die package among a plurality of memory die packages (e.g., memory die packages (100)) configured to implement an integrated memory architecture receive memory requests from the processors. In step (1130), one or more memory controllers (e.g., memory physical interfaces (116)) access a portion of the distributed memory in response to the received memory requests.

[0039] Exemplary computer-readable media

[0040] The present disclosure has described various exemplary circuits in detail above. The present disclosure is intended to cover not only embodiments comprising such circuits but also computer-readable storage media comprising design information defining such circuits. Accordingly, the present disclosure is intended to support claims covering storage media defining circuits in a format recognized by devices comprising the disclosed circuits, computing systems configured to generate simulation models of hardware circuits, manufacturing systems configured to generate hardware (e.g., integrated circuits) comprising the disclosed circuits. Claims for such storage media are intended to cover entities that generate circuit designs, for example, but do not perform full operations themselves, such as design simulation, design synthesis, or circuit manufacturing.

[0041] Now, referring to FIG. 12, a block diagram of an exemplary non-transient computer-readable storage medium for storing circuit design information is depicted. In the illustrated embodiment, a computing system (1240) is configured to process the design information. This may include executing instructions contained in the design information, interpreting instructions contained in the design information, compiling, converting, or otherwise updating the design information. Thus, in some embodiments, the design information controls the computing system (1240) (e.g., by programming the computing system (1240)) to perform the various operations discussed below.

[0042] In the illustrated example, the computing system (1240) processes design information to generate both a computer simulation model of the hardware circuit (1260) and lower-level design information (1250). In other embodiments, the computing system (1240) may generate only one of these outputs, generate other outputs based on the design information, or both. In relation to the computing simulation, the computing system (1240) may execute instructions of a hardware description language including register transfer level (RTL) code, behavior code, structure code, or a combination of some of these. The simulation model may perform functions specified by the design information, facilitate verification of the functional accuracy of the hardware design, generate power consumption estimates, generate timing estimates, and the like.

[0043] In the illustrated example, the computing system (1240) also processes design information to generate lower-level design information (1250) (e.g., gate-level design information, netlist, etc.). This may include synthetic operations as illustrated, such as constructing a multi-level network, optimizing the network using technology-independent techniques, technology-dependent techniques, or both, and outputting a network of gates (with potential constraints based on gates available in a technology library, sizing, delay, power, etc.). Based on the lower-level design information (1250) (potentially among other inputs), the semiconductor manufacturing system (1220) is configured to manufacture an integrated circuit (1230) (which may correspond to the function of the simulation model (1260)). Note that the computing system (1240) may generate different simulation models based on design information at various levels of description, including information (1250, 1215), etc. Data representing design information (1250) and a model (1260) may be stored on a medium (1210) or on one or more other media.

[0044] In some embodiments, lower-level design information (1250) controls (e.g., programs) a semiconductor manufacturing system (1220) to manufacture an integrated circuit (1230). Thus, when processed by the manufacturing system, the design information can program the manufacturing system to manufacture a circuit including various circuit parts disclosed herein.

[0045] The non-transient computer-readable storage medium (1210) may include any of various suitable types of memory devices or storage devices. The non-transient computer-readable storage medium (1210) may be an installation medium, e.g., CD-ROM, floppy disks, or tape device; computer system memory or random access memory, e.g., DRAM, DDR RAM, SRAM, EDO RAM, Rambus RAM, etc.; non-volatile memory such as flash, magnetic media, e.g., hard drive, or optical storage; registers, or other similar types of memory elements, etc. The non-transient computer-readable storage medium (1210) may also include other types of non-transient memory or combinations thereof. Thus, the non-transient computer-readable storage medium (1210) may include two or more memory media; and such media may reside in different locations—e.g., different computer systems connected via a network.

[0046] Design information (1215) may be specified using any of various suitable computer languages, including hardware description languages ​​such as VHDL, Verilog, SystemC, SystemVerilog, RHDL, M, MyHDL, etc., without limitation. Various formats of design information may be recognized by one or more applications executed by a computing system (1240), a semiconductor manufacturing system (1220), or both. In some embodiments, design information may also include one or more cell libraries that specify the synthesis, layout, or both of an integrated circuit (1230). In some embodiments, design information is specified wholly or partially in the form of a netlist that specifies cell library elements and their connectivity. Design information discussed herein, taken alone, may or may not contain sufficient information for manufacturing the corresponding integrated circuit. For example, design information may specify circuit elements to be manufactured but may not be their physical layout. In such cases, design information may be combined with layout information to actually manufacture the specified circuit part.

[0047] In various embodiments, the integrated circuit (1230) may include one or more custom macrocells, such as memories, analog or mixed-signal circuits, etc. In such cases, design information may include information related to the included macrocells. Such information may include, without limitation, a schematic capture database, mask design data, behavior models, and device or transistor level netlists. The mask design data may be formatted according to a graphical data system (GDSII) or any other suitable format.

[0048] The semiconductor manufacturing system (1220) may include any of various suitable elements configured to manufacture integrated circuits. This may include elements for, for example, depositing semiconductor materials (e.g., on a wafer that may include masking), removing materials, changing the shape of the deposited materials, modifying materials (e.g., by doping the materials or modifying dielectric constants using ultraviolet processing). The semiconductor manufacturing system (1220) may also be configured to perform various tests of the manufactured circuits for accurate operation.

[0049] In various embodiments, the integrated circuit (1230) and the model (1260) are configured to operate according to a circuit design specified by design information (1215), which may include performing any of the functions described herein. For example, the integrated circuit (1230) may include any of the various elements shown in FIGS. 1 through 10. Additionally, the integrated circuit (1230) may be configured to perform the various functions described herein together with other components. Additionally, the functions described herein may be performed by a plurality of connected integrated circuits.

[0050] As used herein, a phrase of the form “design information specifying the design of a circuit configured to do so” does not imply that the circuit must be manufactured for the elements to be satisfied. Rather, such phrase indicates that the design information describes a circuit that, at the time of manufacture, will be configured to perform the indicated actions or will include the specified components. Similarly, referring to “executable” “instructions of a hardware description programming language” to program a computing system to generate a computer simulation model does not imply that the instructions must be executed for the elements to be satisfied, but rather specifies the characteristics of the instructions. In this context, additional characteristics related to the model (or the circuit represented by the model) may similarly relate to the characteristics of the instructions. Accordingly, an entity selling a computer-readable medium having instructions satisfying the enumerated characteristics may provide an infringing product even if another entity actually executes the instructions on the medium.

[0051] It should be noted that the given design can be implemented using a number of different gate arrangements, circuit techniques, etc., at least in the context of digital logic. However, once a digital logic design is specified, those skilled in the art do not need to perform substantial experiments or studies to determine such implementations. Rather, those skilled in the art understand the procedures for reliably and predictably generating one or more circuit implementations that provide the function described by the design information. While different circuit implementations may affect the performance, area, power consumption, etc., of the given design (potentially involving trade-offs between different design goals), the logic function does not change between different circuit implementations of the same circuit design.

[0052] In some embodiments, the instructions included in the design information instructions provide RTL information (or other higher-level design information) and are executable by a computing system to synthesize a gate-level netlist representing a hardware circuit based on the RTL information as input. Similarly, the instructions may provide behavior information and be executable by a computing system to synthesize a netlist or other lower-level design information. The lower-level design information may program a manufacturing system (1220) to manufacture an integrated circuit (1230).

[0053] ***

[0054] The present disclosure includes references to “Examples” or groups of “Examples” (e.g., “Some Examples” or “Various Examples”). Examples are different implementations or instances of the disclosed concepts. References to “Examples,” “One Example,” “A Specific Example,” etc., do not necessarily refer to the same example. A number of possible examples are considered, including modifications or alternatives that fall within the spirit or scope of the present disclosure, as well as those specifically disclosed.

[0055] The present disclosure may discuss potential benefits that may arise from the disclosed embodiments. Not all implementations of these embodiments necessarily represent any or all of the potential benefits. Whether benefits for a particular implementation are realized depends on many factors, some of which are outside the scope of the present disclosure. In fact, there are many reasons why an implementation within the scope of the claims may not represent some or all of any disclosed benefits. For example, a particular implementation may include other circuitry outside the scope of the present disclosure that, together with one of the disclosed embodiments, nullifies or weakens one or more of the disclosed benefits. Furthermore, suboptimal design practices of a particular implementation (e.g., implementation techniques or tools) may also nullify or weaken the disclosed benefits. Even assuming a skilled implementation, the realization of benefits may still depend on other factors, such as the environmental circumstances in which the implementation is deployed. For example, inputs supplied to a particular implementation may prevent one or more of the problems addressed in the present disclosure from occurring at a particular opportunity, and as a result, the benefits of the solution may not be realized. Given the existence of possible factors outside of the present disclosure, it is expressly intended that any potential benefits described herein should not be interpreted as claim limitations that must be satisfied to prove infringement. Rather, the identification of such potential benefits is intended to exemplify the type(s) of improvement available to designers who benefit from the present disclosure. That such benefits are described as acceptable (e.g., that a particular benefit is referred to as “may occur”) is not intended to convey doubt as to whether such benefits can actually be realized, but rather to acknowledge the technical reality that the realization of such benefits often depends on additional factors.

[0056] Unless otherwise noted, the embodiments are non-limiting. That is, the disclosed embodiments are not intended to limit the scope of claims drawn up based on the present disclosure, even if only a single example is described for a particular feature. The disclosed embodiments are intended to be illustrative rather than limiting, without any statement in the present disclosure to the contrary. Accordingly, the present application is intended to allow not only claims covering the disclosed embodiments but also such alternatives, modifications, and equivalents that will be apparent to those skilled in the art having an interest in the present disclosure.

[0057] For example, the features in the present application may be combined in any suitable manner. Accordingly, new claims may be formed during the examination of the present application (or the application claiming priority thereto) for any combination of such features. In particular, with reference to the appended claims, features from dependent claims may be combined with features of other dependent claims where appropriate, including claims dependent on other independent claims. Similarly, features from individual independent claims may be combined where appropriate.

[0058] Accordingly, while the appended dependent claims may each be drafted to depend on a single other claim, additional dependencies are also taken into account. Any combination of features in dependencies consistent with the present disclosure is taken into account and may be claimed in such or other applications. In short, the combinations are not limited to those specifically enumerated in the appended claims.

[0059] Where appropriate, claims written in one format or statutory type (e.g., device) are also considered to be intended to support corresponding claims in another format or statutory type (e.g., method).

[0060] ***

[0061] Because the present disclosure is a legal document, various terms and phrases may be subject to administrative and judicial interpretation. Accordingly, it is given in this specification that the definitions provided throughout the present disclosure, as well as the following paragraphs, will be used to determine how to interpret claims drafted based on the present disclosure.

[0062] References to the singular form of an article (i.e., a noun or noun phrase preceded by "a," "an," or "the") are intended to mean "one or more" unless clearly indicated otherwise by the context. Accordingly, references to "items" in a claim do not exclude additional instances of the items without accompanying circumstances. "Plural" items refer to sets of two or more items.

[0063] The word "can" is used in this specification not in a mandatory sense (i.e., must), but in a permissive sense (i.e., having the possibility of, able to).

[0064] The terms "comprising" and "including," and their forms, are open-ended and mean "including but not limited to."

[0065] When the term “or” is used in this disclosure with respect to a list of these options, it shall be understood to be used in a generally inclusive sense unless the context otherwise provides. Accordingly, references to “x or y” are equivalent to “x or y, or both,” and thus cover 1) x but not y, 2) y but not x, and 3) both x and y. On the other hand, phrases such as “either x or y, but neither” make it clear that “or” is used in an exclusive sense.

[0066] References to “w, x, y, or z, or any combination thereof” or “... at least one of w, x, y, and z” are intended to cover all possibilities involving a single element up to the total number of elements in the set. For example, given the set [w, x, y, z], these phrases cover any single element of the set (e.g., w but not x, y, or z), any two elements (e.g., w and x but not y or z), any three elements (e.g., w, x, and y but not z), and all four elements. Thus, the phrase “... at least one of w, x, y, and z” refers to at least one element of the set [w, x, y, z], thereby covering all possible combinations of elements within this list. Such phrases should not be interpreted to require that there be at least one instance of w, at least one instance of x, at least one instance of y, and at least one instance of z.

[0067] In this disclosure, various “labels” may precede nouns or noun phrases. Unless the context otherwise provides, different labels used for a feature (e.g., “first circuit,” “second circuit,” “specific circuit,” “given circuit,” etc.) refer to different instances of the feature. Additionally, when applied to a feature, the “first,” “second,” and “third” labels do not imply any type of ordering (e.g., spatial, temporal, logical, etc.) unless otherwise noted.

[0068] The phrase "based on" is used to describe one or more factors that influence a decision. This term does not exclude the possibility that additional factors may influence the decision. That is, the decision may be based on specific factors alone, or on specific factors as well as other unspecified factors. Consider the phrase "determine A based on B." This phrase specifies that B is a factor used to determine A or that influences the determination of A. This phrase does not exclude that the determination of A may also be based on some other factor, such as C. Furthermore, this phrase is intended to cover embodiments where A is determined based solely on B. As used herein, the phrase "based on" is synonymous with the phrase "at least partially based."

[0069] The phrases “in response to” and “in response to” describe one or more factors that trigger an effect. These phrases do not exclude the possibility that additional factors may influence or otherwise trigger an effect, either jointly with or independently of specific factors. That is, the effect may respond only to these factors, or may respond to specific factors as well as other unspecified factors. Consider the phrase “perform A in response to B.” This phrase specifies that B is a factor that triggers the performance of A or triggers a specific result for A. This phrase does not exclude that performing A may also respond to some other factor, such as C. This phrase also does not exclude that performing A may be performed jointly in response to B and C. This phrase is also intended to cover embodiments where A is performed only in response to B. As used herein, the phrase “in response” is synonymous with the phrase “at least partially in response.” Similarly, the phrase "in response to" is synonymous with the phrase "at least partially in response."

[0070] ***

[0071] Within this disclosure, different entities (which may be variously referred to as “units,” “circuits,” other components, etc.) may be described or claimed to be “configured” to perform one or more tasks or operations. The expression “[entity] configured to perform [one or more tasks]” is used in this specification to refer to a structure (i.e., a physical one). More specifically, this expression is used to indicate that such a structure is arranged to perform one or more tasks during operation. A structure may be said to be “configured” to perform some tasks even if the structure is not currently operating. Accordingly, an entity described or mentioned as being “configured” to perform some tasks refers to a physical thing, such as a device, a circuit, a system having memory and a processor unit that stores program instructions executable to implement the task. Such phrases are not used in this specification to refer to intangible things.

[0072] In some cases, various units / circuits / components may be described herein as performing a set of tasks or operations. It is understood that these entities are "configured" to perform such tasks / operations, even if not specifically mentioned.

[0073] The term "configured" is not intended to mean "configurable." For example, an unprogrammed FPGA would not be considered "configured" to perform a specific function. However, such an unprogrammed FPGA can be "configurable" to perform that function. After proper programming, it can then be said that the FPGA is "configured" to perform a specific function.

[0074] For the purposes of U.S. patent applications based on the present disclosure, reference in a claim that a structure is "configured" to perform one or more tasks is expressly intended not to apply 35 USC §112(f) to the relevant claim elements. If the applicant wishes to apply Section 112(f) during the examination of a U.S. patent application based on the present disclosure, the claim elements will be enumerated using a structure that is a "means" for "performing" [a function].

[0075] Different “circuits” may be described in this disclosure. These circuits or “circuit parts” constitute hardware comprising various types of circuit elements, such as combinational logic, clocked storage devices (e.g., flip-flops, registers, latches, etc.), finite state machines, memory (e.g., random access memory, embedded dynamic random access memory), programmable logic arrays, etc. Circuit parts may be custom-designed or taken from standard libraries. In various implementations, circuit parts may appropriately include digital components, analog components, or a combination of both. Specific types of circuits may generally be referred to as “units” (e.g., decoding unit, arithmetic logic unit (ALU), function unit, memory management unit (MMU), etc.). Such units also refer to circuits or circuit parts.

[0076] Accordingly, the disclosed circuits / units / components and other elements illustrated in the drawings and described herein include hardware elements such as those described in the preceding paragraph. In many instances, the internal arrangement of hardware elements within a particular circuit may be specified by describing the function of the circuit. For example, a particular "decode unit" may be described as performing the function of "processing the opcode of an instruction and routing the instruction to one or more of a plurality of function units," which implies that the decode unit is "configured" to perform this function. Specification of such function is sufficient to suggest to those skilled in the art of computers a set of possible structures for the circuit.

[0077] In various embodiments, as discussed in the previous paragraph, circuits, units, and other elements may be defined by functions or behaviors configured to be implemented. The arrangement of such circuits / units / components relative to one another and the manner in which they interact form a microarchitecture definition of hardware that is ultimately fabricated in an integrated circuit or programmed into an FPGA to form a physical implementation of the microarchitecture definition. Thus, a microarchitecture definition is recognized by those skilled in the art as a structure from which many physical implementations can be derived, all of which belong to the broader structure described by the microarchitecture definition. That is, a person skilled in the art who receives the microarchitecture definition provided according to this disclosure may implement the structure by coding the descriptions of the circuits / units / components in a hardware description language (HDL), such as Verilog or VHDL, without excessive experimentation and by applying ordinary techniques. HDL descriptions are often expressed in a manner that can appear functional. However, to those skilled in the art, such HDL descriptions are a method used to translate the structure of a circuit, unit, or component into the next level of implementation details. Such HDL descriptions may take the form of behavioral code (which is typically not synthesizable), Register Transfer Language (RTL) code (which, in contrast to behavioral code, is typically synthesizable), or structural code (e.g., a netlist specifying logic gates and their connections). HDL descriptions can be sequentially synthesized against a library of cells designed for a given integrated circuit manufacturing technology and modified for timing, power, and other reasons to create a final design database, which can then be sent to a foundry to generate masks and ultimately produce an integrated circuit.Some hardware circuits or parts thereof can also be custom-designed using a schematic editor and captured into an integrated circuit design along with the synthesized circuit parts. Integrated circuits may include transistors and other circuit elements (passive elements such as capacitors, resistors, inductors, etc.) and interconnections between the transistors and circuit elements. Some embodiments may implement multiple integrated circuits coupled together to implement the hardware circuits, and / or in some embodiments, separate elements may be used. Alternatively, the HDL design can be synthesized into a programmable logic array, such as a field programmable gate array (FPGA), and implemented on the FPGA. This decoupling between the design of a group of circuits and the subsequent low-level implementation of these circuits generally results in a scenario where the circuit or logic designer does not specify a particular set of structures for the low-level implementation beyond a description of what the circuit is configured to do, because this process is performed at a different stage of the circuit implementation process.

[0078] The fact that many different low-level combinations of circuit elements can be used to implement the same specifications of a circuit results in multiple equivalent structures for that circuit. As mentioned, these low-level circuit implementations can vary depending on changes in manufacturing technology, the foundry selected to fabricate the integrated circuit, the library of cells provided for a specific project, and so on. In many cases, the choices made by different design tools or methodologies to generate these different implementations can be arbitrary.

[0079] Furthermore, it is common for a single implementation of a specific functional specification of a circuit to involve a large number of devices (e.g., millions of transistors) for a given embodiment. Consequently, the enormous volume of this information makes it impractical to provide a complete description of the low-level structure used to implement a single embodiment, not to mention the vast array of equivalent possible implementations. For this reason, the present disclosure describes the structure of circuits using functional shorthand commonly used in the industry.

Claims

Claim 1 A computing system comprising: a plurality of compute die packages including processors configured to execute program instructions that operate on data stored in distributed memory accessible through an integrated memory architecture; and a plurality of memory die packages configured to implement the integrated memory architecture, wherein a given memory die package among the memory die packages comprises one or more optical dies including one or more optical interfaces configured to receive memory requests from the processors; and a memory controller die coupled to the one or more optical dies through one or more die-to-die interconnects, wherein the memory controller die comprises a cache and one or more memory controllers configured to access a portion of the distributed memory in response to the received memory requests; and wherein the plurality of compute die packages are distinct from the plurality of memory die packages. Claim 2 A computing system according to claim 1, wherein the cache is configured to store a subset of the data stored in the portion of the distributed memory, and the given memory die package is configured to service one or more of the memory requests from the cache. Claim 3 A computing system according to claim 1, wherein the given memory die package comprises one or more memories coupled to the one or more memory controllers and corresponding to the portion of the distributed memory. Claim 4 A computing system according to claim 1, wherein the given memory die package is configured to perform address hashing to distribute data across a plurality of memories coupled to the one or more memory controllers. Claim 5 A computing system according to claim 1, further comprising a memory tray including a plurality of memory die packages among the memory die packages. Claim 6 A computing system according to claim 5, further comprising a memory rack, wherein the memory tray is one of a plurality of memory trays inserted into the memory rack. Claim 7 A computing system according to claim 1, wherein the processors comprise one or more central processing units (CPUs). Claim 8 A computing system according to claim 7, wherein the processors further comprise one or more graphics processing units (GPUs), application-specific integrated circuits (ASICs), or field-programmable gate arrays (FPGAs). Claim 9 A computing system according to claim 1, wherein a given compute die package among the plurality of compute die packages comprises: a compute die including one or more processors and a die-die interconnect; and an optical die coupled to the compute die through the die-die interconnect and including one or more optical interfaces. Claim 10 A computing system according to claim 9, further comprising a compute tray including the plurality of compute die packages. Claim 11 A computing system according to claim 10, further comprising a compute rack, wherein the compute tray is one of a plurality of compute trays inserted into the compute rack. Claim 12 A computing system according to claim 11, further comprising a plurality of interconnected compute nodes, wherein a given compute node among the compute nodes comprises one or more compute racks coupled to one or more memory racks. Claim 13 A computing system according to claim 1, wherein the processors are configured to access any address within the integrated address space defined by the integrated memory architecture. Claim 14 A computing system according to claim 1, wherein a given page within an integrated address space defined by the integrated memory architecture is distributed among a plurality of memory die packages. Claim 15 A computing system according to claim 1, wherein the compute die packages and the memory die packages are coupled together through an optical interconnection fabric comprising at least two networks having heterogeneous operating characteristics. Claim 16 An integrated circuit comprising: a memory die package having one or more co-packaged dies, wherein the memory die package comprises one or more optical dies having an optical interface configured to receive memory requests from one or more processors in one or more compute die packages separated from the memory die package via an optical fiber; and a memory controller die coupled to the one or more optical dies via one or more die-die interconnects, wherein the memory controller die comprises a memory controller coupled to a memory and configured to service the memory requests; and a cache of the memory configured to store a portion of data stored in the memory. Claim 17 In claim 16, the memory die package is an integrated circuit configured to implement an integrated memory architecture of distributed memory accessible to one or more processors. Claim 18 In claim 17, the memory die package is an integrated circuit configured to store a given page within an integrated address space defined by the integrated memory architecture, such that the given page is distributed among the memory die package and one or more additional memory die packages. Claim 19 delete Claim 20 delete

Images