Disaggregated heterogeneous memory

Abstract

Systems and methods are provided herein for providing additional memory to GPUs or other computing devices. This may be accomplished by one or more chips on a first substrate having access to at least two memory devices that are not located on the first substrate. For example, a first microelectronic device may comprise a first substrate, a first interposer placed on the first substrate, a first chip placed on a first portion of the first interposer, and a second chip placed on a second portion of the first interposer. The second chip may be a logic chip electronically coupled to the first chip. The second chip may be configured to access at least two memory devices that are not located on the first substrate via the first chip.

Description

DISAGGREGATED HETEROGENEOUS MEMORYBackground

[0001] The present disclosure relates to advanced packaging for and interconnections among microelectronic devices, and in particular, to providing additional memory for computation.Summary

[0002] Recent advancements in natural language processing (e.g., the development and deployment of large language models (LLMs)) has resulted in increased computational, and memory demands for graphic processing units (GPUs). LLMs rely heavily on extensive neural networks that require substantial computing power and large memory capacities for optimal performance. The deployment of LLMs continues to increase, and in some instances the computational and memory requirements have outpaced the ability of existing GPU and other computational architectures. The limited available space along the periphery, or shoreline, of GPUs may hinder the performance of increasingly complex LLMs. This problem is exacerbated by the required architecture of some GPUs. For example, some GPUs require memory chips (e.g., high bandwidth memory (HBM) chips) to be attached along the shorelines of the GPU chips. However, the shorelines of these chips are limited, and the memory chips that are located on the shoreline of the logic chips are restricted in their capacity due to the finite space available along the periphery of GPUs. In view of these deficiencies, there exists a need to provide additional memory resources to GPUs beyond what can be supported along the shoreline.

[0003] Accordingly, techniques are disclosed herein for a system to provide additional memory to GPUs or other computing devices. One general aspect includes a system comprising a first microelectronic device. The first microelectronic device may comprise a first interposer attached to a first substrate. The first microelectronic device may further comprise a first chip attached to a first portion of the first interposer, a second chip attached to a second portion of the first interposer, and a second interposer attached to the first substrate. The first chip may be a memory chip and may be adjacent to the second chip. The first chip may be electronically coupled to the second chip using one or more first connections disposed within the first interposer. The second chip may be a logic chip and the second interposer may be an interposer comprising optical waveguides and supportingcomponents (e.g., electrical to optical converters, (optical to electrical converters, etc.). The second interposer may be attached to the first substrate adjacent to the first interposer. The first microelectronic device may further comprise a third chip attached to the top of the second interposer. The third chip may be electronically coupled to the second chip using one or more shoreline connections.

[0004] The third chip may be a field programable gate area (FPGA) or other chip containing serializers / deserializers (SerDes) and may be electronically coupled to the second interposer. The second interposer may further comprise a first optical input and a first optical output. The third chip may deserialize information received from the first optical input through the second interposer. For example, a second microelectronic device, not located within the device package of the first microelectronic device, may transmit information via one or more optical connections to the second interposer. The second interposer may transmit the information received from the second microelectronic device to the third chip via the first optical input. The third chip may also serialize information received from the second chip and output the serialized information, via the first optical output, to the second interposer. The second interposer may then transmit the serialized information in optical format to one or more devices (e.g., a second microelectronic device) via one or more optical connections. In some embodiments, the second chip is configured to access additional memory storage (e.g., the second microelectronic device) via the third chip, the second interposer, and / or one or more optical connections. Accordingly, one or more logic chips (e.g., the second chip) has access to additional memory storage (e.g., the second microelectronic device) beyond that which can be attached along its shoreline.

[0005] In some embodiments, the system comprises a second microelectronic device and / or a third microelectronic device optically connected to the first microelectronic device. In some embodiments, the second and / or third microelectronic device are HBM memory devices. In some embodiments, the second microelectronic device comprises a third interposer attached to a second substrate. The second microelectronic device may further comprise a fourth chip attached to a first portion of the third interposer, a fifth chip attached to a second portion of the third interposer, and a fourth interposer attached to the second substrate. The fourth chip may be a second memory chip and may be adjacent to the fifth chip. The fourth chip may be electronically coupled to the fifth chip using one or more second connections disposed within the second interposer. The fifth chip may be a secondlogic chip and the fourth interposer may be a second optical interposer. The fourth interposer may be attached to the second substrate adjacent to the third interposer. The second microelectronic device may further comprise a sixth chip attached to the top of the fourth interposer. The sixth chip may be electronically coupled to the fifth chip using one or more shoreline connections.

[0006] The sixth chip may be a second FPGA or other chip containing SerDes and may be electronically coupled to the fourth interposer. The fourth interposer may further comprise a second optical input and a second optical output. The sixth chip may deserialize information received from the second optical input through the fourth interposer. For example, the first microelectronic device may transmit information via the one or more optical connections to the fourth interposer. The fourth interposer may transmit the information received from the first microelectronic device to the sixth chip via the second optical input. The sixth chip may also serialize information received from the fifth chip and output the serialized information, via the second optical output, to the fourth interposer. The fourth interposer may then transmit the serialized information to one or more devices (e.g., the first microelectronic device) via the one or more optical connections. In some embodiments, the second chip is configured to access additional memory storage (e.g., the fourth chip) via the third chip, the second interposer, the one or more optical connections, the fourth interposer, the sixth chip, and / or the fifth chip. Accordingly, one or more logic chips (e.g., the second chip) on the first microelectronic device has access to additional memory storage (e.g., fourth chip) beyond that which can be attached along its shoreline.

[0007] In some embodiments, the second and / or third microelectronic device are double data rate (DDR) memory devices. In some embodiments, the third microelectronic device comprises a fifth interposer, a seventh chip, and one or more graphics DDR (GDDR) devices. The seventh chip may be a third FPGA or other chip containing SerDes and may be electronically coupled to the fifth interposer. The fifth interposer may be an interposer comprising optical waveguides and supporting components such as electrical to optical and optical to electrical converters. The fifth interposer may further comprise a third optical input and a third optical output. The seventh chip may deserialize information received from the third optical input. For example, the first microelectronic device may transmit information via the one or more optical connections to the fifth interposer. The fifth interposer may transmit the information received from the first microelectronic device to the seventh chip viathe second optical input. The seventh chip may also serialize information received from one or more GDDR devices and output the serialized information, via the third optical output, to the fifth interposer. The fifth interposer may then transmit the serialized information to one or more devices (e.g., the first microelectronic device) via the one or more optical connections. In some embodiments, the second chip is configured to access additional memory storage (e.g., the GDDR devices) via the third chip, the second interposer, the one or more optical connections, the fifth interposer, and / or the seventh chip. Accordingly, one or more logic chips (e.g., the second chip) on the first microelectronic device has access to additional memory storage (e.g., the one or more GDDR devices) beyond that which can be attached along its shoreline.

[0008] In some embodiments, the first substrate of the first microelectronic device is in a shape of a rectangle comprising a first edge, a second edge, a third edge, and a fourth edge. The first edge of the substrate may be parallel to the third edge of the substrate. The first edge of the substrate may be perpendicular to the second edge and the fourth edge of the substrate. In some embodiments, a first plurality of memory chips are aligned along the first edge of the first substrate and a second plurality of memory chips are aligned along the second edge of the first substrate. A plurality of logic chips may be located between the first plurality of memory chips and the second plurality of memory chips. In some embodiments, the second interposer is placed directly on the first substrate along the second edge of the first substrate.Brief Description of the Drawings

[0009] The accompanying drawings provide additional details related to some embodiments of the disclosure described herein. The drawings are provided for purposes of illustration only and merely depict typical or example embodiments. For example, FIGS. 1-8 display one or more microelectronic devices and / or portions of microelectronic devices, in accordance with embodiments of the disclosure. These drawings are provided to facilitate an understanding of the concepts disclosed herein and should not be considered limiting of the breadth, scope, or applicability of these concepts. It should be noted that for clarity and ease of illustration, these drawings are not necessarily made to scale.

[0010] FIG. 1 depicts a system with disaggregated memory, in accordance with some embodiments of this disclosure.

[0011] FIG. 2 depicts a cross-sectional view of devices of a system with disaggregated memory, in accordance with some embodiments of this disclosure.

[0012] FIG. 3 depicts a plan view of a system with disaggregated memory, in accordance with some embodiments of this disclosure.

[0013] FIG. 4 depicts a plan view of a microelectronic device, in accordance with some embodiments of this disclosure.

[0014] FIG. 5 depicts a plan view of a microelectronic device, in accordance with some embodiments of this disclosure.

[0015] FIG. 6 depicts a plan view of a microelectronic device, in accordance with some embodiments of this disclosure.

[0016] FIG. 7 depicts a system with disaggregated memory, in accordance with some embodiments of this disclosure.

[0017] FIG. 8 depicts a cross-sectional view of microelectronic device, in accordance with some embodiments of this disclosure.

[0018] FIG. 9 depicts a cross-sectional view of microelectronic device, in accordance with some embodiments of this disclosure.

[0019] FIG. 10 depicts a cross-sectional view of microelectronic device, in accordance with some embodiments of this disclosure.

[0020] FIG. 11 depicts a cross-sectional view of microelectronic device, in accordance with some embodiments of this disclosure.

[0021] FIG. 12 depicts a cross-sectional view of microelectronic device, in accordance with some embodiments of this disclosure.Detailed Description

[0022] FIG. 1 depicts an illustrative example of a general purpose Al processing unit (e.g., GP-AIU), in accordance with some embodiments of this disclosure. In some embodiments, unit 100 (e.g., a GP-AIU) comprises compute tiles (e.g., compute tiles 102a-102h), high bandwidth memory (HBM) chips (e.g., HBM chips 104a-104h), and memory extenders (e.g., memory extenders 106a-106d). In some implementations, unit 100 connects, via connection 108, to memory bank 110. In some embodiments, memory bank 110 comprises memory extender 112 and one or more HBM chips 114. In some implementations, unit 100 connects, via connection 116, to DDR5 memory bank 118. In some embodiments, compute tiles 102a-102h, HBM chips 104a-104h, and memory extenders 106a-106d are connected via shoreline UCIe (e.g., universal chiplet interconnect express) connections to create unit 100. In some implementations, compute tiles 102a-102h, HBM chips 104a-104h, and memory extenders 106a-106d are integrated with a chip-on-wafer-on-substrate (CoWoS) interposer (e.g., a silicon wafer that acts as a platform for stacking system on chip components to facilitate high-speed data transfer and integration), a carrier substrate, an optical interposer, or any other suitable components. In some embodiments, when unit 100 comprises 12 compute tiles, unit 100 comprises 432 GB of HBM cache storage. In some implementations, unit 100 is limited by the package size of substrate and / or interposer. For example, in some embodiments, when unit 100 uses TSMC’s CoWos technology, unit 100 is limited to a 73mmx73mm size.

[0023] In some embodiments, compute tiles 102a-102h comprise modular processing units that can be integrated into a larger computing system to enhance processing efficiency, scalability, and parallelism. In some implementations, compute tiles 102a-102h comprise dedicated processing elements, memory structures, and interconnects designed to facilitate high-speed data exchange and workload distribution. In some embodiments, compute tiles 102a-102h are arranged in a scalable architecture (e.g., as depicted by unit 100), allowing additional tiles to be incorporated dynamically to increase processing power or optimize computational efficiency (e.g., as depicted in FIGS. 3-7). In some implementations, each compute tile (e.g., compute tiles 102a-102h) are configured for general-purpose processing, artificial intelligence (Al) and / or machine learning (ML) acceleration, graphics rendering, or other computational tasks. In some embodiments, compute tiles 102a-102h utilize high- bandwidth, low-latency interconnects that enable efficient data transfer between tiles, thereby reducing bottlenecks and improving overall system performance. In some implementations, compute tiles 102a-102h incorporate dynamic power management techniques, such as selective activation, voltage scaling, and / or workload-based power distribution, to optimize energy consumption.

[0024] In some embodiments, compute tiles 102a-102h comprise different types of compute tiles, such as CPU, GPU, and / or Al accelerators. In some implementations, combining the different types of compute tiles (e.g., compute tiles 102a-102h) within unit 100 provides a heterogeneous computing environment that balances performance and efficiency for diverse workloads. In some embodiments, compute tiles 102a-102h are implementedusing advanced semiconductor manufacturing techniques, including chiplet-based packaging, thereby enabling high-performance integration without the constraints of monolithic die scaling. In some implementations, compute tiles 102a-102h are utilized in a variety of computing domains, such as high-performance computing (HPC), edge computing, data centers, and consumer devices. In some embodiments, compute tiles 102a-102h modular nature allows manufacturers to develop customizable processing architectures tailored to specific performance and power requirements (e.g., unit 100).

[0025] In some embodiments, HBM chips 104a-104h have a 36 gigabyte (GB) storage capacity or any suitable storage capacity. In some implementations, HBM chips 104a-104h have a 1.2 Terabytes per second (TB / s) maximum data transfer bandwidth or any suitable maximum data transfer bandwidth. In some embodiments, HBM chips 104a-104h are integrated into respective compute chips (e.g., compute tiles 102a-102h). In some embodiments, HBM chips 104a-104h comprise vertically stacked memory modules that integrate multiple memory dies using through-silicon vias (TSVs) and an interposer to achieve high data transfer rates, low power consumption, and reduced physical footprint. In some implementations, HBM chips 104a-104h enable efficient communication with processing units, improving system throughput and reducing memory bottlenecks. In some embodiments, HBM chips 104a-104h comprise multiple dynamic random-access memory (e.g., DRAM) layers stacked vertically and connected through TSVs and micro-bumps to increase memory density while minimizing footprint, thereby enabling compact high- performance computing systems. In some implementations, HBM chips 104a-104h have a wide memory interface (e.g., thousands of bits), thereby allowing high data transfer rates.

[0026] In some embodiments, HBM chips 104a-104h operate in parallel across many channels, thereby reducing latency and improving memory throughput (e.g., for computeintensive applications). In some implementations, HBM chips 104a-104h reduce power per bit transferred by using a shorter interconnect path and optimized signaling techniques. In some embodiments, HBM chips 104a-104h enable efficient power delivery through voltage scaling and power management techniques. In some implementations, HBM chips 104a-104h minimize memory access latency by reducing the distance between memory and the processing unit (e.g., CPU, GPU, FPGA). In some embodiments, HBM chips 104a-104h use a high-speed interposer or embedded multi-die interconnect bridge (EMIB) to enhance signal integrity and further reduce data transfer delays. In some implementations, HBM chips 104a-104h are connected to processors (e.g., GPUs, Al accelerators) via a silicon interposer, a high-speed chiplet-based architecture, or any other suitable interposer (e.g., as further discussed in regard to FIG. 2).

[0027] In some embodiments, HBM chips 104a-104h enable direct high-throughput, low- latency communication between memory and compute units (e.g., compute tiles 102a-102h). In some implementations, HBM chips 104a-104h are configured in different capacities (e.g., HBM2, HBM3, future variants), thereby allowing manufacturers to optimize performance, bandwidth, and power consumption for different applications. In some embodiments, HBM chips 104a-104h are integrated within a system to further enhance overall memory bandwidth. In some implementations, HBM chips 104a-104h are widely used in AI / ML accelerators, GPUs, FPGAs, HPC, networking equipment, and edge devices (e.g., where high-speed memory access is critical).

[0028] In some embodiments, memory extenders 106a-106d and / or memory extender 112 enable connections (e.g., connection 108, connection 116) that are e.g., low power and high bandwidth to be made with one or more memory banks (e.g., HBM memory bank, DDR5 memory bank 118, etc.). In some implementations, memory extenders 106a-106d and / or memory extender 112 comprise inputs / outputs to couple with one or more connections (e.g., connection 116) between unit 100 and DDR5 memory bank 118 (e.g., that has a 1.1 TB / s data transfer bandwidth or any suitable data transfer bandwidth). In some embodiments, memory extenders 106a-106d and / or memory extender 112 are coupled to one or more connections (e.g., connection 108) to be made between unit 100 and memory bank 110 (e.g., that has a 4.4 TB / s data transfer bandwidth or any suitable data transfer bandwidth). In some implementations, memory extenders 106a-106d and / or memory extender 112 comprise inputs / outputs to couple (e.g., having a 1.1-2.2 TB / s data transfer bandwidth or any suitable data transfer bandwidth) with a neighboring component (e.g., GPU).

[0029] In some embodiments, connection 108 and / or connection 116 comprise low power, high bandwidth optical links. In some implementations, connection 108 and / or connection 116 comprise a 1.1 TB / s connection (e.g., when connecting unit 100 to DDR5 memory bank 118), a 4.4 TB / s connection (e.g., when connecting unit 100 to memory bank 110), a 1.1-2.2 TB / s connection (e.g., when connecting memory extenders 106a-106d and / or memory extender 112 to a neighboring component). In some implementations, connection 108 and / or connection 116 comprise passive connections (e.g., copper connections). In someembodiments, connection 108 and / or connection 116 comprise nonpassive connections (e.g., optical links) that enable efficient, high-speed data communication by leveraging photonic transmission instead of traditional electrical interconnects. In some implementations, connection 108 and / or connection 116 reduce power consumption while increasing data transfer capacity, thereby addressing bottlenecks in computing and networking systems. In some embodiments, when connection 108 and / or connection 116 comprise optical links, connection 108 and / or connection 116 provide multi-terabit per second (Tb / s) transmission across computing systems (e.g., surpassing traditional copper interconnect Tb / s transmission rates). In some implementations, connection 108 and / or connection 116 minimize resistive losses and heat generation, thereby reducing overall power per bit transferred (e.g., when connection 108 and / or connection 116 comprise optical links). In some embodiments, when connection 108 and / or connection 116 comprise optical links, connection 108 and / or connection 116 allow multiple data channels to be transmitted simultaneously, thereby maximizing bandwidth without increasing the physical footprint.

[0030] In some embodiments, the memory bank 110 comprises one or more HBM memory chips 114 that is disaggregated, per GP-AIU (e.g., unit 100). In some implementations, memory bank 110 has a 144 GB storage capacity (e.g., or any suitable storage capacity) and a 4.8 TB / s maximum data transfer bandwidth (e.g., or any suitable maximum data transfer bandwidth). In some implementations, memory bank 110 delivers 144 GB at 4.8 TB / s. In some implementations, memory bank 110 delivers data over an HBM-UCIe-SerDes-optical link (e.g., or any suitable link). In some embodiments, memory bank 110 is connected to unit 100 via connection 108. In some implementations, memory bank 110 establishes connection 108 with unit 100 via memory extender 112. In some implementations, unit 100 establishes a connection with multiple instances of memory bank 110 (e.g., unit 100 connects to four individual memory banks, each memory bank comprising memory bank 110). For example, in some embodiments, unit 100, via memory extenders 106a-106d, connects to four high bandwidth HBM banks (e.g., memory bank 110).

[0031] In some embodiments, DDR5 memory bank 118 comprises a low cost and supercharged memory bank (e.g., DDR5 memory bank). In some embodiments, , DDR5 memory bank 118 comprises a high capacity DDR5 bank that is disaggregated, per GP-AIU (e.g., unit 100). In some implementations, DDR5 memory bank 118 has a 1-2 TB storage capacity (e.g., or any suitable storage capacity) and a 1 TB / s maximum data transferbandwidth (e.g., or any suitable maximum data transfer bandwidth). In some implementations, DDR5 memory bank 118 delivers 1-2 TB at 1 TB / s. In some implementations, DDR5 memory bank 118 delivers data over a DDR-SerDes-optical link (e.g., or any suitable link). In some embodiments, DDR5 memory bank 118 is connected to unit 100 via connection 116. In some implementations, unit 100 establishes a connection with multiple instances of DDR5 memory bank 118 (e.g., unit 100 connects to four individual memory banks, each memory bank comprising DDR5 memory bank 118). For example, in some embodiments, unit 100, via memory extenders 106a-106d, connects to four low cost and supercharged DDR5 memory banks (e.g., DDR5 memory bank 118). In some implementations, unit 100 (e.g., via memory extenders 106a-106d) connects to four high capacity DDR5 banks (e.g., DDR5 memory bank 118) and four high bandwidth HBM banks (e.g., memory bank 110). In some embodiments, unit 100 establishes any suitable number of connections (e.g., connection 108, connection 116) with any suitable number (e.g., 1, 4, 8, 12) of high capacity DDR5 banks (e.g., DDR5 memory bank 118) and / or any suitable number (e.g., 1, 4, 8, 12) of high bandwidth HBM banks(e.g., memory bank 110).

[0032] FIG. 2 depicts a side view of a general purpose Al processing unit (e.g., GP-AIU), in accordance with some embodiments of this disclosure. In some embodiments, unit 200 (e.g., unit 100 of FIG. 1) comprises compute chip 202 (e.g., compute tiles 102a-102h), HBM chip 204 (e.g., HBM chips 104a-104h), SerDes chip 206, interposer 208, interposer 210, and substrate 212. In some implementations, unit 200 connects, via connection 214 (e.g., connection 108), to HBM memory bank 216 (e.g., memory bank 110), comprising interposer 218, interposer 220, routing and cache unit 222, SerDes chip 224, HBM chip 226, and substrate 228. In some embodiments, unit 200 connects, via connection 230 (e.g., connection 116), to DDR memory bank 232 (e.g., DDR5 memory bank 118) comprising SerDes and cache unit 234 and interposer 236. In some implementations, SerDes and cache unit 234 connects, via connections 238, to (G)DDR memory banks 240. In some embodiments, compute chip 202, HBM chip 204, and SerDes chip 206 are connected via shoreline UCIe connections. In some implementations, routing and cache unit 222, SerDes chip 224, and HBM chip 226 are connected via shoreline UCIe connections. In some embodiments, SerDes and cache unit 234 and interposer 236 are connected via shoreline UCIe connections.

[0033] In some embodiments, SerDes chip 206 and / or SerDes chip 224 comprise highspeed data interface circuits designed to convert parallel data streams into serial data (e.g.,serialization) and reconstruct serial data back into parallel format (e.g., deserialization). In some embodiments, SerDes chip 206 and / or SerDes chip 224 interface with the physical transport medium (PTM). In some implementations, SerDes chip 206 and / or SerDes chip 224 enable efficient, high-bandwidth data transmission across computing and networking systems while minimizing interconnect complexity and power consumption. In some embodiments, SerDes chip 206 and / or SerDes chip 224 allow high-data-rate serial transmission over differential signaling, thereby significantly reducing the number of required physical connections compared to parallel buses. In some implementations, SerDes chip 206 and / or SerDes chip 224 enable transmission speeds in the range of gigabits per second (Gbps) to terabits per second (Tbps), which is critical for AI / ML accelerators, high-performance computing (HPC), and data center interconnects.

[0034] In some embodiments, SerDes chip 206 and / or SerDes chip 224 use advanced equalization techniques (e.g., continuous-time linear equalization (CTLE), decision feedback equalization (DFE)) to compensate for channel losses and distortions over long-distance links. In some implementations, SerDes chip 206 and / or SerDes chip 224 use differential signaling to reduce electromagnetic interference (EMI) and improve signal integrity, thereby allowing for high-speed data transmission across PCBs, fiber optics, and backplanes. In some embodiments, SerDes chip 206 and / or SerDes chip 224 have a chiplet-based SerDes architecture to enable high-bandwidth, low-latency communication between disaggregated computing elements (e.g., multi-die System on Chips, GPUs, Al processors). In some implementations, SerDes chip 206 and / or SerDes chip 224 use die-to-die (D2D) and package- to-package (P2P) interconnects to allow heterogeneous integration in high-performance computing and networking applications. In some embodiments, SerDes chip 206 and / or SerDes chip 224 enable high-speed networking, AI / ML accelerators, data center interconnects, automotive advanced driver-assistance systems (ADAS), wireless communications, storage controllers, and chiplet-based computing architectures where high- bandwidth, low-power data movement is essential.

[0035] In some embodiments, SerDes chip 206 and / or SerDes chip 224 are optimized for UCIe to enable high-bandwidth, low-latency communication between compute dies (e.g., compute chip 202, compute tiles 102a-102h) and memory expansion chiplets (e.g., memory extenders 106a-106d). In some implementations, SerDes chip 206 and / or SerDes chip 224, by leveraging serial high-speed signaling, facilitate seamless memory pooling, disaggregation,and composable architectures. In some embodiments, SerDes chip 206 and / or SerDes chip 224 enable multi-TB / s memory interconnects, thereby providing high-throughput. In some implementations, SerDes chip 206 and / or SerDes chip 224 support serial-to-parallel conversion for wide memory interfaces, thereby allowing multiple chiplets to operate as a unified system. In some implementations, SerDes chip 206 and / or SerDes chip 224 support multi-lane, high-bandwidth interconnects tailored for chiplet-based architectures to enable flexible die-to-die (D2D) communication. In some embodiments, SerDes chip 206 and / or SerDes chip 224 enable data rates of 16-32+ GT / s per lane, thereby allowing aggregate bandwidths in the terabits per second (Tbps) range while maintaining low latency. In some implementations, SerDes chip 206 and / or SerDes chip 224 support UCIe-standardized layers, including PHY, transport, and protocol layers, thereby enabling seamless chiplet interoperability. In some embodiments, SerDes chip 206 and / or SerDes chip 224 are backward compatible with PCIe (Peripheral Component Interconnect Express), CXL (Compute Express Link), and HBM interconnects, thereby ensuring broad applicability in data centers, Al accelerators, and HPC architectures.

[0036] In some embodiments, interposer 208 and / or interposer 218 comprise CoWoS interposers. In some implementations, unit 200 comprises / uses advanced co-packaging such as CoWoS interposers (e.g., interposer 208 and / or interposer 218). In some embodiments, unit 200 uses standard packaging (e.g., such as flip-chip, ball grid array, and organic substrates), thereby decreasing associated costs and risks when manufacturing unit 200. In some implementations, interposer 208 and / or interposer 218 comprise a packaging technology that enables the integration of multiple dies (e.g., compute chip 202, HBM chip 204, SerDes chip 206) on a silicon interposer, thereby providing a high-bandwidth, low- latency, and power-efficient interconnect for heterogeneous computing architectures. In some embodiments, interposer 208 and / or interposer 218 serve as high-density electrical and thermal conduits, thereby facilitating fine-pitch, low-power signaling e.g., between compute, memory, and accelerator chiplets. In some implementations, interposer 208 and / or interposer 218 comprise a silicon interposer with fine-pitch redistribution layers (RDLs) to support high-bandwidth interconnects between compute dies (e.g., compute chip 202, compute tiles 102a-102h) and HBM stacks (e.g., HBM chip 204, HBM chips 104a-104h).

[0037] In some embodiments, interposer 208 and / or interposer 218 enable die-to-die communication with ultra-low latency, thereby significantly reducing data transferbottlenecks. In some implementations, interposer 208 and / or interposer 218 utilize microbump and through-silicon via (TSV) technology to achieve high-bandwidth interconnects, supporting multi-TB / s aggregate bandwidths. In some embodiments, interposer 208 and / or interposer 218 allow mix-and-match integration of chiplets fabricated on different process technologies, thereby facilitating co-packaging of high-performance logic (e.g., CPUs, GPUs, Al accelerators) with memory and specialized ASICs. In some implementations, interposer 208 and / or interposer 218 provide dedicated high-speed pathways between compute chiplets and HBM, thereby ensuring minimal memory latency while maximizing effective bandwidth. In some embodiments, interposer 208 and / or interposer 218 incorporate internal thermal vias and optimized heat dissipation structures to improve heat transfer efficiency and reduce hot spots. In some implementations, interposer 208 and / or interposer 218 feature multi-layer interposer routing, thereby minimizing signal degradation and electromagnetic interference (EMI). In some embodiments, interposer 208 and / or interposer 218 support co-packaged optics (CPO) and silicon photonics (SiPh) integration, thereby enabling optical VO (e.g., connection 108, connection 116, connection 214, connection 230) for high-speed data transfer.

[0038] In some embodiments, interposer 210, interposer 220, and / or interposer 236 comprise optical interposers (e.g., that enable the establishment of connection 214 and / or connection 230). In some implementations, interposer 210, interposer 220, and / or interposer 236 integrate optical waveguides, photonic devices, and electrical interconnects within an interposer to enable high-bandwidth, low-power optical communication between chiplets. In some embodiments, interposer 210, interposer 220, and / or interposer 236 comprise a hybrid electronic-photonic substrate that enables chiplet-to-chiplet communication via integrated optical waveguides, photonic modulators, and / or detectors (e.g., rather than relying solely on conventional electrical interconnects). In some implementations, by leveraging integrated silicon photonics (SiPh), co-packaged optics (CPO), and high-speed optical links, interposer 210, interposer 220, and / or interposer 236 provide low-latency, low-power, and ultra-high- bandwidth interconnects for advanced computing architectures. In some embodiments, interposer 210, interposer 220, and / or interposer 236 integrate embedded optical waveguides within the interposer substrate to enable multi-TB / s chiplet-to-chiplet data transfer. In some implementations, interposer 210, interposer 220, and / or interposer 236 support densewavelength-division multiplexing (DWDM), thereby allowing multiple optical signals to coexist in a single interposer layer and dramatically increasing bandwidth efficiency.

[0039] In some embodiments, interposer 210, interposer 220, and / or interposer 236 use integrated photonic modulators and detectors to allow direct conversion of electrical signals into optical domain for near-lossless interconnects. In some implementations, interposer 210, interposer 220, and / or interposer 236 support both electrical and optical interconnects to facilitate hybrid photonic-electronic chiplet architectures for, e.g., Al accelerators, high- performance computing (HPC), and data center networking. In some embodiments, interposer 210, interposer 220, and / or interposer 236 enable direct optical interfaces between e.g., CPUs, GPUs, FPGAs, and HBM. In some implementations, interposer 210, interposer 220, and / or interposer 236 incorporate integrated optical couplers and fiber attach points, thereby allowing external optical fibers to interface directly with the interposer. In some embodiments, interposer 210, interposer 220, and / or interposer 236 support co-packaged optics (CPO) for e.g., next-generation network switches. In some implementations, interposer 210, interposer 220, and / or interposer 236 enable vertical stacking of photonic and electrical layers to allow high-density 3D integration of optical waveguides, modulators, and detectors within a single interposer. In some embodiments, interposer 210, interposer 220, and / or interposer 236 use through-silicon vias (TSVs) and / or photonic vias (PVs) to connect optical layers with electrical components, thereby ensuring efficient signal conversion and routing.

[0040] In some embodiments, substrate 212 and / or substrate 228 comprise a carrier substrate. In some implementations, substrate 212 and / or substrate 228 serve as a mechanical support and electrical interface for unit 200 and / or HBM memory bank 216. In some embodiments, substrate 212 and / or substrate 228 provide structural integrity, power distribution, signal routing, and thermal management (e.g., for unit 200 and / or HBM memory bank 216). In some implementations, substrate 212 and / or substrate 228 provides a rigid or semi-rigid foundation for integrating semiconductor dies (e.g., compute chip 202, HBM chip 204, SerDes chip 206 and / or routing and cache unit 222, SerDes chip 224, and HBM chip 226) to ensure proper alignment and mechanical stability during fabrication and operation. In some embodiments, substrate 212 and / or substrate 228 implement fine-pitch redistribution layers (RDLs), embedded traces, and vias for high-speed signal transmission between e.g., integrated dies and external interfaces. In some implementations, substrate 212 and / or substrate 228 support high-bandwidth electrical signaling through microbumps, through-substrate vias (TSVs), or via hybrid bonding (e.g., for minimal signal loss and high data rates). In some embodiments, substrate 212 and / or substrate 228 comprise organic laminates, silicon, glass, ceramic, and / or composite materials (e.g., depending on the thermal, electrical, and mechanical requirements). In some implementations, substrate 212 and / or substrate 228 enable flip-chip ball grid arrays (FCBGA), fan-out wafer-level packaging (FOWLP), and embedded multi-die interconnect bridge (EMIB) architectures (e.g., to enable advanced semiconductor integration). In some embodiments, substrate 212 and / or substrate 228 facilitate multi-chip module (MCM) designs by supporting the co-packaging of e.g., logic (e.g., compute tiles 102a-102h, compute chip 202), memory (e.g., HBM chip 204, HBM chip 226, HBM chips 104a-104h), and / or accelerators for Al and HPC applications.

[0041] In some embodiments, routing and cache unit 222 is connected to HBM chip 226 via a shoreline UCIe connection. In some implementations, routing and cache unit 222 is connected to SerDes chip 224 via a shoreline UCIe connection. In some embodiments, routing and cache unit 222 enables HBM memory bank 216 to route and cache data. For example, in some implementations, routing and cache unit 222 manage data flow and interconnects between different processing elements (e.g., SerDes chip 224), memory (e.g., HBM chip 222), and any other suitable component. In some embodiments, routing and cache unit 222 enables efficient communication between dies using high-bandwidth interconnects (e.g., UCIe, HBM, SerDes, optical links). In some implementations, routing and cache unit 222 manages traffic between compute units using e.g., optical waveguides or electrical traces for high-speed data movement. In some embodiments, routing and cache unit 222 stores frequently used data. In some implementations, routing and cache unit 222 employs scratchpad memory or SRAM (static RAM) based caches to minimize data-fetch overhead from DRAM. In some embodiments, routing and cache unit 222 is embedded into interposer 218 (e.g., CoWos, silicon interposer) e.g., to facilitate high-bandwidth communication.

[0042] In some embodiments, SerDes and cache unit 234 is connected to interposer 236 by a shoreline UCIe connection. In some implementations, SerDes and cache unit 234 is connected to (G)DDR memory banks 240 by connection 238 (e.g., a low power, high bandwidth optical link). In some embodiments, SerDes and cache unit 234 enables highspeed, long-distance, low-power data transmission across interconnects such as UCIe, PCIe, CXL, Ethernet, and optical links. In some implementations, SerDes and cache unit 234 optimizes memory access latency, bandwidth efficiency, and compute throughput, reducingdependency on external memory systems like DRAM or HBM. In some embodiments, SerDes and cache unit 234 performs the functions of SerDes chip 206 and / or SerDes chip 224 (e.g., by embedding SerDes chip 206 and / or SerDes chip 224 in a cache unit). In some implementations, SerDes and cache unit 234 integrates SerDes-based memory links for cache-to-cache communication between chiplets.

[0043] In some embodiments, (G)DDR (e.g., graphics double data rate) memory banks 240 allow for parallel access (e.g., by SerDes and cache unit 234 and ultimately unit 200) to respective portions of stored data, thereby enabling high-throughput operations without bottlenecks. In some embodiments, (G)DDR memory banks 240 comprise any of 4 (G)DDR memory banks, 8 (G)DDR memory banks, 16 (G)DDR memory banks, 32 (G)DDR memory banks, or any suitable number of (G)DDR memory banks. In some embodiments, the (G)DDR memory banks that comprise (G)DDR memory banks 240 are divided into rows and columns to allow for parallel access. In some implementations, each(G)DDR memory bank of (G)DDR memory banks 240 is accessed independently, thereby allow the GPU or memory controller to read / write to multiple banks simultaneously. In some embodiments, access to (G)DDR memory banks 240 is governed by row access strobe (RAS) and column access strobe (CAS) signals that control the timing and sequencing of memory operations, thereby enabling efficient data access.

[0044] In some embodiments, unit 200 does not include HBM chip 204. In some implementations, compute chip 202 has SRAM (e.g., when HBM chip 204 is not a part of unit 200). In some embodiments, interposer 208 uses advanced packaging or is in a standard packaging (e.g., organic, flip chip). In some implementations, SerDes chip 206 is included in unit 200 as a single chip. In some embodiments, compute chip 202 is integrated with the physical transport medium (PTM) through a SerDes chip on the compute chip (e.g., SerDes chip 206 is integrated with compute chip 202). In some implementations, compute chip 202 is integrated with the PTM through a wide parallel bus. In some embodiments, compute chip 202 is integrated with the PTM through a separate, distinct SerDes chip (e.g., SerDes chip 206, SerDes chip 224) through a wide parallel interface. In some implementations, SerDes chip 206 connects to the PTM. In some embodiments, the PTM is a passive medium (e.g., copper, Printed Circuit Board (PCB) traces, Twinax cabling, coaxial cable, or flexible substrates). In some implementations, SerDes chip 224 is connected to interposer 220 (e.g.,that supports a copper or optical connection) via an organic package substrate (e.g., or other suitable connection).

[0045] In some embodiments, the PTM is an optical medium (e.g., a non-passive medium due to the active circuits used in optics)designed for signal transmission, such as optical fibers, optical waveguides, and / or photonic substrates. In some implementations, the PTM comprises an active optical system (e.g., that incorporates integrated photonic circuits) such as the optical substrates being developed by Celestial Al and / or Lightmatter. For example, in some embodiments, the optical substrate comprises a substrate that sits under a chip (e.g., as being developed by Celestial Al and Lightmatter). In some embodiments, the optical substrate comprises co-packaged copper and / or co-packaged optics that sit adjacent to e.g., compute chip 202 and connect to compute chip 202 through a parallel bus, such as UCIe. In some embodiments, SerDes chip 206 facilitates optical interconnections (e.g., even when the optics are sitting adjacent to the chip). In some embodiments, co-packaged optics (CPO) sit adjacent to e.g., compute chip 202, HBM chip 204, and / or SerDes chip 206 and connects via a UCIe connection. In some implementations, the optical interconnect uses e.g., SerDes chip 206 to connect to the optics (e.g., even when adjacent to the chip). In some embodiments, unit 200 comprises pluggable form factor optics (e.g., Quad Small Form-factor Pluggable, Octal SFP, C Form-factor Pluggable, Small Form-factor Pluggable). In some implementations, DDR memory bank 232 comprises any suitable type of dynamic randomaccess memory (DRAM), such as graphics double data rate (GDDR), double data rate (DDR), and / or low power double data rate (LPDDR) (e.g., the selected DRAM being suitable for supporting the functionality of unit 200). For example, when unit 200 is used for GPUs, Al, and / or HPC, DDR memory bank 232 comprises GDDR. Further, for example, when unit 200 is used for CPUs (e.g., PCs or servers), DDR memory bank 232 comprises DDR. In some embodiments, when unit 200 is used for mobile and / or low-power devices, DDR memory bank 232 comprises LPDDR.

[0046] In some embodiments, when unit 200 does not include HBM chip 204 and SerDes chip 204 has been integrated within compute chip 202, unit 200 comprises a single chip (e.g., compute chip 202) placed on an interposer (e.g., interpose 108, interposer 110). In some implementations, compute chip 202 interfaces with the PTM by SerDes chip 204 and / or a wide parallel interface. In some embodiments, SerDes chip 206 (e.g., and / or SerDes chip 224) connects to a compute chip (e.g., compute chip 202) through a wide parallel interfaceand SerDes chip 206 then connects to the PTM (e.g., effectively connecting the compute chip to the PTM). In some implementations, compute chip 202 and the PTM connect through a wide parallel bus BOW (e.g., bunch of wires). In some embodiments, the wide parallel bus BOW comprises an ultra-low-power, high-bandwidth die-to-die (D2D) interface designed for efficient communication between chiplets. In some implementations, the wide parallel bus BOW is on compute chip 202 and / or SerDes chip 206.

[0047] In some embodiments, unit 200 comprises a first microelectronic device, substrate 212 comprises a first substrate, interposer 208 comprises a first interposer (e.g., that is placed directly on a first portion of substrate 212), compute chip 202 comprises a first chip (e.g., that is placed directly on a first portion of interposer 208), SerDes chip 206 comprises a second chip (e.g., that is placed directly on a second portion of interposer 208). In some implementations, SerDes chip 206 is a logic chip that is adjacent to compute chip 202 and electronically coupled to compute chip 202 via one or more connections. In some embodiments, SerDes chip 206 is configured to access HBM memory bank 216 and / or DDR memory bank 232 (e.g., memory devices that are not located on substrate 212).

[0048] FIG. 3 depicts an illustrative example of a general purpose Al processing unit (e.g., GP-AIU), in accordance with some embodiments of this disclosure. In some embodiments, unit 300 (e.g., unit 100) comprises compute tiles 302a-302c (e.g., compute tiles 102a-102h), HBM chips 304a-3041 (e.g., HBM chips 104a-104h), and memory extenders 306a-3061 (e.g., memory extenders 106a-106d). In some implementations, unit 300 connects, via connection 308 (e.g., connection 116), to memory bank 310 (e.g., DDR5 memory bank 118). In some embodiments, memory bank 310 comprises memory extender 312 and DDR5 chips 314a- 314e. In some implementations, unit 300 has a length of 60 mm and a width of 71 mm. In some embodiments, compute tiles 302a-302c have a length of 18 mm and a width of 23 mm. In some embodiments, HBM chips 304a-3041 have a 36 GB storage capacity or any suitable storage capacity. In some implementations, HBM chips 304a-3041 have a 1.2 TB / s maximum data transfer bandwidth or any suitable maximum data transfer bandwidth. In some embodiments, unit 300 (e.g., by memory extenders 306a-3061) supports a 57 Tbs / s (e.g., TB / s) transmission rate (e.g., or any other suitable transmission rate), uses 3 picojoules (pJ) per bit (b) processed optically (e.g., or any other suitable energy consumption rate), and user 6 pJ / b processed by a SerDes chip (e.g., SerDes chip 206) (e.g., or any other suitable energy consumption rate). In some implementations, DDR5 chips 314a-314e have a 128 GB storagecapacity or any suitable storage capacity. In some implementations, DDR5 chips 314a-314e have a 64 GB / s maximum data transfer bandwidth or any suitable maximum data transfer bandwidth.

[0049] FIG. 4 depicts an illustrative example of a general purpose Al processing unit (e.g., GP-AIU), in accordance with some embodiments of this disclosure. In some embodiments, unit 400 (e.g., unit 100) comprises compute tiles 402a-402d (e.g., compute tiles 102a-102h), HBM chips 404a-4041 (e.g., HBM chips 104a-104h), and memory extenders 406a and 406b (e.g., memory extenders 106a-106d). In some embodiments, compute tiles 402a-402d have a length of 23 mm and a width of 17 mm (e.g., or any other suitable dimensions). In some embodiments, the portion unit 400 comprising compute tiles 402a-402d and HBM chips 404a-4041 has a length of 71 mm and a width of 71 mm (e.g., due a maximum size of 73 mm by 73 mm as imposed by an interposer or substrate that the portion of unit 400 is placed on). In some implementations, memory extenders 406a and 406b enable 4 TB / s optical links to be established. In some embodiments, unit 400 comprises a 6x reticle size super carrier interposer (e.g., CoWoS). In some implementations, unit 400 has 2346 mm2of total compute power, 648 GB (e.g., 0.276 GB / mm2) of VRAM, and 29.6 TB / s of bandwidth (e.g., 21.6 TB / s + 8 TB / s from memory extender 406a and 406b).

[0050] FIG. 5 depicts an illustrative example of a general purpose Al processing unit (e.g., GP-AIU), in accordance with some embodiments of this disclosure. In some embodiments, unit 500 (e.g., unit 100) comprises compute tiles 502a-502d (e.g., compute tiles 102a-102h), HBM chips 504a-5041 (e.g., HBM chips 104a-104h), and memory extender 506 (e.g., memory extenders 106a-106d). In some implementations, unit 500 has a length of 64 mm and a width of 64 mm (e.g., is a square). In some embodiments, compute tiles 502a-502d have a length of 13 mm and a width of 26 mm (e.g., or any other suitable dimensions). In some implementations, memory extender 506 has a length of 12 mm and a width of 12 mm (e.g., is a square).

[0051] FIG. 6 depicts an illustrative example of a general purpose Al processing unit (e.g., GP-AIU), in accordance with some embodiments of this disclosure. In some embodiments, unit 600 (e.g., unit 100) comprises compute tiles 602a-602h (e.g., compute tiles 102a-102h), HBM chips 604a-604h (e.g., HBM chips 104a-104h), and memory extender 606a-606d (e.g., memory extenders 106a-106d). In some implementations, unit 600 has a length of 47 mm and a length of 55 mm (e.g., or any other suitable dimensions). In some implementations,compute tiles 602a-602h have a length of 11 mm and a width of 15 mm (e.g., or any other suitable dimensions). In some embodiments, memory extenders 606a-606d enable 1 TB / s links to be established. In some implementations, unit 600 has 1320 mm2of total compute, 288 GB (e.g., 0.218 GB / mm2) of total VRAM, and 9.6 TB / s of total bandwidth.

[0052] FIG. 7 depicts an illustrative example of a general purpose Al processing unit (e.g., GP-AIU), in accordance with some embodiments of this disclosure. In some embodiments, unit 700 (e.g., unit 100) comprises compute tiles 702a-702jj (e.g., compute tiles 102a-102h), each compute tile (e.g., compute tile 702b) comprising HBM connector 704 (e.g., that connects unit 700 to DDR5 memory bank 718), and DDR connector 706a and DDR connector 706b (e.g., which connect unit 700 to memory bank 710). In some implementations, unit 700 connects, via connection 708 (e.g., connection 108) to memory bank 710 (e.g., memory bank 110). In some embodiments, memory bank 710 comprises memory extender 712 (e.g., memory extender 112) and one or more HBM chips 714 (e.g., HBM chips 114). In some implementations, unit 700 connects, via connection 716 (e.g., connection 116), to DDR5 memory bank 718 (e.g., DDR5 memory bank 118). In some embodiments, each compute tile is connected to each neighboring compute tile (e.g., compute tile 702b is connected to compute tile 702c) by a 10 TB / s UCIe connection. In some implementations, each compute tile (e.g., compute tile 702b) has a length of 16 mm and a width of 16 mm (e.g., or any other suitable dimensions). In some embodiments, unit 700 has a length of 101 mm and a width of 101 mm (e.g., is a square). In some implementations, HBM connector 704 enables a 2.2 TB / s connection to be made (e.g., with one or more HBM chips 714 via memory extender 712). In some embodiments, DDR connector 706a and DDR connector 706b each enable a 1.1 TB / s connection to be made (e.g., with DDR5 memory bank 718). In some implementations, unit 700 has 9216 mm2of total compute, 5.2 TB of total HBM, 72 TB of total DDR, 79.2 TB / s of total HBM bandwidth, and 39.6 TB / s of total DDR bandwidth (e.g., or any other suitable values). In some embodiments, unit 700 supports 15.3 PFLOPS (e.g., Peta Floating Point Operations Per Second), 22.5 PFLOPS, 477 TeraFLOPS (TFLOPS), and / or 704 TFLOPS (e.g., or any other suitable value).

[0053] FIG. 8 depicts a side view of a general purpose Al processing unit (e.g., GP-AIU), in accordance with some embodiments of this disclosure. In some embodiments, unit 800 (e.g., unit 200) comprises substrate 802 (e.g., substrate 212), interposer 804 (e.g., interposer208, interposer 210), HBM chip 808 (e.g., HBM chip 204), compute chip 810 (e.g., compute chip 202), and SerDes chip 812 (e.g., SerDes chip 206).

[0054] FIG. 9 depicts a side view of a general purpose Al processing unit (e.g., GP-AIU), in accordance with some embodiments of this disclosure. In some embodiments, unit 900 (e.g., unit 200) comprises substrate 902 (e.g., substrate 212), interposer 904 (e.g., interposer 208), interposer 906 (e.g., interposer 210), HBM chip 908 (e.g., HBM chip 204), and compute chip 910 (e.g., compute chip 202). In some implementations, a SerDes chip (e.g., SerDes chip 812, SerDes chip 206) is embedded in compute chip 910 or otherwise integrated with compute chip 910. In some embodiments, HBM chip 908 is placed on a first portion of interposer 904. In some implementations, compute chip 910 is placed on interposer 906 and a portion of interposer 904. In some embodiments, interposer 904 and interposer 906 are placed on substrate 902. In some embodiments, HBM chip 908 and compute chip 910 are connected by a shoreline UCIe connection.

[0055] FIG. 10 depicts a side view of a general purpose Al processing unit (e.g., GP-AIU), in accordance with some embodiments of this disclosure. In some embodiments, unit 1000 (e.g., unit 200) comprises substrate 1002 (e.g., substrate 212), interposer 1004 (e.g., interposer 208, interposer 210), HBM chip 1008 (e.g., HBM chip 204), and compute chip 1010 (e.g., compute chip 202). In some implementations, a SerDes chip (e.g., SerDes chip 812, SerDes chip 206) is embedded in compute chip 1010 or otherwise integrated with compute chip 910. In some embodiments, an optical interposer (e.g., interposer 210, interposer 906) is embedded or otherwise integrated with interposer 1004. In some embodiments, HBM chip 1008 is placed on a first portion of interposer 1004 and compute chip 1010 is placed on a second portion of interposer 1004. In some embodiments, interposer 1004 is placed on substrate 1002. In some embodiments, HBM chip 1008 and compute chip 1010 are connected by a shoreline UCIe connection.

[0056] FIG. 11 depicts a side view of a general purpose Al processing unit (e.g., GP-AIU), in accordance with some embodiments of this disclosure. In some embodiments, unit 1100 (e.g., unit 200) comprises substrate 1102 (e.g., substrate 212), interposer 1104 (e.g., interposer 208), interposer 1106 (e.g., interposer 210), compute chip 1110 (e.g., compute chip 202), and SerDes chip 1112 (e.g., SerDes chip 206). In some implementations, a HBM chip (e.g., HBM chip 808, HBM chip 204) is embedded in compute chip 1110 or otherwise integrated with compute chip 1110. In some embodiments, compute chip 1110 is placed oninterposer 1104 and SerDes chip 1112 is placed on interposer 1106. In some embodiments, interposer 1104 and interposer 1106 are placed on substrate 1102. In some embodiments, compute chip 1110 and SerDes chip 1112 are connected by a shoreline UCIe connection.

[0057] FIG. 12 depicts a side view of a general purpose Al processing unit (e.g., GP-AIU), in accordance with some embodiments of this disclosure. In some embodiments, unit 1200 (e.g., unit 200) comprises substrate 1202 (e.g., substrate 212), interposer 1204 (e.g., interposer 208), compute chip 1210 (e.g., compute chip 202), and SerDes chip 1212 (e.g., SerDes chip 206). In some implementations, a HBM chip (e.g., HBM chip 808, HBM chip 204) is embedded in compute chip 1210 or otherwise integrated with compute chip 1210. In some embodiments, an optical interposer (e.g., interposer 210, interposer 1106) is embedded or otherwise integrated with interposer 1204. In some embodiments, compute chip 1210 and SerDes chip 1212 are placed on interposer 1204. In some embodiments, interposer 1204 is placed on substrate 1202. In some embodiments, compute chip 1210 and SerDes chip 1212 are connected by a shoreline UCIe connection.

Claims

What is Claimed is:

1. A system comprising: a first microelectronic device, wherein the first microelectronic device comprises: a first substrate; a first interposer placed directly on a first portion of the first substrate; a first chip placed directly on a first portion of the first interposer; and a second chip placed directly on a second portion of the first interposer, wherein: the second chip is a logic chip; the second chip is located adjacent to the first chip; the second chip is electronically coupled to the first chip using one or more first connections disposed within the first interposer; and the second chip is configured to access at least two memory devices that are not located on the first substrate.

2. The system of claim 1, wherein the first chip is a first memory chip.

3. The system of claim 2, wherein the first microelectronic device further comprises: a second interposer placed directly on a second portion of the first substrate, wherein: the second interposer comprises one or more first optical connections and a first one or more supporting components; the second interposer is located adjacent to the first interposer; and the second interposer is optically coupled to at least two memory devices via the one or more first optical connections, wherein the at least two memory devices are not located on the first substrate; and a third chip placed directly on at least a portion of the second interposer, wherein: the third chip is located adjacent to the second chip; the third chip is electronically coupled to the second chip and the second interposer using one or more second connections not disposed within the first interposer;the second chip is configured to access the at least two memory devices via the second interposer, the third chip, and the one or more first optical connections.

4. The system of claim 3, further comprising: a second microelectronic device not located on the first substrate, wherein the second microelectronic device comprises: a second substrate; a third interposer placed directly on a first portion of the second substrate; a fourth chip placed directly on a first portion of the third interposer, wherein the fourth chip is a second memory chip; a fifth chip placed directly on a second portion of the third interposer, wherein: the fifth chip is located adjacent to the fourth chip; and the fifth chip is electronically coupled to the fourth chip using one or more third connections disposed within the third interposer; a fourth interposer placed directly on a second portion of the second substrate, wherein: the fourth interposer comprises one or more second optical connections and a second one or more supporting components; the fourth interposer is located adjacent to the third interposer; and the fourth interposer is optically coupled to the second interposer via the one or more second optical connections; and a sixth chip placed directly on a portion of the fourth interposer, wherein: the sixth chip is located adjacent to the fifth chip; and the sixth chip is electronically coupled to the fifth chip and the fourth interposer using one or more fourth connections not disposed within the third interposer.

5. The system of claim 4, wherein: the third chip comprises a first optical input and a first optical output; the third chip deserializes information received from the first optical input; and the third chip serializes information received from the second chip and outputs the serialized information via the first optical output.

6. The system of claim 5, wherein: the sixth chip comprises a second optical input and a second optical output; the sixth chip deserializes information received from the second optical input; and the sixth chip serializes information received from the fifth chip and outputs the serialized information via the second optical output.

7. The system of claim 6, further comprising: a third microelectronic device not located on the first substrate, wherein the third microelectronic device comprises: a fifth interposer; and a seventh chip on a portion of the fifth interposer, wherein; the seventh chip comprises a third optical input and a third optical output; the seventh chip is electronically coupled to the fifth interposer; and the seventh chip deserializes information received from the third optical input.

8. The system of claim 7, wherein the third microelectronic device is a double data rate (DDR) memory device.

9. The system of claim 3, wherein: the first substrate is a rectangle comprising a first edge, a second edge, a third edge, and a fourth edge; the first edge of the first substrate is parallel to the third edge of the first substrate; the first edge of the first substrate is perpendicular to the second edge of the first substrate and the fourth edge of the first substrate; a first plurality of memory chips are aligned along the first edge of the first substrate; a second plurality of memory chips are aligned along the third edge of the first substrate; a plurality of logic chips are placed directly on the first interposer between the first plurality of memory chips and the second plurality of memory chips; and the second interposer is placed directly on the first substrate along the second edge of the first substrate.

10. The system of claim 3, wherein: the first chip has a first thickness; the second chip has a second thickness; the second interposer has a third thickness; the third chip has a fourth thickness; and the first thickness, is the same or similar to the second thickness and the fourth thickness.

11. The system of claim 10, wherein the third thickness is the same or similar to a thickness of the first interposer.

12. The system of claim 3, wherein the first chip, the second chip, and the third chip are located within a first horizontal plane.

13. The system of claim 12, wherein: the first interposer and the second interposer are located within a second horizontal plane; and the first horizontal plane is located above the second horizontal plane.

14. The system of claim 1, wherein the first microelectronic device is a GPU.

15. The system of claim 14, further comprising a second microelectronic device not located on the first substrate, wherein the second microelectronic device is a second GPU.

16. The system of claim 1, wherein the first one or more supporting components comprise at least one electrical to optical converter.

17. The system of claim 1, wherein the first one or more supporting components comprise at least one optical to electrical converter.

18. The system of claim 1, wherein the first one or more supporting components comprise:at least one electrical to optical converter; and at least one optical to electrical converter.

19. A system comprising: a first microelectronic device, wherein the first microelectronic device comprises: a first substrate; a first interposer placed directly on a first portion of the first substrate; a first chip placed directly on the first interpose; a second interposer placed directly on a second portion of the first substrate, wherein: the second interposer is located adjacent to the first interposer; and the second interposer is coupled to at least two memory devices via one or more connections, wherein the at least two memory devices are not located on the first substrate; and a second chip placed directly on at least a portion of the second interposer, wherein: the second chip is electronically coupled to the first chip and the second interposer; and the first chip is configured to access the at least two memory devices via the second interposer, the second chip, and the one or more connections.

20. The system of claim 19, wherein the first chip is a logic chip.

21. The system of claim 20, wherein the first chip comprises static random-access memory.

22. The system of claim 19, wherein the one or more connections are optical connections.

23. The system of claim 19, wherein the one or more connections are copper connections.

24. A system comprising: a first microelectronic device, wherein the first microelectronic device comprises: a first substrate; a first interposer placed directly on a first portion of the first substrate;a first chip placed directly on a first portion of the first interpose; and a second chip placed directly on a second portion of the first interposer, wherein: the second chip is electronically coupled to the first chip; and the first chip is configured to access at least two memory devices via the the second chip, and one or more connections.

25. The system of claim 24, wherein: the first chip is a logic chip; and the second chip is serializer deserializer chip.

26. The system of claim 25, wherein the first chip comprises static random-access memory.

27. The system of claim 25, wherein the one or more connections are optical connections.

28. The system of claim 25, wherein the one or more connections are copper connections.

Images