Aggregation of multiple memory modules for system on chip

By configuring multiple memory modules to share interfaces and high bandwidth data transmission of intermediary substrates in the on-chip system, the problem of increasing the size of the SoC die is solved, and the memory capacity and bandwidth are improved and cost reduction is achieved.

CN119065590APending Publication Date: 2024-12-03AVAGO TECHNOLOGIES INTERNATIONAL SALES PTE LTD
View PDF 4 Cites 0 Cited by

Patent Information

Application Number
CN202410553793.3
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Priority Date
2023-10-04
Filing Date
2024-05-07
Publication Date
2024-12-03

AI Technical Summary

Technical Problem

When existing system-on-chip (SoC) adds multiple high-bandwidth memory modules, larger peripheral edges are required to accommodate the increase in memory interfaces, resulting in an increase in the size of the SoC die, which in turn increases costs.

Method used

By configuring multiple memory modules on the substrate, using a shared memory interface to enable multiple memory modules to communicate with the processor, reducing the need for the size of the SoC chip, and realizing high-bandwidth data transmission through the interposer substrate, aggregating the data paths to Supports connection of more memory modules.

Benefits of technology

This achieves the compact size of the SoC die while increasing memory capacity and bandwidth, reducing costs and improving the high bandwidth communication efficiency of the memory.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119065590A_ABST
    Figure CN119065590A_ABST
Patent Text Reader

Abstract

The invention relates to aggregation of multiple memory modules for a system on a chip. A system includes a substrate including a first circuit. The system also includes an integrated circuit formed in a first die disposed on the substrate. The integrated circuit includes at least a processor, a controller, and a first memory interface. The first memory interface is positioned in a first edge of the first die and configured to couple to the first circuit. The system also includes a first buffer circuit formed in a second die disposed on the interposer substrate adjacent to the first edge of the first die. The first buffer circuit includes a second memory interface configured to be coupled to the first connection circuit. The system further includes a plurality of memory modules disposed on the second die. Each of the plurality of memory modules at least partially shares the second memory interface to communicate with the integrated circuit.
Need to check novelty before this filing date? Find Prior Art

Description

[0001] CROSS REFERENCE TO RELATED APPLICATIONS

[0002] This application claims priority to U.S. Provisional Application No. 63 / 505,928, filed on June 2, 2023, which is commonly assigned and incorporated herein by reference for all purposes. Technical Field

[0003] The present invention relates to semiconductor integrated circuits and manufacturing methods. Background Art

[0004] A system on a chip (SoC) is an integrated circuit that combines various electronic components and functionality onto a single chip. It is designed to provide a complete system solution in a compact package commonly used in electronic devices such as smartphones, tablet computers, IoT devices, and embedded systems. SoC technology has revolutionized the semiconductor industry by enabling highly integrated and efficient systems with reduced power consumption. A variety of specific function systems can be built using SoC dies coupled with multiple discrete high-performance memory (HPM) dies. The HPM die is typically connected to the SoC via a memory interface through an interposer or package.

[0005] Typically, each memory interface in a SoC involves many data paths connected to the SoC die. These data paths facilitate data transfer between the SoC and the memory, including read and write operations. In addition, the memory interface includes address lines that link specific bits to specified memory cells, and control signals for configuring the memory modules and interacting with the memory modules. Allocating a specific area on the SoC die, usually located at one of the peripheral edges, is necessary to establish a connection with each memory die and form a memory interface. However, incorporating multiple high-bandwidth memory (HBM) modules requires a larger periphery of the SoC die to accommodate the increasing number of memory interfaces. This results in a larger SoC die size, which translates into higher costs. Therefore, there is a need for an enhanced system and method for coupling a SOC to multiple discrete high-bandwidth memory modules while minimizing the impact on size and cost. Summary of the invention

[0006] In one aspect, the present disclosure relates to a system comprising: a substrate comprising a first circuit; a second circuit formed in a first die disposed on the substrate, the second circuit comprising at least a processor and a first memory interface, the first memory interface disposed in a first edge of the first die and configured to couple the processor with the first circuit; a third circuit formed in a second die disposed on the substrate adjacent to the first edge of the first die, the third circuit comprising a second memory interface configured to couple to the first circuit; and a plurality of memory modules disposed on the second die, each of the plurality of memory modules at least partially sharing the second memory interface to communicate with the processor via the first circuit and the first memory interface.

[0007] On the other hand, the present disclosure relates to an integrated circuit, comprising: a substrate comprising a first circuit; a second circuit comprising at least a first memory interface, the second circuit being configured as a first die disposed on the substrate, wherein the first memory interface is configured to couple to the first circuit; a third circuit comprising a second memory interface, the third circuit being configured as a second die disposed on the substrate adjacent to the first die, wherein the second memory interface is configured to couple to the first circuit; and a plurality of memory circuits disposed on the second die, each of the plurality of memory circuits at least partially sharing the second memory interface to communicate with the second circuit via the first circuit and the first memory interface.

[0008] On the other hand, the present disclosure relates to a method of coupling a system-on-chip circuit to a separate memory circuit, the method comprising: providing a substrate comprising a first circuit; providing a second circuit comprising a first memory interface; placing the second circuit on the substrate, the first memory interface being configured to couple to the first circuit; providing at least a third circuit comprising a second memory interface, the second memory interface comprising a first number of data paths; placing the third circuit on the substrate, the third circuit being positioned adjacent to the second circuit, the second memory interface being configured to couple to the first circuit; placing two or more memory circuits on the third circuit laterally side by side or vertically above and below each other, each memory circuit comprising a second number of data paths; and coupling the second number of data paths for each memory circuit to the first number of data paths in the second memory interface to allow each memory circuit to communicate with the second circuit, the first number of data paths being less than the sum of the second number of data paths. BRIEF DESCRIPTION OF THE DRAWINGS

[0009] A further understanding of the nature and advantages of particular embodiments may be achieved by reference to the remainder of the specification and drawings, wherein the same reference numerals are used to refer to similar components. In some instances, a sub-label is associated with a reference numeral to represent one of multiple similar components. When a reference numeral is cited without specifying an existing sub-label, it is intended to refer to all such multiple similar components.

[0010] Figure 1 is a schematic diagram illustrating both a top view and a cross-sectional view of a system-on-chip (SOC) die coupled with two high bandwidth memory (HBM) dies in a conventional configuration.

[0011] Figure 2 is a schematic diagram illustrating the size correlation between the SOC die and each HBM die in a conventional configuration.

[0012] Figure 3 is a schematic diagram illustrating both a top view and a cross-sectional view of an integrated circuit including a SOC die coupled with four HBM dies in accordance with an embodiment of the present technology.

[0013] Figure 4 is a schematic diagram illustrating a top view of a SOC die coupled with two HBM dies adjacent a first peripheral edge and four HBM dies adjacent a second peripheral edge in accordance with an embodiment of the present technology.

[0014] Figure 5 is a schematic diagram illustrating both a top view and a cross-sectional view of a SOC die coupled with two HBM dies adjacent a first peripheral edge and four HBM dies adjacent a second peripheral edge in accordance with another embodiment of the present technology. DETAILED DESCRIPTION

[0015] The present technology relates to semiconductor integrated circuits and methods for manufacturing the same. Specifically, it introduces an integrated circuit comprising a system-on-chip (SOC) die connected to a first memory interface via a connection circuit positioned on an interposer substrate. On the same interposer substrate, there is a buffer die that accommodates a second memory interface. Multiple memory modules are positioned on the buffer die, and these memory modules utilize a shared second memory interface to establish communication with the system-on-chip. This arrangement allows the system-on-chip to maintain a compact size, thereby achieving cost savings, while increasing its memory capacity by incorporating additional memory modules with high bandwidth. It is important to note that there are other alternative embodiments as well.

[0016] The following description is presented to enable one of ordinary skill in the art to make and use the invention and incorporate it into the context of a particular application. Various modifications and various uses in different applications will be readily apparent to one of ordinary skill in the art, and the general principles defined herein may be applied to a wide range of embodiments. Therefore, the present technology is not intended to be limited to the embodiments presented, but should be accorded the widest scope consistent with the principles and novel features disclosed herein.

[0017] In the following detailed description, a number of specific details are set forth in order to provide a more thorough understanding of the present technology. However, it will be appreciated by those skilled in the art that the present technology may be practiced without being limited to these specific details. In other examples, well-known structures and devices are shown in block diagram form rather than in detail to avoid obscuring the present technology.

[0018] The reader's attention is directed to all papers and documents filed concurrently with this specification and disclosed for public inspection with this specification, and the contents of all such papers and documents are incorporated herein by reference. Unless otherwise expressly stated, all features disclosed in this specification (including any attached claims, abstracts and drawings) may be replaced by alternative features for the same, equivalent or similar purposes. Therefore, unless otherwise expressly stated, each feature disclosed is only an example of a series of generally equivalent or similar features.

[0019] Furthermore, any element in a claim that does not explicitly recite “means for performing a specified function” or “step for performing a specified function” should not be construed as a “means” or “step” clause as specified in 35 U.S.C. Section 112, paragraph 6. Specifically, the use of “the step of” or “the action of” in the claims herein is not intended to invoke the provisions of 35 U.S.C. Section 112, paragraph 6.

[0020] When an element is referred to herein as being "connected" or "coupled" to another element, it is understood that the element may be directly connected to the other element, or have intervening elements present between the elements. In contrast, when an element is referred to as being "directly connected" or "directly coupled" to another element, it is understood that there are no intervening elements in the "direct" connection between the elements. However, the presence of a direct connection does not exclude other connections in which intervening elements may be present.

[0021] When an element is referred to herein as being "disposed" in some manner relative to another element (e.g., disposed on, disposed between, disposed below, disposed adjacent to, or disposed in some other relative manner), it is understood that the element may be disposed directly relative to the other element (e.g., disposed directly on the other element) or have intervening elements present between the elements. In contrast, when an element is referred to as being "directly disposed" relative to another element, it is understood that in the "direct" instance there are no intervening elements. However, the presence of a direct disposition does not exclude other instances in which there may be intervening elements.

[0022] Similarly, when an element is referred to herein as being "engaged" to another element, it is understood that the element may be directly joined to the other element (without any intervening elements), or have intervening elements present between the joined elements. In contrast, when an element is referred to as being "directly joined" to another element, it is understood that there are no intervening elements in the "direct" joining between the elements. However, the presence of direct joining does not exclude other forms of joining in which intervening elements may be present.

[0023] Likewise, when an element is referred to herein as a "layer," it is understood that the layer may be a single layer or may include multiple layers. For example, a conductive layer may include multiple different conductive materials or multiple layers of different conductive materials, and a dielectric layer may include multiple dielectric materials or multiple layers of dielectric materials. When a layer is described as being coupled or connected to another layer, it is understood that the coupled or connected layer may include intervening elements that exist between the coupled or connected layers. In contrast, when a layer is referred to as being "directly" connected or coupled to another layer, it is understood that there are no intervening elements between the layers. However, the presence of a directly coupled or connected layer does not exclude other connections in which intervening elements may exist.

[0024] In addition, the terms left, right, front, back, top, bottom, forward, reverse, clockwise, and counterclockwise are used for explanation purposes only and are not limited to any fixed direction or orientation. Instead, they are only used to indicate the relative position and / or direction between the various parts of an object and / or component.

[0025] In addition, for ease of description, the methods and processes described herein may be described in a particular order. However, it should be understood that unless the context otherwise dictates, intervening processes may occur before and / or after any portion of the process, and that various other procedures may be reordered, added, and / or omitted according to various embodiments.

[0026] Unless otherwise indicated, all numbers used herein to express quantities, dimensions, and the like are to be understood as being modified in all instances by the term "about". In this application, unless specifically stated otherwise, the use of the singular includes the plural, and the use of the terms "and" and "or" means "and / or" unless otherwise indicated. Furthermore, the terms "include" and "have" and other forms such as "includes / included" and "has / have / had" are to be considered non-exclusive. Furthermore, terms such as "element" or "component" encompass both elements and components comprising one unit and elements and components comprising more than one unit, unless specifically stated otherwise.

[0027] As used herein, the phrase "at least one of..." preceding a series of items, with the terms "and" or "or" used to separate any item, modifies the list as a whole, rather than modifying each member of the list (i.e., each item). The phrase "at least one of..." does not require selection of at least one of each listed item; rather, the phrase allows for a meaning that includes at least one of any item and / or at least one of any combination of items. For example, the phrase "at least one of A, B, and C" or "at least one of A, B, or C" each refers to only A, only B, or only C; and / or any combination of A, B, and C. In instances where selection of "at least one of each of A, B, and C" or, alternatively, "at least one of A, at least one of B, and at least one of C" is desired, it is explicitly described as such.

[0028] One general aspect includes a system that relates to a system on a chip coupled with a plurality of discrete memory modules. The system also includes a substrate that may include a first circuit. The system also includes a second circuit formed in a first die disposed on the substrate, the second circuit may include at least a processor and a first memory interface disposed in a first edge of the first die and configured to couple the processor with the first circuit. The system also includes a third circuit formed in a second die disposed on the substrate adjacent to the first edge of the first die, the third circuit may include a second memory interface configured to couple to the first circuit. The system also includes a plurality of memory modules disposed on the second die, each of the plurality of memory modules at least partially sharing the second memory interface to communicate with the processor via the first circuit and the first memory interface. The substrate is an interposer substrate, which is an electrical interface that redirects or "rewires" connections from one standard form factor to another standard form factor. In the context of semiconductor and microelectronic packaging, an interposer is typically a thin slice of silicon (or other material) having circuitry that connects the chip above it to the circuit below it. For example, when used with high bandwidth memory (HBM) technology, the interposer acts as a bridge between the HBM stack and the processor (such as a GPU). The interposer contains fine-grained wiring, which facilitates the dense connections required for the HBM stack and processor communication.

[0029] The implementation may include one or more of the following features. The second circuit may include a central processing unit (CPU). The second circuit may include a graphics processing unit (GPU). The second circuit may include an artificial intelligence (AI) accelerator. The first circuit may include a connection interposer and a connection trace. The second circuit formed in the first die may include a system on chip. A plurality of memory modules may include two or more memory dies disposed on the second die side by side and / or vertically above and below each other. The second circuit may include a third memory interface disposed in the second edge of the first die. The system may include a fourth circuit formed in a third die, the fourth circuit may include a fourth memory interface, the third die is disposed on a substrate adjacent to the second edge of the first die, and the substrate may include a fifth circuit configured to couple the fourth memory interface with the third memory interface. The system may include a plurality of memory modules disposed on the third die, the plurality of memory modules at least partially sharing a fourth memory interface to communicate with the second circuit via the fifth circuit and the third memory interface. The first memory interface is configured to support a bandwidth of 128Gb / s or higher. The second memory interface may include a number of data paths for communicating data, addresses, and control signals, wherein the number of data paths is less than the sum of data paths for the plurality of memory modules and may include a subset of the data paths for communicating in common with the plurality of memory modules.

[0030] Another general aspect includes an integrated circuit. The integrated circuit also includes a substrate, which may include a first circuit. The circuit also includes a second circuit, which may include at least a first memory interface, the second circuit is configured as a first die disposed on the substrate, wherein the first memory interface is configured to couple to the first circuit. The circuit also includes a third circuit, the third circuit includes a second memory interface, the third circuit is configured as a second die disposed on the substrate adjacent to the first die, wherein the second memory interface is configured to couple to the first circuit. The circuit also includes a plurality of memory circuits, the plurality of memory circuits are disposed on the second die, each of the plurality of memory circuits at least partially sharing the second memory interface to communicate with the second circuit via the first circuit and the first memory interface. In an embodiment, the first die is configured as a system on chip (SOC) chip. A SOC chip generally refers to an integrated circuit (IC) that integrates most components of a computer or other electronic system into a single chip. It combines various discrete components such as processors and dedicated hardware accelerators into a unified architecture for the purpose of delivering a more optimized and space-saving solution. In an embodiment, the second die is configured as a buffer chip. A buffer chip is an electronic component designed to manage, modify or amplify the voltage level of a signal passing through it. The main purpose of a buffer is to serve as an interface between two circuits with different electrical characteristics, thereby ensuring signal integrity and preserving the properties of the original signal. Buffer chips can be used as standalone integrated circuits (ICs), or as part of a larger IC, where they perform their role along with other functionality.

[0031] The embodiment may include one or more of the following features. The integrated circuit in which the first die is configured as a system on chip may include at least one integrated circuit of a CPU, a GPU, an artificial intelligence accelerator, a computing system, a networking system, and an autonomous control system. A plurality of memory circuits may include two or more memory modules disposed on the second die side by side and / or vertically above and below each other. Each of the plurality of memory circuits may include a high bandwidth memory (HBM) module capable of supporting a bandwidth of 128Gb / s or higher. The second circuit may include a third memory interface. The integrated circuit may include a fourth circuit, the fourth circuit may include a fourth memory interface, the fourth circuit is configured to be disposed adjacent to the first die on a substrate, and the substrate may include a fifth circuit configured to couple the fourth memory interface to the third memory interface. The integrated circuit may include a plurality of memory circuits disposed on a third die, each of the plurality of memory circuits at least partially sharing a fourth memory interface to communicate with the second circuit via the fifth circuit and the third memory interface.

[0032] Yet another general aspect includes a method of coupling a system-on-chip circuit to a separate memory circuit. The method also includes providing a substrate that may include a first circuit. The method also includes providing a second circuit that may include a first memory interface. The method also includes disposing the second circuit on the substrate, the first memory interface being configured to couple to the first circuit. The method also includes providing at least a third circuit that may include a second memory interface, the second memory interface may include a first number of data paths. The method also includes disposing the third circuit on the substrate, the third circuit being positioned adjacent to the second circuit, the second memory interface being configured to couple to the first circuit. The method also includes disposing two or more memory circuits on the third circuit laterally side by side or vertically above and below each other, each memory circuit may include a second number of data paths. The method also includes coupling the second number of data paths for each memory circuit to the first number of data paths in the second memory interface to allow each memory circuit to communicate with the second circuit, the first number of data paths being less than the sum of the second number of data paths. Each of the first number of data paths is configured to have a higher data rate than each of the second number of data paths in each memory circuit.

[0033] When an element is referred to herein as a "die," it is understood that it refers to an individual silicon wafer containing an integrated circuit or a specific functional unit of one or more integrated circuits. During the semiconductor manufacturing process, multiple copies of the same integrated circuit or functional unit are manufactured on a single silicon wafer. Each of these individual copies is referred to as a die. For example, a die is composed of various electronic components (such as transistors, resistors, capacitors, and interconnects) etched or deposited onto a silicon substrate. For another example, a die may contain a single functional unit formed on a single silicon wafer, such as a processor, controller, memory, and interface. For another example, a die may contain a system of multiple functional units all integrated on a single silicon wafer, such as a system-on-chip die.

[0034] A system-on-chip (SOC) die is an integrated circuit that combines various electronic components and functionality onto a single chip. Integrated circuits are designed to provide complete system solutions in compact packages commonly used in electronic devices such as smartphones, tablet computers, IoT devices, and embedded systems. A SOC die includes several components that may vary depending on specific applications and requirements. Typically, it includes a processor, such as a central processing unit (CPU), that is responsible for executing instructions and performing calculations. It may consist of one or more processor cores, each of which is capable of independently executing instructions. In another example, a processor may be provided as a graphics processing unit (GPU) that is responsible for handling graphics calculations and accelerating graphics-intensive tasks. Basically, a SOC die may also include an input / output (I / O) interface to enable communication with external devices and peripherals. This includes interfaces such as USB, HDMI, Ethernet, Wi-Fi, Bluetooth, and various sensor interfaces. These interfaces facilitate data transfer, connectivity, and interaction with the external environment. In general, a SOC die may also incorporate various types of memory to store data and instructions. The SOC die may also include analog / digital converters (ADCs) and digital / analog converters (DACs) to convert analog signals to digital and vice versa. This enables the SOC to interface with analog sensors and actuators, allowing it to interact with the physical world. In addition, the SOC die may include an internal bus architecture that enables communication and data transfer between various components within the chip. These buses facilitate efficient and high-speed data exchange while managing the flow of information between different subsystems.

[0035] In addition to the components integrated within the SOC itself, the SOC design can also be coupled to multiple discrete memory chips to meet the memory requirements of the system. These memory chips are generally used to expand memory capacity and provide additional storage options. In particular, high-bandwidth memory (HBM) modules are used to meet the growing demand for higher memory performance in SOCs that are configured as modern computing systems such as graphics processing units (GPUs), high-performance computing (HPC) systems, and advanced AI accelerators. HBM is a high-speed computer memory interface for 3D stacked (one vertically on top of another) dynamic random access memory (DRAM), which achieves high bandwidth in a much smaller form factor than DDR memory solutions such as DDR (double data rate) 4 or GDDR5, while consuming less power. HBM memory stacks use TSV interconnects. These are vertical electrical connections that pass completely through a silicon wafer or die. The HBM stack is placed on a substrate adjacent to the processor (e.g., a GPU on a SOC die). They are interconnected via an interposer, which is a silicon layer with wiring that connects the processor to the HBM stack. This proximity reduces the data transfer distance, thereby further reducing latency and power consumption. Key features and benefits of HBM include increased bandwidth, allowing parallel data access and reduced memory latency, while power consumption is reduced due to shorter data paths and lower signal capacitance. HBM is designed to be scalable, allowing memory capacity and bandwidth to be increased by adding more memory stacks to the design. This scalability benefits applications with high memory requirements, such as GPU computing and deep learning.

[0036] The interface between the SOC and the HBM module is a key element to achieve efficient communication and data transfer between these components. The specific interface used may depend on the implementation scheme and technology involved. A common interface for connecting the SOC die and the HBM die is a high-bandwidth memory interface, which provides a high-speed and high-bandwidth data path for data transfer between the SOC die and the HBM die, and provides a separate command and address bus for transmitting control signals and memory addresses from the SOC to the HBM. The SOC die may include a memory controller to manage communication and data transfer between the SOC and multiple HBM modules.

[0037] Figure 11 is a schematic diagram illustrating both a top view and a cross-sectional view of a SOC die coupled with two HBM dies in a conventional configuration. Typically, the SOC die 110 is disposed on a substrate 101, and two HBM dies 121 and 122 are also disposed on the substrate 101. The SOC die 110 includes a first memory interface (HBM I / F) 1101 and a second memory interface 1102. Both memory interfaces are disposed at the first edge of the periphery of the SOC die 110. For the two HBM dies 121 and 122, each also includes an HBM interface 1211 and 1221. The two HBM dies 121 and 122 are positioned adjacent to the first peripheral edge of the SOC die 110, so that the HBM interface 1211 has a short physical distance from the HBM interface 1101, and the HBM interface 1221 also has a short physical distance from the HBM interface 1102. In an embodiment, substrate 101 is an interposer substrate including one or more connection circuits containing a medium for rewiring or redirecting electrical connections. The traces or paths on the interposer substrate are much finer than those that may be on a typical printed circuit board. Interposer substrates generally use Si as a common material because it is compatible with semiconductor processes, and other materials such as organic or glass substrates can be used. A pair of HBM interfaces 1101 and 1211 are coupled via a first connection circuit 1011 built inside the interposer substrate 101 (see Figure 1 Another pair of HBM interfaces 1102 and 1221 are also connected through another connection circuit ( Figure 1 In this example, two discrete HBM interfaces need to be formed in the SOC die to couple the two HBM interfaces respectively for data communication between the SOC die and the two discrete HBM dies. Accordingly, the two discrete HBM dies 121 and 122, each having a limited size, need to be arranged side by side adjacent to the first peripheral edge to maintain a short physical distance between the two corresponding HBM interfaces to facilitate high-speed high-bandwidth data transmission.

[0038] Figure 2 is a schematic diagram illustrating the size correlation between the SOC die and each HBM die in a conventional configuration. Figure 1 Only two HBM dies coupled with the SOC die are shown adjacent to one peripheral edge. Figure 2 Such an example is shown where the number of HBM dies is increased from 2 to 4, and the second peripheral edge of the SOC die 210 is used to form two additional HBM interfaces 2103 and 2104 for respectively coupling two HBM interfaces 2231 and 2241 of two additional HBM dies 223 and 224. This is essentially the same configuration of coupling two HBM dies 221 and 222 to the SOC die 210 via two pairs of HBM interfaces, e.g., 2101-2211 and 2102-2221, respectively.

[0039] It is often necessary to increase the number of HBM dies to increase system memory capacity, memory bandwidth, or both. Due to the limited size Y1 of each HBM module and associated HBM interface, and the requirement to form a matching HBM interface on the SOC die to connect to the corresponding HBM interface on each HBM module, adding HBMs requires the (first or second) peripheral edge size Y2 of the SOC die to grow as a function of the HBM module size Y1 and the number of HBMs (i.e., Y2=f(Y1)). For example, if the number of HBM dies increases from 4 to 6, then the required peripheral edge size Y2 must increase to accommodate the increase based on Figure 2 The same configuration shown in FIG. 1 is coupled to at least a third HBM die at each of the first and second peripheral edges of the SOC die. A limitation of current approaches is that increasing the size of the SOC die increases costs.

[0040] Figure 3 is a schematic diagram illustrating both a top view and a cross-sectional view of an integrated circuit including a SOC die coupled with four HBM dies according to an embodiment of the present technology. This figure is merely an example, which should not unduly limit the scope of the claims. One of ordinary skill in the art will recognize many variations, alternatives, and modifications. In an embodiment, in addition to a first die 310 provided as an integrated circuit on a substrate 301 as a SOC die (e.g., an integrated circuit on a chip), a second die 321 (e.g., a circuit) is provided as a buffer chip and is also disposed on the substrate 301 adjacent to the first peripheral edge of the first die 310. In an embodiment, the substrate 301 is an interposer substrate. A buffer chip generally refers to a chip containing all conductive traces or data lines for forming an interface circuit system, which acts as a buffer or intermediate connection between two main circuits. The buffer chip is also used as a base to support other circuits disposed (side by side) or stacked (one on top of the other) on top of itself. As Figure 3 , the first die 310 includes a first memory interface 3101 formed in a first peripheral edge of the first die. The second die 321 includes a second memory interface 3301 formed along a peripheral edge of the second die and adjacent to a first peripheral edge of the first die. The interposer substrate 301 includes a first connection circuit 3011 embedded therein (see Figure 3). The first connection circuit 3011 is configured to couple to the first memory interface 3101 and the second memory interface 3301 in the first die 310. Optionally, the first connection circuit 3011 includes a first through silicon via (TSV) group (not shown) to establish a vertical connection between a first surface contact group (not shown) on the second die 321 and a conductive trace bundle embedded in the interposer substrate 301. The first connection circuit 3011 also includes a second TSV group to connect between a second surface contact group on the first die 310 and a conductive trace bundle embedded in the interposer substrate to establish a matching coupling between the second memory interface 3301 and the first memory interface 3101.

[0041] In addition, if Figure 3 As shown in the cross-sectional view of , multiple memory modules can be disposed on the second die 321. Figure 3 In the embodiment, the plurality of memory modules include two high bandwidth memory (HBM) modules 331 and 332 disposed side by side laterally on a second die 321. Each of the two HBM modules (also referred to as HBM dies) is configured to at least partially share a second memory interface 3301 in the second die 321 for data / signal communication with the integrated circuit in the first die 310. In an embodiment, the second memory interface 3301 is configured to include a first number of data paths for passing data, addresses, and control signals. Figure 1 and Figure 2 Unlike the HBM die shown in FIG. 3 , each HBM die 331 or 332 does not need to include its own memory interface. Alternatively, the buffer circuit in the second die 321 is designed to include all surface contacts required for coupling each HBM die and all data paths required to directly connect those surface contacts to the second memory interface 3301. In some embodiments, each HBM module requires a second number of data paths for coupling to the second memory interface to transmit data, address information, and control signals. The first number of data paths in the buffer circuit is aggregated to be less than the sum of the second number of data paths of the multiple memory modules on the second die, and each data path in the first number of data paths supports a higher bandwidth. In some embodiments, the first number of data paths includes a single set of data paths for supporting the communication of common address information and control signals of multiple memory modules. The aggregation or merging of the data paths in the second memory interface will achieve a reduced size and eliminate the need for multiple independent memory interfaces for multiple HBM modules.

[0042] Therefore, in the first die 310, the first memory interface 3101 can be redesigned to have a higher bandwidth and a smaller number of data paths to match the second memory interface 3301. Therefore, according to the floorplanning of the integrated circuits in the first die 310, the aggregation of data paths of multiple HBM dies on the second die 321, and the data rate, the size or aspect ratio of the first memory interface 3101 can be optimized to X3 and Y3 (see Figure 3 ). For example, the first memory interface 3101 is configured to support 128GB / s or higher bandwidth, which can be scaled based on the number of multiple memory modules on the second die 321. The (X3, Y3) optimization of the first memory interface 3101 on the first die 310 will decouple the size Y2 (Y2>Y3) of the SOC die (at the first peripheral edge along which the first memory interface 3101 is located) from the size Y1 of each HBM die. In other words, the size of the SOC die can be independent of the number of HBM dies (placed adjacent to the first peripheral edge of the SOC die). Therefore, the SOC die can be kept in a relatively compact size based on the optimization of the layout plan of the integrated circuit and the associated first memory interface.

[0043] Optionally, as more HBM modules are added for coupling with the first (SOC) die 310, the integrated circuit in the first die 310 may include a third memory interface 3102 formed along a second peripheral edge of the first die 310. Figure 3 , a third die 322 is provided as a buffer circuit, which is also disposed on the interposer substrate 301 adjacent to the second peripheral edge of the first die 310. The third die 322 includes a fourth memory interface 3401 formed along the peripheral edge of the third die adjacent to the second peripheral edge of the first die. The interposer substrate 301 also includes a second connection circuit 3012 (see Figure 3 ). The second connection circuit 3012 is configured to couple to the third memory interface 3102 and the fourth memory interface 3401 in the first die 310. Based on the third die 322, more HBM modules 341 and 342 can be disposed on the third die. Figure 3Two added HBM dies 341 and 342 are shown to be disposed side by side laterally on the third die 322. Each added HBM die does not have its own independent memory interface, but is coupled to the fourth memory interface 3401 in the third die 322 to establish data / signal communication with the integrated circuit in the first die 310. The fourth memory interface 3401 includes a data path for transmitting an aggregate number of data, addresses, and control signals, wherein the number of aggregated data paths is less than the sum of the data paths of multiple memory modules, and each of the aggregated number of data paths supports a higher bandwidth. The aggregated number of data paths includes a single set of data paths for common addresses and control signals of multiple memory modules. The SOC die 310 can still be kept in a relatively compact size to reduce its cost based on the optimization of the layout planning of the integrated circuit and the associated first memory interface and the third memory interface.

[0044] Figure 4 is a schematic diagram illustrating a top view of a SOC die coupled with two HBM dies adjacent to a first peripheral edge and four HBM dies adjacent to a second peripheral edge in accordance with an embodiment of the present technology. This figure is merely an example, which should not unduly limit the scope of the claims. Those of ordinary skill in the art will recognize many variations, alternatives, and modifications. Figure 4 Display, based on Figure 3 The method for coupling a SOC die to a plurality of discrete memory modules proposed in the invention can add an unlimited number of multiple memory modules via buffer circuits to couple with the SOC die from at least one peripheral edge using an aggregate memory interface. Figure 4 In a specific example, the SOC die 410 includes a first memory interface 4101 formed along a first peripheral edge thereof. A second die 421 is provided as a first buffer circuit, which includes a second memory interface 4211 disposed adjacent to the first peripheral edge of the first die 410. Both the first die (SOC die) 410 and the second die 421 are disposed on the interposer substrate 401. Two HBM modules 431 and 432 are added side by side on the second die 421. The two HBM modules are able to communicate with the SOC die using the second memory interface 4211, which is coupled to the first memory interface via a connection circuit in the interposer substrate.

[0045] like Figure 44, the same SOC die 410 also includes a third memory interface 4102 formed along the second peripheral edge. A third die 422 is provided as a second buffer circuit, which includes a fourth memory interface 4221 disposed adjacent to the second peripheral edge of the first die 410. The third die 422 is disposed on the interposer substrate, wherein the fourth memory interface 4221 is adjacent to the third memory interface 4102 in the first die. Four HBM modules 441, 442, 443, and 444 are added, all of which are laterally arranged side by side on the third die 422 and all use the aggregated fourth memory interface 4221 to communicate with the SOC die through another connection circuit inside the interposer substrate 401. Additional HBM modules can be added to the second die or the third die in a similar configuration, provided that the memory interface there is appropriately scaled in terms of the aggregated number of data paths and the upgraded bandwidth. In some embodiments, the SOC die may include a memory controller that manages communication and data transfer between the SOC integrated circuit and the HBM memory module. The memory controller translates the memory request of the SOC into appropriate commands and addresses for the HBM die. For example, four HBM modules coupled to a SOC die can provide a bandwidth range of approximately 128 GB / s to 160 GB / s. In another example, four second-generation HBM modules coupled to a SOC die in a similar configuration can provide a bandwidth range of approximately 256 to 320 GB / s.

[0046] Figure 5 is a schematic diagram illustrating both a top view and a cross-sectional view of a SOC die coupled with two HBM dies adjacent a first peripheral edge and four HBM dies adjacent a second peripheral edge in accordance with another embodiment of the present technology. Figure 5 Provides a method for coupling a SOC die with multiple memory modules Figure 4 In a particular embodiment, at least some of the plurality of memory modules added on one of the buffer dies have been vertically stacked on each other. The plurality of memory modules include at least two or more memory dies disposed side by side laterally on the buffer die and / or two or more memory dies stacked vertically on the buffer die.

[0047] In some embodiments, HBM utilizes a stacked memory architecture in which multiple memory dies are stacked vertically on top of each other. In some embodiments, the HBM interface facilitates communication with these stacked dies, thereby enabling simultaneous access to large memory capacities while minimizing the physical footprint. The HBM interface can support multiple channels or data paths, thereby allowing parallel data transfer and increasing throughput. For example, the memory interface may have up to 1024 or 2048 data I / O pins, respectively. In some embodiments, the HBM interface may be integrated with a memory controller positioned in a SOC die, which is responsible for managing data transfer between the processor of the SOC die and the HBM die. The memory controller handles tasks such as issuing read and write commands, managing data flows, and handling error correction. In some embodiments, the HBM interface may be designed to support scalability, thereby allowing the integration of multiple HBM stacks or higher capacity HBM dies in future implementations. This scalability enables systems made of SOC dies coupled to multiple HBM dies to meet the growing demand for memory-intensive applications.

[0048] In some embodiments, Figure 5 An integrated circuit is provided that includes an interposer substrate (501), the interposer substrate (501) including a first connection circuit (5011). The integrated circuit also includes a system-on-chip circuit (510) including at least a first memory interface (5101). The system-on-chip circuit (510) is disposed on the interposer substrate (501). The first memory interface (5101) is configured to couple to the first connection circuit (5011). In some embodiments, the integrated circuit also includes a first buffer circuit (521), the first buffer circuit (521) including a second memory interface (5211) configured to match the first memory interface (5101). The first buffer circuit (521) is also disposed on the interposer substrate (501) at a position adjacent to the system-on-chip circuit (510). Specifically, the second memory interface (5211) formed on an edge portion of the first buffer circuit (521) is adjacent to the first memory interface (5101) formed on a first peripheral edge of the system-on-chip circuit (510). The second memory interface (5211) is configured to be coupled to the first connection circuit (5011) in the interposer substrate (501). The integrated circuit further includes a plurality of memory circuits (e.g., HBM die 531 and HBM die 532) disposed on the first buffer circuit (521). Optionally, the HBM die 531 and the HBM die 532 are disposed laterally side by side on the first buffer circuit (521). Each of the plurality of memory circuits at least partially shares the second memory interface (5211) to communicate with the on-chip system circuit (510) via the first connection circuit (5011) and the first memory interface (5101).

[0049] In an embodiment, the system-on-chip circuit (510) includes a central processing unit (CPU). In another embodiment, the system-on-chip circuit (510) includes a graphics processing unit (GPU). In yet another embodiment, the system-on-chip circuit (510) includes an artificial intelligence (AI) accelerator. In yet another embodiment, the system-on-chip circuit (510) includes at least one of a high-performance computing system, a high-bandwidth networking system, an autonomous driving system, a machine learning system, and a data-intensive computing system.

[0050] In some embodiments, Figure 5 As shown in FIG. 5 , the system-on-chip circuit ( 510 ) further includes a third memory interface ( 5102 ) disposed on a second peripheral edge of the system-on-chip circuit. Figure 5 The integrated circuit provided in the embodiment may also include a second buffer circuit (522), the second buffer circuit (522) including a fourth memory interface (5221) formed on another edge portion of the second buffer circuit. The second buffer circuit (522) is also disposed on the interposer substrate (501) adjacent to the system-on-chip circuit (510). The interposer substrate (501) also includes a second connection circuit (5012) configured to couple the fourth memory interface (5221) to the third memory interface (5102). The integrated circuit may also include a plurality of memory circuits (e.g., HBM die 541, HBM die 542, HBM die 543, and HBM die 544) disposed on the second buffer circuit (522). Some of the plurality of memory circuits (e.g., HBM die 541 and HBM die 542) are disposed laterally side by side on the second buffer circuit (522). Some of the plurality of memory circuits (e.g., HBM die 541 and HBM die 543) are vertically stacked on the second buffer circuit 522. Each of the plurality of memory circuits at least partially shares the fourth memory interface 5221 to communicate with the on-chip system circuit 510 via the second connection circuit 5012 and the third memory interface 5102.

[0051] In some embodiments, Figure 5 A method of coupling a system-on-chip circuit to a separate memory circuit is also provided. The method includes the step of providing a substrate (501) including a first circuit (5011). Optionally, the substrate is an interposer substrate. The first circuit is an electrically connected circuit comprising a plurality of conductive traces embedded in the substrate coupled to at least two sets of surface contacts disposed at two discrete locations via one or more interposers, such as through silicon vias (TSVs). The method also includes the step of providing a second circuit (510) including a first memory interface (5101). Optionally, the second circuit is configured as a system-on-chip. The first memory interface (5101) is formed along a first edge of the system-on-chip (510).

[0052] The method further comprises the step of disposing a second circuit (510) on the substrate (501). The first memory interface (5101) is configured to be coupled to the first circuit (5011). Optionally, the second circuit 510 is a system on a chip configured to be disposed at a location on the interposer substrate (501) such that the first memory interface (5101) is coupled to a first set of surface contacts on the interposer substrate (501) to connect with the first circuit (5011).

[0053] The method further comprises the step of providing at least a third circuit (521) comprising a second memory interface (5211). Optionally, the third circuit is configured as a buffer chip, i.e., the chip contains all the conductive traces or data lines for supporting the second memory interface (5211) used as a buffer or an intermediate connection between two main circuits. The buffer chip is also a base for supporting other circuits arranged (side by side) or stacked (one on top of the other) on top of itself. Optionally, the second memory interface (5211) is formed along an edge portion of the buffer chip (521). The method further comprises the step of placing the buffer circuit (521) on the interposer substrate (501). Optionally, the buffer circuit (521) is positioned adjacent to the system-on-chip circuit (510) such that the second memory interface (5211) at the edge portion of the buffer chip (521) is adjacent to the first memory interface (5101) at the first edge of the system-on-chip (510). In addition, the buffer chip (521) is positioned at a position such that the second memory interface is coupled to the second set of surface contacts on the substrate (501) and to the first circuit (5011). Therefore, the second memory interface (5211) is configured to be coupled to the first memory interface (5101) via the first circuit (5011).

[0054] The method further includes the step of placing two or more memory circuits (e.g., HBM die 531 and HBM die 532) on the buffer circuit (521) side by side laterally or vertically above and below each other. Each memory circuit includes a second number of data paths for data transfer and signal communication. The method further includes the step of forming a first number of data paths in the interposer substrate, the first number of data paths aggregating the sum of the first number of data paths for each memory circuit. The first number of data paths is coupled to the second memory interface (5211) for communicating with the on-chip system (510) via the connection circuit (5011) and further through the first memory interface (5101).

[0055] While the above is a complete description of specific embodiments, various modifications, alternative constructions, and equivalents may be used. Therefore, the above descriptions and illustrations should not be taken as limiting the scope of the present technology, which is defined by the appended claims.

Claims

1. A system comprising: a substrate comprising a first circuit; a second circuit formed in a first die disposed on the substrate, the second circuit comprising at least a processor and a first memory interface disposed in a first edge of the first die and configured to couple the processor with the first circuit; a third circuit formed in a second die disposed on the substrate adjacent the first edge of the first die, the third circuit comprising a second memory interface configured to couple to the first circuit; and A plurality of memory modules are disposed on the second die, each of the plurality of memory modules at least partially sharing the second memory interface to communicate with the processor via the first circuit and the first memory interface.

2. The system according to claim 1, wherein the second circuit comprises a central processing unit (CPU). 3 . The system of claim 1 , wherein the second circuit comprises a graphics processing unit (GPU).

4. The system of claim 1, wherein the second circuit comprises an artificial intelligence (AI) accelerator. The system of claim 1 , wherein the first circuit comprises a connection interposer and connection traces.

6. The system of claim 1, wherein the second circuit formed in the first die comprises a system on a chip.

7. The system of claim 1, wherein the plurality of memory modules comprises two or more memory dice disposed laterally side by side and / or vertically above and below each other on the second die.

8. The system of claim 1, wherein the second circuit comprises a third memory interface disposed in a second edge of the first die.

9. The system of claim 8, further comprising a fourth circuit formed in a third die, the fourth circuit comprising a fourth memory interface, the third die being disposed on the substrate adjacent to the second edge of the first die, the substrate comprising a fifth circuit configured to couple the fourth memory interface with the third memory interface.

10. The system of claim 9, further comprising a plurality of memory modules disposed on the third die, the plurality of memory modules at least partially sharing the fourth memory interface to communicate with the second circuit via the fifth circuit and the third memory interface.

11. The system of claim 1, wherein the first memory interface is configured to support a bandwidth of 128 GB / s or higher.

12. The system of claim 1 , wherein the second memory interface comprises a plurality of data paths for communicating data, addresses, and control signals, wherein the number of data paths is less than the sum of data paths for the plurality of memory modules and comprises a subset of the data paths for communicating in common with the plurality of memory modules.

13. An integrated circuit comprising: a substrate comprising a first circuit; a second circuit comprising at least a first memory interface, the second circuit being configured as a first die disposed on the substrate, wherein the first memory interface is configured to couple to the first circuit; a third circuit including a second memory interface, the third circuit configured as a second die disposed on the substrate adjacent to the first die, wherein the second memory interface is configured to couple to the first circuit; and A plurality of memory circuits are disposed on the second die, each of the plurality of memory circuits at least partially sharing the second memory interface to communicate with the second circuit via the first circuit and the first memory interface.

14. The integrated circuit of claim 13, wherein the first die is configured as a system on a chip, the system on a chip comprising at least one integrated circuit of a CPU, a GPU, an artificial intelligence accelerator, a computing system, a networking system, and an autonomous control system.

15. The integrated circuit of claim 13, wherein the plurality of memory circuits comprises two or more memory modules disposed laterally side by side and / or vertically above and below each other on the second die. 16 . The integrated circuit of claim 13 , wherein each of the plurality of memory circuits comprises a high bandwidth memory (HBM) module capable of supporting a bandwidth of 128 GB / s or higher.

17. The integrated circuit of claim 13, wherein the second circuit comprises a third memory interface.

18. The integrated circuit of claim 17, further comprising a fourth circuit comprising a fourth memory interface, the fourth circuit being configured to be disposed adjacent to a third die on the substrate on which the first die is disposed, the substrate comprising a fifth circuit configured to couple the fourth memory interface to the third memory interface.

19. The integrated circuit of claim 18, further comprising a plurality of memory circuits disposed on the third die, each of the plurality of memory circuits at least partially sharing the fourth memory interface to communicate with the second circuit via the fifth circuit and the third memory interface.

20. A method of coupling a system-on-chip circuit to a separate memory circuit, the method comprising: providing a substrate including a first circuit; providing a second circuit including a first memory interface; disposing the second circuit on the substrate, the first memory interface being configured to couple to the first circuit; providing at least a third circuit including a second memory interface, the second memory interface including a first number of data paths; disposing the third circuit on the substrate, the third circuit being positioned adjacent to the second circuit, the second memory interface being configured to couple to the first circuit; disposing two or more memory circuits laterally side by side or vertically above and below each other on the third circuit, each memory circuit comprising a second number of data paths; and The second number of data paths for each memory circuit is coupled to the first number of data paths in the second memory interface to allow each memory circuit to communicate with the second circuit, the first number of data paths being less than a sum of the second number of data paths.

Citation Information

Patent Citations

  • Multiplexed memory device interfaces and methods

    CN116137923A

  • Stacked memory routing techniques

    US20200098730A1

  • Stacked memory device and operating method thereof

    US20210240615A1

  • Multiple channel memory system

    WO2022115167A2