A Hybrid Memory Architecture for Advanced 3D Systems

Abstract

Stacked memory dies utilizing a mix of high-temperature and low-temperature operating memory and nonvolatile-based memory dies, as well as chip packages containing the same, are disclosed. High-temperature memory dies, such as those using nonvolatile memory (NVM) technology, are in a memory stack with low-temperature memory dies, such as those with volatile memory technology. In some cases, the high-temperature memory technology may be used together with logic circuits on the same IC die. In one example, a memory stack is provided that includes a first memory IC die having high-temperature memory circuitry, such as nonvolatile memory, stacked below a second memory IC die. The second memory IC die has high-temperature memory circuitry, such as volatile memory circuitry.

Description

[Technical Field]

[0001] SUMMARY OF THE INVENTION Embodiments of the present invention generally relate to stacked memory dies having volatile and non-volatile based memory dies and chip packages containing the same. [Background technology]

[0002] The memory wall (i.e., bandwidth limitations) has been cited as one of the key limiters in pushing the boundaries of computation in modern systems. High-bandwidth memory (HBM) and other stacked dynamic random-access memory (DRAM) memories have been proposed / enabled to reduce off-chip memory access latency and increase memory density. In addition to the traditional DRAM roadmap, technologies that have not yet reached maturity for large-scale manufacturing, such as ferroelectric random-access memory (FeRAM), magnetoresistive random-access memory (MRAM), and phase-change memory (PCM), are being explored. During this technology enablement phase, it is important not only to examine how new technologies "replace" the classical roadmap, but also to examine whether they can assist / complement / address the limitations of existing DRAM without adding much complexity, or how they can be used with existing technologies to enhance specific characteristics and achieve superior system-on-chip (SoC) performance and power efficiency.

[0003] The memory wall problem is currently being addressed by the industry with solutions such as HBM. Stacked DRAM and HBM, as described in JEDEC Solid State Technology Association (e.g., JEDEC) specifications, address memory bandwidth and latency issues by replacing long off-chip connections with stacked memory (e.g., connected through a silicon interposer) closer to the logic die. However, yield challenges and overhead exist due to the nonlinear power increase with increasing memory capacity. Additionally, 3D stacking over logic introduces new thermal challenges that can adversely affect retention in DRAM. Meanwhile, other non-volatile memory (NVM) technologies, such as logic-compatible FeRAM, have no refresh requirements and can withstand high temperatures, but suffer from scalability / capacity and wear-out issues. Meanwhile, static random-access memory (SRAM) is a faster but leaky memory system. Current solutions do not fully utilize hybrid memory systems as disclosed by the inventors herein to take advantage of the unique characteristics of each memory type, and therefore do not maximize performance / power efficiency potential.

[0004] Non-volatile main memory such as FeRAM, MRAM, and volatile memory such as DRAM (including HBM and other stacked variants of DRAM) are being explored and traded off to achieve higher memory density, performance, and lower power.

[0005] DRAM is the most common off-chip memory, but even Double Data Rate 5 Synchronous DRAM (DDR5) has certain performance-power-area (PPA) limitations that require data to be off-chip for access. A typical DRAM bitcell consists of a one-transistor and one-capacitor (1T-1C) structure, where the capacitor is formed by a dielectric layer sandwiched between conductor plates. System interprocess communication (IPC) is often limited by DRAM bandwidth and latency, especially in memory-intensive workloads. HBM has been introduced to provide increased bandwidth and memory density, allowing up to 8-12 layers of DRAM dies, along with optional logic / memory interface dies, to be stacked on top of each other. This memory stack can either be connected to the CPU / GPU through a silicon interposer (Figure 1) or placed on top of the CPU / GPU itself to provide superior connectivity and performance.

[0006] FeRAM is similar to 1T-1C DRAM, except that the capacitors are made of ferroelectric material rather than the (linear) dielectric used in DRAM. Bits "0" and "1" are written by the electric polarization orientation of the ferroelectric material within the dielectric. The advantage of this technology is refresh-free storage, potentially offering higher density and performance than DRAM.

[0007] On the other hand, MRAM has a one transistor and one register (1T-1R) bit cell. Unlike DRAM and FeRAM, MRAM does not have destructive read. However, MRAM is less reliable than FeRAM, and has lower endurance and retention.

[0008] Typically, memory technologies are developed and "optimized" as independent macros or for specific applications, such as deep neural networks (DNNs) in the case of HBM. However, some advances, such as graphics double data rate (GPDDR) versus double data rate (DDR), have been developed to support high-bandwidth memory for graphics applications. The finer optimization of memory technologies with logic technologies and architectures has not been explored in depth, and much work has been done to achieve superior performance and low-power products. Nonlinear power growth and diminishing improvements in performance and memory density per generation require more design and co-optimization to alleviate memory bottlenecks. Summary of the Invention [Means for solving the problem]

[0009] Stacked memory dies utilizing a mix of high-temperature and low-temperature operating memory and nonvolatile-based memory dies, as well as chip packages containing the same, are disclosed. High-temperature memory dies, such as those using nonvolatile memory (NVM) technology, are in a memory stack with low-temperature memory dies, such as those with volatile memory technology. In some cases, the high-temperature memory technology may be used together with logic circuits on the same IC die. In one example, a memory stack is provided that includes a first memory IC die having high-temperature memory circuitry, such as nonvolatile memory, stacked below a second memory IC die. The second memory IC die has high-temperature memory circuitry, such as volatile memory circuitry.

[0010] In another example, a memory stack is provided that includes a first memory IC die stacked on a second memory IC die. The first memory IC die includes memory circuitry that requires a more frequent refresh rate compared to the second memory IC die. In some other examples, the first memory IC die includes memory circuitry that can operate at temperatures above 110 degrees Celsius without increasing the refresh rate compared to operation at 95 degrees Celsius. The second memory IC die includes memory circuitry that requires an increased refresh rate at temperatures above 110 degrees Celsius compared to operation at 95 degrees Celsius.

[0011] In another example, a memory stack is provided that includes a first memory IC die stacked on a second memory IC die, the first memory IC die including ferroelectric random access memory (FeRAM), and the second memory IC die including dynamic random access memory (DRAM) circuitry.

[0012] In yet another example, a chip package is provided having a memory stack mounted on a package substrate. The memory stack includes a plurality of first memory IC dies stacked on a second memory IC die. The second memory IC die includes ferroelectric random access memory (FeRAM) circuitry and, optionally, controller circuitry. The second memory IC die is stacked on the package substrate. The plurality of first memory IC dies includes DRAM circuitry.

[0013] Also disclosed herein are non-volatile memory (NVM) technologies that can be utilized in memory stacks with volatile memory technologies. In some cases, NVM technologies may be used together, and in some cases, as logic circuits on the same IC die. Leveraging the specific characteristics of each of the technologies in stacked memory subsystems can beneficially result in differentiated SoC performance.

[0014] In one example, a memory stack is provided that includes a first memory IC die having non-volatile memory (NVM) circuitry stacked below a second memory IC die, the second memory IC die having volatile memory circuitry.

[0015] In another example of a memory stack, one IC memory die of the memory stack includes ferroelectric random access memory (FeRAM) or static random access memory (SRAM) circuitry, and another IC memory die of the memory stack includes volatile memory circuitry.

[0016] In another example of a memory stack, one IC memory die of the memory stack includes ferroelectric random access memory (FeRAM) or static random access memory (SRAM) circuitry, while another IC memory die of the memory stack includes dynamic random access memory (DRAM) circuitry.

[0017] In another example of a memory stack, a first processing in memory (PIM) circuit is located on a second memory IC die of the memory stack, while a second PIM circuit is located on a third memory IC die of the memory stack.

[0018] In another example of a memory stack, a first buffer IC die is disposed between one pair of memory IC dies and a second buffer IC die is disposed between another pair of memory IC dies.

[0019] In another example, a memory stack includes a first memory IC die including a first ferroelectric random access memory (FeRAM) circuit and a first processing-in-memory (PIM) circuit, a second memory IC die stacked on the first memory IC die, and a third memory IC die stacked on the first memory IC die, the second memory IC die including a second FeRAM circuit and a second PIM circuit, and the third memory IC die including a third FeRAM circuit and a third PIM circuit.

[0020] In yet another example, a chip package is provided that includes a hybrid memory stack mounted on a substrate, the hybrid memory stack including both volatile and non-volatile memory IC dies. [Brief explanation of the drawings]

[0021] [Figure 1] FIG. 1 illustrates a chip package having a memory stack connected to a compute / processor die through an interposer. [Figure 2] FIG. 1 illustrates an IC die stack arranged above a compute / processor chip and a memory interface / controller die. [Figure 3A] FIG. 1 illustrates a high-temperature memory die stacked on top of a compute / processor chip and a memory interface / controller die. [Figure 3B] FIG. 1 illustrates a high temperature memory, computational / processor circuitry, and memory interface / controller circuitry integrated onto a single integrated circuit (IC) die. [Figure 4A] FIG. 1 illustrates a memory die stack disposed above a compute / processor chip and a memory interface / controller die, the memory die stack including at least one low-temperature memory integrated circuit die and at least one high-temperature memory integrated circuit die. [Figure 4B] FIG. 1 illustrates a memory die stack arranged on a compute / processor chip, the memory die stack including at least one low-temperature memory integrated circuit die and at least one high-temperature memory integrated circuit die, the high-temperature memory IC die including controller circuitry. [Figure 4C] FIG. 1 illustrates a memory die stack disposed above a compute / processor chip and a memory interface / controller die, the memory die stack including at least one low-temperature memory integrated circuit die and at least one high-temperature memory integrated circuit die with a buffer IC die disposed therebetween, the buffer IC including logic circuitry and non-volatile memory circuitry. [Figure 5A] FIG. 1 illustrates a memory die stack disposed above a compute / processor chip and a memory interface / controller die, the memory die stack including multiple low-temperature memory integrated circuit dies, one or more of which have processing-in-memory (PIM) circuitry with FeRAM and / or embedded DRAM-based PIM local storage. [Figure 5B] FIG. 5B illustrates a memory die stack disposed above a compute / processor chip and a memory interface / controller die, the memory die stack including multiple low-temperature memory integrated circuit dies, one or more of which have additional PIM circuitry for FeRAM and / or embedded DRAM-based PIM storage compared to the memory die stack of FIG. 5A. [Figure 5C] FIG. 1 illustrates a memory die stack disposed above a compute / processor chip and a memory interface / controller die, the memory die stack including multiple low-temperature memory integrated circuit dies, one or more of which have fine-grained PIM circuitry with FeRAM and / or embedded DRAM-based PIM local storage. [Figure 5D] FIG. 1 illustrates a memory die stack disposed above a compute / processor chip and a memory interface / controller die, the memory die stack including multiple high-temperature memory integrated circuit dies, one or more of which have fine-grained sub-bank level PIM circuitry with FeRAM-based PIM local storage. DETAILED DESCRIPTION OF THE INVENTION

[0022] The disclosure herein addresses the specific challenges of stacked DRAM subsystems with hybrid memory 3D organization and logic co-design. The disclosed technology defines various methods, systems, and devices for designing advanced memory-based systems that are computationally and application-aware. Generally, memory die stacks are disclosed that utilize a mix of high-temperature and low-temperature operating memory die, such that the high-temperature die can act as a thermal buffer with adjacent heat-generating logic die. Specifically, memory die stacks are disclosed that utilize a mix of volatile and non-volatile based memory die, an example of which is stacked DRAM and FeRAM based memory die.

[0023] FIG. 1 shows a chip package 100 having a memory stack 102 connected to a compute / processor IC die 108 through an interposer 110. Any of the memory stacks described herein may be utilized in the chip package 100 shown in FIG. 1 or other suitable memory devices. The chip package 100 shown in FIG. 1, which may include any of the other memory stacks described below, includes a package substrate 114 on which the interposer 110 is mounted. The package substrate 114 of the chip package 100 may be coupled to a printed circuit board (PCB) 136 to form an electronic system 180, such as, but not limited to, a graphics card shown in FIG. 1.

[0024] The memory stack 102 generally includes at least one low temperature memory integrated circuit (LTMIC) die 104 stacked with at least one high temperature memory IC (HTMIC) die 106. The spaces shown in FIG. 1 between the dies 104 and 106 are used to make solder connections (not shown) between the dies 104 and 106. Alternatively, the dies 104, 106 may be stacked in direct contact with each other using hybrid bonding techniques. The HTMIC die 106 and the LTMIC die 104 may be relatively defined by at least one of the following definitions: In one example, the HTMIC die 106 has a longer memory refresh requirement than another memory die in the memory stack 102, and the memory die with the shorter memory refresh requirement is relatively referred to as the LTMIC die 104. In another example, the HTMIC die 106 has a longer period between refreshes (i.e., a longer refresh period) than recommended by the Joint Electron Device Engineering Council (JEDEC) Solid State Technology Association standard JESD21-C, and a memory die having a memory refresh requirement in accordance with JESD21-C is relatively referred to as an LTMIC die 104. In another example, the HTMIC die 106 has a memory refresh requirement of more than 60 microseconds between memory refreshes, and a memory die requiring a memory refresh every 60 microseconds is relatively referred to as an LTMIC die 104. In yet another example, the HTMIC die 106 is non-volatile memory, while the LTMIC die 104 is volatile memory. In yet another example, the LTMIC die 104 can be defined as a memory die that can operate at a temperature of up to 110 degrees Celsius (i.e., operating temperature) without needing to increase the refresh rate. At temperatures above 110 degrees Celsius, the LTMIC die 104 requires an increased refresh rate compared to operation at 110 degrees Celsius (compared to operation at 95 degrees Celsius).An example of an LTMIC die 104 is a dynamic random access memory (DRAM) die. Other examples of an LTMIC die 104 include volatile memory dies such as system random access memory (SRAM), among others.

[0025] In yet another example, the HTMIC die 106 has a higher operating temperature than the LTMIC die 104. Defined differently, an HTMIC die 106 is a memory die that can operate at temperatures above 110 degrees Celsius without the need to increase its refresh rate (compared to operation at 95 degrees Celsius). An example of an HTMIC die 106 is a ferromagnetic random access memory (FeRAM) die. Other examples of HTMIC die 106 include non-volatile memory dies such as magnetoresistive random access memory (MRAM), phase change memory (PCM), flash memory, and resistive random-access memory (RRAM), among others.

[0026] The memory stack 102 may optionally include at least one controller IC die 120 stacked with the LTMIC die 104 and the HTMIC die 106. The IC dies 104, 106, 120 may be electrically and mechanically connected by solder ball and / or hybrid bonding techniques so that functional circuits within the IC dies 104, 106, 120 can communicate with each other and / or transmit data signals, power and / or ground therethrough.

[0027] The functional circuits within the IC memory die 104, 106 are arranged into multiple memory banks. Each bank has multiple rows, and each row has multiple columns. A memory cell resides at each unique memory location within a bank. A memory cell can be addressed using its unique identifying row and cell location within a particular bank of the memory die 104, 106.

[0028] Functional circuitry within IC dies 104, 106, and 120 is coupled to functional circuitry within compute / processor IC die 108 via routing 112 formed within interposer 110. Routing 112 within interposer 110 connects to functional circuitry within IC dies 104, 106, and 120 via solder connections 118. Routing 112 within interposer 110 also connects the functional circuitry within IC dies 108 and 120 to routing 122 formed within package substrate 114 via solder connections 118. Solder balls 116 are utilized to connect routing 122 within package substrate 114 to routing 124 formed on PCB 136.

[0029] In other embodiments where an interposer is not present, the chip stack 102 and IC die 108 may be mounted directly to the package substrate 114 .

[0030] The IC die 120 is generally a heat-generating device. That is, the IC die 120 generates heat during use. Because the performance of the LTMIC die 104 can be degraded due to the heat generated by the IC die 120, the performance of the chip package 100 is enhanced by separating the LTMIC die 104 from the heat-generating IC die 120 by one or more HTMIC die 106. Because the HTMIC die 106 is generally more heat-tolerant than the LTMIC die 104, the HTMIC die 106 can be located adjacent to the heat-generating IC die 120 without a significant reduction in performance, while the LTMIC die 104 can be significantly separated from the heat-generating IC die 120 and still maintain a robust level of performance.

[0031] 1, the chip stack 102 includes one HTMIC die 106 disposed between multiple LTMIC dies 104 and a controller IC die 120. Although four IC memory dies 104, 106 are shown in the single chip stack 102 depicted in FIG. 1, the number of LTMIC dies 104 and HTMIC dies 106 comprising the chip stack 102 can vary from one to as many as can fit within the chip package 100. Additionally, although only one chip stack 102 is shown in FIG. 1, one or more additional chip stacks can be disposed adjacent to the chip stack 102 depicted in FIG.

[0032] The controller IC die 120 contains functional logic circuitry that provides commands that allow the rows and columns identifying each bank of the memory dies 104, 106 to be addressed. The controller IC die 120 controls write / read operations from each memory bank.

[0033] The HBM memory can be put into a low-power mode by the row address bus to save I / O driver power. To further reduce power consumption, the clock can be gated when in power-down or self-refresh mode.

[0034] 2 shows an IC die stack 202 disposed above the compute / processor IC die 108 and the memory interface / controller IC die 120. The IC die stack 202 may be used in place of the IC die stack 102 in the chip package 100 and electronic system 180 shown in FIG. 1 and may be mounted on the interposer 110 or package substrate 114.

[0035] In the example shown in FIG. 2 , an IC die stack 202 that may be used in an electronic system 180 includes a high-bandwidth memory (HBM) cube 204 stacked on top of a compute IC die 108 and a controller IC die 120. The compute IC die (e.g., compute chip) 108 may be a central processing unit (CPU), a graphics processing unit (GPU), a field programmable gate array (FPGA), or other accelerator. The HBM cube 204 includes multiple LTMIC dies 104 stacked vertically. Each LTMIC die 104 includes functional circuitry configured as a memory circuit 220. In one example, the memory circuit 220 is configured as a volatile memory circuit, such as dynamic random access memory (DRAM) and system random access memory (SRAM), among others. In an example where the memory circuit 220 is configured as a DRAM circuit, the LTMIC die 104 may be referred to as a DRAM IC die. The stacked DRAM IC dies may be connected by solder connections, hybrid bonding, or other suitable connections. Although the HBM cube 204 shown in FIG. 2 is shown with three DRAM IC dies, the HBM cube 204 may alternatively have more or less than three DRAM IC dies.

[0036] One or more HTMIC die 106 are sandwiched between the LTMIC die 104 and the controller IC die 120 within the HBM cube 204. While one HTMIC die 106 is shown in FIG. 2 , more may be utilized. The HTMIC die 106 includes functional circuitry configured as memory circuitry 222. In one example, the memory circuitry 222 is configured as non-volatile random access memory such as ferroelectric random access memory (FeRAM), magnetoresistive random access memory (MRAM), phase change memory (PCM), flash memory, and resistive random access memory (RRAM), among others. In examples where the memory circuitry 222 is configured as FeRAM, the HTMIC die 106 may be referred to as a FeRAM IC die.

[0037] The memory circuit 222 of the HTMIC die 106 has a higher operating temperature than the operating temperature of the memory circuit 220 of the LTMIC die 104. Defined differently, the memory circuit 222 of the HTMIC die 106 is a memory circuit that can operate at temperatures above 110 degrees Celsius without needing to increase its refresh rate (compared to operation at 95 degrees Celsius), while the memory circuit 220 of the LTMIC die 104 is a memory circuit that cannot operate at temperatures above 110 degrees Celsius without needing to increase its refresh rate (compared to operation at 95 degrees Celsius).

[0038] In one example, the HTMIC die 106 is a non-volatile random access memory IC die that has a faster refresh rate compared to the LTMIC die 104, which has volatile random access memory. Thus, in addition to the HTMIC die 106 performing better than the LTMIC die 104 when placed closer to the controller IC die 120, the faster refresh rate allows for faster communication with the controller IC die 120, beneficially reducing latency within the IC die stack 202 and ultimately within the chip package 100 and electronic system 180.

[0039] In the example shown in FIG. 2, the HTMIC die 106 has memory circuitry 222 configured as an FeRAM circuit, while the LTMIC die 104 has memory circuitry 220 configured as a DRAM circuit.

[0040] FIG. 3A illustrates another example of a memory stack 302 including an HTMIC die 106 stacked on top of a compute / processor IC die 108 and a memory interface / controller IC die 120. Although not shown in FIG. 3A, multiple LTMIC dies 104 can be stacked on top of the HTMIC die 106 shown in FIG. 3A in the manner illustrated in FIG. 2 to complete a memory stack 322. The IC die stack 302 can be used in place of the IC die stack 102 in the chip package 100 and electronic system 180 illustrated in FIG. 1 and mounted on the interposer 110 or package substrate 114. In the example illustrated in FIG. 3A, the HTMIC die 106 includes a memory circuit 222 configured as an FeRAM circuit. In the example illustrated in FIG. 3A, the HTMIC die 106 is vertically stacked on top of the memory interface / controller IC die 120, while the memory interface / controller IC die 120 is vertically stacked on top of the compute / processor IC die 108. The memory circuitry 222 of the HTMIC die 106 includes FeRAM or other suitable circuitry that is compatible with the controller circuitry 224 of the memory interface / controller IC die 120. Interconnections between the dies 104, 106, 108, 120 may be made by solder connections, hybrid bonding, or other suitable connections. The HTMIC die 106 stacked on top of the compute / processor IC die 120 forms a hybrid memory-logic assembly that may later be stacked with the LTMIC die 104, as shown in FIG. 2.

[0041] FIG. 3B illustrates another example of a memory stack 322 including a memory circuit 222, a compute / processor circuit 324, and a memory interface / controller circuit 224 integrated on a single HTMIC die 106. While not shown in FIG. 3B, multiple LTMIC dies 104 may be stacked on the HTMIC die 106 shown in FIG. 3B in the manner illustrated in FIG. 2 to complete the memory stack 322. The additional dies stacked on the HTMIC die 106 shown in FIG. 3B may be one or more FeRAM dies, one or more LTMIC dies 104 (such as DRAM IC dies), and / or one or more other types of memory dies. In one example, the memory circuit 222 is configured as an FeRAM circuit that is compatible with the memory interface / controller circuit 224 so that the circuits 222, 224 can coexist within the same HTMIC die 106. Similarly, the FeRAM circuit 222 is also compatible with the compute / processor circuit 324 so that the FeRAM circuits 222, 324, 224 can coexist within the same HTMIC die 106. In one example, the memory circuit 222 is disposed between the compute / processor circuit 324 and the memory interface / controller circuit 224. Optionally, the compute / processor circuit 224 may reside on an IC die adjacent to the HTMIC die 106 that includes both the FeRAM and controller circuits 222, 224.

[0042] Advanced memory technology roadmaps target increased memory density and bandwidth while minimizing the impact on power and performance to alleviate memory bottlenecks to system performance. With advances in memory technology, memory stacking, and novel non-volatile memories like FeRAM, updated circuitry, architecture, and memory interface principles are essential to keep pace with the memory technology itself. Described below are improvements in memory technology that leverage the inherent enhancements of HBM / other forms of stacked high-temperature memory by integrating low-temperature memory technology to create hybrid memory stacks. In one example, stacked DRAM memory can be integrated with FeRAM-based memory to form a hybrid memory stack or hybrid memory-logic assembly.

[0043] For example, hybrid memories and hybrid memory-logic assemblies are disclosed that utilize a mix of memory technologies that can be stacked on top of a logic die. For example, a hybrid memory cube having a non-volatile memory (such as FeRAM) IC die and a volatile memory (such as DRAM IC) IC die has the HTMIC die 106 beneficially located closest to the logic IC die 120 because the HTMIC die 106 can withstand the higher heat dissipated from the logic die, while the LTMIC die 104 located above the HTMIC die 106 can be located closer to a heat spreader in a chip package (such as the chip package 100 shown in FIG. 1 ) to minimize temperature gradients and the impact on performance and refresh rate associated with the LTMIC die 104.

[0044] 4A illustrates an example of a memory die stack 400 arranged above a compute / processor IC die 108 and a memory interface / controller IC die 120. The memory die stack 400 includes at least one HTMIC die 106, such as a non-volatile memory IC die, and multiple LTMIC die 104, such as volatile memory die. Another alternative hybrid approach is to arrange the HTMIC die 106 and LTMIC die 104 in the same memory die stack 400 ranked based on latency, with the fastest die closer to the memory interface / controller IC die 120. The ranked memory IC die 104, 106 may include SRAM, DRAM, and non-volatile memory (NVM) IC die all arranged by latency in the same memory die stack 400. This results in a hierarchical hardware-managed cache for the stacked memory cubes, e.g., the LTMIC die 104, such as the DRAM IC die in the memory die stack 400.

[0045] FIG. 4B illustrates a memory die stack 410 disposed on top of a compute / processor IC die 108. The memory die stack 410 includes at least one HTMIC die 106, such as a non-volatile memory IC die, and multiple LTMIC die 104, such as volatile memory die. The HTMIC die 106 of the memory die stack 410 includes both a memory circuit 222 and a controller circuit 224. For example, the memory circuit 222 of the HTMIC die 106 may include FeRAM (or other non-volatile memory) circuitry and the controller circuit 224 integrated on the same die. Such an arrangement is illustrated in the memory die stack 410 of FIG. 4B and is made possible by the FeRAM configuration of the memory circuit 222 being compatible with the logic technology of the controller circuit 224.

[0046] Alternatively, the IO / SA logic on each memory die 104 may be separated into a buffer IC die 422 to achieve higher performance and yield, and may also include logic-compatible FeRAM memory blocks. Such an example is shown in FIG. 4C , which shows a memory die stack 420 disposed above the compute / processor IC die 108 and memory interface / controller IC die 120. The memory die stack 420 includes at least one HTMIC die 106, such as a non-volatile memory IC die, and at least one LTMIC die 104, such as a volatile memory die, with a buffer IC die 422 disposed therebetween. The buffer IC die 422 includes both logic 224 and non-volatile memory circuitry 222. The buffer IC die 422 may be hybrid bonded on each side to other dies 104 comprising the HBM cube 204 of the die stack 420.

[0047] Memory stacks and technologies may be selected and designed in a "hierarchical" manner (hardware-managed cache) to use faster memory closer to the logic IC die 120 that does not require refresh to act as a "middle" tier and transfer data to denser, slower memory on the top tier of the LTMIC die 104, away from the logic IC die 120. This may help hide the latency and overhead due to the refresh required for the LTMIC die 104, such as the DRAM IC die above the memory die stack 420.

[0048] When FeRAM or other nonvolatile memory dies are used with multi-bit cell storage (e.g., NAND flash stores multiple bits in one cell), wear out can be a concern because each cell is accessed "n" times (where "n" is the number of bits in a single cell) compared to a single-bit cell scenario. Therefore, DRAM / SRAM and other volatile memories with increased endurance can be used as a "standby" cache or hardware-managed cache for the nonvolatile memory dies. This allows multiple writes (called write leveling) to be combined into a single write to the nonvolatile memory multi-bit cells, which beneficially reduces the number of writes to a single cell in the nonvolatile memory IC die and increases the lifespan of the nonvolatile memory circuitry. Reads can be combined in substantially the same manner.

[0049] 5A-5D illustrate some non-limiting examples of processing-in-memory (PIM) circuits 502 utilized within a hybrid memory assembly, i.e., memory die stack 500. The tradeoff for PIM is typically between increased speed due to computation near the memory versus area overhead / impact on memory density and leakage due to integration of the PIM circuitry. The PIM circuitry 502 includes a processor or other logic circuit integrated with the memory circuits 220 / 222 on a single IC die of the memory die stack 500. The PIM circuitry 502 includes local storage circuitry 506.

[0050] The local storage circuit 506 of the PIM circuit 502 can be FeRAM and / or embedded DRAM (eDRAM). Advantageously, the FeRAM and / or eDRAM-based local storage circuit 506 is generally low-leakage logic-compatible storage compared to local logic-based high-leakage registers. This allows for area scaling of the PIM circuit 502 and reduced leakage compared to traditional PIMs that use register-in-logic-based logic storage. Thus, the amount of processing in memory can be increased within the same IC die area allocated to the PIM circuit 502.

[0051] This can also be used for finer grained PIM (e.g., at the sub-bank level; currently PIM is performed at the bank level) or to increase memory density due to reduced PIM area.

[0052] 5A, a memory die stack 500 is shown disposed above a compute / processor IC die 108 and a memory interface / controller IC die 120. The memory die stack 500 includes at least one HTMIC die 106, such as a non-volatile memory IC die, and multiple LTMIC die 104, such as volatile memory die. One or more of the LTMIC die 104 includes processing-in-memory (PIM) circuitry 502 having FeRAM and / or embedded DRAM-based PIM local storage circuitry 506.

[0053] 5B, a memory die stack 510 is shown disposed above the compute / processor IC die 108 and the memory interface / controller IC die 120. The memory die stack 510 includes at least one HTMIC die 106, such as a non-volatile memory IC die, and multiple LTMIC die 104, such as volatile memory die. One or more of the LTMIC die 104 has additional PIM circuitry 502 relative to FeRAM and / or embedded DRAM-based PIM local storage circuitry 506 compared to the memory die stack 500 of FIG. 5A.

[0054] 5C, a memory die stack 520 is shown disposed above the compute / processor IC die 108 and the memory interface / controller IC die 120. The memory die stack 520 includes at least one HTMIC die 106, such as a non-volatile memory IC die, and multiple LTMIC die 104, such as volatile memory die. One or more of the LTMIC die 104 includes fine-grained (PIM) circuitry 502 with FeRAM and / or embedded DRAM-based PIM local storage circuitry 506.

[0055] In FIG. 5D , a memory die stack 530 is shown disposed above the compute / processor IC die 108 and the memory interface / controller IC die 120. The memory die stack 530 includes at least one HTMIC die 106, such as a non-volatile memory IC die, and multiple LTMIC die 104, such as volatile memory die. One or more of the LTMIC die 104 includes a fine-grained sub-bank-level PIM circuit 502 with a FeRAM-based PIM local storage circuit 506. In an alternative hybrid approach, the multiple LTMIC die 104 includes a DRAM IC memory die stacked on top of one or more SRAM IC memory die. The SRAM IC memory die can be configured to function as a hardware-managed cache for the DRAM IC memory die or as part of an address space providing very low latency. In the example shown in FIG. 5D , the HTMIC die 106 includes a FeRAM memory array, which allows for easier integration of the PIM circuit 502 into the logic-compatible FeRAM circuitry of the HTMIC die 106. All of the above examples enable increased SoC performance and power efficiency.

Claims

1. A memory stack comprising: a first memory IC die including a memory; a second memory IC die stacked on the first memory IC die, the second memory IC die including memory circuitry requiring a refresh rate more frequent than the refresh rate of the first memory IC die; Memory stack.

2. the memory circuit of the first memory IC die is a non-volatile memory circuit; The memory stack of claim 1.

3. the non-volatile memory circuit is a ferroelectric random access memory (FeRAM) or a static random access memory (SRAM) circuit; The memory stack of claim 2.

4. the memory circuit of the second memory IC die is a volatile memory circuit; The memory stack of claim 3.

5. the volatile memory circuit is a dynamic random access memory (DRAM) circuit; The memory stack of claim 4.

6. a controller die stacked below the first memory IC die and in contact with the first memory IC die; a processor die stacked below and in contact with the controller die, the processor die including processor circuitry that communicates with memory circuitry of the first memory IC die and the second memory IC die via controller circuitry of the controller die; The memory stack of claim 1.

7. the first memory IC die includes a controller circuit; The memory stack of claim 1.

8. a processor die stacked below and in contact with the first memory IC die, the processor die including processor circuitry that communicates with the first memory IC die and memory circuitry of the second memory IC die via controller circuitry of the first memory IC die; The memory stack of claim 7.

9. a third memory IC die stacked on the second memory IC die, the third memory IC die including dynamic random access memory (DRAM) circuitry; The memory stack of claim 1.

10. the third memory IC die has a greater latency than the second memory IC die, and the second memory IC die has a greater latency than the first memory IC die; The memory stack of claim 9.

11. a first processing-in-memory (PIM) circuit disposed on the second memory IC die; a second PIM circuit disposed on the third memory IC die; The memory stack of claim 9.

12. the first memory IC die includes a controller circuit; The memory stack of claim 1.

13. a processor die stacked below and in contact with the first memory IC die, the processor die including processor circuitry that communicates with the first memory IC die and memory circuitry of the second memory IC die via controller circuitry of the first memory IC die; The memory stack of claim 12.

14. a first buffer IC die disposed between the first memory IC die and the second memory IC die; The memory stack of claim 1.

15. A chip package comprising: A package substrate; a memory stack stacked on the package substrate; the memory stack comprises a plurality of first memory IC dies stacked on a second memory IC die; the second memory IC die having ferroelectric random access memory (FeRAM) circuitry and optionally controller circuitry; the second memory IC die is stacked on the package substrate; the plurality of first memory IC dies include DRAM circuits; Chip package.

Images