Hybrid memory architecture for advanced 3D systems

Abstract

Disclosed herein are stacked memory dies that utilize a mixture of high and low operating temperature memories and non-volatile based memory dies, and chip packages including the same. High temperature memory dies, such as those using non-volatile memory (NVM) technology, are located in a memory stack having low temperature memory dies, such as those having volatile memory technology. In some cases, high temperature memory techniques may be used together, in some cases, on the same IC die as the logic circuitry. In one example, a memory stack is provided that includes a first memory IC die stacked under a second memory IC die, the first memory IC die having a high temperature memory circuit, such as a non-volatile memory. The second memory IC die has a high temperature memory circuit, such as a volatile memory circuit.

Description

Technical Field

[0001] Embodiments of the present invention generally relate to stacked memory dies having volatile and nonvolatile based memory dies and chip packages including the same. Background Art

[0002] The memory wall (i.e., bandwidth limitation) is considered one of the key limiting factors in pushing the limits of computing in modern systems. High-bandwidth memory (HBM) and other stacked dynamic random access memory (DRAM) memories have been proposed / implemented to alleviate off-chip memory access latency and increase memory density. In addition to the traditional DRAM roadmap, several other memories are being explored that are not yet mature for large-scale manufacturing, such as technologies such as ferroelectric random access memory (FeRAM), magnetoresistive random access memory (MRAM), phase change memory (PCM), etc. At this technology implementation stage, it is critical to not only examine how the new technology can "replace" the classic roadmap, but also whether the new technology can help / complement / solve the limitations of existing DRAM without adding too much complexity, or how the new technology can be used with existing technologies to enhance specific properties to achieve excellent system-on-chip (SoC) performance and power efficiency.

[0003] The industry is currently addressing the memory wall problem with solutions like HBM. Stacked DRAM and HBM, described in JEDEC Solid State Technology Association (e.g., JEDEC) specifications, address memory bandwidth and latency issues by replacing long off-chip connections with stacked (e.g., connected through a silicon interposer) memory closer to the logic die. However, there are yield challenges and overhead because power increases nonlinearly with memory capacity. In addition, 3D stacking on logic introduces new thermal challenges that may negatively impact retention in DRAM. On the other hand, other non-volatile memory (NVM) technologies, such as logic-compatible FeRAM, have no refresh requirements and can tolerate high temperatures; but have issues with scalability / large capacity and loss, while static random access memory (SRAM) is a faster but leaky memory system. Current solutions do not fully utilize the hybrid memory system disclosed by the inventors herein to exploit the unique properties of each memory type; and therefore fail to maximize performance / power efficiency potential.

[0004] To achieve higher memory density, performance, and lower power, non-volatile main memory such as FeRAM, MRAM, and volatile memory such as DRAM (including HBM and other stacked variants of DRAM) are being considered and weighed.

[0005] DRAM has become the most popular off-chip memory, however, even double data rate 5 synchronous DRAM (DDR5) has certain performance-power-area (PPA) limitations, namely that data must be accessed off-chip. A typical DRAM bit cell consists of a one transistor and one capacitor (1T-1C) structure, where the capacitor is formed by a dielectric layer sandwiched between conductor plates. System inter-process communication (IPC) is often limited by DRAM bandwidth and latency, especially in memory-intensive workloads. HBM was introduced to provide increased bandwidth and storage density, allowing up to 8-12 layers of DRAM dies to be stacked on top of each other with the aid of optional logic / memory interface dies. This memory stack can be connected to the CPU / GPU (CPU / GPU) through a silicon interposer. Figure 1 ), can also be placed on top of the CPU / GPU itself to provide excellent connectivity and performance.

[0006] FeRAM is similar to 1T-1C DRAM, except that the capacitors are made of ferroelectric materials instead of the (linear) dielectrics used in DRAM. Bits "0" and "1" are written by the electrical polarization orientation of the ferroelectric material in the dielectric. The benefit of this technology is refresh-free storage, which has the potential to provide higher density and performance than DRAM.

[0007] On the other hand, MRAM has a one transistor and one resistor (1T-1R) bit cell. Unlike DRAM and FeRAM, MRAM has no destructive reads. However, MRAM is less reliable and has lower endurance and retention than FeRAM.

[0008] Typically, memory technologies are developed and “optimized” as a standalone macro or for specific applications (such as deep neural networks (DNNs) in the case of HBM). Although some advances such as Graphics Double Data Rate (GPDDR) and Double Data Rate (DDR) have been developed to support high bandwidth memory for graphics applications. However, finer grained optimization of memory technologies with logic technology and architecture has not been explored in depth, and there is still much work to be done to achieve excellent performance and lower power products. The non-linear power increase and generational reduction of performance and memory density improvements require more design and co-optimization to drive relief of memory bottlenecks. Summary of the invention

[0009] Disclosed herein are stacked memory dies utilizing a mix of high and low operating temperature memory and non-volatile based memory dies and chip packages including the same. High temperature memory dies, such as those using non-volatile memory (NVM) technology, are located in a memory stack with low temperature memory dies, such as those with volatile memory technology. In some cases, the high temperature memory technologies may be used together, in some cases, on the same IC die as logic circuits. In one example, a memory stack is provided, the memory stack comprising a first memory IC die stacked below a second memory IC die, the first memory IC die having a high temperature memory circuit, such as a non-volatile memory. The second memory IC die has a high temperature memory circuit, such as a volatile memory circuit.

[0010] In another example, a memory stack is provided that includes a first memory IC die stacked on a second memory IC die. The first memory IC die includes memory circuits that require a more frequent refresh rate than the second memory IC die. In some other examples, the first memory IC die includes memory circuits that are operable at temperatures greater than 110°C with no increase in refresh rate compared to operation at 95°C. The second memory IC die includes memory circuits that require an increased refresh rate at temperatures greater than 110°C compared to operation at 95°C.

[0011] In another example, a memory stack is provided, the memory stack including a first memory IC die stacked on a second memory IC die. The first memory IC die includes a ferroelectric random access memory (FeRAM). The second memory IC die includes a dynamic random access memory (DRAM) circuit.

[0012] In yet another example, a chip package is provided having a memory stack mounted to a package substrate. The memory stack includes a plurality of first memory IC dies stacked on a second memory IC die. The second memory IC die includes a ferroelectric random access memory (FeRAM) circuit and an optional controller circuit. The second memory IC die is stacked on the package substrate. The plurality of first memory IC dies include DRAM circuits.

[0013] Also disclosed herein are non-volatile memory (NVM) technologies that can be utilized in memory stacks with volatile memory technologies. In some cases, NVM technologies can be used together, in some cases, on the same IC die as logic circuits. In stacked memory subsystems, utilizing the specific properties of each of these technologies can beneficially result in different SoC performance.

[0014] In one example, a memory stack is provided that includes a first memory IC die stacked below a second memory IC die, the first memory IC die having non-volatile memory (NVM) circuitry. The second memory IC die has volatile memory circuitry.

[0015] In another example of a memory stack, one IC memory die of the memory stack includes ferroelectric random access memory (FeRAM) or static random access memory (SRAM) circuits and another IC memory die of the memory stack includes volatile memory circuits.

[0016] In another example of a memory stack, one IC memory die of the memory stack includes ferroelectric random access memory (FeRAM) or static random access memory (SRAM) circuits, while another IC memory die of the memory stack includes dynamic random access memory (DRAM) circuits.

[0017] In another example of a memory stack, a first process-in-memory (PIM) circuit is disposed in a second memory IC die of the memory stack, and a second PIM circuit is disposed in a third memory IC die of the memory stack.

[0018] In another example of a memory stack, a first buffer IC die is disposed between a pair of memory IC dies and a second buffer IC die is disposed between another pair of memory IC dies.

[0019] In another example, a memory stack includes a first memory IC die including a first ferroelectric random access memory (FeRAM) circuit and a first process-in-memory (PIM) circuit; a second memory IC die stacked on the first memory IC die; and a third memory IC die stacked on the first memory IC die. The second memory IC die includes a second FeRAM circuit and a second PIM circuit. The third memory IC die includes a third FeRAM circuit and a third PIM circuit.

[0020] In yet another example, a chip package is provided that includes a hybrid memory stack mounted on a substrate. The hybrid memory stack includes volatile and non-volatile memory IC dies. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] Figure 1 Depicted is a chip package with a memory stack connected to a compute / processor die through an interposer.

[0022] Figure 2Depicted is an IC die stack disposed on top of a compute / processor chip and a memory interface / controller die.

[0023] Figure 3A Depicted is a high temperature memory die stacked on top of a compute / processor chip and a memory interface / controller die.

[0024] Figure 3B Depicted are high temperature memory, computing / processor circuitry, and memory interface / controller circuitry integrated into a single integrated circuit (IC) die.

[0025] Figure 4A A memory die stack disposed on top of a compute / processor chip and a memory interface / controller die, the memory die stack including at least one low temperature memory integrated circuit die and at least one high temperature memory integrated circuit die.

[0026] Figure 4B A memory die stack disposed on top of a computing / processor chip includes at least one low temperature memory integrated circuit die and at least one high temperature memory integrated circuit die, wherein the high temperature memory integrated circuit die includes a controller circuit.

[0027] Figure 4C A memory die stack is disposed on top of a compute / processor chip and a memory interface / controller die, the memory die stack comprising at least one low temperature memory integrated circuit die and at least one high temperature memory integrated circuit die with a buffer IC die disposed therebetween, the buffer IC comprising logic and non-volatile memory circuits.

[0028] Figure 5A A memory die stack is disposed on top of a compute / processor chip and a memory interface / controller die, the memory die stack comprising a plurality of cryogenic memory integrated circuit dies, one or more of the cryogenic memory integrated circuit dies having a processing-in-memory (PIM) circuit having a PIM local storage based on FeRAM and / or embedded DRAM.

[0029] Figure 5B A memory die stack disposed on top of a computing / processor chip and a memory interface / controller die, the memory die stack comprising a plurality of cryogenic memory integrated circuit dies, with Figure 5A One or more of the cryogenic memory integrated circuit dies have additional PIM circuitry relative to FeRAM and / or embedded DRAM based PIM storage compared to the stack of memory dies.

[0030] Figure 5C A memory die stack is disposed on top of a computing / processor chip and a memory interface / controller die, the memory die stack comprising a plurality of cryogenic memory integrated circuit dies, one or more of the cryogenic memory integrated circuit dies having a fine-grained PIM circuit having a PIM local storage based on FeRAM and / or embedded DRAM.

[0031] Figure 5D A memory die stack is disposed on top of a computing / processor chip and a memory interface / controller die, the memory die stack comprising a plurality of high temperature memory integrated circuit dies, one or more of the high temperature memory integrated circuit dies having a fine-grained sub-group level PIM circuit having a FeRAM-based PIM local storage. DETAILED DESCRIPTION

[0032] The disclosure herein addresses specific challenges of stacked DRAM subsystems with hybrid memory 3D organization and logic co-design. The disclosed technology defines various methods, systems, and apparatuses to design systems based on advanced memory that is both compute and application-aware. In general, memory die stacking utilizing a mix of high and low operating temperature memory die is disclosed, such that the high temperature die can be used as a temperature buffer for adjacent heat-generating logic die. In particular, memory die stacking utilizing a mix of volatile and non-volatile based memory die is disclosed, an example of which is a memory die based on stacked DRAM and FeRAM.

[0033] Figure 1 Depicted is a chip package 100 having a memory stack 102 connected to a computing / processor IC die 108 via an interposer 110. Any of the memory stacks described herein may be used in Figure 1 The chip package 100 depicted in FIG. 1 or other suitable memory device may include any of the following other memory stacks: Figure 1 The chip package 100 depicted in FIG. 1 also includes a package substrate 114 on which the interposer 110 is mounted. The package substrate 114 of the chip package 100 may be coupled to a printed circuit board (PCB) 136 to form an electronic system 180, such as but not limited to Figure 1 Graphics card depicted in .

[0034] The memory stack 102 generally includes at least one low temperature memory integrated circuit (LTMIC) die 104 stacked with at least one high temperature memory IC (HTMIC) die 106 . Figure 1The space between the dies 104, 106 shown in the memory stack 102 is used to form a solder connection (not shown) between the dies 104, 106. Alternatively, the dies 104, 106 can be stacked in direct contact with each other using a hybrid bonding technique. The HTMIC and LTMIC dies 106, 104 can be defined comparatively by at least one of the following definitions. In one example, the HTMIC die 106 has a longer memory refresh requirement than another memory die in the memory stack 102, and the memory die with the shorter memory refresh requirement is comparatively referred to as the LTMIC die 104. In another example, the HTMIC die 106 has a longer period between refreshes (i.e., a longer refresh cycle) than recommended by the Joint Electron Device Engineering Council (JEDEC) Solid State Technology Association standard JESD21-C, and the memory die with the memory refresh requirement that complies with JESD21-C is comparatively referred to as the LTMIC die 104. In another example, the HTMIC die 106 has a memory refresh requirement of more than 60 microseconds between memory refreshes, and the memory die requiring a memory refresh every 60 microseconds is comparatively referred to as the LTMIC die 104. In yet another example, in another example, the HTMIC die 106 is a non-volatile memory, and the LTMIC die 104 is a volatile memory. In yet other examples, the LTMIC die 104 may be defined as a memory die that can operate at a temperature (i.e., an operating temperature) up to 110°C without having to increase the refresh rate. At temperatures above 110°C, the LTMIC die 104 requires an increased refresh rate compared to operation at 110°C (compared to operation at 95°C). An example of the LTMIC die 104 is a dynamic random access memory (DRAM) die. Other examples of the LTMIC die 104 include volatile memory die such as system random access memory (SRAM), etc.

[0035] In still other examples, the operating temperature of the HTMIC die 106 is higher than the operating temperature of the LTMIC die 104. In other words, the HTMIC die 106 is a memory die that can operate at a temperature higher than 110° C. without having to increase the refresh rate (compared to operation at 95° C.). One example of the HTMIC die 106 is a ferromagnetic random access memory (FeRAM) die. Other examples of the HTMIC die 106 include non-volatile memory devices such as magnetoresistive random access memory (MRAM), phase change memory (PCM), flash memory, and resistive random access memory (RRAM).

[0036] The memory stack 102 may optionally include at least one controller IC die 120 stacked with the LTMIC die 104 and the HTMIC die 106. The IC dies 104, 106, 120 may be electrically and mechanically connected via solder balls and / or hybrid bonding techniques so that functional circuits within the IC dies 104, 106, 120 may communicate with each other and / or transmit data signals, power, and / or ground therethrough.

[0037] The functional circuits within the IC memory die 104, 106 are arranged into a plurality of memory groups. Each group has a plurality of rows, and each row has a plurality of columns. A memory cell resides at each unique memory location within a group. The memory cell can be addressed using the unique identification row and cell location of the memory cell within a particular group of memory dies 104, 106.

[0038] The functional circuitry within the IC dies 104, 106, 120 is coupled to the functional circuitry of the computing / processor IC die 108 via wiring 112 formed in the interposer 110. The wiring 112 of the interposer 110 is connected to the functional circuitry within the IC dies 104, 106, 120 via solder connections 118. The wiring 112 of the interposer 110 is also connected to the functional circuitry of the IC dies 108, 120 via solder connections 118 to wiring 122 formed in the package substrate 114. Solder balls 116 are used to connect the wiring 122 of the package substrate 114 with wiring 124 formed in the PCB 136.

[0039] In other embodiments where no interposer is present, the chip stack 102 and IC die 108 may be mounted directly to the package substrate 114 .

[0040] The IC die 120 is typically a heat generating device. That is, the IC die 120 generates heat when in use. Since the performance of the LTMIC die 104 may be degraded due to the heat generated by the IC die 120, the performance of the chip package 100 is enhanced by separating the LTMIC die 104 from the heat generating IC die 120 with one or more HTMIC dies 106. Since the HTMIC die 106 is typically more heat resistant than the LTMIC die 104, the HTMIC die 106 can be located near the heat generating IC die 120 without significantly degrading performance, while enabling the LTMIC die 104, which is significantly spaced apart from the heat generating IC die 120, to also maintain a stable performance level.

[0041] exist Figure 1 In the example depicted in FIG. 1 , the chip stack 102 includes one HTMIC die 106 disposed between a plurality of LTMIC dies 104 and a controller IC die 120. Figure 1Although four IC memory dies 104, 106 are illustrated in the single chip stack 102 shown in FIG. 1 , the number of LTMIC dies 104 and the number of HTMIC dies 106 constituting the chip stack 102 may vary from one to as many as can be assembled in the chip package 100. Figure 1 Only one chip stack 102 is shown, but one or more additional chip stacks may be provided in Figure 1 1 and 2. The vicinity of the chip stack 102 is shown in FIG.

[0042] The controller IC die 120 includes functional logic circuitry that provides commands that enable the rows and columns identifying each group of memory die 104, 106 to be addressed. The controller IC die 120 controls write / read operations from each memory group.

[0043] The HBM memory can be put into low power mode via the row address bus to save power on the I / O drivers. To further reduce power consumption, the clock can be gated in power-down or self-refresh mode.

[0044] Figure 2 Depicted is an IC die stack 202 disposed on top of the computing / processor IC die 108 and the memory interface / controller IC die 120. The IC die stack 202 may be used in place of the IC die stack 102 and mounted on a Figure 1 On the interposer 110 or the package substrate 114 in the chip package 100 and the electronic system 180 illustrated in FIG.

[0045] exist Figure 2 In the example depicted in , an IC die stack 202 that can be used in an electronic system 180 includes a high bandwidth (HBM) cube 204 stacked on top of a computing IC die 108 and a controller IC die 120. The computing IC die (e.g., a computing chip) 108 can be a central processing unit (CPU), a graphics processing unit (GPU), a field programmable gate array (FPGA), or other accelerator. The HBM cube 204 includes multiple LTMIC dies 104 stacked vertically. Each LTMIC die 104 includes functional circuits configured as memory circuits 220. In one example, the memory circuit 220 is configured as a volatile memory circuit, such as a dynamic random access memory (DRAM) and a system random access memory (SRAM), etc. In an example where the memory circuit 220 is configured as a DRAM circuit, the LTMIC die 104 may be referred to as a DRAM IC die. The stacked DRAM IC dies may be connected by solder connections, hybrid bonding, or other suitable connections. Although in Figure 2Three DRAM IC dies are illustrated in the HBM cube 204 depicted in FIG. 1 , but the HBM cube 204 may alternatively have more or less than three DRAM IC dies.

[0046] One or more HTMIC dies 106 are sandwiched between the LTMIC die 104 and the controller IC die 120 in the HBM cube 204. Figure 2 In the embodiment of the present invention, one HTMIC die 106 is shown, but multiple HTMIC die 106 may be utilized. The HTMIC die 106 includes functional circuits configured as a memory circuit 222. In one example, the memory circuit 222 is configured as a non-volatile random access memory, such as a ferroelectric random access memory (FeRAM), a magnetoresistive random access memory (MRAM), a phase change memory (PCM), a flash memory, and a resistive random access memory (RRAM). In one example where the memory circuit 222 is configured as a FeRAM, the HTMIC die 106 may be referred to as a FeRAM IC die.

[0047] The operating temperature of the memory circuit 222 of the HTMIC die 106 is higher than the operating temperature of the memory circuit 220 of the LTMIC die 104. In other words, the memory circuit 222 of the HTMIC die 106 is a memory circuit that can operate at a temperature higher than 110° C. without having to increase the refresh rate (compared to operation at 95° C.), while the memory circuit 220 of the LTMIC die 104 is a memory circuit that must increase the refresh rate (compared to operation at 95° C.) to operate at a temperature higher than 110° C.

[0048] In one example, the HTMIC die 106 is a non-volatile random access memory IC die that has a faster refresh speed than the LTMIC die 104 having a volatile random access memory. Therefore, in addition to the HTMIC die 106 performing better than the LTMIC die 104 when placed closer to the controller IC die 120, the faster refresh speed enables faster communication with the controller IC die 120, which beneficially reduces latency within the IC die stack 202, and ultimately reduces latency within the chip package 100 and the electronic system 180.

[0049] exist Figure 2 In the example depicted in , the HTMIC die 106 has a memory circuit 222 configured as a FeRAM circuit, while the LTMIC die 104 has a memory circuit 220 configured as a DRAM circuit.

[0050] Figure 3AAnother example of a memory stack 302 is depicted that includes a HTMIC die 106 stacked on top of a compute / processor IC die 108 and a memory interface / controller IC die 120. Figure 3A 1, but the plurality of LTMIC dies 104 may be provided as follows: Figure 2 The example is stacked in Figure 3A 106 to complete the memory stack 322. IC die stack 302 can be used in place of IC die stack 102 and mounted on Figure 1 On the interposer 110 or the package substrate 114 in the chip package 100 and the electronic system 180 illustrated in FIG. Figure 3A In the example depicted in , the HTMIC die 106 includes a memory circuit 222 configured as a FeRAM circuit. Figure 3A , the HTMIC die 106 is stacked vertically on top of the memory interface / controller IC die 120, which in turn is stacked vertically on top of the compute / processor IC die 108. The memory circuit 222 of the HTMIC die 106 includes FeRAM or other suitable circuitry that is compatible with the controller circuit 224 of the memory interface / controller IC die 120. The interconnections between the dies 104, 106, 108, 120 may be accomplished by solder connections, hybrid bonding, or other suitable connections. The HTMIC die 106 stacked on top of the compute / processor IC die 120 forms a hybrid memory-logic assembly that may subsequently be stacked with the LTMIC die 104, such as Figure 2 exemplified.

[0051] Figure 3B Another example of a memory stack 322 including memory circuitry 222, computation / processor circuitry 324, and memory interface / controller circuitry 224 integrated into a single HTMIC die 106 is depicted. Figure 3B 1, but the plurality of LTMIC dies 104 may be provided as follows: Figure 2 The example is stacked in Figure 3B 106 to complete the memory stack 322. Figure 3BThe additional die on the HTMIC die 106 shown in the figure may be one or more FeRAM dies and / or one or more LTMIC dies 104 (such as DRAM IC dies) and / or one or more other types of memory dies. In one example, the memory circuit 222 is configured as a FeRAM circuit that is compatible with the memory interface / controller circuit 224, so that the circuits 222, 224 can be co-located within the same HTMIC die 106. Similarly, the FeRAM circuit 222 is also compatible with the computing / processor circuit 324, so that the circuits 222, 324, 224 can be co-located within the same HTMIC die 106. In one example, the memory circuit 222 is disposed between the computing / processor circuit 324 and the memory interface / controller circuit 224. Optionally, the computing / processor circuit 224 may reside in an IC die adjacent to the HTMIC die 106 including the FeRAM and controller circuits 222, 224.

[0052] The goal of the advanced memory technology roadmap is to increase memory density and bandwidth while minimizing the impact on power and performance to alleviate memory bottlenecks in system performance. With advances in memory technology, memory stacking, and new non-volatile memories such as FeRAM, it is imperative to update circuits, architectures, and memory interface principles to keep pace with the memory technology itself. The following describes improvements to memory technology that leverage enhancements specific to HBM / other forms of stacked high-temperature memory by integrating low-temperature memory technology to create a hybrid memory stack. In one example, stacked DRAM memory can be integrated with FeRAM-based memory to form a hybrid memory stack or hybrid memory-logic component.

[0053] Hybrid memories and hybrid memory-logic components are disclosed that utilize a mix of memory technologies that can be stacked, for example, on top of a logic die. For example, a hybrid memory cube having a non-volatile memory (such as FeRAM, etc.) IC die and a volatile memory (such as DRAM IC, etc.) die has a HTMIC die 106 that is beneficially disposed closest to the logic IC die 120 because the HTMIC die 106 can withstand higher heat dissipation from the logic die, while the LTMIC die 104 disposed on top of the HTMIC die 106 can be closer to a chip package (such as a DRAM IC, etc.). Figure 1 The heat sink is placed in the chip package 100 depicted in FIG. 1 to minimize temperature gradients and the impact on the performance and refresh rate associated with the LTMIC die 104.

[0054] Figure 4AAn example of a memory die stack 400 disposed on top of a compute / processor IC die 108 and a memory interface / controller IC die 120 is depicted. The memory die stack 400 includes at least one HTMIC die 106 (such as a non-volatile memory IC die) and a plurality of LTMIC dies 104 (such as volatile memory dies). Another alternative hybrid approach is to arrange the HTMIC and LTMIC dies 106, 104 within the same memory die stack 400 sorted based on latency, with the fastest die closer to the memory interface / controller IC die 120. The sorted memory IC dies 104, 106 may include SRAM, DRAM, and non-volatile memory (NVM) IC dies, all arranged by latency in the same memory die stack 400. This results in a hierarchical hardware managed cache of LTMIC dies 104 (such as DRAM IC dies) within a stacked memory cube (e.g., memory die stack 400).

[0055] Figure 4B 4 is a memory die stack 410 disposed on top of a computing / processor IC die 108. The memory die stack 410 includes at least one HTMIC die 106 (such as a non-volatile memory IC die) and a plurality of LTMIC dies 104 (such as volatile memory dies). The HTMIC die 106 of the memory die stack 410 includes a memory circuit 222 and a controller circuit 224. For example, the memory circuit 222 of the HTMIC die 106 may include a FeRAM (or other non-volatile memory) circuit and a controller circuit 224 integrated on the same die. This arrangement Figure 4B , which is achieved by the FeRAM configuration of the memory circuit 222 being compatible with the logic technology of the controller circuit 224.

[0056] Alternatively, the IO / SA logic on each memory die 104 may be separated into a buffer IC die 422 to achieve higher performance and yield, and may also include logic compatible FeRAM memory blocks. Figure 4C , which shows a memory die stack 420 disposed on top of a compute / processor IC die 108 and a memory interface / controller IC die 120, the memory die stack 420 including at least one HTMIC die 106 (such as a non-volatile memory IC die) and at least one LTMIC die 104 (such as a volatile memory die) with a buffer IC die 422 disposed therebetween. The buffer IC die 422 includes logic and non-volatile memory circuits 224, 222. The buffer IC die 422 can be hybrid bonded on each side to the other dies 104 of the HBM cube 204 comprising the die stack 420.

[0057] Memory stacking and techniques may be selected and designed in a “hierarchical” fashion (hardware managed cache) to use faster memory or memories closer to the logic IC die 120 that do not require refresh as a “middle” tier, moving data to denser, slower memories on upper LTMIC dies 104 away from the logic IC die 120. This may help hide the latency and overhead caused by the refresh required on the LTMIC die 104 (such as a DRAM IC die) on the top of the memory die stack 420.

[0058] In the case where FeRAM or other nonvolatile memory dies are used with multi-bit cell storage (e.g., NAND flash stores multiple bits in a cell), wear can become an issue because each cell will be accessed "n" times compared to the single bit cell case, where "n" is the number of bits in a single cell. Therefore, DRAM / SRAM and other volatile memories with enhanced endurance can be used as a "backup" or hardware managed cache for nonvolatile memory dies. This allows multiple writes (known as write leveling) to be combined into a single write to a nonvolatile memory multi-bit cell, which beneficially reduces the number of writes to a single cell in the nonvolatile memory IC die and increases the life of the nonvolatile memory circuit. Reads can be combined in essentially the same manner.

[0059] FIG. 5A to FIG. 5D Some non-limiting examples of processing-in-memory (PIM) circuits 502 utilized within a hybrid memory assembly (i.e., memory die stack 500) are illustrated. The tradeoff of PIM is generally between the speedup provided by near-memory computations and the area overhead / impact on memory density and leakage provided by PIM circuit integration. PIM circuits 502 include processors or other logic circuits integrated with memory circuits 220 / 222 on a single IC die of memory die stack 500. PIM circuits 502 include local storage circuits 506.

[0060] The local storage circuit 506 of the PIM circuit 502 may be FeRAM and / or embedded DRAM (eDRAM). Advantageously, the FeRAM and / or eDRAM based local storage circuit 506 is typically a low leakage logic compatible memory compared to the local logic based high leakage registers. This allows for area scaling and leakage reduction of the PIM circuit 502 compared to conventional PIMs that use registers in logic based logic memory. Thus, within the same IC die area allocated to the PIM circuit 502, the amount of processing in memory may be increased.

[0061] This can also be used for finer-grained PIM (eg, sub-group level; currently PIM is performed at the group level) or increased memory density due to reduced PIM area.

[0062] First reference Figure 5A , a memory die stack 500 is shown disposed on top of a compute / processor IC die 108 and a memory interface / controller IC die 120. The memory die stack 500 includes at least one HTMIC die 106 (such as a non-volatile memory IC die) and a plurality of LTMIC dies 104 (such as volatile memory dies). One or more of the LTMIC dies 104 have a process-in-memory (PIM) circuit 502 having a PIM local storage circuit 506 based on FeRAM and / or embedded DRAM.

[0063] exist Figure 5B , a memory die stack 510 is shown disposed on top of a compute / processor IC die 108 and a memory interface / controller IC die 120. The memory die stack 510 includes at least one HTMIC die 106 (such as a non-volatile memory IC die) and a plurality of LTMIC dies 104 (such as volatile memory dies). Figure 5A Compared to the memory die stack 500 of FIG. 1 , one or more of the LTMIC dies 104 have additional PIM circuits 502 relative to the FeRAM and / or embedded DRAM based PIM local storage circuits 506 .

[0064] exist Figure 5C , a memory die stack 520 is shown disposed on top of a compute / processor IC die 108 and a memory interface / controller IC die 120. The memory die stack 520 includes at least one HTMIC die 106 (such as a non-volatile memory IC die) and a plurality of LTMIC dies 104 (such as volatile memory dies). One or more of the LTMIC dies 104 have a fine-grained PIM circuit 502 having a PIM local storage circuit 506 based on FeRAM and / or embedded DRAM.

[0065] exist Figure 5D, a memory die stack 530 is shown disposed on top of a compute / processor IC die 108 and a memory interface / controller IC die 120. The memory die stack 530 includes at least one HTMIC die 106 (such as a non-volatile memory IC die) and a plurality of LTMIC dies 104 (such as a volatile memory die). One or more of the LTMIC dies 104 have a fine-grained sub-group level PIM circuit 502 having a FeRAM-based PIM local storage circuit 506. In an alternative hybrid approach, the plurality of LTMIC dies 104 include a DRAM IC memory die stacked on top of one or more SRAM IC memory dies. The SRAM IC memory die can be configured to function as a hardware managed cache for the DRAM IC memory die, or as part of an address space that provides extremely low latency. In Figure 5D In the example depicted in , the HTMIC die 106 includes FeRAM memory arrays that allow for easier integration of the PIM circuits 502 within the logic-compatible FeRAM circuits of the HTMIC die 106. All of the above examples achieve enhanced SoC performance and power efficiency.

Claims

1. A memory stack, the memory stack comprising: a first memory IC die, the first memory IC die comprising a memory, and A second memory IC die is stacked on the first memory IC die, the second memory IC die including memory circuits that require a more frequent refresh rate than the first memory IC die.

2. The memory stack of claim 1 wherein the memory circuit of the first memory IC die is a non-volatile memory circuit.

3. The memory stack of claim 2, wherein the non-volatile memory circuit is a ferroelectric random access memory (FeRAM) or a static random access memory (SRAM) circuit.

4. The memory stack of claim 3 wherein the memory circuit of the second memory IC die is a volatile memory circuit.

5. The memory stack of claim 4, wherein the volatile memory circuit is a dynamic random access memory (DRAM) circuit.

6. The memory stack according to claim 1, further comprising: a controller die stacked below and in contact with the first memory IC die; and A processor die is stacked below and in contact with the controller die, the processor die including processor circuitry that communicates with memory circuitry of the first and second memory IC dies through controller circuitry of the controller IC die.

7. The memory stack of claim 1 wherein the first memory IC die comprises a controller circuit.

8. The memory stack according to claim 7, further comprising: A processor die is stacked below and in contact with the first memory IC die, the processor die including processor circuitry that communicates with memory circuitry of the first memory IC die and the second memory IC die through controller circuitry of the first memory IC die.

9. The memory stack according to claim 1, further comprising: A third memory IC die is stacked on the second memory IC die, the third memory IC die including dynamic random access memory (DRAM) circuitry.

10. The memory stack of claim 9, wherein the third memory IC die has a greater latency than the second memory IC die, and the second memory IC die has a greater latency than the first memory IC die.

11. The memory stack according to claim 9, further comprising: a first processing-in-memory (PIM) circuit disposed in the second memory IC die; and A second PIM circuit is disposed in the third memory IC die.

12. The memory stack of claim 1 wherein the first memory IC die comprises a controller circuit.

13. The memory stack according to claim 12, further comprising: A processor die is stacked below and in contact with the first memory IC die, the processor die including processor circuitry that communicates with memory circuitry of the first memory IC die and the second memory IC die through the controller circuitry of the first memory IC die.

14. The memory stack according to claim 1, further comprising: A first buffer IC die is disposed between the first memory IC die and the second memory IC die.

15. A chip package, comprising: Package substrate; and A memory stack, wherein the memory stack is stacked on the packaging substrate, and the memory stack comprises: A plurality of first memory IC dies are stacked on a second memory IC die, the second memory IC die having a ferroelectric random access memory (FeRAM) circuit and an optional controller circuit, the second memory IC die being stacked on the package substrate, the plurality of first memory IC dies including a DRAM circuit.

Images