High capacity memory circuit with low effective latency

Abstract

A first circuit formed on a first semiconductor substrate is wafer bonded with a second circuit formed on a second memory circuit, where the first circuit includes a quasi-volatile or non-volatile memory circuit, the second memory circuit includes a fast memory circuit having a lower read latency than the quasi-volatile or non-volatile memory circuit, and a logic circuit. The volatile and non-volatile memory circuits can include static random access memory (SRAM) circuits, dynamic random access memory (DRAM) circuits, embedded DRAM (eDRAM) circuits, magnetic random access memory (MRAM) circuits, embedded MRAM (eMRAM), or any suitable combination of these circuits.

Description

[0001] Cross-referencing related applications

[0002] This application is related to and claims priority to U.S. provisional patent application filed on February 7, 2020, entitled “High Capacity Memory Circuit With Low Effective Latency” (“Parent Provisional Application”, Serial No. 62 / 971,720).

[0003] This application also relates to (i) U.S. non-provisional application filed January 29, 2020 (“Non-Provisional Application I”), Serial No. 16 / 776,279, entitled “Device with Embedded High-Bandwidth, High-Capacity Memory using Wafer Bonding”; and (ii) U.S. patent application filed September 25, 2019 (“Non-Provisional Application II”), Serial No. 16 / 582,996, entitled “Memory Circuit, System and Method for Rapid Retrieval of Data Sets”; and (iii) U.S. non-provisional patent application filed October 4, 2019 (“Non-Provisional Application III”), Serial No. 16 / 593,642, entitled “Three-dimensional Vertical NOR Flash Thin-film Transistor”. (iv) U.S. non-provisional patent application filed January 15, 2020 (“Non-provisional Application IV”), Serial No. 16 / 744,067, entitled “Implementing Logic Function and Generating AnalogSignals Using NOR Memory Strings”;

[0004] This application also relates to U.S. Provisional Application (“Provisional Application”), filed on December 12, 2019, serial number 62 / 947,405, entitled “Vertical Thin-film Transistor and Application as Bit Line Connector for 3-Dimensional Memory Arrays”.

[0005] The publication of the parent provisional application, the provisional application, and the non-provisional applications I-IV are incorporated herein by reference in their entirety. Technical Field

[0006] This invention relates to memory circuits and computing systems. Specifically, this invention relates to very high-capacity memory circuits while providing low effective latency and memory-computer system interaction comparable to existing dynamic random access memory (“DRAM”) circuits. Background Technology

[0007] Non-provisional applications II and III each disclose high-capacity three-dimensional thin-film memory circuits that can be configured as quasi-volatile memory circuits. While quasi-volatile memory circuits have shorter data retention times (e.g., minutes) compared to non-volatile memory circuits (e.g., years), they offer faster write and erase operations, greater endurance, lower read latency, and comparable circuit density compared to conventional non-volatile circuits. Non-provisional applications II and III also disclose quasi-volatile memory circuits formed as three-dimensional arrays of thin-film storage transistors on a semiconductor substrate, along with analog and digital support circuitry such as various power supply circuits, drivers, sense amplifiers, word line and bit line decoding circuits, data latches, multiplexers, select transistors, and input and output circuits. Some of these circuits can operate at high voltages (e.g., 8.0–16.0 volts), while others can operate at medium voltages (e.g., 2.0–6.0 volts) and low voltages (e.g., 0.6–1.2 volts). In this specification, the circuitry formed in the semiconductor substrate beneath a three-dimensional memory array of thin-film storage transistors is generally referred to as “circuit under array” (“CuA”). Typically, for non-volatile or quasi-volatile thin-film memory arrays, high-voltage circuitry is relatively low-density (i.e., large-area) circuitry, while low-voltage transistors are relatively high-density. Of these transistor types, low-voltage transistors typically offer the highest performance (i.e., the fastest) and provide the densest circuitry. In one embodiment disclosed in Non-Provisional Application II, the storage transistors of each three-dimensional array are organized as parallel stacks of NOR memory strings, wherein the stack has eight or more NOR memory strings, one disposed on top of another, separated by a dielectric layer. The storage transistors in each NOR memory string share a common drain region and a common source region. The common drain region of each NOR memory string, also commonly referred to as a “bit line,” extends along a direction parallel to the surface of the semiconductor substrate. Connections to the gate electrodes of the storage transistors are provided by conductors (“word lines”) shared by the many NOR memory strings. Each word line extends along a direction substantially perpendicular to the surface of the semiconductor substrate. In this detailed description, the memory array of non-temporary application II is referred to as an HNOR memory array based on their essentially "horizontal" common drain and common source regions.

[0008] As disclosed in Non-Provisional Application II, the memory transistors in a three-dimensional memory array form memory portions (“array portions”) and contact portions (“step portions”). The step portions are so named because as the distance between the bit lines and the surface of the semiconductor substrate increases, the amount by which each bit line of each stack of NOR memory strings extends beyond the array portion decreases sequentially, thus forming a step structure. Electrical contacts for the bit lines can be provided in the step portions. The step portions of each stack of NOR memory strings can have two step structures on opposite sides of the array portions.

[0009] In one disclosed embodiment of non-provisional application III, the storage transistors of each three-dimensional array are organized into parallel columns of NOR memory strings, each column having at least one NOR memory string, wherein the storage transistors share a common drain region and a common source region. The common drain region, or bit line, of each NOR memory string extends along a direction substantially perpendicular to the surface of the semiconductor substrate. In this detailed description, the memory array of non-provisional application III is referred to as a VNOR memory array based on their substantially "vertical" common drain and common source regions. Similar to HNOR memory arrays, the storage transistors in a three-dimensional VNOR memory array also form storage portions ("array portions") and contact portions ("stepped portions"). The stepped portions of the VNOR memory array provide electrical contacts with word lines. Electrical contacts to bit lines may be provided in the stepped portions. The stepped portions of the VNOR memory array may have two stepped structures on opposite sides of the array portions.

[0010] Forming thin-film memory arrays on CuA presents challenges. For example, fabricating quasi-volatile and non-volatile memory arrays on a substrate requires high-temperature steps (“thermal cycling”). Since CuA is first formed in the substrate, it is also exposed to thermal cycling before forming the quasi-volatile and non-volatile memory arrays. Dense low-voltage logic circuits are particularly susceptible to degradation due to exposure to thermal cycling. For example, sense amplifiers are particularly prone to degradation under thermal treatment, which adversely affects their sensitivity and signal integrity. Therefore, CuA imposes a limit on the thermal budget allowed for forming memory arrays to prevent thermal cycling from degrading the performance of high-performance, low-voltage, and other types of transistors in CuA. Generally, high-voltage and medium-voltage circuits can withstand thermal cycling without experiencing any significant adverse effects.

[0011] The numerous manufacturing steps required to form CuA and memory circuits adversely affect potential yield and performance. Non-provisional application I discloses an integrated circuit formed by wafer-level hybrid bonding of a semiconductor die. Using wafer-level or chip-level hybrid bonding, memory circuits and their associated CuA (“memory chips”) and logic circuits (“accompanying chips”) can be independently fabricated on separate semiconductor substrates and aggregated together by aligned hybrid bonds provided on their respective bonding surfaces. In this detailed description, the terms “bond” or “bonding” can refer to any wafer-level bonding technology, chip-level bonding, or any combination of wafer-level bonding and chip-level bonding (e.g., wafer-to-wafer hybrid bonding, chip-to-chip hybrid bonding, and chip-to-wafer hybrid bonding). Non-provisional application I demonstrates that such combinations not only alleviate the challenges in manufacturing steps but also enable higher performance and new applications of memory circuits that were previously impossible.

[0012] U.S. Patent Application Publication 2019 / 0057974 (“Lu”), filed July 26, 2018, by Z. Lu et al. entitled “Hybrid Bonding Contact Structure Of Three-Dimensional Memory Device,” discloses a three-dimensional (3-D) NAND memory device formed by bonding two semiconductor substrates. In Lu, a three-dimensional NAND memory array is fabricated on the planar surface of a first substrate, and “peripheral circuitry” is fabricated on a second substrate. The two substrates are bonded using hybrid bonding in a “flip-chip” manner. Directly beneath the bonding surface of each substrate, Lu teaches the formation of interconnect structures such that when the two substrates are bonded, the hybrid bonding connects the two interconnect structures together to form an interconnect network connecting the peripheral circuitry and the 3-D NAND memory array.

[0013] Lu discloses that the peripheral circuitry formed on the second substrate includes “page buffers, decoders (e.g., row decoders and column decoders), latches, sense amplifiers, drivers, charge pumps, current or voltage references, or any active or passive components of the circuitry (e.g., high-voltage and low-voltage transistors, diodes, resistors, or capacitors).” In some embodiments, one or more peripheral circuits may be formed on the second substrate 510 using complementary metal-oxide-semiconductor (CMOS) technology (also referred to as a “CMOS chip”) (Lu, in paragraph

[0125] ). Note that page buffers, decoders, and sense amplifiers are low-voltage logic circuits that can take full advantage of the optimal performance of advanced manufacturing process nodes, as described above. Drivers, charge pumps, and current or voltage references are typically medium- and high-voltage analog circuits required in 3-D NAND memory circuitry, for example, for generating programming, erasing, reading, and inhibiting voltages. Medium- or high-voltage circuits are generally not as scalable as low-voltage circuits, making them less cost-effective to manufacture at advanced manufacturing process nodes. Additionally, multi-oxide-CMOS technology requires accommodating both high-voltage and low-voltage transistors on the same chip. Such a process compromises the scaling and performance of low-voltage transistors, which would otherwise be possible. Therefore, by placing high-voltage, medium-voltage, and low-voltage circuits on a second substrate, the peripheral circuits of the LU can only be fabricated on the second substrate using manufacturing processes capable of forming all low-voltage logic circuits as well as medium-voltage and high-voltage analog circuits, thus sacrificing high-voltage and low-voltage transistors. This approach prevents low-voltage logic circuits from utilizing better performance and circuit density at more advanced manufacturing process nodes. Summary of the Invention

[0014] According to one embodiment of the invention, a first circuit formed on a first semiconductor substrate is bonded to a second circuit formed on a second semiconductor substrate, wherein the first circuit includes a quasi-volatile or non-volatile memory circuit, and wherein the second memory circuit includes a memory circuit that is faster than the quasi-volatile or non-volatile memory circuit. This faster memory circuit can be a volatile or non-volatile memory circuit. The faster memory circuit can include static random access memory (SRAM) circuits, dynamic random access memory (DRAM) circuits, embedded DRAM (eDRAM) circuits, magnetic random access memory (MRAM) circuits, embedded MRAM (eMRAM) circuits, spin-transfer torque MRAM (ST-MRAM) circuits, phase-change memory (PCM), resistive random access memory (RRAM), conductive bridged random access memory (CBRAM), ferroelectric resistance random access memory (FRAM), carbon nanotube memory, or any suitable combination of these circuits. Bonding the first and second circuits can be accomplished using conventional techniques, such as wafer-level or chip-level hybrid bonding.

[0015] Because quasi-volatile memory circuitry on a memory chip can achieve high data density, high endurance, and high-speed access, the integrated circuit of this invention enables many new applications. The accompanying faster on-chip memory circuitry provides even faster access times. This combination effectively produces a high-density, low-latency memory circuit, essentially a heterogeneous memory with advantages that can be utilized in new applications. For example, the integrated circuit of this invention is particularly suitable for in-memory computing or near-memory computing applications.

[0016] The invention can be better understood by taking into account the following detailed description in conjunction with the accompanying drawings. Attached Figure Description

[0017] Figure 1(a) illustrates an integrated circuit 120 operating under the control or supervision of a host processor 103 according to an embodiment of the present invention, which includes a memory chip 101 and an accompanying chip 102 interconnected by a hybrid bond.

[0018] Figure 1(b) illustrates an alternative implementation of an integrated circuit 120 according to an embodiment of the present invention, wherein each modular logic circuit 106 is provided with access to two fast memory circuits 107.

[0019] Figure 1(c) illustrates another alternative implementation of an integrated circuit 120 according to an embodiment of the present invention, wherein a single modular logic circuit 106 is provided with access to each fast memory circuit 107.

[0020] Figure 1(d) shows a functional representation of one of the fast memory circuits 107 according to an embodiment of the present invention.

[0021] Figure 1(e) illustrates the functional organization of the memory bank group BG(0)-BG(3) of the fast memory circuit in a memory chip 101 according to an embodiment of the present invention.

[0022] Figure 1(f) shows four half-banks 188a-188d on the accompanying chip 102 (e.g., half-bank group 182-1a in Figure 1(e)) according to an embodiment of the present invention, and corresponding half-banks 189a-189d of the quasi-volatile memory circuit in the memory chip 101.

[0023] Figure 1(g) shows four half-memory banks 187a-187d of a half-memory bank group (e.g., half-memory bank group 182-1a of Figure 1(e)) on an accompanying chip 102 according to another embodiment of the present invention, and corresponding half-memory banks 189a-189d of the quasi-volatile memory circuit in memory chip 101.

[0024] Figure 1(h) illustrates a functional configuration of a memory chip set 170 based on a 16-memory array as described above in conjunction with Figure 1(e) according to an embodiment of the present invention.

[0025] Figure 1(i) shows an alternative organization of the memory chip group 170 of Figure 1(e) with 16 memory cells according to an embodiment of the present invention, wherein modular logic circuits 172 are distributed among modular memory circuits 171.

[0026] Figure 1(j) shows a circuit 190 according to an embodiment of the present invention, wherein integrated circuits 120a and 120b respectively implement data-intensive in-memory computation and massively parallel search in CAM.

[0027] Figure 2(a) generally illustrates a hybrid-bonded memory chip 101 and an accompanying chip 102 of an integrated circuit 120 according to an embodiment of the present invention; in this embodiment, the memory chip and the accompanying chip 102 are bonded in a "flip-chip" orientation.

[0028] Figure 2(b) shows array structures 202-a and 202b in more detail, which represent any two adjacent array structures from array structures 202-(1,1) to 202-(n,m).

[0029] Figure 2(c) generally illustrates a hybrid-bonded memory chip 101 and an accompanying chip 102 of an integrated circuit 120 according to another embodiment of the present invention; in this embodiment, the memory chip and the accompanying chip 102 are bonded in a “stacked” orientation.

[0030] Figure 2(d) generally illustrates a hybrid bonding memory chip 101 and an accompanying chip 102 of an integrated circuit 120 according to a third embodiment of the present invention; in this third embodiment, the memory chip 101 includes a VNOR memory array.

[0031] Figure 2(e) generally illustrates a hybrid bonding memory chip 101 and an accompanying chip 102 of an integrated circuit 120 according to a fourth embodiment of the present invention; in this fourth embodiment, the memory chip 101 includes a VNOR memory array and a vertical thin-film transistor (TFT).

[0032] Figure 3 A portion of the integrated circuit 120 in Figure 2(a) is shown in more detail.

[0033] Figure 4 A top view of the accompanying chip 102 is shown, showing the mixed bond stripes 203 and the metal layer 206.

[0034] Figure 5(a) illustrates a sense amplifier, latch, SRAM array and various logic circuits formed on the surface 212 of the substrate 211 in the accompanying chip 102 according to a first embodiment of the present invention.

[0035] Figure 5(b) illustrates a sense amplifier, latch, SRAM array and various logic circuits on the surface 212 of the substrate 211 of the accompanying chip 102 according to a second embodiment of the present invention.

[0036] Figure 6 A memory module 600 according to an embodiment of the present invention is generally shown, wherein each of the memory chipsets 601-0 to 601-15 includes a memory chip 101 and an accompanying chip 102 coupled together; the memory module may be provided in the format of a dual in-line memory module (DIMM).

[0037] Figure 7 An integrated circuit 160 is shown, which includes an integrated circuit 120 formed by a memory chip 101 and an accompanying chip 102, and a non-memory chip 161.

[0038] Figure 8 An integrated circuit 120 and a non-memory chip 161 interconnected via a silicon interposer 801 are shown.

[0039] Figure 9 This is a schematic representation of a computing system 900, which can be a larger host system (e.g., Figure 6 Subsystems within the host system (603).

[0040] Figure 10 This is a schematic representation of a memory chipset 170, with the memory chipset 125 provided on a battery 1001 on an accompanying chip 102.

[0041] Figure 11 A paging system using a fast memory circuit (e.g., SRAM circuit) and a quasi-volatile memory circuit of an integrated circuit 120 is illustrated schematically according to an embodiment of the present invention. Detailed Implementation

[0042] According to one embodiment of the invention, an integrated circuit can be formed by combining a high-density quasi-volatile or non-volatile memory circuit formed on a first semiconductor die (“memory chip”) and a faster memory circuit (e.g., SRAM, DRAM, eDRAM, MRAM, eMRAM, PCM, or any other suitable memory circuit) formed on a second semiconductor die (“companion die”). The quasi-volatile or non-volatile memory circuit on the memory chip is preferably constructed at high density, such as through three-dimensional construction. Conversely, the faster memory circuit on the companion chip is preferably constructed for high performance, such as through higher-level logic processing nodes. For example, the memory chip and companion chip can be clustered together by high-density hybrid bonding.

[0043] Importantly, in one embodiment of the invention, both the memory chip and the accompanying chip are organized in modular blocks, commonly referred to as “tiles”. In this embodiment, there is a one-to-one correspondence between the tiled memory chip and the tiled accompanying chip. Each tiled region in the accompanying chip—whose area corresponds to the corresponding tiled memory chip—provides a sense amplifier and other logic support circuitry for the quasi-volatile memory circuitry within the corresponding tiled chip. Furthermore, each tiled accompanying chip includes fast memory circuitry (e.g., SRAM circuitry) placed within a specific “pocket” region on the tiled chip. As a result, the corresponding tiled memory chips and accompanying chips form a very high-density, very low-latency heterogeneous memory circuitry (i.e., the three-dimensional structure of the memory circuitry (e.g., quasi-volatile memory circuitry) in the memory chip provides high density, while the fast memory circuitry provides very low latency (e.g., SRAM circuitry)). The memory circuitry on the memory chip may include 3D NAND, 3D PCM, 3D HNOR memory, 3D VNOR memory, or other suitable non-volatile or quasi-volatile memory circuitry types. The accompanying on-chip memory circuitry may include volatile memory circuitry (e.g., SRAM or DRAM), or high-performance, non-volatile memory circuitry (e.g., MRAM, ST-MRAM, or FRAM), or any suitable combination of these types of memory circuitry.

[0044] According to one embodiment of the invention, high-performance low-voltage transistors are provided on a companion chip rather than on the memory chip to (i) avoid degradation of the high-performance low-voltage logic transistors during thermal cycling in the fabrication of memory arrays on the memory chip, and (ii) benefit from advanced manufacturing nodes optimized for their production. Since low-voltage transistors form sense amplifiers, registers or data latches, high-performance data path circuitry, input and output interfaces, error correction circuitry, and fast logic circuitry (e.g., low-voltage decoders and multiplexers, state machines and sequencers, and input and output circuitry), these circuits can best utilize one or more generations of manufacturing process nodes that are more advanced (albeit more expensive) than those capable of fabricating high- and medium-voltage transistors. Furthermore, depending on the intended application or manufacturing technology, the memory chip can be hybrid-bonded to a companion chip specifically configured for that intended application, or it can be fabricated using that manufacturing process (e.g., a sufficiently advanced or cost-effective CMOS manufacturing process node). High-performance low-voltage transistors are particularly susceptible to degradation during thermal cycling in memory array fabrication. Decoupling the low-voltage transistors from high- and medium-voltage transistors by fabricating them on separate chips provides an advantageous solution.

[0045] In one embodiment, while medium- and high-voltage transistors are fabricated as CuA in the memory chip using, for example, a minimum design rule from 65-nm to 28-nm, high-performance low-voltage transistors on the companion chip can be achieved using a faster and denser low-voltage-only design rule from 28-nm to below 5-nm. In this approach, the companion chip not only provides the traditional support circuitry for the memory array within the memory chip, but the density achievable using more advanced manufacturing nodes allows for the inclusion of other circuitry (e.g., SRAM circuitry, arithmetic and logic circuitry, Reduced Instruction Set Computer (RISC) circuitry, and other suitable logic circuitry), which is effective, for example, in in-memory computing or near-memory applications. Furthermore, by providing low-voltage circuitry in the companion chip, the CuA on the memory chip only needs to provide high- and medium-voltage transistors, allowing the memory chip to benefit from reduced die size and simpler manufacturing processes, resulting in higher yields.

[0046] In this embodiment, the word line-related circuitry and its connections reside within the memory chip and do not require word line-related hybrid bonding connections with the accompanying chip. Without such word line-related hybrid bonding connections, the number of hybrid bonds required in this embodiment of the invention is significantly less than that required for the 3-D NAND memory devices of the LU discussed above, which require hybrid bonding connections for all word line signals and all bit line signals to be received in or generated from supporting circuitry (e.g., a signal decoder) in the accompanying chip. Interconnect layers in the accompanying chip route signals to and from circuitry in and from the accompanying chip substrate. Therefore, routing both word line-related and bit line-related signals to the accompanying chip results in fewer hybrid bonds and routing tracks in the accompanying chip that can be used for other signals or other purposes. The present invention avoids this problem.

[0047] One embodiment of the present invention is illustrated by FIG1(a). FIG1(a) illustrates an integrated circuit 120 operating under the control or supervision of a host processor 103, comprising a memory chip 101 and an accompanying chip 102 bonded together (e.g., using hybrid bonding). (Other suitable bonding techniques include, for example, microbump or direct interconnect bonding.) In the detailed description below, the integrated circuit 120 may be referred to as a “memory chipset.” The host processor 103 may be, for example, a conventional central processing unit (CPU), graphics processing unit (GPU), field-programmable gate array (FPGA), or memory controller. As shown in FIG1(a), the memory chip 101 may include any quasi-volatile or non-volatile memory circuitry, for example, any type described in non-provisional applications II and III. Examples of such quasi-volatile memory circuitry include HNOR memory string arrays and VNOR memory string arrays. The quasi-volatile memory circuitry may include a three-dimensional array of multiple thin-film memory transistors formed on a single-crystal semiconductor substrate. The semiconductor substrate may also have suitable support circuitry (CuA) formed therein, such as a voltage source for generating signals used in read, program, or erase operations. As described below, the low-voltage, fast logic circuitry, sense amplifier, and other support circuitry for the quasi-volatile memory circuitry can be implemented in the accompanying chip 102.

[0048] When implemented using quasi-volatile memory circuitry, the high-density memory array on memory chip 101 offers the benefit of high endurance. However, in read-intensive applications, the high-density memory array on memory chip 101 can be implemented using non-volatile memory circuitry or a combination of quasi-volatile and non-volatile memory circuitry. In such a combination, the non-volatile memory circuitry is used to store data that changes infrequently, for which long-term retention is more important than high endurance. For example, examples of three-dimensional non-volatile and quasi-volatile memory circuitry that can be used on memory chip 101 are described in non-provisional applications II and III.

[0049] The accompanying chip 102 may include fast memory circuitry 107, as shown in FIG1(a) as modular fast memory circuits 107-1, 107-2, ..., 107-n. The quasi-volatile memory circuitry on the accompanying chip 102 and the supporting circuitry of the fast memory circuitry 107 may be interconnected to CuA on the memory chip 101 using hybrid bonds. FIG1(a) shows each of the modular fast memory circuits tightly coupled to the corresponding memory circuit blocks of the memory chip 101. For example, in the memory chip 101, memory banks 110-1, 110-2, ..., 110-n (i.e., memory banks bank[0], bank[1], ..., bank[n]), each of which may be a bank of quasi-volatile or non-volatile memory cells, are shown to be physically tightly connected to the modular fast memory circuits 107-1, 107-2, ..., 107-n respectively by, for example, hybrid bonds 111-1, 111-2, ..., 111-n. In one embodiment, each modular fast memory circuit on the companion chip 102 is tightly coupled to a corresponding memory circuit block in the memory chip 101. Therefore, the modular fast memory circuit 107 becomes part of the corresponding quasi-volatile or non-volatile memory bank 110. In a practical implementation, the memory chip 101 and the companion chip 102 will be bonded to each other such that minimal resistance results in a conductor (e.g., hybrid bond 111) between the memory bank 110 in the memory chip 101 and the fast memory circuit 107 on the memory chip 101. As shown in Figure 1(a), the logic circuit 106 can also be modularized and arranged as modular logic circuits 106-1, 106-2, ..., 106-n, each associated with a corresponding one of the modular fast memory circuits 107-1, 107-2, ..., 107-n via closely spaced and low-resistivity interconnecting conductors 112-1, 112-2, ..., 112-n, which support the operation of their respective modular logic circuits. The modular logic circuits 106-1, 106-2, ..., 106-n can be any suitable logic circuit, such as multiplexers, adders, multipliers, Boolean logic units, RISC processors, math coprocessors, and FPGAs. Such modular logic circuits 106 operate together with their associated modular fast memory circuits 107, forming what is sometimes called "in-memory computing" elements. In-memory computing elements provide the computational operations that dominate neural networks, which are widely used in many machine learning, classification, and other artificial intelligence (AI) applications. In one embodiment, the computational complexity required for each logic circuit 106 may be sufficient to require the implementation of an embedded processor (e.g., a RISC processor, a math coprocessor, or a microcontroller).

[0050] As shown in Figure 1(a), additional control circuitry and data paths, generally designated as control and data circuitry 108, may also be provided. The control and data circuitry 108, logic circuitry 106, fast memory circuitry 107, and bonding pads via hybrid bonds 111, along with the circuitry on the memory chip 101, are interconnected on the companion chip 102 via various interconnect conductors 112, 113, and 114 and interconnect structures 105. The companion chip 102 communicates with the host processor 103 or controller via input and output interfaces 109. The host processor 103 or controller may be provided on a separate integrated circuit. The memory interface 104 may be an industry-standard memory interface (e.g., DDR4, DDR5, or PCIe), a through-silicon via (TSV), microbumps, direct interconnects, or a second set of hybrid bonds.

[0051] In this embodiment, the three-dimensional memory array in memory chip 101 and its associated CuA are organized in modular building blocks, commonly referred to as "circuit blocks," which are arranged in a two-dimensional form on a semiconductor substrate. Each circuit block can implement one or more three-dimensional memory arrays, as well as bit lines and word lines for accessing the memory arrays of the circuit block. Because the word lines and bit lines for accessing the three-dimensional memory arrays of the circuit block are provided within the circuit block, their necessary short lengths are much smaller than the impedance they would cause if they were wired long distances on the semiconductor die. The smaller impedance contributes to lower read and write latency for the memory cells in the memory array. In earlier circuit block implementations, control circuitry, including drivers, decoders, and multiplexers, was provided in the CuA beneath the memory array of the circuit block. However, as described above, a portion of the control circuitry (e.g., sense amplifiers, registers, and data latches) is provided in the accompanying chip 102, thereby significantly reducing the area required for the CuA of the circuit block. In this embodiment, the reduced area required to implement the CuA also results in a smaller circuit block.

[0052] Furthermore, circuit blocks can be organized into memory banks, each bank having multiple rows of circuit blocks that can be addressed together by the same set of word lines. In one implementation, each row can have 18 circuit blocks, each processing 2 at a time. 10 1 kilobit ("1 kbit") of data input or output to process 2 11 Pages of user data (“2-K bytes”) plus overhead (e.g., providing limited error correction and redundant spare circuitry block capabilities). Some control structures (e.g., column or bit-line decoders) can be shared between groups of multiple memory banks (“memory banks”). In one implementation, each memory bank group can be configured to have 2, 4, 8, or 16 memory banks.

[0053] In Figure 1(a), modular logic circuits 106-1, 106-2, ..., 106-n each directly access a corresponding fast memory circuit 107-1, 107-2, ..., 107-n via one of the interconnect conductors 112-1, 112-2, ..., 112-n. Depending on the computational needs of the application, such as the computational power requirements of modular logic circuit 106, or the nature of the data to be stored in fast memory circuit 107, other organization may be more efficient. For example, Figure 1(b) shows an organization in which modular logic circuits 106-1, 106-2, ..., 106-(n-1) each directly access two corresponding fast memory circuits 107-1, 107-2, ..., 107-n via two of the interconnect conductors 112-1, 112-2, ..., 112-n. Alternatively, in Figure 1(c), a single modular logic circuit 106 is provided to directly access each of the fast memory circuits 107-1, 107-2, ..., 107-n via interconnect conductors 112 and interconnect structures 105. Of course, Figures 1(a)-1(c) The alternative configurations are not exhaustive; many changes and modifications can be made depending on the needs of the application.

[0054] Figure 1(d) shows a functional representation of one of the fast memory circuits 107 according to an embodiment of the present invention. Figure 1(d) shows a sense amplifier 150, which represents a sensed data value retrieved from a corresponding bank in the quasi-volatile memory of the memory chip 101 via hybrid bonding. In each read activation cycle, each bank in the memory chip 101 delivers a fixed number of bits (e.g., 1024 bits) from each circuit block to the sense amplifier 150. The data value is latched into a master-slave register 151, which allows active data to be stored in the slave latch of the master-slave register 151, while the master latch of the master-slave register 151 can be used to receive the next active data value. The multiplexer 152 then selects a predetermined number of bits from the slave latches and places the selected bits on a compute data bus 154, including a true bus 154a and a complement bus 154b, representing each bit in true and complement form. Each bit on the computational data bus 154 and its complement appear on the true and complement bit lines of a memory cell in the fast memory array 153 (e.g., an SRAM array). For example, from the perspective of the host processor 103, the fast memory array 153 maps to the address space of the quasi-volatile memory. (As described below...) Figure 5(a) and 5(b) As discussed, SRAM array 153 may reside in a reserved portion of the address space if needed. When enabled, word line 155 writes data from compute data bus 154 into the corresponding memory cell of fast memory array 153.

[0055] Memory array 153 can be used as a bitwise multiplier (without carry) that multiplies a first operand represented by bits on word line 155 with a second operand represented by selected bits from a slave latch in master-slave register 151. For example, in a matrix multiplication operation, the selected bits from the slave latch can represent elements in a row (or a portion of a row) of a matrix, and the bits on the word line can represent columns (or a portion of columns) of a matrix. During operation in multiplier mode, the enable bit of word line 155 writes the corresponding bit of the second operand to its corresponding memory cell, while the disable bit in word line 155 triggers a reset signal each time, causing a zero value to be written to the corresponding memory cell. The result stored in fast memory array 153 constitutes the product term of the multiplication operation. Adders and carry circuits in computational circuitry 106 (e.g., arithmetic and logic circuits 106-1, 106-2, ..., 106-4) can provide the sum of the product terms to complete the multiplication operation. The result of the multiplication operation can then be written back from the compute bus 154 to the fast memory array 153. The multiplier pattern is particularly advantageous in applications that heavily utilize matrix multiplication, such as many AI applications.

[0056] Figure 1(e) illustrates the functional organization of memory bank groups BG(0)-BG(3) in an accompanying chip 102 according to an embodiment of the present invention. As shown in Figure 1(e), the memory bank 180 of the fast memory circuit 107 on the accompanying chip 102 can be organized into portions 180a and 180b, which share data path control and input / output interface circuitry 181, representing... Figures 1(a)-1(c)Each of these includes control and data circuitry 108 and input and output interfaces 109. Section 180a includes memory banks BG(0) and BG(1), while section 180b includes memory banks BG(2) and BG(3), such that sections 180a and 180b together present four memory banks. In this embodiment, the fast memory circuitry 107 on the accompanying chip 102 can serve 64 Gbits of quasi-volatile memory cells on the memory chip 101. Each memory bank is divided into two half-memory banks, as shown in FIG1(e) as half-memory banks 182-1a, 182-2a, 182-3a, 182-4a, 182-1b, 182-2b, 182-3b, and 182-4b. Specifically, half-memory groups 182-1a and 182-3a form memory group BG[0], half-memory groups 182-2a and 182-4a form memory group BG[1], half-memory groups 182-1b and 182-3b form memory group BG[2], and half-memory groups 182-2b and 182-4b form memory group BG[3]. A general purpose input / output bus, GIO bus 184 (indicated by GIO buses 184A and 184b in FIG. 1(e)) allows access from input and output interface 109. Furthermore, for data transfer between memory banks, such as for computation using arithmetic and logic circuitry 106, a 256-bit internal data bus DIO 183 is provided (represented in FIG. 1(e) by 128-bit half-buses 183-1a and 183-2a in portion 180a and 128-bit half-buses 183-1b and 183-2b in portion 180b, respectively). In this embodiment, each half-memory bank may include four 8-circuit-block-wide half-memory banks, each half-memory bank having 4-8 megabits of fast memory cells. In this embodiment, the GIO bus 183 delivers one page of data (2 KB) to the host processor 103 per cycle via input and output interface 109 using an industry-standard bus protocol (e.g., DDR5).

[0057] Figure 1(f) illustrates four half-memory banks 188a-188d of a half-memory bank group (e.g., half-memory bank group 182-1a of Figure 1(e)) on an accompanying chip 102, and corresponding half-memory banks 189a-189d of the quasi-volatile memory circuitry in memory chip 101, according to an embodiment of the present invention. As shown in Figure 1(f), each of the half-memory banks 188a-188d is bordered on one side by a sense amplifier section (e.g., sense amplifier section 190-1) and on the other side by an arithmetic and logic circuit section (e.g., arithmetic and logic section 191-1). Each sense amplifier serves data retrieved from the corresponding quasi-volatile memory cell half-memory bank (e.g., half-memory bank 189a) in memory chip 101 via a hybrid bond or microbump. In one embodiment, a sense amplifier for 4096 bits of user data is provided in each half-memory bank. In addition to allowing host access from input and output interface 109, GIO bus 184 also allows read and write operations between each half-bank of the fast memory circuit accompanying chip 102 and its corresponding half-bank of the quasi-volatile memory circuit on memory chip 101. In this way, the fast memory circuit can be used as a cache for the corresponding quasi-volatile memory circuit, or independently for storing frequently accessed data (“hot data”; for example, data that is more than ten times more frequently accessed than the data stored in the quasi-volatile memory circuit), or for storing configuration or control data (“metadata”) of the corresponding quasi-volatile memory circuit. Such metadata improves the performance and reliability of the quasi-volatile memory circuit.

[0058] Figure 1(g) illustrates four half-memory banks 187a-187d of a half-memory bank group (e.g., half-memory bank group 182-1a of Figure 1(e)) on an accompanying chip 102, and corresponding half-memory banks 189a-189d of the quasi-volatile memory circuitry in memory chip 101, according to an alternative embodiment of the invention. As shown in Figure 1(g), unlike the embodiment of Figure 1(f), half-memory banks 187a-187d do not have the same construction. A sense amplifier in each of the half-memory banks 187a-187d is provided as a sense amplifier portion on both sides of each half-memory bank (e.g., sense amplifier portions 190-1a and 190-1b in half-memory bank 187a). Instead of providing arithmetic and logic circuitry in each half-memory bank, the arithmetic and logic circuitry is concentrated in half-memory banks 187-d. Apart from its configuration, this alternative embodiment operates in the same manner as described above in conjunction with the embodiment of Figure 1(f). For some applications, this alternative embodiment may provide performance comparable to or better than that of the embodiment in FIG1(f). For other applications, the embodiment in FIG1(f) may provide better performance than that of the embodiment in FIG1(g).

[0059] like Figure 1(e) and 1(f) As shown, each half-memory cell within each half-memory cell group (e.g., half-memory cell 188a of half-memory cell group 182-1a) is provided with access to a computation bus (typically indicated by computation bus 184), which is a shared bus between the sense amplifier section, the fast memory circuitry, and the arithmetic and logic circuitry. In one embodiment, the computation data bus is 256 bits wide per block column, and each half-memory cell group is eight block wide. Of course, the widths of the computation data bus and the half-memory cell groups may vary depending on the requirements of the intended application. Thus, significant on-chip data bandwidth is provided within the half-memory cell groups for data transfer between the sense amplifier section (which transmits data read from quasi-volatile memory circuitry in the memory chip), the fast memory circuitry, and the arithmetic and logic circuitry. In this way, large amounts of data can be streamed into the fast memory circuitry as operands for arithmetic and logical operations with other operands, which are other data already stored in the fast memory circuitry or the quasi-volatile memory circuitry, or previous calculation results. For example, in AI applications, data can be stored in quasi-volatile memory and output via a sensed amplification section during read operations. This data, along with weights stored in fast memory circuitry, is then used to perform matrix multiplication, for example, using on-chip arithmetic and logic circuitry and a computation bus. This contrasts sharply with prior art practices that require data to be transferred to or from DRAM to a processor (e.g., a CPU or GPU). In embodiments of the present invention, such computations can be performed without transferring data to or from memory or accompanying chips to the CPU or GPU.

[0060] The computation bus 184 enables the execution of massively parallel computational operations (“in-memory computations”) without operand fetching and the resulting memory operations involving the host interface bus. In this embodiment, since each memory bank group comprises four memory banks, four sets of in-memory computations can be performed in parallel within each memory bank group. Each circuit block column can be configured for computations in the same or different memory banks as other circuit block columns. The results of these in-memory computations can then be sent to the host via input and output interfaces. Concurrently performed in-memory computations can be independent or part of a coordinated computation (i.e., in-memory computations for each memory bank can involve an entire data page). These in-memory computations not only significantly improve power and performance but also make the integrated circuit 120 particularly advantageous for many applications, such as many AI applications previously considered difficult to handle. For example, neural networks can be implemented using in-memory computations, using input data fetched from quasi-volatile memory circuitry, along with neuron weights and intermediate results already stored or readily available from fast memory circuitry. As another example, recursive computations (e.g., those involved in recursive neural networks) can also be implemented using in-memory computations. By utilizing quasi-volatile memory (e.g., 64 Gbit) on memory chip 101 and a large number of on-chip fast memory circuits (e.g., 64 Gbit SRAM) on accompanying chip 102, their combination (i.e. integrated circuit 120) achieves performance and computing applications that have been impossible to achieve for existing applications to date.

[0061] The accompanying chip 102 enables the integrated circuit 120 to be essentially a computing platform with high-density (e.g., greater than 64 Gbytes) quasi-volatile or non-volatile memory, offering significantly greater bandwidth compared to conventional high-performance computing platforms using DRAM modules (e.g., HBM modules) connected to the host processor via an interpolator. Figure 1(h) illustrates a functional configuration of a memory chipset 170 based on an organization such as that described above in conjunction with Figure 1(e) according to an embodiment of the invention, comprising 16 memory banks including computing memory banks 170-1, 170-2, ..., 170-16. As shown in Figure 1(h), a representative computing memory bank 170-1 includes representative modular memory circuits 171-1, 171-2, and 171-3 (e.g., SRAM circuits) that constitute a memory bank, such as any of the memory banks discussed above in Figure 1(e). Furthermore, compute memory bank 170-1 also includes representative modular logic circuitry 172, connected to modular memory circuitry 171-1, 171-2, and 171-3 via a local compute bus 173 (e.g., compute bus 154 described above). The number of modular memory circuitry in each memory bank in Figure 1(h) is for illustrative purposes only; any suitable number of modular memory or logic circuitry is possible. The local bus 173 in each of compute memory banks 170-1 to 170-16 has access to a data bus within the memory bank (e.g., GIO bus 184 or DIO bus 183 described above) to allow data transfer between compute memory banks. In this configuration, modular logic circuitry 172 can form any suitable compute circuitry, such as an ALU core, GPU core, or any suitable embedded controller or microprocessor. Modular logic circuitry 172 can be implemented, for example, via an FPGA. In the configuration of Figure 1(h), compute memory 170-1 can form a CPU with a 16MB SRAM cache supporting 16GB of memory provided by quasi-volatile or non-volatile memory of memory chip 101. One advantage of compute memory 170-1 is that it allows modular logic circuitry 172 (e.g., ALU or GPU core) to be very close to fast memory circuitry 171-1, 171-2, and 171-3, facilitated by local compute bus 173. In fact, even greater advantages can be achieved by distributing modular logic circuitry among the modular fast memory circuitry, as shown in Figure 1(i), to provide greater proximity between the modular memory circuitry and the modular logic circuitry. Data transfer between compute memory units can be performed on inter-memory data bus 175.

[0062] As shown in Figure 1(i), each of the computational memory banks 170-1, ..., 170-16 includes modular fast memory circuits 171-1, 171-2, ..., 171-n and modular logic circuits 172-1, 172-2, ..., 172-n. In addition to the local computational bus 173 within the memory bank, modular data buses 174-1, 174-2, ..., 174-n can also be provided, each data bus allowing data transfer between the modular memory circuits and their adjacent modular logic circuits. Therefore, each modular logic circuit can be connected to a neighboring processor core.

[0063] A 16-memory computing platform can be configured to operate in a pipelined manner. For example, a deep neural network may include many layers. In one embodiment, a computing memory bank can be used for each layer of such a deep neural network. The weight matrix of the neurons in that layer of the neural network can be stored in the fast memory circuitry of the computing memory bank. When the layer computation of the neural network is completed, its result is forwarded to the next computing memory bank. The forwarding of data from one computing bank to another can be done synchronously, i.e., at a specified edge of the clock signal. In this way, after an initial delay of the first 16 cycles, the result of the deep neural network can appear in every subsequent cycle. For this type of computation, conventional processors are limited by the total amount of data that can be placed in fast memory circuitry (e.g., SRAM), and must then leave the chip to fetch new data from DRAM.

[0064] Non-Provisional Application IV discloses a logic function that can be implemented using NOR memory strings (e.g., Content-Addressable Memory (CAM)). CAM allows for parallel data searching. Due to the high density achievable in memory chip 101, CAM can be implemented on integrated circuit 120 to enable massively parallel data searching, as disclosed in Non-Provisional Application IV. Figure 1(j) shows circuit 190, in which integrated circuits 120a and 120b—two copies of the aforementioned integrated circuit 120—implement data-intensive in-memory computation and massively parallel searching in CAM, respectively. Both integrated circuits 120a and 120b are controlled by host processor 103 via memory interface 104. For example, integrated circuit 120a can be assigned highly data-intensive computational tasks, such as image classification. The results of the data-intensive computation can be transferred to integrated circuit 120b via memory interface 104 under the control of host processor 103, where massively parallel searching of image data storage banks stored in the CAM circuitry in memory chip 101 can be performed. For the reasons already explained above and in Non-Provisional Application IV, these two operations, both independently and in combination, are expected to provide very fast execution. One can also envision using many copies of integrated circuit 120, some programmed for logic functions, while the rest implement CAM. In this configuration, the logic function integrated circuits can be programmed to perform various computational tasks in parallel or in one or more pipelines, with the results provided via one or more high-bandwidth memory interface buses for parallel searching.

[0065] Figure 2(a) generally illustrates a flip-chip or face-to-face bonded memory chip 101 and an accompanying chip 102 of integrated circuit 120. In the embodiment of Figure 2(a), the accompanying chip 102, instead of the memory chip 101, implements a sense amplifier (shown in Figure 2(a) as some of circuit elements 208-1 to 208-n) to support the operation of the quasi-volatile or non-volatile memory array 202 of the memory chip 101. The memory chip 101 also implements bit-line control logic circuitry (shown in Figure 2(a) as some of circuit elements 208-1, ..., 208-n) on or near the surface 212 of the substrate 211. The accompanying chip 102 may also route an external high-voltage signal (not shown) from CuA of the memory chip 101 to provide the array 202 of quasi-volatile or non-volatile memory cells in the memory chip 101. For example, a high-voltage bit line select (BLSEL) transistor is provided in the CuA of memory chip 101. Each transistor multiplexes multiple bit line signals of the quasi-volatile memory array 202 onto a bit line interior (BLI) node, which is then routed as an input signal to a corresponding sense amplifier on the accompanying chip 102 via a hybrid key. In the accompanying chip 102, the BLI node is connected to the input of the sense amplifier via a conductor-filled via (represented by via 215 in FIG. 2(a), represented by via 217 and circuit element 208-2 in FIG. 2(a), respectively.

[0066] The sense amplifier and its associated data latch are formed on the companion chip 102 using advanced manufacturing process nodes optimized to CMOS logic technology, and are not exposed to thermal cycling during the formation of the quasi-volatile memory array of the memory chip 101, thus avoiding performance degradation due to thermal cycling. Since the additional capacitance of the BLI node is very small (e.g., less than 2%), this capacitance has no substantial impact on the performance or operation of the sense amplifier. In this configuration, the CuA on the memory chip 101 implements the high-voltage word line and bit line decoders, drivers, and multiplexers. As a result, the "division of labor" between the memory chip 101 and the companion chip 102 not only reduces the area requirement of the CuA on the memory chip 101, but also significantly reduces the number of hybrid bonds required to route bit line signals to the companion chip 102 through signal multiplexing via the BLI node. This contrasts sharply with the use of hybrid bonds to route bit line signals as taught by Lu discussed above. In this embodiment, instead of requiring approximately 20,000 hybrid bonds per circuit block (as taught in Lu) in a non-reused manner, approximately 1,000 hybrid bonds per circuit block are needed to route bitline signals to the companion chip 102, while enjoying the advantage of high signal integrity due to the absence of exposure of high-performance, low-voltage circuitry (e.g., sense amplifiers) to thermal cycling during the fabrication of the quasi-volatile memory array. This significant reduction in the number of hybrid bonds required to route signals to the companion chip 102 essentially frees up a significant number of routing paths in the metal interconnect layer of the companion chip 102. The absence of high-performance, low-voltage logic circuitry in the memory chip 101 also reduces the number of masking steps required in fabricating the memory chip 101, resulting in a simpler fabrication process (i.e., higher yield) and lower wafer handling costs when fabricating the memory chip 101.

[0067] The proximity of the sense amplifier, high-performance, low-voltage fast memory circuitry 107, and logic circuitry 106 of the memory array 202 with memory chip 101 on the accompanying chip 102 provides the following advantages: (i) it allows these circuits to be fabricated in a process optimized for their performance; (ii) it avoids the power-consuming and time-consuming computational operations of carrying data from memory chip 101 to the accompanying chip 102 and back to memory chip 101; (iii) it provides greater noise immunity from the high-voltage circuitry still residing on memory chip 101, resulting in greater sensing sensitivity; (iv) it utilizes the fast memory circuitry and sense amplifier in the accompanying chip to perform write operations in parallel in the quasi-volatile memory circuitry (i.e., serving read operations from the fast memory circuitry while performing write operations involving data on the same page in parallel in the quasi-volatile memory circuitry); and (v) it utilizes the fast memory circuitry and sense amplifier to monitor the health of the quasi-volatile memory circuitry, thereby improving the reliability and durability of the quasi-volatile memory circuitry.

[0068] In one embodiment, memory chip 101 has a storage capacity of 64-Gbits in a three-dimensional quasi-volatile memory array, which is divided into 1024 circuit blocks, each with 64Mbits of random access quasi-volatile memory cells, the supporting circuitry of which is located in CuA (except for the sense amplifier). The read latency to a location in the quasi-volatile memory array is approximately 100 nanoseconds, with approximately 10 10Durability of programming and erasing cycles. In this embodiment, each circuit block in memory chip 101 is connected via hybrid bonding to a corresponding one of 1024 SRAM modules on companion chip 102. On companion chip 102, each circuit block has (i) a 64 K-bit SRAM cell and (ii) a sense amplifier for supporting the quasi-volatile memory cell in the corresponding circuit block of memory chip 101. The read latency to the location in the SRAM cell of the circuit block is approximately 25 nanoseconds, with substantially infinite durability. Using the SRAM modules on companion chip 102 as fast cache memory, uniquely mapped to the quasi-volatile memory array in the corresponding designated circuit block, results in a heterogeneous memory circuit that can provide the best advantages of both memory types, namely, (i) significantly higher density of quasi-volatile memory cells and (ii) significantly faster read access time and significantly higher durability in the SRAM circuit. Therefore, for applications operating on large datasets, relying solely on SRAM circuitry may be too expensive, or relying solely on quasi-volatile memory circuitry may be too slow or lack sufficient durability to support high-frequency, read-intensive, or write-intensive applications. Heterogeneous memory circuitry combining memory types can provide a superior solution. This invention includes circuitry and methods for distributing data between fast memory circuitry (e.g., SRAM) and slower memory circuitry (e.g., quasi-volatile memory), and for moving data between one type of memory circuitry and another without involving the host.

[0069] As shown in Figure 2(a), the memory chip 101 includes an n×m formation of circuit blocks, each circuit block having a CuA structure and an associated array structure. Therefore, Figure 2(a) shows CuA structures 201-(1,1) to 201-(n,m) and array structures 202-(1,1) to 202-(n,m). Each CuA structure may include, for example, various voltage sources and various high- and medium-voltage analog and logic circuits to support its corresponding circuit block. Sequencing and control modules 209-1 to 209-n are provided on one side of this formation of the circuit block, each including a sequencer (Seq) and bit line and word line control circuitry for the memory bank (BCU). As described above, each array structure includes a three-dimensional array of memory cells organized as quasi-volatile or non-volatile NOR memory strings, and a stepped structure allowing electrical access to the common drain region or bit lines of each NOR memory string. Figure 2(b) shows array structures 202-a and 202b in more detail, representing any two adjacent array structures from 202-(1,1) to 202-(n,m). As shown in Figure 2(b), array structures 202-a and 202b each include a memory cell array (for example, arrays 251a and 251b, respectively) and steps on their opposite sides (for example, steps 252a and 252b). Figure 2(b) also shows that signals from CuA of memory chip 101 are routed to hybrid bonds 253a and 253b on interconnect conductor layer 256 through vias 254a and 254b filled with conductors, wherein segments 256a and 256b of interconnect conductor layer overlap steps 252a and 252b, respectively.

[0070] Memory chip 101 and companion chip 102 are bonded by stripes 203-1 to 203-n of hybrid bonds, each stripe extending along the word line (WL) direction and positioned above the space between memory cell arrays of adjacent array structures, overlapping their respective steps. These hybrid bonds connect signals propagating "vertically" (i.e., substantially perpendicular to the semiconductor substrate surface) through vias filled with conductors. In one embodiment, signals connected between the memory chip and companion chip via hybrid bonds are multiplexed and demultiplexed if needed to share and increase the effective number of interconnects connected via hybrid bonds and overcome the density limitations of current hybrid bond technology. Figure 2(a) also shows metal layers 204-207 in companion chip 102. Metal layer 204 provides an interconnect layer for distributing signals to destinations in memory chip 101 and companion chip 102, including high-voltage signals from CuA in memory chip 101. Metal layer 205 provides a substantial ground layer that shields other circuitry in companion chip 102 from interference from these high-voltage signals. Metal layer 206 provides each parallel interconnect conductor (“feed-thru conductor”) extending along the bit line (BL) direction to allow bit line signals to be routed to metal layer 207, the second interconnect network having interconnect conductors extending along the WL direction.

[0071] More specifically, hybrid bonds 203-1 to 203-n connect bit lines of array structures 202-(1,1) to 202-(n,m) in memory chip 101 to sense amplifiers at the surface 212 of substrate 211 in companion chip 102 and between the circuitry in CuA of memory chip 101 and the circuitry at the surface of substrate 211 in companion chip 102. Hybrid bonds 203-1 to 203-n also route high-voltage signals from a voltage source at the surface 212 of the semiconductor substrate in memory chip 101 through metal layer 204 in companion chip 102 to other parts of memory chip 101. Substrate 211 may be a semiconductor wafer thinned after the circuitry of companion chip 102 is formed to an insulating layer (e.g., silicon oxide layer). Alternatively, substrate 211 may be formed by implanting oxygen atoms into a semiconductor wafer after annealing to form an oxide layer. After the circuitry of companion chip 102 is formed at surface 212, substrate 211 may be mechanically separated from the semiconductor wafer. The substrate 211 is referred to as a silicon-on-insulator (SOI) substrate. Bonding pads 210-1 to 210-n can then be formed on the cleaved surface 213.

[0072] Figure 2(a) also shows bonding pads 210-1 to 210-n on surface 213 of substrate 211 opposite surface 212, where circuit elements 208-1 to 208-n are formed. Bonding pads 210-1 and 210-n are each provided to allow access to signals from the circuit formed on surface 212 of substrate 211 via TSVs, such as TSVs 214-1 to 214-n shown in Figure 2(a). Bonding pads 210-1 to 210-n can allow wafer-level or chip-level bonding to another substrate. Suitable bonding techniques can be hybrid bonding, direct interconnect, or microbump bonding. In Figure 2(a), for illustrative purposes, bonding pad 210-n is represented by a bonding pad suitable for hybrid bonding. Bonding pad 210-1 is represented by a microbump suitable for microbump bonding.

[0073] Figure 2(c) generally illustrates a hybrid-bonded memory chip 101 and a companion chip 102 of an integrated circuit 120 according to another embodiment of the present invention; in this embodiment, the memory chip 101 and the companion chip 102 are bonded in a “stacked” orientation. As shown in Figure 2(c), the memory chip 101 and the companion chip 102 each contain substantially the same circuitry as described above in conjunction with Figure 2(a), except that bonding pads (or microbumps for microbump bonding, as appropriate) for hybrid bonding are formed on the “back side” of the substrate 211. This is achieved, for example, by fabricating the companion chip 102 on an SOI substrate that has been sufficiently thinned (e.g., to 3 micrometers or less). Connectors (e.g., bonding pads or microbumps) are then formed on surface 213 of the substrate 211 to mate with corresponding connectors on the memory chip 101 via hybrid bonding (or microbump bonding). Connectors on surface 213 of substrate 211 are connected to circuitry on surface 212 via miniaturized high-density TSVs through conductor-filled vias passing through substrate 211. Compared to the "flip-chip" embodiment shown in FIG. 2(a), this embodiment has the advantage of significantly simplifying or substantially avoiding the complexity of signal routing (e.g., "feedthrough" routing) in metal layers 204, 205, 206, and 207.

[0074] In Figures 2(a) and 2(c), memory chip 101 implements an HNOR memory string array. The invention can also be practiced with memory chip 101 implementing a quasi-volatile or non-volatile VNOR memory string array. For example, various embodiments of VNOR memory string arrays are described in Non-Provisional Application III. Figure 2(d) generally illustrates a hybrid-bonded memory chip 101 and accompanying chip 102 of an integrated circuit 120 according to a third embodiment of the invention; in this third embodiment, memory chip 101 includes a VNOR memory string array. As shown in Figure 2(d), rows 220 in a circuit block of one or more quasi-volatile or non-volatile VNOR memory string arrays include memory string pairs 228-1, 228-2, ..., and 228-n, with two VNOR memory strings formed on opposite sides of each memory string pair.

[0075] As shown in Figure 2(d), the VNOR memory strings in each memory string pair share a common source line and a common bit line, indicated in Figure 2(d) by bit line (BL) 222-1, 222-2, ..., and 222-n, and source lines (SL) 223-1, 223-2, ..., and 223-n. Each of the memory string pairs 228-1, 228-2, ..., and 228-n forms two channel regions on either side of the common bit line and common source line, each channel region being isolated from the word line conductor stack by a charge trapping layer. In Figure 2(d), a word line conductor stack is represented by word line conductors 221-1, 221.20, ..., and 221-m. Across row 220, the common source line and common bit line of the memory string pair alternate between the beginning and end of row 220. A pair of conductors (“global bit lines”) 224-1 and 224-2 connect the common bit lines of row 220 before and after the memory string pairs 228-1, 228-2, ..., 228-n. In this embodiment, each of the common source lines 223-1, 223-2, ..., 223-n is precharged by a voltage applied to an associated one of the common bit lines 222-1, ..., 222-n or by a voltage source hard-wired (not shown) to CuA in the memory chip 101, as described in Non-Provisional Application III.

[0076] Bit line selector circuit 225, each connected to a global bit line of a multi-row VNOR memory string in the circuit block, is provided in CuA below the VNOR memory string array to select a signal from one of the global bit lines 224-1 and 224-2 in the circuit block. Bit line selector circuit 225 performs essentially the same function as a multiplexer, which selects from the bit line signals to provide the selected bit line signal to the BLI described above in conjunction with FIG. 2(a). In this embodiment of FIG. 2(d), the selected signal is provided to the bit line signal BLI represented by conductor-filled via 226, which is connected at the bonding surface of memory chip 101 to one of the bonding pads (or microbumps) 227. The bonding pads (or microbumps) 227 are connected to corresponding bonding pads (or microbumps) in the accompanying chip 102 via hybrid bonding (or microbump bonding) in essentially the same manner as described above in conjunction with FIG. 2(a).

[0077] Figure 2(e) generally illustrates a hybrid bonding memory chip 101 and an accompanying chip 102 of an integrated circuit 120 according to a fourth embodiment of the present invention; in this fourth embodiment, the memory chip 101 includes a VNOR memory string array and a vertical thin-film transistor (TFT) serving as an additional layer for bit line selection circuitry. In Figure 2(e), an additional conductor layer of global bit lines, represented by global bit lines 224-1 and 224-2, is provided in a metal layer (“global bit line layer 224”) above the VNOR memory string array. In this embodiment, these additional global bit lines are not connected to the bonding pad 227 via a bit line selector circuit in the CuA of the memory chip, but rather via a vertical TFT, represented by a vertical TFT 229 formed above the global bit line layer 224 in Figure 2(e). A vertical TFT for bit line selection is described in the provisional application. The presence of source line selection circuitry 230 and bit line selection circuitry in the vertical TFT 229 allows for greater flexibility in routing bit line signals to the sense amplifier in the companion chip 102 via the BLI node. This reduces the number of hybrid bonds required for this routing, thereby reducing the footprint of the memory chip 101 and the companion chip 102, resulting in the advantage of denser circuitry. The vertical TFT can also be used in HNOR memory string arrays to efficiently select bit lines and route them to the companion chip 102.

[0078] Figure 3 A portion of the integrated circuit 120 in Figure 2(a) is shown in more detail. (As shown) Figure 3As shown, the mixed-bond stripes 203-1, 203-2, and 203-3 are arranged adjacent to array structures 202-(1,1) and 202-(2,1), representing any two adjacent array structures 202-(1,1) to 202-(n,m) in FIG. 2(a). Some signals connected by the mixed-bond stripes 203-1, 203-2, and 203-3 are routed vertically through openings in the metal layers 204-207 accompanying the chip 102 via vias filled with conductors to the circuitry at the surface 212 of the substrate 211. Other signals are fanned out by the feedthrough metal layer 206. As described above, the metal layer 204 also allows high-voltage signals to be routed back to the memory chip 101, as shown in signal path 302 connecting signals in the conductors of the metal layer 204 to the array structure 202-(1,1). Figure 3 Regions 301-1, 301-2, and 301-3 are also shown; they are projections of stripes 203-1, 203-2, and 203-3 onto the semiconductor substrate accompanying chip 102. The gaps (“pocket regions”) between adjacent pairs of regions 301-1, 301-2, and 301-3 are relatively large areas on the semiconductor substrate accompanying chip 102.

[0079] Figure 4 A top view of the accompanying chip 102 is shown, illustrating the mixed-bond stripes 203 and the metal layer 206. (See attached image.) Figure 4 As shown, stripe 203 includes hybrid bonds 503. Some of the hybrid bonds 503 are used for routing to BLI nodes in the accompanying chip 102 via vias (“BLI vias”) 215 filled with conductors. Signals routed on metal layers 204-207 must be routed around the BLI vias 215 (i.e., “feedthrough” routing), as shown by conductor 505 on metal layer 206, which is considered to be “jogging” around the two BLI node vias 215. Figure 4 Not shown, the signal lines in metal layer 204 are provided for routing high-voltage signals. Each high-voltage signal is routed by a conductor between two grounding conductors on the same metal layer (i.e., metal layer 204), which provides additional shielding (not shown in addition to the ground plane in metal layer 205). Figure 4 (As shown in the figure). Interconnect conductor 501 is an interconnect conductor in feedthrough metal layer 206.

[0080] According to one embodiment of the invention, the pocket region can be used to enable integrated circuit 120 to have capabilities previously unavailable for memory circuitry. For example, according to one embodiment of the invention, FIG5(a) shows a circuit at surface 212 of substrate 211 of accompanying chip 102. FIG5(a) shows representative circuit module groups 510a and 510b in the circuit at surface 212 of substrate 211 of accompanying chip 102, separated by a region (“pad region”) providing input and output interfaces for integrated circuit 120 (e.g., data input and output buses for communication with host processor 103). Each of circuit module groups 510a and 510b comprises a two-dimensional array of circuit modules, wherein each column of circuit modules (i.e., along the WL direction) occupies a pocket region between adjacent stripes of mixed bonds. In FIG5(a), each of circuit module groups 510a and 510b includes circuit module types 521 and 522. Circuit module type 521 may be a circuit module that includes volatile memory circuitry (e.g., an SRAM array). Circuit module type 522 includes column decoder circuits (i.e., through locally related specific circuit blocks) that serve adjacent memory banks in the same column of volatile memory circuits and quasi-volatile memory cells in the corresponding array structure in memory chip 101.

[0081] Figure 5(a) also shows variations 531 and 532 of the type 521 circuit module. Each of variations 531 and 532 includes one or more SRAM arrays 541 and a sense amplifier and data latch circuitry 543. The sense amplifier and data latch can be shared among multiple memory cells in the memory array using a multiplexer. Variation 531 implements a single-port SRAM array, while variation 532 implements a dual-port SRAM array.

[0082] In one embodiment, all SRAM arrays 541 in the accompanying chip 102 may occupy a different address space than the quasi-volatile memory cells in the memory chip 101, as shown in address space mapping 550. In address space mapping 550, the SRAM arrays 541 are mapped to lower addresses, while the quasi-volatile memory cells in the memory chip 101 are mapped to higher addresses. Therefore, the quasi-volatile memory cells and the SRAM arrays 541 together form an extended address space, integrating and sharing data lines within the same memory cell. The extended address space enables read and write operations from the SRAM arrays 541 during programming, erasing, or refresh operations in the quasi-volatile memory circuitry.

[0083] Optionally, the circuit module can also additionally implement arithmetic and logic circuitry 544 (e.g., adders, multipliers, dividers, subtractors, RISC processors, math coprocessors, and logic gates such as XOR). Circuit modules with both SRAM arrays and arithmetic logic circuitry are particularly well-suited for implementing in-memory and near-memory computations required in many applications, such as machine learning, classification, neural networks, and other AI applications. Significantly higher performance is achieved compared to traditional processor architectures because of the much higher bandwidth between the SRAM array 541 and the arithmetic and logic circuitry 544—that is, data retrieved from and written back to memory is routed between memory and processing units via on-chip signal routing, without the limited bandwidth of a traditional memory interface bus (“Von Neumann bottleneck”). Utilizing battery or capacity backup power, the SRAM array retains its data even during power loss, allowing unrestricted access to the same data without conflicting with the need to perform refresh operations, which is particularly suitable for storing system data as well as application and operating system software. Furthermore, the large storage capacity of quasi-volatile memory circuitry and fast SRAM circuitry can be used to perform recursive computation operations for training in AI applications. Furthermore, the quasi-volatile memory circuit can be part of a larger memory having quasi-volatile and non-volatile memory portions, wherein the non-volatile memory portions store weights that do not change frequently.

[0084] Alternatively, each of the SRAM arrays 541 can be used as a cache for quasi-volatile memory cells in a corresponding array structure within a corresponding memory bank. Since the memory chip 101 and the accompanying chip 102 are interconnected via a hybrid key, this hybrid key can be organized to provide a high-bandwidth internal data bus (e.g., a 256-bit or 1024-bit wide bus per circuit block) between the corresponding quasi-memory circuitry of the memory chip 101 and the SRAM arrays in the accompanying chip 102. To implement the cache functionality, circuitry can be provided in each circuit module to directly transfer data from the memory bank to the corresponding SRAM array (e.g., one page at a time) via these high-bandwidth internal data buses. In one embodiment, each SRAM array has a 64k-bit storage capacity and is used as a 64M-bit cache for the quasi-volatile memory circuitry. In this embodiment, a row of 16 circuit blocks (plus overhead) is activated together to provide 2-K byte pages loaded or written together. In this way, a single activation at the corresponding quasi-volatile memory bank prefetches data pages (after readout at the sense amplifier) ​​into the SRAM array 541. If the host processor 103 accesses data with a conventional cache line size (e.g., 64 bytes) and has reference locality, each prefetch can serve many read accesses. If the SRAM array 541 holds multiple pages of the corresponding quasi-volatile memory bank in the memory chip 101, the effective read latency of the integrated circuit 120—averaging the activation time of the quasi-volatile memory bank over many host accesses—is close to the read latency of the SRAM array. The activation time of the SRAM bank (e.g., 2 ns or less) is very short relative to the activation time of the corresponding quasi-volatile memory circuitry. Furthermore, write operations can be deferred until pages of the quasi-volatile memory bank cached in the SRAM array 541 need to be swapped out or “evict”.

[0085] Since writing or erasing pages once is preferred and sometimes necessary in quasi-volatile memory arrays, this delayed writing of cached data from the SRAM array 541 is particularly advantageous from a performance and endurance point of view. From a performance point of view, amortizing the write access time of the quasi-volatile memory bank over many host computer accesses provides integrated circuit 120 with performance similar to SRAM circuitry. As a result, by utilizing the multi-page cache in the SRAM array 541, the combined performance of volatile and quasi-volatile memory is effectively the performance of the SRAM memory circuitry. Furthermore, since the SRAM array dissipates minimal power when not actively reading or writing, integrated circuit 120, which incorporates both SRAM and quasi-volatile memory circuitry, is highly energy-efficient. This combination of SRAM and quasi-volatile memory circuitry reduces power consumption because fewer read, write, and erase operations are performed on the quasi-volatile memory circuitry. By performing fewer read, write, and erase operations on the quasi-volatile memory circuitry, the frequencies of erase suppression interference, write suppression interference, and read interference in the quasi-volatile memory are correspondingly reduced. Furthermore, greater durability is achieved because the quasi-volatile memory cells are significantly less exposed to high-voltage electric field stress during write and erase operations.

[0086] As described in non-provisional applications I and II, quasi-volatile memory circuitry requires refresh operations to retain data beyond its retention time (e.g., minutes). Naturally, a "refresh conflict" occurs when a data read operation is performed on a page at the time the refresh operation for that memory cell expires. Those skilled in the art will understand that refresh conflicts (e.g., those occurring in DRAM) are sometimes resolved by delaying the read operation until the refresh operation is complete. Therefore, refresh conflicts are an overhead that adversely affects memory performance. However, by using an SRAM array as a cache for the corresponding quasi-volatile memory array in the memory circuitry, read operations can be serviced outside the SRAM cache without requiring access to the quasi-volatile memory circuitry, thus substantially avoiding most refresh conflicts. Since the retention time of quasi-volatile memory circuitry is already relatively long than that of DRAM, the effective performance that can be achieved using an SRAM cache combined with quasi-volatile memory, as provided by this invention, may exceed that of conventional memory systems such as DRAM.

[0087] Prior art caches primarily comprise fast dedicated memory circuits (e.g., SRAM or SRAM-like circuits), which are independent of the memory circuits that cache data. Typically, such caches have their own data path and address space, thus limiting their ability to operate as another independent storage or memory circuit. However, as shown in FIG1(a), an SRAM array provided on accompanying chip 102 shares a data path and address space with the quasi-volatile memory circuits of memory chip 101. In this arrangement, even when operating as a cache for the quasi-volatile memory circuits in memory chip 101 (i.e., mapped into the quasi-memory circuit address space), the SRAM array can still be used as a fast access memory circuit accessible from the independent SRAM address space discussed above. Furthermore, cache and fast access memory operations can be performed on a shared data path. As described above, access to the host processor 103 is available for both cache access and fast memory access via input and output interface 109 (e.g., an industry-standard DDR5 interface or a high-bandwidth memory (HBM) interface).

[0088] In one embodiment, the high-bandwidth internal data bus used for data transfer between memory chip 101 and companion chip 102 can also be used to transfer data between SRAM arrays in companion chip 102 in a massively parallel manner. This capability is particularly advantageous for computational operations in memory. These internal buses transfer large amounts of data to high-speed logic, RISC processors, math coprocessors, or arithmetic circuit modules on companion chip 102 each execution cycle without involving data movement across input and output interfaces 109. Such an arrangement allows host processor 103 to perform arithmetic or logical operations executed by logic or arithmetic circuit modules on companion chip 102 without data movement across input and output interfaces 109, thus bypassing the well-known “von Neumann bottleneck.”

[0089] In one embodiment, the SRAM array in the accompanying chip 102 is used only as a cache memory for quasi-volatile memory circuitry in a one-to-one correlated cache mode (i.e., addressable memory cells such as "pages" are the same in both quasi-memory arrays as in the SRAM array). However, this approach may not be ideal for some applications. For example, the SRAM array in the accompanying chip 102 may be configured to address on a "page" basis, which can be 2KB, as in some embodiments discussed above. In some operating system software, a page may be defined as 512 bytes or 1KB. As another example, under an industry standard, addressable data cells based on the width of an industry-standard memory interface bus (e.g., 128 bits) may be preferred. In one embodiment, a portion of the SRAM array may be configured to address on a "page-by-page" basis, where the page size is configurable, or any suitable addressable data cell to suit the requirements of the host processor 103, the operating system, or any suitable application. The addressing scheme can be fixed or can be configured by software, firmware, or host commands based on "runtime" (i.e., dynamically), for example by setting the configuration register in the accompanying chip 102.

[0090] Due to the number of high-bandwidth internal data buses available, parallel multi-memory bank operations (whether concurrent or non-concurrent) are possible. While the high-speed arithmetic or logic circuit modules on the accompanying chip 102 transfer large amounts of data for arithmetic and logic operations, the next set of data can be fetched in parallel from the quasi-volatile memory circuitry in memory chip 101 and loaded into the SRAM array in the accompanying chip 102. Organizing the SRAM array and logic and arithmetic circuit modules by rows and columns, parallel computing tasks (e.g., those used in AI applications) can be based on various segments of memory banks (e.g., less than all logic circuit blocks at once), based on columns of circuit blocks, or on multiple memory banks at once. This operation of the SRAM array can be controlled or allocated by firmware or circuitry (e.g., a state machine) on the accompanying chip 102 or by a set of commands issued by the host processor 103.

[0091] In one embodiment, the SRAM array group can be organized into a 256-row by 16-column array of circuit blocks, such that a 256-bit internal data bus is associated with a column of SRAM circuit blocks. In this configuration, 16 parallel 256-bit arithmetic or logical operations can be performed simultaneously for data associated with each memory bank. Furthermore, in one embodiment, the 16 columns can be divided into four memory bank segments, for example, such that the 16 parallel operations are four distinct groups of operations, each group corresponding to a memory bank segment. The SRAM array on the accompanying chip 102 can also be organized into memory bank groups, with each memory bank group having multiple memory banks. Independent and parallel operations can be performed on a memory bank group basis. In this way, the SRAM array in the memory chipset of the present invention can be easily allocated in many possible configurations to simultaneously perform cache operations and in-memory computation operations.

[0092] Figure 5(b) illustrates additional variations 533 and 534 that can be implemented at the surface 212 of the substrate 211 of the accompanying chip 102 for the type 521 circuit module of Figure 5(a) according to an embodiment of the present invention.

[0093] Some or all of the SRAM array 541 may be replaced by an array of eDRAM, MRAM, phase-change memory, resistive random access memory, conductive bridged random access memory, or ferroelectric resistance random access memory, or any suitable combination of these circuits. In other embodiments of the invention, some of these memory arrays may provide comparable results.

[0094] Figure 6 A memory module 600 according to an embodiment of the present invention is generally shown, which can be provided in the format of a dual in-line memory module (DIMM). Figure 6 As shown, the memory module 600 includes controller circuitry 602 and memory chipsets 601-0 to 601-15, each chipset being a memory chip bonded to an accompanying chip (e.g., the integrated circuit described above). The memory module 600 may be mechanically attached to a printed circuit board providing an electrical connection to a host computing system 603 (e.g., via an industry-standard data bus). The host computing system 603 may be any computing system, such as a server and mobile device, or any other suitable computing device (e.g., any telecommunications switch, router, or gene sequencer). Although Figure 6Sixteen memory chipsets are shown, but this number is merely illustrative and not intended to limit the invention. Memory module 600 may include memory chipsets with quasi-volatile memory circuitry. In some embodiments, the chipset may include quasi-volatile memory circuitry, non-volatile circuitry, and circuitry for another memory type (e.g., DRAM). Specific memory configurations can be optimized to suit the expected workload and power requirements of host system 603. Controller circuitry 602 may be provided as a separate integrated circuit. Controller circuitry 602 may be a conventional memory controller or may be specific to the operation of a chipset having quasi-volatile memory circuitry with computational or mathematical operation capabilities on the chipset.

[0095] According to one embodiment of the present invention, Figure 7 An integrated circuit 160 is shown, which includes a non-memory chip 161 and a memory chipset (e.g., the integrated circuit 120 above, which includes a memory chip 101 and an accompanying chip 102). The non-memory chip 161 may include one or more CPUs, GPUs, FPGAs, image sensors, baseband and other signal processors, Ethernet and other data communication circuitry, or any other suitable logic circuitry. In integrated circuit 160, integrated circuit 120 and the non-memory chip 161 may be bonded together, and signals between the memory chipset and the non-memory chip 161 are electrically connected using, for example, through-silicon vias (TSVs), which improves signal communication speed and reduces latency between integrated circuit 120 and the non-memory chip 161 during operation. Another embodiment may use another conventional interconnect, bonding, or bumping technology. For example, integrated circuit 120 and the non-memory chip 161 may be configured to use any suitable interface technology (e.g., DDR, HBM, or register-to-register data transfer technology). An interface that implements a register-to-register data transfer protocol can optimize software or hardware performance (e.g., software in an operating system or an application running on a host computer system, or packet switching circuitry in a telecommunications router).

[0096] According to another embodiment of the present invention, such as Figure 8 As shown, integrated circuit 800 includes integrated circuit 120 and non-memory chips 161, which are interconnected via silicon interposers (illustrated by way of interposer 801). Silicon interposer 801 serves as a silicon substrate providing interconnect conductors, similar to a printed circuit board. Silicon interposer 801 can provide electrical connections to additional memory chipsets and additional non-memory chips. Silicon interposer 801 offers the advantage of fast signal communication between interconnected chips while avoiding packaging challenges such as heat dissipation.

[0097] Figure 9This is a schematic representation of a computing system 900. For example, the computing system 900 could be a larger host system (e.g., Figure 6 Subsystems within the host system 603. The computing system 900 can execute specialized applications (e.g., gene sequencing, telecommunications, or automotive and Internet of Things (IoT) applications). The computing system 900 demonstrates that the accompanying chip 102 can be customized and optimized to meet workloads generated by the software application 903, the operating system 902, and the firmware 901 of the host processor 103. In the computing system 900, SRAM arrays or other buffered or cached memory circuitry within integrated circuits 120 associated with the quasi-volatile or non-volatile memory bank 110 of the memory chip 101 can be managed and configured outside of the integrated circuit group 120. Management optimization, for example, can be achieved through machine learning or digital signal processing techniques.

[0098] Figure 10 This is a schematic representation of memory chipset 125, with memory chipset 170 provided on a battery 1001 or capacitor on an accompanying chip 102. Memory chipset 125 is advantageous for applications where the accompanying chip 102 stores system information (e.g., memory management information, including the location of bad blocks, lookup tables, and registers). When power is lost, memory chipset 125 prevents data loss. Battery 1001 holds data in any SRAM array or other volatile memory circuitry on the accompanying chip 102 or memory chip 101. In the event of power loss, battery 1001, firmware on accompanying chip 102, and dedicated quasi-volatile or other non-volatile backup memory on memory chip 101 allow memory chipset 125 to write such system information (e.g., memory management information) to non-volatile memory circuitry. The stored system information can be recovered upon the next power-on.

[0099] One advantage of the SRAM array on chip 102 is power saving. When the host system (e.g., Figure 6 When the host system (603) is idle, standard DDR5 for the memory modules allows refresh operations to be paused. Some embodiments of the invention allow selected quasi-volatile memory blocks to be shut down. When refresh pause is allowed, a user can transfer critical data (e.g., firmware for the memory chipset or metadata about the latest state of the memory circuit blocks) from the quasi-volatile memory circuits to the SRAM array, thereby pausing refresh operations on the quasi-volatile memory circuits to save power. When power is restored, the firmware in the accompanying SRAM array can quickly restart normal operation. Alternatively, refresh operations on all quasi-memory circuits can be stopped except for a selected few circuits. Critical information for resuming operations (e.g., firmware for the memory chipset) can be stored in a selected few blocks that maintain refresh operations.

[0100] According to the present invention, the integrated circuit 120 can support paging schemes in virtual memory systems. Figure 11 A paging system using a fast memory circuit (e.g., SRAM circuit) and a quasi-volatile memory circuit of integrated circuit 120 is illustrated schematically according to an embodiment of the present invention. In one embodiment of the paging scheme, the accompanying chip 102 maintains an appropriate number of blocks of SRAM circuit 1101 (within an appropriate block size, such as 1 byte, 64 bits, 128 bits, 2K bytes, or any suitable addressable unit) based on the requirements of the intended application or operating system to serve the next input data read or write command from the memory chip 101 of the host processor 103 at a specific location associated with the quasi-volatile memory circuit 1157.

[0101] exist Figure 11 In this document, flowchart 1103 is provided to illustrate the operation of the paging system. First, in step 1151, multiple blocks (“memory blocks”) of the SRAM circuitry 1101 are allocated. For this purpose, page tables and appropriate data structures (e.g., “heap,” “stack,” “list,” or any other suitable data structure known to those skilled in the art) can be used to manage or allocate the blocks of the SRAM circuitry. To improve performance, from the perspective of the host processor 103, memory operation control circuitry (e.g., state machine-based control circuitry) and accompanying chip 102 (see [link to documentation]) can be provided in the data path. Figure 1(a)-Figure 1(j) The control circuit 108 of the quasi-volatile memory circuit 1157. Recall that the actual write operation of the quasi-volatile memory circuit 1157 may take up to, for example, 100 nanoseconds, even though data can be read from a copy stored in the SRAM circuit 1101 in a very short time (e.g., 10 nanoseconds). Therefore, the accompanying chip 102 avoids service interruptions to the host processor 103 by scheduling the slower write operations of the quasi-volatile memory circuit 1157 in the background. In particular, it is necessary to allow the memory block holding the data to be written to the quasi-volatile memory circuit 1157 to complete its entire contents write operation to the quasi-volatile memory circuit 1157. This requires having a sufficient number of memory blocks available to serve an appropriate number of read or write commands from the host processor 103 for the next number of inputs.

[0102] In step 1152, the memory operation control circuitry determines the number of unallocated memory blocks, and in step 1153, it determines whether the number of unallocated memory blocks exceeds a threshold. If so, in step 1154, there are enough unallocated memory blocks remaining, and the currently allocated memory blocks are not required to write their contents back to the quasi-volatile memory circuitry 1157 to free up space. Otherwise, in step 1155, the currently allocated memory block is selected based on an "eviction" policy, and its data is either "evictioned" or written back to the corresponding location in the quasi-volatile memory circuitry 1157 within the memory chip 101. For example, a suitable eviction policy could be "least recently accessed" (i.e., no block among all allocated blocks has been read the longest). In step 1156, the data in the selected memory block is written back to the corresponding location in the quasi-volatile memory circuitry 1157 (as indicated in the page table). During this period, the memory operation control circuit monitors the "ready or busy" state of the applicable quasi-volatile memory bank, and when the bank is not busy, the accompanying chip 102 considers the write operation complete and returns to step 1152. Since there are enough unallocated memory blocks to handle read and write access requests from the host processor 103, and multiple incomplete write operations returning to the quasi-volatile memory circuit 1157 may be performed in parallel, read and write requests from the host processor 103 will not be stalled due to incomplete write operations.

[0103] The method illustrated in flowchart 1103 is also applicable to and beneficial for cache operations. Of course, in cache applications, it is typically not necessary to select which memory block to write back.

[0104] While the above detailed description provides HNOR memory string arrays (e.g., those described in Non-Provisional Application II) as primary examples of quasi-volatile and non-volatile memory circuitry on a memory chip, other types of quasi-volatile and non-volatile memory circuitry (e.g., VNOR memory string arrays described in Non-Provisional Application III) can also be used in various embodiments of the invention and achieve the aforementioned advantages. For example, hybrid bonding allows for high-bandwidth interconnects between the VNOR memory array and the SRAM array and computational logic elements in the accompanying chip (e.g., the SRAM array 541 and arithmetic and logic circuitry 544 on the accompanying chip 102 of Figures 5(a) and 5(b)). Regardless of whether an HNOR memory string array or a VNOR memory string array is used to provide the quasi-volatile and non-volatile memory circuitry, sense amplifiers and other high-performance, low-voltage logic circuitry can be implemented on the accompanying chip and electrically connected via hybrid bonding to provide data, taking advantage of the proximity of data from the sense amplifier to the SRAM circuitry and computational logic circuitry.

[0105] The above detailed description is provided to illustrate specific embodiments of the invention and is not intended to be limiting. Many variations and modifications of the invention are possible. For example, SRAM circuits are extensively mentioned or used in this detailed description and drawings to illustrate the invention. However, the invention is also applicable to other fast memory circuits. The use of SRAM circuits herein to illustrate fast memory circuits is not intended to be limiting. The invention is set forth in the appended claims.

Claims

1. An integrated circuit, comprising: A first semiconductor die having a first type of memory circuit formed thereon, wherein the first type of memory circuit has at least one layer of memory cells formed above a substrate layer in the semiconductor die; and A second semiconductor die has arithmetic and logic circuits and a second type of memory circuit formed thereon, wherein the second type of memory circuit has a lower read latency than the first type of memory circuit, and wherein the first type of memory circuit and the second type of memory circuit are interconnected by wafer-level or chip-level bonding formed between the first semiconductor die and the second semiconductor die, wherein the arithmetic and logic circuit accesses the second type of memory circuit via an internal bus. The second type of memory circuit is modularized into a memory module. The memory module includes multiple memory cells, multiple word lines, and multiple bit lines for selecting memory cells for read or write access. The memory module is configured to function as a bit multiplier, receiving a data value represented by a selected set of word lines as a first operand and a second operand represented by the selected set of bit lines. Each assertion value of the first operand on the selected set of word lines causes a product term formed by shifting the second operand to be written to the selected memory cell. Each deassertion value of the first operand on the selected set of word lines causes zero to be written to the selected memory cell. In this embodiment, the memory circuits on the first semiconductor die are each modularized into memory modules, and each memory module in the first semiconductor die is connected by a key to a data output circuit in the second semiconductor die dedicated to the memory module, and the data output circuit provides data from the memory module to an associated memory module in the second semiconductor die.

2. The integrated circuit according to claim 1, wherein, The wafer-level or chip-level bonding includes one of the following: hybrid bonding, direct interconnect bonding, and microbump bonding.

3. The integrated circuit according to claim 1, wherein, The first type of memory circuit includes quasi-volatile memory circuits or non-volatile memory circuits, and the second type of memory circuit includes one or more of the following: static random access memory (SRAM) circuits, dynamic random access memory (DRAM) circuits, magnetic random access memory (MRAM) circuits, phase change memory (PCM), resistive random access memory (RRAM), conductive bridged random access memory (CBRAM), ferroelectric resistive random access memory (FRAM), and carbon nanotube memory.

4. The integrated circuit according to claim 1, wherein, The second type of memory circuit includes one or more of the following: embedded DRAM circuit, embedded MRAM circuit, spin-transfer torque MRAM circuit.

5. The integrated circuit according to claim 1, wherein, The second semiconductor die is manufactured using a manufacturing process optimized for producing CMOS logic circuits.

6. The integrated circuit according to claim 1, wherein, The second semiconductor die also includes a sense amplifier, a register or data latch, and logic circuitry formed thereon.

7. The integrated circuit according to claim 3, wherein, Signals communicating between the first semiconductor die and the second semiconductor die are multiplexed and demultiplexed to share a key interconnect.

8. The integrated circuit according to claim 1, wherein, The integrated circuit also includes multiple internal data buses on the second semiconductor die, each internal data bus providing read and write access to a set of memory modules.

9. The integrated circuit according to claim 8, wherein, The arithmetic and logic circuits are modularized into logic modules.

10. The integrated circuit according to claim 1, wherein, The data output circuit includes a sensing amplifier.

11. The integrated circuit according to claim 1, wherein the first semiconductor die further includes a multiplexer circuit for selecting data from the memory module to be sent to the data output circuit in the second semiconductor die.

12. The integrated circuit of claim 1, further comprising a master-slave register provided in the second semiconductor die between a data output circuit and a memory module, wherein, The slave latch of each master-slave register holds the current data output of the data output circuit, while the master latch of each master-slave register receives the next data output from one of the data output circuits.

13. The integrated circuit according to claim 1, wherein, The data output circuit provides the data via the internal data bus of the associated memory module in the second semiconductor die.

14. The integrated circuit according to claim 1, wherein, Each logic module is associated with one or more memory modules in the second semiconductor die, wherein the logic module accesses the associated memory module via the internal data bus of the associated memory module in the second semiconductor die.

15. The integrated circuit according to claim 1, wherein, Multiple logic modules are provided to sum the product values ​​written to selected memory cells.

16. The integrated circuit according to claim 1, wherein, Each memory module in the second semiconductor die is configured as one of the following: a dual-port memory circuit and a single-port memory circuit.

17. The integrated circuit according to claim 1, wherein, Each logic module in the second semiconductor die is configured as one of the following: an adder circuit, a divider circuit, a Boolean arithmetic unit circuit, a multiplier circuit, a subtractor circuit, a RISC processor, a math coprocessor, and a multiplexer circuit.

18. The integrated circuit of claim 1, further comprising an on-chip control circuit in the second semiconductor die, the on-chip control circuit controlling the operation of the memory and logic module.

19. The integrated circuit according to claim 18, wherein, The on-chip control circuit implements high-speed caching or paging of data from the quasi-volatile or non-volatile memory circuit of the first semiconductor die in the fast memory circuit of the second semiconductor die.

20. The integrated circuit according to claim 19, wherein, Data caching or paging is performed using a block size determined by the page size fixed in the quasi-volatile or non-volatile memory circuitry.

21. The integrated circuit according to claim 20, wherein, Data caching or paging is performed using programmable block sizes.

22. The integrated circuit according to claim 21, wherein, The programmable block size is one of a plurality of block sizes, each block size corresponding to a unit of data transmission in one or more industry-standard memory interface protocols.

23. The integrated circuit according to claim 21, wherein, The size of the programmable block is set by the host processor.

24. The integrated circuit according to claim 19, wherein, The on-chip control circuit allocates the fast memory circuit in blocks, maintaining at least a predetermined number of unallocated blocks.

25. The integrated circuit according to claim 24, wherein, When the number of unallocated blocks is below a predetermined number, the on-chip control circuit selects one of the allocated blocks for data transfer to the quasi-volatile or non-volatile memory circuit.

26. The integrated circuit according to claim 25, wherein, The on-chip control circuitry selects the blocks used for data transmission based on an "eviction" strategy.

27. The integrated circuit according to claim 25, wherein, The on-chip control circuitry monitors data transfers in the quasi-volatile or non-volatile memory circuitry and schedules data transfers in the cache based on whether one or more previous data transfers have been completed.

28. The integrated circuit according to claim 27, wherein, The data transmission is performed in multiples of the programmable block size.

29. The integrated circuit of claim 18, further comprising a memory interface formed in the second semiconductor die, the memory interface being accessible by a host processor, and a data path connecting the first semiconductor die and the memory modules of the semiconductor die, as well as the logic module and the memory interface.

30. The integrated circuit according to claim 29, wherein, The memory interface includes an industry-standard memory interface.

31. The integrated circuit according to claim 29, wherein, The host processor accesses the memory modules in the first semiconductor die and the second semiconductor die through the memory interface using an access scheme based on first and second non-overlapping portions of addresses in the address space, respectively mapping the memory modules in the first semiconductor die and the memory modules in the second semiconductor die to the first and second non-overlapping portions of addresses in the address space.

32. The integrated circuit according to claim 31, wherein, Hot data or metadata is stored at an address in the second part of the address space and hot data or metadata is read from the memory.

33. The integrated circuit according to claim 31, wherein, The host processor requests data from a location in the memory circuitry of the first semiconductor die by presenting an address mapped to a location in the address space, wherein the on-chip control circuitry provides data retrieved from the memory module of the second semiconductor die as the requested data.

34. The integrated circuit according to claim 32, wherein, The memory module that retrieves the data from the second semiconductor die is allocated as a cache memory for use as a memory module in the first semiconductor die to which the address is mapped.

35. The integrated circuit according to claim 33, wherein, The memory modules on the first semiconductor die are organized into a memory storage body, and the memory modules in the second semiconductor die that serve as cache memory for the memory modules in the memory storage body of the first semiconductor die form a corresponding memory storage body in the second semiconductor die.

36. The integrated circuit according to claim 35, wherein, In the two semiconductor dies, each memory module occupies a two-dimensional area on its respective semiconductor die.

37. The integrated circuit according to claim 36, wherein, Each circuit block in the first semiconductor die is mapped one-to-one to a corresponding circuit block in the second semiconductor die, and wherein one or more signals from each circuit block in the first semiconductor die are provided to the circuit at the substrate of the corresponding circuit block via wafer-level or chip-level bonding and through-silicon vias.

38. The integrated circuit according to claim 37, wherein, The one-to-one mapping maps the circuit blocks in the first semiconductor die to the closest circuit block in the second semiconductor die.

39. The integrated circuit according to claim 36, wherein, Each circuit block includes a fast memory circuit section in which fast memory circuitry is placed, and one or more data output sections provided on one or both sides of the fast memory section in which data output circuitry is placed.

40. The integrated circuit according to claim 39, wherein, The fast memory circuit in the fast memory circuit section serves as a high-speed cache for the quasi-volatile or non-volatile memory circuit in the corresponding circuit block of the second semiconductor die.

41. The integrated circuit according to claim 40, wherein, The circuit blocks are arranged in rows and columns on each semiconductor die, and each internal bus on the second semiconductor die provides read and write access to the memory modules in a column of circuit blocks.

42. The integrated circuit according to claim 41, wherein, Each logic module also occupies a circuit block.

43. The integrated circuit according to claim 42, wherein, The circuit blocks are arranged to form one or more computing units, each computing unit comprising a row of a first plurality of consecutive circuit blocks in the second semiconductor die, each row of circuit blocks comprising one of the memory modules; The second semiconductor die comprises a second plurality of consecutive rows of circuit blocks, each row of circuit blocks including a logic module and a plurality of computing data buses, each computing data bus facilitating data transfer between memory module circuit blocks and logic module circuit blocks.

44. The integrated circuit according to claim 43, wherein, For each memory module circuit block, each computing unit also includes a corresponding circuit block in the first semiconductor die, the circuit block including a quasi-volatile or non-volatile memory circuit connected to the memory module circuit block via a data output circuit, such that the computing data bus also facilitates data transmission between the quasi-volatile or non-volatile memory circuit, the fast memory module circuit block and the data output circuit of the logic module circuit block.

45. The integrated circuit according to claim 44, wherein, Each computing unit can be configured to implement a neural network with neurons, each neuron being parameterized by a parameter value matrix stored in the fast memory module circuit block, wherein input data is stored as a vector in the quasi-volatile or non-volatile memory circuit, and wherein the logic module implements matrix multiplication between the parameter value matrix and the vector.

46. ​​The integrated circuit according to claim 43, wherein, Each computing data bus connects the memory module circuit block and the logic module circuit block, the logic module circuit block being a column-aligned circuit block.

47. The integrated circuit according to claim 43, wherein, The computing unit has a first configuration and a second configuration, wherein each logic module circuit block in the first configuration is closer to its nearest fast memory module circuit block than each logic module circuit block in the second configuration is closer to its nearest fast memory module circuit block.

48. The integrated circuit according to claim 43, wherein, Each logic module circuit block includes one of the following: a central processing unit (CPU) core, a graphics processing unit (GPU) core, a field-programmable gate array (FPGA), and an embedded controller.

49. The integrated circuit according to claim 41, wherein, Each memory bank comprises a memory module consisting of circuit blocks spanning multiple rows and columns.

50. The integrated circuit according to claim 49, wherein, Multiple memory banks form a memory bank group, the memory bank group further comprising a first data bus accessible to each of the memory banks within the memory bank group.

51. The integrated circuit according to claim 50, wherein, The integrated circuit includes a plurality of said memory bank groups, wherein the integrated circuit further includes a second data bus accessible by each of the memory bank groups, and wherein each memory bank in each memory bank accesses the second data bus through a first data bus of its own memory bank group.

52. The integrated circuit according to claim 49, wherein, Data associated with memory circuitry in a row of circuit blocks within a memory cell forms a data page, wherein each data transfer between a memory cell in the first semiconductor die and a corresponding memory cell in the second semiconductor die comprises an integer number of data pages.

53. The integrated circuit according to claim 9, wherein, Each logic module operates on data read from a selected group of one or more memory modules on the second semiconductor die via the internal data bus.

54. The integrated circuit according to claim 53, wherein, Each memory module in the second semiconductor die can be configured as a cache memory for a corresponding memory module in the first semiconductor die, or as a data memory accessed by the logic module during operation of the logic module.

55. The integrated circuit according to claim 54, wherein, Data transmitted from the memory module of the first semiconductor die via its dedicated data output circuit is provided on an internal data bus accessible by the corresponding logic module.

56. The integrated circuit of claim 55, further comprising on-chip control circuitry in the second semiconductor die, wherein, The operation of each logic module, data transmission from the memory module of the first semiconductor die, and read and write access to the memory module of the second semiconductor die are controlled by the on-chip control circuit, which executes on a set of prescribed software and firmware commands.

57. The integrated circuit according to claim 55, wherein, Under the control of the host processor, data transfer operations are performed between the memory module of the first semiconductor die and the corresponding memory module of the second semiconductor die through the memory interface.

58. The integrated circuit according to claim 56, wherein, The memory interface includes an industry-standard memory interface.

59. The integrated circuit according to claim 35, wherein, The host processor includes a central processing unit, a graphics processing unit, a field-programmable gate array (FPGA), or a memory controller.

60. The integrated circuit according to claim 3, wherein, Each of the quasi-volatile or non-volatile memory circuits includes a three-dimensional array of one or more NOR memory strings, each NOR memory string having at least one layer of quasi-volatile thin-film memory cells formed on a substrate layer of the first semiconductor die.

61. The integrated circuit according to claim 60, wherein, The NOR memory string includes a VNOR memory string.

62. The integrated circuit according to claim 60, wherein, The NOR memory string includes an HNOR memory string.

63. The integrated circuit of claim 62 further includes a support circuit formed on the surface of the substrate layer of the first semiconductor die under the three-dimensional array.

64. The integrated circuit according to claim 63, wherein, The first semiconductor die and the second semiconductor die are bonded in a "flip-chip" or "face-to-face" orientation.

65. The integrated circuit according to claim 64, wherein, The first semiconductor die is bonded to a first surface on a first side of a substrate in the second semiconductor die, and wherein a support circuit in the second semiconductor die is fabricated on a second surface of a substrate on a second side of the substrate, the first side and the second side being opposite sides of the substrate.

66. The integrated circuit according to claim 65, wherein, The substrate includes an insulator.

67. The integrated circuit according to claim 66, wherein, The substrate is separated from the semiconductor wafer by splitting, mechanical grinding, or chemical etching.

68. The integrated circuit according to claim 67, wherein, The splitting is accomplished by creating an insulator through the injection of oxygen atoms.

69. The integrated circuit according to claim 66, wherein, Provide through-silicon vias (TSVs) to connect the support circuitry to the wafer-level or chip-level bonding.

70. The integrated circuit of claim 63, further comprising a selection transistor, the selection transistor including a vertical thin-film transistor for routing signals from a first type of memory circuit in the first semiconductor die to the wafer-level or chip-level bond.

71. The integrated circuit according to claim 63, wherein, The support circuitry includes a voltage source for reading, programming, or erasing.

72. The integrated circuit according to claim 71, wherein, The support circuit also includes logic circuitry that operates at a voltage less than the voltage of the voltage source for programming and erasing.

73. The integrated circuit according to claim 70, wherein, The support circuitry also includes a selection transistor for routing signals from a first type of memory circuit in the first semiconductor die to the wafer-level or chip-level bond.

74. The integrated circuit according to claim 69, wherein, The support circuit also includes word line related control circuitry.

75. The integrated circuit according to claim 74, wherein, The word line related control circuit includes a word line decoder.

76. The integrated circuit according to claim 63, wherein, A sensing amplifier for sensing the first type of memory circuit is formed on the planar surface of the second semiconductor die and is connected to the first type of memory circuit on the first semiconductor die via wafer-level or chip-level bonding.

77. The integrated circuit of claim 71, further comprising a first conductor layer formed between the wafer-level or chip-level bond and the planar surface of the second semiconductor die, wherein, One or more signals generated by the voltage source for reading, programming, and erasing are routed by the conductors of the first conductor layer to a first type of memory circuit in the first semiconductor die.

78. The integrated circuit of claim 77, wherein each of the signals is generated by the voltage source for reading, programming and erasing, and is routed by the conductors of the first conductor layer and between the ground conductors providing shielding.

79. The integrated circuit of claim 77, further comprising a second conductor layer formed between the wafer-level or chip-level bond and the first conductor layer, wherein, The conductor of the second conductor layer is grounded to provide shielding for signals routed on the first conductor layer to a first type of memory circuit in the first semiconductor die.

80. The integrated circuit of claim 77, further comprising a second conductor layer that distributes signals routed via wafer-level and chip-level bond interconnect circuitry formed on the planar surface of the second semiconductor die.

81. The integrated circuit of claim 1, further comprising non-memory circuitry formed on a third semiconductor die, wherein, The third semiconductor die is bonded to the second semiconductor die to allow electrical interconnection between the non-memory circuitry and the circuitry on the second semiconductor die.

82. The integrated circuit according to claim 81, wherein, Through-silicon vias are formed on one or both of the second and third semiconductor dies to provide signal paths.

83. The integrated circuit according to claim 81, wherein, The non-memory circuitry is part of the computing system.

84. The integrated circuit according to claim 83, wherein, The computing system includes one or more of the following: a central processing unit, a graphics processing unit, a memory controller, a RISC processor, a math coprocessor, a server, a mobile device, a telecommunications switch, a router, and a gene sequencer.

85. The integrated circuit of claim 1, further comprising non-memory circuitry formed on a third semiconductor die, wherein, Both the second semiconductor die and the third semiconductor die are bonded to the silicon interposer substrate to allow non-memory circuits and circuits on the second semiconductor die to be electrically interconnected through conductors formed in the silicon interposer substrate.

86. The integrated circuit according to claim 85, wherein, The non-memory circuitry is part of the computing system.

87. The integrated circuit according to claim 86, wherein, The computing system includes one or more of the following: a central processing unit, a graphics processing unit, a memory controller, a RISC processor, a math coprocessor, a server, a mobile device, a telecommunications switch, a router, and a gene sequencer.

88. The integrated circuit of claim 1, further comprising a battery for providing backup power to the integrated circuit.

89. The integrated circuit according to claim 35, wherein, Each of the memory banks is configured to allow reading from the fast memory circuitry associated with the memory bank when the quasi-volatile memory circuitry or non-volatile memory of the memory bank is refreshed, programmed, or erased.

90. The integrated circuit according to claim 35, wherein, Each of the memory banks is configured to allow reading from a fast memory circuit associated with the memory bank when a quasi-volatile memory circuit or non-volatile memory of the memory bank is written to.

91. The integrated circuit according to claim 18, wherein, The on-chip control circuit configures the memory module and the logic module to perform computational operations on programmable block-sized data.

92. The integrated circuit according to claim 18, wherein, The memory module and the logic module on the second semiconductor die are modularized into circuit blocks each occupying a predetermined area on the second semiconductor die, wherein the circuit blocks are functionally organized into memory banks, and wherein the on-chip control circuitry is configured to perform in-memory computations on a per-circuit-block, per-memory-bank, or multi-memory-bank basis.

93. The integrated circuit according to claim 6, wherein, A conductor layer is provided on the second semiconductor die to allow signals to be demultiplexed from the bond interconnect.

94. A computing system comprising a plurality of integrated circuits according to claim 44 interconnected via a memory interface bus, wherein, A subset of the integrated circuits is configured to operate in a pipelined manner.

95. The computing system according to claim 94, wherein, The first subset of the integrated circuits is configured such that the quasi-volatile or non-volatile memory circuits include NOR memory strings that perform logic functions.

96. The computing system according to claim 95, wherein, The first subset of the integrated circuit implements one or more content-addressable memory circuits.

97. The computing system according to claim 95, wherein, The first subset of the integrated circuit is also configured to perform a parallel search function on the content-addressable memory.

98. The computing system according to claim 97, wherein, The second subset of integrated circuits is configured to perform data-intensive computations.

99. The computing system according to claim 98, wherein, The computing system is configured to transfer the results of the data-intensive computation to be used in the parallel search function on the content-addressable memory.

100. A memory module having input and output interfaces, comprising: Multiple integrated circuits, wherein one of the integrated circuits is provided by the integrated circuit according to claim 1; and A controller manages the integrated circuit to provide access to the integrated circuit through the input and output interfaces.

101. The memory module according to claim 100, wherein, The memory module conforms to the dual in-line memory module format.

102. A storage system, comprising: Quasi-volatile memory circuits or non-volatile memory circuits; A fast memory circuit integrated with the quasi-volatile memory circuit; A data path circuit shared by the quasi-volatile memory circuit or the non-volatile memory circuit and the fast memory circuit is used to access read and write operations of the quasi-volatile memory circuit or the non-volatile memory circuit and the fast memory circuit in an extended address space, wherein the fast memory circuit is accessed by logic circuitry capable of accessing the fast memory circuitry to perform in-memory computations. The fast memory circuit is modularized into a memory module, which includes multiple memory cells, multiple word lines, and multiple bit lines for selecting memory cells for read or write access. The memory module is configured to function as a bit multiplier, receiving a data value represented by a selected set of word lines as a first operand and a second operand represented by the selected set of bit lines. Each assertion value of the first operand on the selected set of word lines causes a product term formed by shifting the second operand to be written to the selected memory cell. Each deassertion value of the first operand on the selected set of word lines causes zero to be written to the selected memory cell. The quasi-volatile memory circuit or the non-volatile memory circuit is each modularized into a memory module, and each memory module in the quasi-volatile memory circuit or the non-volatile memory circuit is bonded to a dedicated data output circuit in the fast memory circuit, and the data output circuit provides data from the memory module to an associated memory module in the fast memory circuit.

103. An integrated circuit, comprising: The first semiconductor die includes: A quasi-volatile memory circuit or a non-volatile memory circuit is formed above the planar surface of the substrate of the first semiconductor die; and The support circuitry for the quasi-volatile memory circuit is formed on the planar surface of the substrate; and The second semiconductor die includes: A fast memory circuit whose read latency is less than that of the quasi-volatile memory circuit; Logic circuitry capable of accessing the fast memory circuitry to perform in-memory computations; Input and output buses are used for external processors to access and configure the fast memory circuit, the logic circuit, and the quasi-volatile memory circuit; Configurable logic for standard and AI applications that require high-capacity memory; and The quasi-volatile or non-volatile memory circuit is modularized into a memory module, which is configured to operate as a bit-by-bit multiplier. The bit multiplier receives a data value represented by a selected set of word lines as a first operand and a second operand represented by a selected set of bit lines. Each assertion value of the first operand on the selected set of word lines causes a product term formed by shifting the second operand to be written to a selected memory cell. Each deassertion value of the first operand on the selected set of word lines causes zero to be written to the selected memory cell. The memory module includes multiple memory cells, multiple word lines, and multiple bit lines for selecting memory cells for read or write access. The first semiconductor die and the second semiconductor die are bonded using hybrid bonding or through-silicon via (TSV) technology. The first semiconductor die is manufactured using a first manufacturing process optimized for the supporting circuit. The second semiconductor die is manufactured using a manufacturing process that supports low-voltage, high-performance CMOS circuits. In this embodiment, the memory circuits on the first semiconductor die are each modularized into memory modules, and each memory module in the first semiconductor die is connected by a key to a data output circuit in the second semiconductor die dedicated to the memory module, and the data output circuit provides data from the memory module to an associated memory module in the second semiconductor die.

104. A semiconductor die, comprising: (i) A quasi-volatile memory circuit formed above the planar surface of the substrate of the semiconductor die; (ii) A support circuit for a quasi-volatile memory circuit formed at the planar surface of the substrate; and (iii) a modular bonding interface that allows bonding to any of a plurality of accompanying semiconductor dies using hybrid bonding or through-silicon via (TSV) technology, wherein each accompanying semiconductor die includes a die having dedicated configurable circuitry formed therein. The accompanying semiconductor die is configured for bonding with the semiconductor die and includes: A fast memory circuit whose read latency is less than that of the quasi-volatile memory circuit; Arithmetic and logic circuits are able to access the fast memory circuitry to perform in-memory computations; An internal data bus, accessible by the arithmetic and logic circuitry to perform in-memory computations; and Input and output buses are used for external processors to access and configure the fast memory circuit, the logic circuit, and the quasi-volatile memory circuit or the non-volatile memory circuit. The internal data bus and the input and output buses operate independently of each other and simultaneously. The fast memory circuit is modularized into a memory module, which is configured to perform as a bit multiplier. The bit multiplier receives data values ​​represented by a selected set of word lines as a first operand and a second operand represented by a selected set of bit lines. Each assertion value of the first operand on the selected set of word lines causes a product term formed by shifting the second operand to be written to a selected memory cell. Furthermore, each deassertion value of the first operand on the selected set of word lines causes zero to be written to the selected memory cell. The memory circuits on the semiconductor die are each modularized into memory modules, and each memory module in the semiconductor die is connected by a key to a data output circuit in the accompanying semiconductor die dedicated to the memory module, and the data output circuit provides data from the memory module to an associated memory module in the accompanying semiconductor die.

105. The semiconductor die according to claim 104, wherein, The memory module is configured to perform computational operations on data of a programmable block size, wherein the programmable block size is any of the following: 4 bits or an integer multiple thereof, up to 2KB.

Images