High-speed three-dimensional memory and system

Abstract

The invention provides a high-speed three-dimensional memory (3D-M) (100) and a system (1000). The word line number nW of each storage array (100ij) in the high-speed 3D-M chip (100) is less than or equal to 1000, and the bit line number nB is less than or equal to 1000; in each read cycle, at least 1 / 4 of all the storage arrays are activated at the same time. The high-speed 3D-M system (1000) comprises a high-speed 3D-M chip (100) and a computing chip (300) which are bonded face to face, and the working temperature is larger than or equal to 85 DEG C. The high-speed 3D-M chip (100) can reach the bandwidth of HBM, and meanwhile high-bandwidth coupling is achieved through hybrid bonding and the computing chip (300).

Description

Technical Field

[0001] The present invention relates to the field of integrated circuits, and more particularly to a memory and a system. Background Art

[0002] High-performance graphics cards (or AI accelerator cards) are high-computing systems. Figure 1A As shown, a high-performance graphics card (or AI accelerator card) 00 generally contains a computing chip (including CPU, GPU and other functions) 70 and one or more HBM (high bandwidth memory) stacks 50, which are electrically coupled through an interposer (silicon dielectric layer) 90. Each HBM stack 50 contains a logic chip ( Figure 1A The DRAM chips 10a-10h are stacked vertically and connected to each other through TSV (through substrate vias). Figure 1A The interface between the DRAM chips 10a-10h in the HBM stack 50 is at least 1024 bits, which brings a large bandwidth. Figure 1B As shown in (Table 1), the bandwidth of HBM2 reaches 256GB / s, the bandwidth of HBM2e reaches 460GB / s, and the bandwidth of HBM3 reaches 1075GB / s.

[0003] There are many problems with high-performance graphics cards (or AI acceleration cards). First, the HBM stack 50 and the computing chip 70 are placed side by side, the signal transmission distance is long (cm level), and the data transmission power consumption is high; secondly, the number of interface bits that can be achieved by TSV and microbump 30 is in the thousands, and there is still room for improvement. The ideal situation is: to bond the memory chip and the computing chip face to face, so that the number of interface bits can be increased to millions, and the signal transmission distance is short (um level). But this faces several problems: (1) DRAM storage density is 2GB (16Gb) / cm 2 , considering the computing chip area is 8cm 2 If face-to-face bonding is used, only one layer of DRAM chips (e.g. 8 1cm 2 DRAM chips are placed side by side), it can only reach a storage capacity of 16GB, which is far from enough to store large AI models (which require tens or even hundreds of GB); (2) Even worse, the operating temperature of the computing chip 70 is >100 o C. At operating temperature>85 o C, DRAM leakage current will increase significantly, leading to misoperation, so there are technical barriers to directly stacking DRAM chips and computing chips (such as face-to-face coupling); (3) DRAM is a volatile memory, and the information stored in it will be lost when the power is off. The large AI model needs to be uploaded every time the power is turned on, which brings inconvenience in use.

[0004] Three-dimensional memory (3D-M) has been developing rapidly recently. For example, Intel's 3D-XPoint has been put into mass production. 3D-XPoint is a phase change memory. Its first generation product has a capacity of 16GB (128Gb); the second generation product has a capacity of 32GB (256Gb). The chip area of ​​both generations is ~2cm 2 3D-XPint’s storage density is 4-8 times higher than DRAM. 2 The storage capacity in terms of area is ~100GB, which is enough to store large AI models. The latency of 3D-XPoint is close to that of DRAM, at the 100ns level. These indicators are acceptable. However, the bandwidth of 3D-XPoint is only 1.6Gbps, which is far lower than HBM (~1TB / s). According to the current technical level, 3D-XPoint is not enough to support the operation of large AI models, and a new technical solution is needed to increase its bandwidth by three orders of magnitude.

[0005] The reason why HBM can achieve such a high bandwidth (~1TB / s) is that DRAM can read multiple bits (m bits, m>>1) of data from the same memory array at the same time. At its root, DRAM uses three-terminal devices (such as transistors) as switches in the memory element, and the word line does not need to provide current to the bit line when reading (the bit line current is directly provided by the power supply). Therefore, compared with reading only 1 bit of data at a time from the same memory array, reading m bits of data from the same memory array at the same time, the read latency remains unchanged, and the read bandwidth can be increased by m times. However, 3D-M uses different devices and arrays than DRAM: it uses two-terminal devices (such as gate transistors) as switches, and the bit line current is completely provided by the word line. If the same technical means as DRAM is used - reading m bits of data simultaneously from the same storage array, then compared with reading only 1 bit of data at a time from the same storage array, the current on each read bit line (the bit line coupled to the storage element being read) will be diluted by m times (the current provided by the word line is limited. In order to shorten the delay, the word line generally uses the maximum current that the word line driver can provide), resulting in an m-fold increase in read delay, and the final result is that the bandwidth is not increased (bandwidth = number of storage elements being read / read delay). Since it cannot use the technical means of DRAM to increase bandwidth (reading multiple bits of data simultaneously from the same storage array), 3D-M has a low bandwidth and is not sufficient for high-performance graphics cards (or AI accelerator cards). Summary of the invention

[0006] The main object of the present invention is to provide a high bandwidth three-dimensional memory (3D-M).

[0007] Another object of the present invention is to provide a 3D-M with a bandwidth reaching or exceeding HBM (High Bandwidth Memory).

[0008] Another object of the present invention is to provide a low-latency 3D-M.

[0009] Another object of the present invention is to provide a 3D-M with a delay lower than 100ns.

[0010] Another object of the present invention is to provide a high computing power 3D-M system.

[0011] Another object of the present invention is to provide a high-computing-power 3D-M system with a storage capacity of hundreds of GB.

[0012] Another object of the present invention is to provide a high-computing-power 3D-M system with a large number of interface bits between a storage chip and a computing chip.

[0013] Another object of the present invention is to provide a high-computing-power 3D-M system in which the number of interface bits between a storage chip and a computing chip reaches millions.

[0014] Another object of the present invention is to provide a high-computing-power 3D-M system with a short data transmission distance.

[0015] Another object of the present invention is to provide a high-computing-power 3D-M system with a data transmission distance of micrometers or tens of micrometers.

[0016] To achieve these and other purposes, the present invention provides a high-speed 3D-M and its system. Since 3D-M cannot use the technical means of DRAM to increase bandwidth (reading multiple bits of data from the same storage array at the same time), the present invention provides a new method to increase the bandwidth of 3D-M: (1) using a small storage array (the number of word lines in the storage array is n W and the number of bit lines n B (2) In a read cycle, at least 1 / 4 of all the memory arrays (total N) are activated simultaneously. The present invention utilizes massively parallel read operations to enable the bandwidth of 3D-M to reach or exceed the level of HBM. In addition, since a single-layer 3D-M chip (referring to a chip that does not require similar Figure 1A Stacking multiple layers of DRAM chips 100a-100h, only one layer of unstacked 3D-M chip is enough to store large AI models, and 3D-M can be used at high temperatures (>85 o C) works normally, so the high-speed 3D-M chip can be bonded face-to-face with the computing chip to maximize the number of interface bits and minimize the signal transmission distance.

[0017] Accordingly, the present invention proposes a three-dimensional memory 3D-M (100), characterized in that it comprises: a semiconductor substrate (0) and a plurality of memory cells (130) stacked on the semiconductor substrate (0), wherein the plurality of memory cells (130) constitute a plurality of memory arrays (100aa ... 100nn); the memory cell (130) is a two-terminal device, and the two ports of the two-terminal device are respectively coupled to a word line (110) and a bit line (120); in all the plurality of memory arrays (100aa ... 100nn) of the three-dimensional memory 3D-M (100), the number of word lines and the number of bit lines of each memory array (100ij) do not exceed 1000; in a read cycle, at least 1 / 4 of all the plurality of memory arrays (100aa ... 100nn) in the three-dimensional memory 3D-M (100) are activated simultaneously.

[0018] The present invention also provides a three-dimensional memory 3D-M (100), characterized by comprising: a semiconductor substrate (0), and a first storage layer (160) and a second storage layer (260) stacked on the semiconductor substrate (0), wherein the first and second storage layers (160, 260) are stacked on each other and isolated by an insulating dielectric film (140); the first and second storage layers (160, 260) contain a plurality of storage cells (130); the storage cell (130) is a two-terminal device, and the two ports of the two-terminal device are respectively coupled to a word line (110) and a bit line (120); the first storage layer (160) contains a first storage array layer (150), and the second storage layer (260) contains a second storage array layer (250); a plurality of storage arrays, including a plurality of first storage arrays (100aa1-100nn1) located in the first storage array layer (150), and a plurality of second storage arrays (100aa2-100nn2) located in the second storage array layer (250), each of the first storage arrays (100ij1,) and the corresponding second storage array (100ij2) overlap in projection on the semiconductor substrate (0) to form a storage array stack (190ij); in a read cycle, the three-dimensional memory 3D-M In (100), at least 1 / 4 of all the multiple storage arrays are activated simultaneously.

[0019] The present invention also proposes a three-dimensional memory 3D-M (100), characterized in that it comprises: a semiconductor substrate (0) and a plurality of memory cells (130) stacked on the semiconductor substrate (0), each memory cell being a two-terminal device, wherein two ports of the two-terminal device are respectively coupled to a word line (110) and a bit line (120); the plurality of memory cells (130) constitute a plurality of mutually stacked memory layers, and the plurality of memory layers are mutually coupled; the plurality of memory cells (130) constitute a plurality of memory arrays (100aa ... 100nn), and the plurality of memory arrays (100aa ... 100nn) constitute only one memory array layer (160); in a read cycle, at least 1 / 4 of all the plurality of memory arrays in the three-dimensional memory 3D-M (100) are activated simultaneously.

[0020] The present invention further proposes a three-dimensional memory 3D-M system (1000), characterized by comprising: a module substrate (500); a three-dimensional memory 3D-M chip (100) stacked on the module substrate (500) and coupled to the module substrate (500), wherein the three-dimensional memory 3D-M chip (100) contains a plurality of through-substrate vias (TSVs) (800); a computing chip (300) stacked on the three-dimensional memory 3D-M chip (100) and bonded face-to-face with the three-dimensional memory 3D-M chip (100), wherein the computing chip (300) is coupled to the module substrate (500) via the through-substrate vias (TSVs) (800); the operating temperature of the computing chip (300) exceeds 85 o C. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] Figure 1A It is a circuit block diagram of a high-performance graphics card (or AI accelerator card); Figure 1B (Table 1) is the technical indicators of HBM (High Bandwidth Memory).

[0022] Figure 2A - Figure 2C Represents an isolated three-dimensional horizontal memory (3D-M H ): Figure 2A is its cross-sectional view; Figure 2B A top view of the storage layer 160 (storage array layer 150 ); Figure 2C 2 is a top view of the storage layer 260 (storage array layer 250 ).

[0023] Figure 3A - Figure 3B Represents an interleaved three-dimensional horizontal memory (3D-M H ): Figure 3A is its cross-sectional view; Figure 3BIt is a top view of its storage layer 260 (single storage array layer 250).

[0024] Figure 4A - Figure 4B Represents a three-dimensional vertical memory (3D-M V ): Figure 4A For along Figure 4B Cross-sectional view of AA`; Figure 4B Is the 3D-M V Top view at the wordline 110f layer.

[0025] Figure 5 is a storage array 100ij k Circuit diagram of the .

[0026] Figure 6 is the read delay of 3D-M and the number of circuit word lines m W relationship curve diagram.

[0027] Fig. 7A is a top view of the memory arrays contained in the two memory array layers 150, 250 in the 3D-M chip 100; Figure 7B FIG. 1 is a top view of a memory array included in a single memory array layer 150S in the 3D-M chip 100 .

[0028] Fig. 8A - Fig.8D It means that when the activation rate AR=25%, the 3D-M chip with multiple storage array layers ( Fig. 7A ) The activation status of each storage array within the 4 read cycles T1-T4.

[0029] Fig. 9A - Fig. 9B It means that when the activation rate AR=50%, the 3D-M chip with multiple storage array layers ( Fig. 7A ) The activation status of each storage array within the two read cycles T1-T2.

[0030] Fig.10 It means that when the activation rate AR=100%, the 3D-M chip with multiple storage array layers ( Fig. 7A ) The activation status of each storage array within a read cycle T1.

[0031] Fig.11 It means that when the activation rate AR=100%, the 3D-M chip with a single storage array layer ( Figure 7B ) The activation status of each storage array within a read cycle T1.

[0032] Fig. 12A - Fig. 12B It means that when the activation rate AR=50%, the 3D-M chip with a single storage array layer ( Figure 7B ) The activation status of each storage array within the two read cycles T1-T2.

[0033] Fig.13A - Fig.13D It means that when the activation rate AR=25%, the 3D-M chip with a single storage array layer ( Figure 7B ) The activation status of each storage array within the 4 read cycles T1-T4.

[0034] Fig.14A - Fig. 14B (Table 2A-Table 2B) are interlaced 3D-M H and Isolation 3D-M H technical indicators.

[0035] Fig.15 This is a cross-sectional view of a high-computing-power 3D-M system.

[0036] Fig.16A - Fig. 16B Cross-sectional views of two high-computing 3D-M systems containing memory chips.

[0037] Note that these drawings are only schematic diagrams and are not drawn to scale. For the sake of visibility and convenience, some sizes and structures in the drawings may be enlarged or reduced. In different embodiments, the same symbols generally represent corresponding or similar structures. Implementation

[0038] Three-dimensional memory (3D-M) is a monolithic memory with multiple memory layers stacked on its substrate. Through three-dimensional integration (vertical stacking of memory layers), the storage density of 3D-M is much greater than that of traditional two-dimensional memory. The storage array of 3D-M is defined as: a collection of memory cells, any two memory cells in this collection can be coupled to other memory cells through several continuous address lines. According to the direction of its address lines, 3D-M is divided into horizontal 3D-M (3D-M H ) and vertical 3D-M (3D-M V ). 3D-M H All address lines of 3D-M are horizontal (parallel to the substrate); V At least one set of address lines is perpendicular to the substrate. Based on whether there is an insulating layer between the storage layers, 3D-M H It is divided into isolation 3D-M H , Interlaced 3D-M H 3D-M H and 3D-M V For a specific example and its storage array, see Figure 2A - Figure 4B .

[0039] Figure 2A - Figure 2C Represents an isolated 3D-M H 100. Figure 2A As shown, isolated 3D-M H 100 contains a substrate circuit layer 0K and N V Storage layers 160, 260 (N V is the total number of storage layers. In this embodiment, N V =2). These storage layers 160, 260 are stacked on each other and on the substrate circuit layer 0K. The substrate circuit layer 0K contains transistors 0t and interconnects 0i. The transistors 0t are formed in a semiconductor substrate 0; the interconnects 0i are located above the semiconductor substrate 0 and contain multiple interconnect layers 0M1-0M3. Each storage layer (such as 160) contains multiple word lines (such as 110, along the x direction), multiple bit lines (such as 120, along the y direction) and multiple storage cells (such as 130). The word lines and bit lines are collectively referred to as address lines. In 3D-M H In the embodiment, all address lines 110, 120 are horizontal and parallel to the semiconductor substrate 0. Each memory cell (such as 130) is a two-terminal device, and its two ports are electrically coupled to the corresponding word line (such as 110) and bit line (such as 120) respectively. The memory cell 130 generally contains a switch and a programmable membrane. The switch is a diode or a selector; the resistance of the programmable membrane is variable. When the voltage difference across the memory cell exceeds a threshold, the switch is closed, the memory cell is turned on, and its resistance is determined by the programmable membrane; when the voltage difference across the memory cell is lower than the threshold, the switch is opened, and the memory cell is not turned on and is in a high-resistance state. Note that the memory layers 160 and 260 are coupled to the semiconductor substrate 0 through contact channel holes 110v and 210v, respectively.

[0040] exist Figure 2A In the embodiment, the storage layers 160 and 260 are electrically isolated by the insulating layer 140, so it is an isolated 3D-M H . In Isolation 3D-M H In the embodiment, the first storage layer 160 includes a plurality of first storage arrays 100ij1 (100ij k The subscript k indicates that it is located at the kth storage array layer). Figure 2B As shown, the first storage array 100ij1 contains n W root word lines 110a-110h, n B There are 120a-120h bit lines. W The root word lines 110a-110h refer to the number of word lines that cross a continuous bit line (such as 120a); n BThe root bit lines 120a-120h refer to the number of bit lines that cross a continuous word line (such as 110a). The second storage layer 260 includes a plurality of second storage arrays 100ij2. Figure 2C As shown, the second storage array 100ij2 also contains n W root word lines 210a-210h, n B The root bit lines 220a-220h. Fig. 7A , all first memory arrays 100aa1 ... 100nn1 constitute a first memory array layer 150, and all second memory arrays 100aa2 ... 100nn2 constitute a second memory array layer 250. H , the number of storage layers (e.g., 160) contained in each storage array layer (e.g., 150) (n L ) is 1, the entire isolated 3D-M H The number (N) of memory array layers (150, 250) included in chip 100 Z ) is 2.

[0041] Figure 3A - Figure 3B Represents an interlaced 3D-M H 100. Figure 3A As shown, interlaced 3D-M H 100 contains a substrate circuit layer 0K and N V Storage layers 160, 260 (N V is the total number of storage layers. In this embodiment, N V =2). These storage layers 160, 260 are stacked on each other and on the substrate circuit layer OK. H The difference is that the memory layers 160 and 260 share the address line 110, that is, the memory layers 160 and 260 are interleaved. H The structure is better than the isolated 3D-M H More compact and with lower production cost. Due to the staggered structure, the address lines 110, 120, 220 and the storage elements 130, 230 constitute a storage array 100ij. The number of storage layers (n L ) is 2. Reference Figure 7B , Isolate 3D-M H The chip 100 includes a plurality of memory arrays 100aa ... 100nn, which are placed side by side on the semiconductor substrate 0. These memory arrays 100aa ... 100nn constitute a single memory array layer 150S, namely N Z =1. Figure 3B As shown, a single memory array 100ij contains n W root word line, n Broot bit line, n L storage tiers. W A root word line refers to the number of word lines that cross a continuous bit line (such as 220a); n B Root bit lines refer to the number of bit lines that cross a continuous word line (eg, 110a).

[0042] exist Figure 2A - Figure 2C and Figure 3A - Figure 3B 3D-M H Using a cross-point array, the storage element area is 4F 2 (F is the address line width, i.e., the process feature size). In addition, the use of three-dimensional integration (i.e., multi-layer stacking, N V >1), the capacity of 3D-M is much larger than that of DRAM. The single-core capacity of DRAM is 2GB, while the single-core capacity of 3D-M can reach 16GB (N V =2) or 32GB (N V =4). Therefore, the storage capacity of a 3D-M chip can be equivalent to 8-16 DRAM chips.

[0043] Figure 4A - Figure 4B Represents a 3D-M V 100. Figure 4A As shown, 3D-M V 100 contains a substrate circuit layer 0K and N V Storage layers (N V is the total number of storage layers. In this embodiment, N V =8). Each storage layer is actually a layer of word lines (one from 110a-110h), so the number of word lines n W =N V These memory layers (word line layers 110a-110h) are stacked on each other and on the substrate circuit layer 0K. The memory hole 123 penetrates all the word line layers 110a-110h, forms a memory film (including a switch, such as a gate) 125 on its sidewall, and then fills with conductive material to form a bit line 120. All memory cells located on the same memory hole 123 constitute a memory string 127. Since the bit line 120 is perpendicular to the semiconductor substrate 0, the 3D-M in this embodiment is 3D-M. V 100. In 3D-M V In 100, all word lines 110a-110h are electrically coupled to bit lines 120 in the Z direction. Therefore, word lines 110a-110h and all bit lines 120 coupled thereto constitute a memory array 100ij, which contains a number of memory layers (n L ) is 8, that is, n L =n W =NV =8.

[0044] Figure 4B The memory array 100ij is described in more detail. Each word line 110f is a conductive plate that is connected to n B The root bit lines 120 cross to form n B There are storage elements 130. In this figure, n B =12, n in actual products B Can be much larger than 12. Figure 4A In combination, the word lines (conductor plates) 110a-110h (their projections on the semiconductor substrate 0 overlap) form a word line group, and all the memory cells thereon form a memory array 100ij. Another word line (conductor plate) 111f and all the word lines above and below it and having overlapping projections on the semiconductor substrate 0 form another word line group, and all the memory cells thereon form another memory array 100ii. Figure 7B , 3D-M V The chip 100 includes a plurality of memory arrays 100aa ... 100nn, which are placed side by side on the semiconductor substrate 0. These memory arrays 100aa ... 100nn constitute a single memory array layer 150S, namely, 3D-M V The number of memory array layers 150S included in the chip 100 (N Z ) is 1.

[0045] In this specification, 3D-M 100 can be represented by the following formula: W ×n B ×n L ×N Z (n W is the number of word lines that cross a continuous bit line; n B is the number of word lines crossing a continuous word line; n L is the number of storage layers contained in a storage array; n Z is the number of storage array layers contained in the 3D-M chip in the Z direction. V = n L ×N Z ).like Figure 2A - Figure 2C Isolation 3D-M H It can be expressed as n W ×n B ×1×2; Figure 3A - Figure 3B Interlaced 3D-M H It can be expressed as n W ×n B ×2×1; Figure 4A - Figure 4B 3D-M VIt can be expressed as n W ×n B ×1×1.

[0046] In addition to isolating 3D-M H (n L =1,N V =N Z )、Interlaced 3D-M H and 3D-M V (N Z =1,N V =n L =n W ), there can also be hybrid 3D-M, where some of the memory layers are interleaved and others are isolated. Figure 3A - Figure 3B Interlaced 3D-M H , the storage layers 160 and 260 form a set (two layers) of staggered storage layers, on which another set of staggered storage layers is further stacked, and the two sets of storage layers are separated by an insulating layer. The hybrid 3D-M is represented by n W ×n B ×2×2. For simplicity, this specification does not draw the hybrid 3D-M. All the conclusions about 3D-M can be easily extended to the hybrid 3D-M.

[0047] Figure 5 Represents a storage array 100ij k This diagram is for Figure 2A - Figure 2C Isolation 3D-M H , storage array 100ij k The subscript k indicates the storage array layer where it is located. Figure 3A - Figure 3B Interlaced 3D-M H and Figure 4A - Figure 4B 3D-M V , the storage array layer is a single storage array layer, and the subscript k in this figure can be omitted. The storage array 100ij circuit contains m W root word line 110, m B The root bit lines 120 and m W ×m B There are storage cells 130. Here, m W is the number of circuit word lines, used to distinguish the number of word lines n in the physical memory array W ;m B is the number of circuit bit lines, used to distinguish the number of bit lines n in the physical storage array B :for Figure 2A - Figure 2C Isolation 3D-MH , m W =n W 、m B =n B ;for Figure 3A - Figure 3B Interlaced 3D-M H , m W =n W 、m B =2×n B ;for Figure 4A - Figure 4B 3D-M V , m W =n W 、m B =n B The word line 110 is coupled to a peripheral circuit (such as an X decoder) 170ij, and the bit line 120 is coupled to a peripheral circuit (such as a Y decoder / readout circuit) 170ij. The peripheral circuit 170ij can be folded into the memory array 100ij. k Under this condition, the storage array is reduced by 100ij k Note that the word line 110 includes not only data word lines (transmitting data) but also dummy word lines (used for planarization process), etc.; the bit line 120 includes not only data bit lines (transmitting data) but also dummy bit lines (used for planarization process), reference bit lines (providing reference voltage for differential sense amplifier), etc.

[0048] During the read operation, the voltage on the word line 110z is raised to the read voltage V R , the read current I provided by word line 110z z By sending the read memory cell (such as 130zx) to the read bit line (such as 120x), and the parasitic capacitance (such as C x ) is charged, and when the voltage on the read bit line (such as 120x) rises to the trigger voltage V T , the read circuit 180 is triggered to generate a read signal. Whether a memory cell is being read is determined by the initial voltage on the bit line coupled to the memory cell. If the bit line 120x is initially 0V, the memory cell 130zx coupled thereto is being read, and the bit line 120x is the read bit line and flows through the read current I x ; If the bit line 120y is initially V R / 2, the memory cell 130zy coupled thereto is not read, the bit line 120y is an unread bit line, and there is no read current I y In a read cycle, if there is only one memory cell 130zx being read on the word line 110z (the other memory cells on the word line 110z are not read), the read current I on the read bit line 120x is x =I zIf there are only two read memory cells 130zx and 130zy on word line 110z, the read current I on read bit lines 120x and 120y is x =I y =I z / 2; Similarly, if there are m memory cells being read on word line 110z, the read current on the read bit line is I z / m, that is, it is diluted m times.

[0049] The read delay T caused by the above read operation D By T D =C X ×V T / I x It is obvious that the reading current I x The larger the read delay T D The smaller the read delay T D In each read cycle, it is best that only one bit of data is read from each memory array. This is significantly different from DRAM and other memories. DRAM memory cells use three-terminal devices (such as transistors) as switches, and the word line does not need to provide current to the bit line. Therefore, in a read cycle, the same memory array can read m (m>>1) bits of data at the same time, which will not affect its read delay T D , and can also increase the read bandwidth by m times (relative to reading only 1 bit of data at a time from the same storage array). In comparison, 3D-M uses a two-terminal device (such as a gate tube) as a switch, and the read current on the bit line is completely provided by the word line. If the same technical means as DRAM is used - reading m bits of data from the same storage array at the same time, the read current on each read bit line will be diluted by m times, resulting in a read delay T D Increasing m times does not increase the bandwidth (bandwidth = number of memory cells being read / read latency). Therefore, for 3D-M, in order to reduce latency, it is best for only one bit of data to be read from the memory array in one read cycle. Note that although only one bit of data is read in one read cycle, since the memory array may use a differential sense amplifier, two bit lines (data bit line and dummy bit line) may be activated at the same time and flow through the read current.

[0050] Another factor that affects read latency is T D The factor is the parasitic capacitance C of the bit line x . Bit line parasitic capacitance C x The number of word lines in the circuit m W Directly proportional. Figure 6 is the read latency of 3D-M D The number of word lines in the circuit m W The relationship curve diagram. D With n W Proportional to: n W The smaller the T DThe smaller the current 3D-XPoint n W is ~2k, read latency T D is about 110ns. W From 2k to 0.5k (~500 roots), the read delay T D Down to ~35ns, fast enough to compete with DRAM.

[0051] Fig. 7A - Figure 7B It shows the placement of the memory array in the 3D-M chip 100 . Fig. 7A For Isolation 3D-M H Two of them (N Z =2) Storage array layers 150, 250 (such as Figure 2A - Figure 2C ). The first memory array layer 150 contains first memory arrays 100aa1-100nn1; the second memory array layer 250 is located above the first memory array layer 150 and contains second memory arrays 100aa2-100nn2 (they are in different planes from the first memory arrays 100aa1-100nn1, so they are represented by dotted lines). The projections of each first memory array (such as 100aa1) and the corresponding second memory array (such as 100aa2) on the semiconductor substrate 0 overlap, and they constitute a memory array stack (such as 180aa). In each memory array layer, there are N in the x direction in 3D-M 100. X There are N storage arrays in the y direction. Y storage arrays, the total number of storage arrays N=N X ×N Y ×N Z , or a total of N X ×N Y Storage array stacks 180aa-180nn. Figure 7B For a single (N Z =1) Storage array layer (such as Figure 3A - Figure 3B and Figure 4A - Figure 4B ) case. It only contains a single memory array layer 150S, so the memory array stack is equivalent to the memory array. The 3D-M 100 as a whole has N in the x direction. X There are N storage arrays in the y direction. Y storage arrays, the total number of storage arrays N = N X ×N Y .

[0052] Since 3D-M cannot use the DRAM method to improve bandwidth (reading multiple bits of data from the same memory array at the same time), the present invention proposes a new method to improve the bandwidth of 3D-M: (1) using a small memory array (the number of word lines in the memory array is n W and the number of bit lines n B (2) In a read cycle, at least 1 / 4 of all memory arrays (total N) are activated simultaneously. The present invention utilizes massively parallel read operations to enable the bandwidth of 3D-M to reach or exceed the level of HBM. Small memory arrays refer to n W and n B All ≤ 1000, preferably ≤ 600. W and n B The same value or different values ​​can be used. The activation rate AR of the storage array is defined as the number N of storage arrays that are activated simultaneously in a read cycle. 激活 Divide by the total number of storage arrays N, that is, AR=N 激活 / N. In order to reach the level of HBM, the activation rate AR needs to be no less than 25%.

[0053] Fig. 8A - Fig.10 Indicates that for multiple (N Z =2) Storage array layer 150, 250, and storage array activation status under three activation rates AR. Fig. 8A - Fig.8D It indicates the activation status of the storage arrays 100aa1-100nn1, 100aa2-100nn2 in 4 consecutive read cycles T1-T4 when the activation rate AR=25%. The storage arrays 100aa1-100nn1, 100aa2-100nn2 are divided into multiple storage array groups 190A-190H, and each storage array group (such as 190A) has the same activation mode in each read cycle. Among them, the storage array group (such as 190A) includes 4 adjacent storage arrays. In this specification, "adjacent" means adjacent in the x direction, adjacent in the y direction, or adjacent in the z direction. Among them, "layer adjacent" means that two storage arrays are located in two adjacent storage array layers, and their projections on the semiconductor substrate 0 overlap, such as storage arrays 100aa1 and 100aa2. "Edge adjacent" means that two storage arrays are located in the same storage layer and are adjacent on an edge line, such as storage arrays 100aa1 and 100ab1. "Corner adjacent" means that two storage arrays are located in the same storage layer and are adjacent at a corner, such as storage arrays 100aa1 and 100bb1. Fig. 8A - Fig.8DIn the embodiment, the storage array group 190A includes storage arrays 100aa1-100ab1 and 100aa2-100ab2. The storage arrays 100aa1-100ab1 are adjacent to each other, and the storage arrays 100aa2-100ab2 are adjacent to each other, and the layers are adjacent to each other. The storage array group 190A may also include other combinations such as 100aa1-100bb1.

[0054] Fig. 8A It means that in the first read cycle T1, only the storage array 100aa1 in the storage array group 190A is activated (the activated lower storage array is indicated by a right-slanting line, and the unactivated storage array is not underlined); only the storage array 100am1 in the storage array group 190B is activated; only the storage array 100bb1 in the storage array group 190C is activated; only the storage array 100bn1 in the storage array group 190D is activated... In order to ensure thermal balance during reading (that is, the heat generated by each activated storage array is evenly distributed, and there is no local heat concentration), in the same read cycle T1, the farther the distance between the activated storage arrays is, the better: it is best that two corner adjacent storage arrays (such as 100aa1 and 100bb1) are activated at the same time, and two side adjacent storage arrays (such as 100aa1 and 100ab1) are preferably not activated at the same time. Figure 8B This means that in the second read cycle T2, only the storage array 100ab1 in the storage array group 190A is activated; only the storage array 100an1 in the storage array group 190B is activated; only the storage array 100ba1 in the storage array group 190C is activated; only the storage array 100bm1 in the storage array group 190D is activated... In adjacent read cycles (T1, T2), it is best that two adjacent storage arrays (such as 100aa1, 100ab1) are activated in sequence.

[0055] Figure 8C It indicates that in the third read cycle T3, only the storage array 100aa2 in the storage array group 190A is activated (the activated upper storage array is indicated by a left oblique line, and the unactivated storage array has no line); only the storage array 100am2 in the storage array group 190B is activated; only the storage array 100bb2 in the storage array group 190C is activated; only the storage array 100bn2 in the storage array group 190D is activated…. Fig.8D It means that in the fourth read cycle T4, only the storage array 100ab2 in the storage array group 190A is activated; only the storage array 100an2 in the storage array group 190B is activated; only the storage array 100ba2 in the storage array group 190C is activated; only the storage array 100bm2 in the storage array group 190D is activated... Generally speaking, for AR=25%, only one storage array in a storage array group is activated.

[0056] Fig. 9A - Fig. 9B It shows the activation of the T1-T2 storage array in two consecutive read cycles when the activation rate AR=50%. The storage array is divided into a plurality of storage array groups 190A`..., each storage array group (such as 190A`) is composed of two adjacent layers of storage arrays (such as 100aa1-100aa2). In this embodiment, the storage array group 190A` is equivalent to the storage array stack 180aa ( Fig. 7A ). Fig. 9A It indicates that in the first read cycle T1, only the storage array 100aa1 in the storage array group 190A' is activated...; Fig. 9B This indicates that in the second read cycle T2, only the storage array 100aa2 in the storage array group 190A' is activated. Generally speaking, for AR=50%, only one storage array is activated in a storage array group (such as 190A') or a storage array stack (such as 180aa).

[0057] Fig.10 It shows the activation of the storage array in one read cycle T1 when the activation rate AR=100%. At this time, all storage arrays 100aa1-100nn1, 100aa2-100nn2 are activated at the same time, which corresponds to the maximum bandwidth BW of the 3D-M chip 100. max When AR<100% (e.g. Fig. 8A - Fig. 9B ), the bandwidth BW of the 3D-M chip 100 = BW max ×AR. For example, when AR=25%, BW=BW max / 4; when AR=50%, BW=BW max / 2.

[0058] Fig.11 - Fig.13D For a single (N Z =2) Storage array layer 150S, activation status of storage array under three activation rates AR. Fig.11 It shows the activation of the memory array in one read cycle T1 when the activation rate AR=100%. At this time, all memory arrays 100aa-100nn are activated at the same time (the activated memory arrays are indicated by cross-hatching, and the inactivated memory arrays are not crossed), which corresponds to the maximum bandwidth BW of the 3D-M chip 100. max Similarly, when AR<100%, the bandwidth BW of the 3D-M chip 100 = BW max ×AR.

[0059] Fig. 12A - Fig. 12BIt indicates the activation of the T1-T2 storage array in two consecutive read cycles when the activation rate AR = 50%. The storage array is divided into multiple storage array groups 190A*-190H*, and each storage array group (such as 190A*) is composed of two adjacent storage arrays (such as 100aa-100ab). Fig. 12A It indicates that in the first read cycle T1, only the storage array 100aa in the storage array group 190A* is activated; only the storage array 100am in the storage array group 190B* is activated...; Fig. 12B It means that in the second read cycle T2, only the storage array 100ab in the storage array group 190A* is activated; only the storage array 100an in the storage array group 190B* is activated... In order to ensure thermal balance during reading, in the same read cycle T1, two storage arrays adjacent to each other at the corners (such as 100aa and 100bb) are preferably activated at the same time, while two storage arrays adjacent to each other at the sides (such as 100aa and 100ab) are preferably not activated at the same time; in adjacent read cycles (T1, T2), it is best that two storage arrays adjacent to each other at the sides (such as 100aa and 100ab) are activated in sequence. Generally speaking, in a storage array group, only one storage array is activated.

[0060] Fig.13A - Fig.13D It shows the activation status of the storage array in four consecutive read cycles T1-T4 when the activation rate AR=25%. The storage array is divided into multiple storage array groups 190A**-190D**, and each storage array group (such as 190A**) includes four adjacent storage arrays (such as 100aa-100bb). Fig.13A It indicates that in the first read cycle T1, only the memory array 100aa in the memory array group 190A** is activated; only the memory array 100am in the memory array group 190B** is activated. Fig. 13B It indicates that in the second read cycle T2, only the memory array 100ab in the memory array group 190A** is activated; only the memory array 100an in the memory array group 190B** is activated. Fig. 13C It indicates that in the third read cycle T3, only the memory array 100ba in the memory array group 190A** is activated; only the memory array 100bm in the memory array group 190B** is activated. Fig.13D It indicates that in the fourth read cycle T4, only the memory array 100bb in the memory array group 190A** is activated; only the memory array 100bn in the memory array group 190B** is activated.... Generally speaking, in one memory array group, only one memory array is activated.

[0061] Fig.14A - Fig. 14B(Table 2A-Table 2B) lists the technical indicators of two types of 3D-M. Figure 3A - Figure 3B ),like Fig.14A As shown in Table 2A, when the total number of storage layers N V = 2, the storage capacity is 128Gb (16GB). W =n B =2k, then the total number of storage arrays N is 16k, and the read latency is ~110ns, corresponding to the maximum bandwidth BW max The data rate is 20GB / s, which is far from HBM. W and n B Halved to 1k, then N=64k, then the read latency is ~60ns, BW max =160GB / s, which is still some distance away from HBM. W and n B Further reduced to 0.5k, then N=256k, then the read latency is ~35ns, BW max The data rate is 1.2TB / s, which can reach the level of HBM3. Even in the case of AR=25%, n W =n B =0.5k corresponding bandwidth BW can also reach 0.3GB / s, which is the level of HBM2; when AR=50%, n W =n B =0.5k corresponding bandwidth BW can reach 0.6GB / s, which is the level of HBM2e. V =4. The storage capacity is 256Gb (32GB). Due to the interleaved structure, there is no obvious improvement in bandwidth.

[0062] For the isolated 3D-M ( Figure 2A - Figure 2C ),like Fig. 14B (Table 2B) shows that when N V =2, the storage capacity is 128Gb (16GB). Due to the use of isolated storage layers 160 and 260, the total number of storage arrays is doubled compared to interleaved 3D-M. W =n B =1k,BW max =320GB / s, reaching the level of HBM2. W =n B =0.5k, BW max =2.4TB / s, exceeding the level of HBM3. V =4, the storage capacity is 256Gb (32GB), and the total number of storage arrays is further doubled. W =nB =1k,BW max =640GB / s; when n W =n B =0.5k, BW max =4.8TB / s, far exceeding the level of HBM3. Therefore, for isolated 3D-M, when n W =n B =1k, the bandwidth level of HBM can be achieved.

[0063] Note that BWmax and n W Inversely proportional to the cube of W =n B ), so we can reduce n W Implementing BW max A huge improvement. W Reduce by 2 times, BW max Increased by 8 times; such as n W Reduced by 4 times, BW max This is why the bandwidth of 3D-M can catch up with HBM. On the other hand, when n W 、n B If the number is too small, the storage array will be too small, which will reduce the array efficiency of the chip 100 (the ratio of the chip area used to store data in the storage chip to the entire storage chip area) and increase the chip area cost. W =n B =0.5k is a relatively ideal value. Of course, this does not exclude the use of other n within the scope of protection of the claims. W and n B combination.

[0064] When 3D-M bandwidth reaches the level of HBM, and the storage capacity of 3D-M far exceeds that of DRAM, a single-layer 3D-M chip can meet the needs of large AI models. Figure 1A The method of stacking multiple layers of DRAM chips 100a-100h only requires one layer of 3D-M chip 100. The 3D-M chip is bonded face to face with the computing chip, which can ensure high bandwidth and short distance between the storage chip and the computing chip, thereby realizing a high computing power 3D-M system.

[0065] Fig.15 A high computing power 3D-M system 1000 is shown. It includes a module substrate 500, a 3D-M chip 100 stacked on the module substrate 500, and a computing chip 300 stacked on the 3D-M chip 100 and bonded face to face. The module substrate 500 may include an interposer (silicon dielectric layer) or other packaging structures. The 3D-M chip 100 is a Figure 2A - Figure 7B The computing chip 300 contains a CPU, a GPU and / or other processing circuits. 2 The computing chip 300 can be made of 4 2cm 2 The 3D-M chip 100 (only one 3D-M chip is shown in this figure) is bonded face to face with the computing chip 300. These four 3D-M chips are placed side by side (meaning they are not stacked on each other) below the computing chip 300, and they are coupled to different parts of the computing chip 300 respectively. In other words, a single computing chip 300 is bonded face to face with multiple 3D-M chips placed in a single layer at the same time. The 3D-M chip 100 contains multiple TSVs 800; the computing chip 300 is coupled to the module substrate 500 through these TSVs 800. In addition, the 3D-M chip 100 is bonded to the computing chip 300 through hybrid bonding 600. The hybrid bonding 600 includes a first dielectric film 610 formed on the functional surface of the 3D-M chip 100 and a first interconnect pillar 620 therein, and a second dielectric film 630 formed on the functional surface of the computing chip 300 (note that the computing chip 300 is inverted) and a second interconnect pillar 640 therein, and the first interconnect pillar 620 is electrically coupled to the second interconnect pillar 640. Note that FIG. 8 only shows a small number of TSVs 800 and interconnect pillars 620, 640 in hybrid bonding 600. In the actual 3D-M system 1000, there will be a large number of TSVs 800 and interconnect pillars 620, 640 in hybrid bonding 600. Since the pitch of TSVs 800 is in the order of ten microns, the number of TSVs 800 is at least in the order of thousands. In addition, since the pitch of interconnect pillars 620, 640 in hybrid bonding 600 is in the order of microns, the number of interconnect pillars 620, 640 is at least in the order of millions. Obviously, hybrid bonding 600 can achieve denser chip-to-chip connections than TSVs 800.

[0066] DRAM represents digital information by storing electric charge. o C, the leakage current of the transistor in the memory cell increases, causing DRAM to malfunction. The operating temperature of the computing chip 300 is generally over 100 o C, so it is difficult for DRAM chips to be directly stacked with computing chips 300. 3D-M uses a non-charge storage mechanism, and its storage elements generally use physical change materials, phase change materials, resistive change materials, magnetic change materials and / or ferroelectric materials to store digital information. These materials are not suitable for working in environments with temperatures exceeding 85 oC, so the 3D-M chip 100 can be directly stacked with the computing chip 300, which is another advantage of the present invention. In the present invention, physical change materials refer to storing information by changing materials (such as changing the material itself through chemical reactions); phase change materials refer to storing information by changing the phase of materials; resistive change materials refer to storing information by changing the resistance of materials; magneto-variable materials refer to storing information by changing the magnetic properties of materials; ferroelectric materials refer to storing information by using the ferroelectric effect. All these storage mechanisms can be collectively referred to as non-charge storage mechanisms, and these storage materials are called non-charge storage materials.

[0067] Fig.16A - Fig. 16B 1 shows two high-computing 3D-M systems 1000 containing memory chips, which also contain memory (such as DRAM) chips 700. Due to the working environment temperature, the memory chip 700 cannot be directly stacked with the computing chip 300. Fig.16A In the embodiment of FIG. 5 , the memory chip 700 is located between the 3D-M chip 100 and the module substrate 500. Since there is a 3D-M chip 100 between it and the computing chip 300, its operating temperature is slightly lower. Note that the memory chip 700 contains TSV 850 to facilitate electrical coupling between the computing chip 300, the 3D-M chip 100 and the module substrate 500. Fig. 16B In the embodiment of the present invention, the memory chip 700 is located on the other side of the module substrate 500 (a side different from the side where the 3D-M chip 100 is located). Since it is farther away from the computing chip 300, its operating temperature is lower, thus ensuring its normal operation.

[0068] It should be understood that, without departing from the spirit and scope of the present invention, the form and details of the present invention can be modified, which does not hinder them from applying the spirit of the present invention. Therefore, except according to the spirit of the attached claims, the present invention should not be restricted in any way.

Claims

1. A three-dimensional memory 3D-M (100), characterized in that include: A semiconductor substrate (0) and a plurality of memory cells (130) stacked on the semiconductor substrate (0), wherein the plurality of memory cells (130) constitute a plurality of memory arrays (100aa . . . 100nn); The storage element (130) is a two-terminal device, and two ports of the two-terminal device are respectively coupled to a word line (110) and a bit line (120); In all the plurality of memory arrays (100aa...100nn) of the three-dimensional memory 3D-M (100), the number of word lines and the number of bit lines of each memory array (100ij) do not exceed 1000; In a read cycle, at least 1 / 4 of all the plurality of memory arrays (100aa . . . 100nn) in the three-dimensional memory 3D-M (100) are activated simultaneously.

2. The three-dimensional memory 3D-M (100) according to claim 1, further characterized by: The number of word lines and the number of bit lines of each storage array (100ij) do not exceed 600.

3. A three-dimensional memory 3D-M (100), characterized in that include: A semiconductor substrate (0), and a first storage layer (160) and a second storage layer (260) stacked on the semiconductor substrate (0), wherein the first and second storage layers (160, 260) are stacked on each other and isolated by an insulating dielectric film (140); the first and second storage layers (160, 260) contain a plurality of storage cells (130); the storage cells (130) are two-terminal devices, and two ports of the two-terminal devices are respectively coupled to a word line (110) and a bit line (120); the first storage layer (160) contains a first storage array layer (150), and the second storage layer (260) contains a second storage array layer (250); A plurality of storage arrays, including a plurality of first storage arrays (100aa1-100nn1) located in the first storage array layer (150), and a plurality of second storage arrays (100aa2-100nn2) located in the second storage array layer (250), wherein each of the first storage arrays (100ij1) overlaps with a projection of the corresponding second storage array (100ij2) on the semiconductor substrate (0) to form a storage array stack (190ij); In a read cycle, at least 1 / 4 of all the multiple storage arrays in the three-dimensional memory 3D-M (100) are activated simultaneously.

4. The three-dimensional memory 3D-M (100) according to claim 3, further characterized by: The plurality of storage arrays have one of the following features A)-I): A) the number of word lines and the number of bit lines in each memory array (100ij) do not exceed 1000; or B) The number of word lines and the number of bit lines in each memory array (100ij) do not exceed 600; or C) In the same read cycle, only one storage array in a storage array group containing four adjacent storage arrays is activated; or D) In ​​the same read cycle, only one storage array in a storage array group containing two adjacent storage arrays is activated; or E) in the same read cycle, at least one storage array in each storage array stack of all the storage array stacks of the three-dimensional memory 3D-M (100) is activated simultaneously; or F) in the same read cycle, all of the plurality of storage arrays in the three-dimensional memory 3D-M (100) are activated simultaneously; or G) Two corner-adjacent storage arrays are activated simultaneously in the same read cycle; or H) In the same read cycle, two adjacent storage arrays are not activated at the same time; or I) In two adjacent read cycles, two adjacent storage arrays are activated in sequence.

5. A three-dimensional memory 3D-M (100), characterized in that include: A semiconductor substrate (0) and a plurality of memory cells (130) stacked on the semiconductor substrate (0), each memory cell being a two-terminal device, two ports of the two-terminal device being respectively coupled to a word line (110) and a bit line (120); the plurality of memory cells (130) forming a plurality of mutually stacked memory layers, the plurality of memory layers being mutually coupled; The plurality of storage cells (130) constitute a plurality of storage arrays (100aa ... 100nn), and the plurality of storage arrays (100aa ... 100nn) constitute only one storage array layer (160); In a read cycle, at least 1 / 4 of all the multiple storage arrays in the three-dimensional memory 3D-M (100) are activated simultaneously.

6. The three-dimensional memory 3D-M (100) according to claim 5, further characterized by: The plurality of storage arrays (100aa...100nn) have one of the following characteristics J)-Q): J) the number of word lines and the number of bit lines in each memory array (100ij) do not exceed 1000; or K) the number of word lines and the number of bit lines of each memory array (100ij) do not exceed 600; or L) In the same read cycle, only one storage array in a storage array group containing four adjacent storage arrays is activated; or M) In the same read cycle, only one storage array in a storage array group containing two adjacent storage arrays is activated; or N) in the same read cycle, all the storage arrays in the three-dimensional memory 3D-M (100) are activated simultaneously; or O) Two corner-adjacent storage arrays are activated simultaneously in the same read cycle; or P) In the same read cycle, two adjacent storage arrays are not activated at the same time; or Q) In two adjacent read cycles, two adjacent storage arrays are activated in sequence.

7. The three-dimensional memory 3D-M (100) according to any one of claims 1 to 6, further characterized by: In a read cycle, each of the activated memory arrays reads out only one bit of data.

8. A three-dimensional memory 3D-M system (1000), characterized in that include: a module substrate (500); A three-dimensional memory 3D-M chip (100) stacked on the module substrate (500) and coupled to the module substrate (500), wherein the three-dimensional memory 3D-M chip (100) contains a plurality of through-substrate vias (TSVs) (800); a computing chip (300) stacked on the three-dimensional memory 3D-M chip (100) and bonded face-to-face with the three-dimensional memory 3D-M chip (100), wherein the computing chip (300) is coupled to the module substrate (500) through the through-substrate via TSV (800); The operating temperature of the computing chip (300) exceeds 85 o C.

9. The three-dimensional memory 3D-M system (1000) according to claim 8, further characterized by Having one of the following R)-S) characteristics: R) the memory cell (130 . . . ) of the three-dimensional memory 3D-M chip (100) contains a non-charge storage material; or S) The three-dimensional memory 3D-M chip (100) and the computing chip (300) are bonded face to face using a hybrid bonding (600) method.

10. The three-dimensional memory 3D-M system (1000) according to claim 8 or 9, further characterized by Contains at least one memory chip (700), wherein the memory chip (700) has one of the following T)-U) characteristics: T) the memory chip (700) is located between the 3D-M chip (100) and the module substrate (500), and is coupled to the 3D-M chip (100) and the module substrate (500) respectively; or U) The memory chip (700) is located on a side of the module substrate (500) different from the side of the 3D-M chip (100).

Images