In-situ persistent three-dimensional memory array

Abstract

The application discloses an in-situ persistent three-dimensional storage array, and belongs to the technical field of semiconductor storage and AI hardware, wherein a through-silicon via (TSV) is arranged at the same position of each layer of horizontal wires, and a vertical through-silicon via (TSV) is formed; an RRAM storage unit is arranged at the intersection of each layer of horizontal wires and the vertical through-silicon via (TSV), and all the RRAM storage units form a logic page, three-dimensional stacking of the storage units is realized, in-situ persistent storage of AI model parameters is realized by using the non-volatile characteristics of the RRAM; in a single read cycle, all the RRAM storage units in the logic page are activated at the same time, the storage states of the RRAM storage units are read out in parallel along the vertical through-TSV downward, and are input into a multiplier input bus of an AI accelerator in the form of current to directly participate in multiply-accumulate operation, data carrying power consumption is reduced, the storage wall limit is broken through, and the parallelism and energy efficiency of AI inference calculation are improved.

Description

Technical Field

[0001] This invention belongs to the field of semiconductor storage and AI hardware technology, specifically relating to an in-situ persistent three-dimensional storage array. Background Technology

[0002] In traditional AI inference systems, model weights are persistently stored in off-chip NAND Flash (NAND flash memory is a type of flash memory that uses a non-linear macrocell pattern internally). Each time the system powers on or a model switches, all weights must be moved from NAND to DRAM (Dynamic Random Access Memory) and then loaded into the AI ​​(Artificial Intelligence) accelerator's SRAM (Static Random-Access Memory). This model loading process consumes milliseconds to seconds and consumes considerable power (typically accounting for more than 30% of the total system power consumption), becoming a major bottleneck in AI chip storage. Summary of the Invention

[0003] The purpose of this invention is to provide an in-situ persistent 3D storage array to solve the problem of existing technologies consuming several milliseconds to several seconds and consuming considerable power during the model loading process.

[0004] This invention provides an in-situ persistent three-dimensional storage array, comprising multilayer horizontal conductors, through-silicon vias (TSVs), and RRAM storage cells; Each of the horizontal conductors in each layer is provided with a through-silicon via (TSV), and the horizontal conductors in each layer are the same size and the TSVs are in the same position to form a vertical through-silicon via (TSV). At the intersection of each horizontal conductor and the vertical through-silicon via (TSV), an RRAM memory cell is provided, and all RRAM memory cells constitute a logic page.

[0005] Furthermore, the horizontal conductors of each layer are physically isolated in the vertical direction and separated by an insulating dielectric layer; the horizontal conductors of each layer are independently led out to the edge of the array to connect to the peripheral control circuit; and in the vertical direction, the geometric dimensions, shape and projection position of the horizontal conductors of each layer are exactly the same and are stacked in sequence, and each layer of horizontal conductors forms an ohmic contact with the sidewall of the vertically penetrating through-silicon via (TSV).

[0006] Furthermore, one end of the RRAM memory cell is electrically connected to the horizontal conductor, and the other end is electrically connected to the sidewall of the through-silicon via (TSV). The RRAM memory cell is disposed at the connection node between the horizontal conductor and the TSV, and the RRAM memory cells in the same logic page are aligned vertically and share the same TSV as a common output terminal.

[0007] Furthermore, all RRAM memory cells in the logical page are activated simultaneously within a single read cycle, and the storage states of all RRAM memory cells are read out in parallel along the vertical through-silicon via (TSV).

[0008] Furthermore, the bottom of the vertical through-silicon via (TSV) is directly connected to the multiplier input bus of the AI ​​accelerator via a wide I / O interface, and participates in the multiplication-accumulation operation directly in the form of current.

[0009] Furthermore, the resistance value of each RRAM storage cell in the logical page represents a different bit of the AI ​​model weight, so as to perform parallel output of multi-bit weights.

[0010] Furthermore, the multiplier input bus of the AI ​​accelerator includes multiple parallel multiplier input channels, each corresponding to a different vertical through-silicon via (TSV). The current signal output from the bottom of each TSV is simultaneously injected into the corresponding multiplier input channel, completing the parallel multiplication and accumulation operation of all weight parameters and input feature vectors within the logic page in a single read cycle.

[0011] Furthermore, the RRAM storage unit retains its resistance value when power is off and can be directly read after power is on.

[0012] Furthermore, the number of RRAM storage cells and corresponding horizontal wires in each logical page is set to 7 or 8.

[0013] Furthermore, the RRAM storage cell employs an HfOx resistive switching layer.

[0014] The beneficial effects of this invention are as follows: This invention provides an in-situ persistent three-dimensional storage array, comprising multiple layers of horizontal conductors, through-silicon vias (TSVs), and Reactive RAM (RRAM) storage cells. Each layer of horizontal conductors has a TSV, with all horizontal conductors of the same size and TSVs positioned at the same location, forming a vertically penetrating TSV. An RRAM storage cell is placed at the intersection of each horizontal conductor and the vertically penetrating TSV. All RRAM storage cells constitute a logical page. The vertically penetrating TSV structure enables three-dimensional stacking of the storage cells, utilizing the non-volatile nature of RRAM to achieve in-situ persistent storage of AI model parameters. During a single read cycle, all RRAM storage cells in the logical page are simultaneously activated, and their storage states are read out in parallel downwards along the vertically penetrating TSV. The bottom of the TSV is directly connected to the multiplier input bus of the AI ​​accelerator, directly participating in multiplication and accumulation operations in the form of current. This reduces data transfer power consumption, overcomes the memory wall limitation, and improves the parallelism and energy efficiency of AI inference computation. Attached Figure Description

[0015] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with the invention and, together with the description, serve to explain the principles of the invention.

[0016] Figure 1 A schematic diagram of an in-situ persistent three-dimensional storage array provided in an embodiment of the present invention; Figure 2 This is a diagram showing the wide I / O interface connection relationship between a three-dimensional RRAM array and an AI accelerator provided in an embodiment of the present invention. Figure 3 A schematic diagram illustrating the structure and parallel readout method of a single logical page provided in an embodiment of the present invention; In the accompanying drawings, 101-horizontal wire, 102-through silicon via (TSV), 103-RRAM memory cell, 201-array bottom TSV lead-out array, 202-multiplier input bus, 203-near-data high-speed direct connection channel, 301-example RRAM cell, 302-logic page range, 303-parallel current read bit line, 401-conventional NAND loading scheme, 402-in-situ persistence scheme of the present invention; The accompanying drawings have illustrated specific embodiments of the invention, which will be described in more detail below. These drawings and descriptions are not intended to limit the scope of the invention in any way, but rather to illustrate the concept of the invention to those skilled in the art through reference to particular embodiments. Detailed Implementation

[0017] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numerals in different drawings denote the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present invention. Rather, they are merely examples of apparatuses and methods consistent with some aspects of the invention as detailed in the appended claims.

[0018] The embodiments of the present invention will now be described in detail with reference to the accompanying drawings.

[0019] like Figure 1 As shown, this embodiment of the invention provides an in-situ persistent three-dimensional storage array, including multilayer horizontal wires 101, through-silicon vias (TSVs) 102, and RRAM storage cells 103; Each of the horizontal conductors 101 in each layer is provided with a through silicon via (TSV) 102, and the horizontal conductors 101 in each layer are of the same size and the TSVs 102 are located in the same position to form a vertical through TSV. At the intersection of each horizontal conductor 101 and the vertical through TSV, an RRAM memory cell 103 is provided, and all RRAM memory cells 103 constitute a logic page.

[0020] This invention provides an in-situ persistent three-dimensional storage array. A through-silicon via (TSV) is disposed on each horizontal conductor layer. The horizontal conductors are of the same size and the TSVs are positioned in the same location, thus forming a vertically penetrating TSV. An irradiated RAM (Reactive RAM) storage cell is disposed at the intersection of each horizontal conductor and the vertically penetrating TSV. All RRAM storage cells constitute a logical page. The vertically penetrating TSV structure achieves three-dimensional stacking of the storage cells, utilizing the non-volatile nature of RRAM to achieve in-situ persistent storage of AI model parameters. During a single read cycle, all RRAM storage cells in the logical page are simultaneously activated, and their storage states are read out in parallel downwards along the vertically penetrating TSV. The bottom of the TSV is directly connected to the multiplier input bus of the AI ​​accelerator, directly participating in multiplication and accumulation operations in the form of current. This reduces data transfer power consumption, overcomes the memory wall limitation, and improves the parallelism and energy efficiency of AI inference computation.

[0021] In an embodiment of the present invention, the horizontal conductors 101 of each layer are physically isolated in the vertical direction and separated by an insulating dielectric layer; the horizontal conductors 101 of each layer are independently led out to the edge of the array to connect to the peripheral control circuit; and in the vertical direction, the geometric dimensions, shapes and projection positions of the horizontal conductors 101 of each layer are exactly the same and are stacked sequentially, and each layer of horizontal conductors 101 forms an ohmic contact with the sidewall of the vertically penetrating through-silicon via (TSV) 102.

[0022] In an embodiment of the present invention, one end of the RRAM memory cell 103 is electrically connected to the horizontal conductor 101, and the other end is electrically connected to the sidewall of the through-silicon via (TSV) 102; the RRAM memory cell 103 is disposed at the connection node between the horizontal conductor 101 and the through-silicon via (TSV) 102, and the RRAM memory cells 103 in the same logic page are aligned in the vertical direction and share the same through-silicon via (TSV) 102 as a common output terminal.

[0023] For example, this array is fabricated using a three-dimensional stacking process. First, a first layer of horizontal conductors 101 is fabricated on a silicon substrate. The horizontal conductors 101 are typically made of a low-resistivity metal such as copper or tungsten and are used to transmit word line drive signals or input voltages. At predetermined locations on the first layer of horizontal conductors 101, vias are fabricated and filled with conductive material to form the bottom layer of through-silicon vias (TSVs) 102. Subsequently, an insulating dielectric layer (such as silicon dioxide) is deposited around the horizontal conductors 101 and the through-silicon vias (TSVs) 102, and chemical mechanical polishing (CMP) is performed to smooth the surface.

[0024] A second layer of horizontal conductors 101 is fabricated on the insulating dielectric layer. Crucially, the geometry, shape, and projected position on the horizontal plane of the second layer of horizontal conductors 101 must be identical to those of the first layer of horizontal conductors 101 to ensure precise connection to the sidewalls of the underlying through-silicon via (TSV) 102. The above steps are repeated to stack multiple layers of horizontal conductors 101 sequentially. Each layer of horizontal conductors 101 forms an ohmic contact with the sidewalls of the vertically penetrating TSV 102. Ultimately, a vertical TSV pillar penetrating the multilayer structure is formed, along with a multilayer structure of horizontal conductors radially distributed around this pillar.

[0025] An RRAM memory cell 103 is disposed at the intersection of each layer's horizontal conductor 101 and through-silicon via (TSV) 102. One end of the RRAM memory cell 103 is electrically connected to the horizontal conductor 101, and the other end is electrically connected to the sidewall of the TSV 102. The physical location of the RRAM memory cell 103 can be located on the upper or lower surface of the horizontal conductor 101, or embedded at the connection node between the horizontal conductor 101 and the TSV 102. Since all layers' RRAM memory cells 103 share the same vertical TSV 102 as their output terminal and are aligned in the vertical direction, this group of RRAM memory cells 103 is defined as a logical page.

[0026] The horizontal conductors 101 of each layer are physically isolated in the vertical direction, separated by an insulating dielectric layer to prevent interlayer short circuits. However, logically, they all serve the same logic page. Each horizontal conductor 101 is independently led out to the edge of the array and connected to peripheral control circuitry (such as a row decoder or voltage driver).

[0027] During write or read operations, the peripheral circuitry can apply a voltage to a specific horizontal conductor 101. Since the through-silicon via (TSV) 102 is a common terminal, when a voltage is applied to a horizontal conductor 101 of a certain layer, a voltage difference is generated across the RRAM memory cell 103 on that layer, resulting in a resistive switching effect (write) or generating a loop current (read).

[0028] This configuration allows each layer of storage cells within a logical page to be controlled independently or simultaneously. One of the core advantages of this invention is simultaneous control. During AI inference operations, the voltage signal corresponding to the input feature vector is simultaneously applied to all horizontal wires 101 within the logical page (or applied rapidly via time-division multiplexing), at which point all RRAM storage cells 103 within the logical page are simultaneously turned on.

[0029] In an embodiment of the present invention, all RRAM memory cells 103 in the logical page are activated simultaneously in a single read cycle, and the storage states of all RRAM memory cells 103 are read out in parallel along the vertical through-silicon via (TSV).

[0030] In an embodiment of the present invention, the bottom of the vertical through-silicon via (TSV) is directly connected to the multiplier input bus of the AI ​​accelerator via a wide I / O interface, and participates in the multiplication-accumulation operation directly in the form of current.

[0031] In embodiments of the present invention, the resistance values ​​of each RRAM storage unit 103 in the logical page represent different bits of the AI ​​model weights, enabling parallel output of multi-bit weights. Different bits refer to different bits in the quantization weights of the AI ​​model. For example, setting up a logical page with 7 layers of RRAM units can correspond to 1 sign bit + 6 data bits of the weight (or 8 layers corresponding to 8-bit weights in INT8 format). The resistance value of each RRAM unit represents a weight value of one bit, and during reading, 7 current channels are output in parallel, thereby outputting multiple weights within one read cycle.

[0032] In an embodiment of the present invention, the multiplier input bus of the AI ​​accelerator includes multiple parallel multiplier input channels, each multiplier input channel corresponding to a different vertical through-silicon via (TSV); the current signal output from the bottom lead of each TSV is simultaneously injected into the corresponding multiplier input channel, and the parallel multiplication and accumulation operation of all weight parameters in the logic page and the input feature vector is completed within the single read cycle.

[0033] In an embodiment of the present invention, the RRAM storage unit 103 maintains its resistance value when power is off and can be directly read after power is on.

[0034] Traditional memory arrays output digital signals (0 / 1), which require a digital-to-analog converter (DAC) before being input to an analog computing unit or directly to a digital multiplier. This invention utilizes the analog characteristics of RRAM and the vertical transmission characteristics of TSV to achieve a direct analog interface connection.

[0035] A vertical through-silicon via (TSV) 102 runs through the entire memory stack layer, extending to the bottom interconnect layer. At the bottom of the TSV 102, instead of connecting to a traditional sensitive amplifier, it connects directly to the multiplier input bus of the AI ​​accelerator. The multiplier can be an analog multiplier (such as an operational amplifier-based circuit) or a mixed-signal processing unit. The memory array consists of an array of multiple TSV 102s. Each TSV 102 corresponds to a logic page and outputs a current signal for a weighted channel. Multiple TSV 102s are arranged in parallel to form a wide I / O interface.

[0036] The multiplier input bus contains multiple parallel input channels, each corresponding to a Through-Silicon Via (TSV). The current signal output from the TSV is directly injected into the corresponding multiplier channel. Inside the multiplier, this current signal can be used in a calculation with another input (such as an activation value) or directly integrated as part of the multiplication-accumulation result.

[0037] Because TSVs have extremely low parasitic capacitance and resistance, signal transmission delay is minimal, enabling high-frequency read operations. This direct coupling between the storage array and the computing unit eliminates intermediate buffering, decoding, and conversion stages, integrating storage and computation into one, and achieving in-situ computing.

[0038] like Figure 2 As shown in the figure, this embodiment of the invention provides a wide I / O interface connection diagram between a three-dimensional RRAM array and an AI accelerator. The upper part of the diagram is the TSV lead-out array 201 at the bottom of the array, and the lower part is the multiplier input bus 202 of the AI ​​accelerator computing unit. The two are connected by a parallel wide arrow 203, which represents a high-speed direct data connection channel.

[0039] In embodiments of the present invention, the number of RRAM storage cells 103 and corresponding horizontal wires 101 in each logical page is set to 7 or 8.

[0040] In an embodiment of the present invention, the RRAM storage cell 103 employs an HfOx resistive switching layer.

[0041] like Figure 3As shown in the figure, this embodiment of the invention provides a schematic diagram of the structure and parallel readout method of a single logic page. Eight RRAM cells 301 arranged vertically in series constitute a logic page. The left curly brace indicates the logic page range 302, the top arrow indicates the direction of the readout activation signal injection, and the eight horizontal lines at the bottom represent the parallel current readout bit lines 303.

[0042] Based on the above technical solution, this embodiment of the invention provides an example, which may include a three-dimensional intersection structure composed of multiple layers of horizontal wires and vertical through-silicon vias (TSVs); RRAM cells are set at the intersection of each layer of horizontal wires and vertical TSVs; a group of RRAM cells serially arranged in the vertical direction constitutes a logic page, which is activated simultaneously in a single read cycle, and the storage state of all cells is read down the TSV in parallel; the bottom of the TSV is directly connected to the multiplier input bus of the AI ​​accelerator through a wide I / O interface, and participates in multiplication and accumulation operations directly in the form of current; when power is off, each RRAM cell automatically maintains its resistance value, and is ready immediately after power-on without any loading operation. In this example, 7 layers of horizontal metal lines are stacked, and the RRAM cells use HfOx resistive switching layers. The RRAM cells at the same position of the 7 layers in the vertical direction are connected in series through TSVs to form a logic page. During reading, all cells of the target logic page are simultaneously sensed by a 1.2V read voltage, generating 7 currents that flow down the TSV in parallel and are directly input to the bottom multiplier array.

[0043] Based on this example, AI model loading latency can be reduced to zero, allowing inference to begin immediately upon power-up; interface energy efficiency can be improved by more than 10 times; and 3D stacking can increase storage density by more than 7 times compared to planar solutions.

[0044] During the model loading latency test, the loading latency of the traditional technical solution was 98.17 microseconds, while the loading latency of the technical solution described in this embodiment of the invention was 52.03 microseconds. It can be seen that the technical solution described in this embodiment of the invention can reduce the model loading latency.

[0045] Based on the technical solutions described in the embodiments of the present invention, the present invention has the following technical effects compared with the prior art: (1) Using RRAM as the storage medium, the non-volatile characteristics of the AI ​​model allow the weight parameters to be retained even after power failure. When power is restored, the system can directly enter the working state without loading from external storage, which significantly reduces system startup time and initialization power consumption, making it particularly suitable for edge AI devices.

[0046] (2) Traditional three-dimensional storage typically employs a stacked, layer-by-layer access method. In contrast, this invention constructs a vertically integrated TSV structure, aligning RRAM cells on multiple horizontal conductors in the vertical direction. Within a single read cycle, all RRAM cells within a logic page are activated simultaneously, and current signals are output in parallel along the TSV. This structure breaks through the bit-line bottleneck of traditional two-dimensional arrays, realizing true three-dimensional parallel data flow.

[0047] (3) The output current at the bottom of the TSV is directly connected to the multiplier input bus of the AI ​​accelerator. The resistance of the RRAM can be directly mapped to the weight. The input voltage is applied to the horizontal wire, and the generated current directly reflects the product of the input and the weight. This design eliminates the complex analog-to-digital conversion and intermediate buffering, thus improving computing efficiency.

[0048] (4) By assigning different layers in the logical page to different bits of the weight, compact storage and parallel output of multi-bit weights are achieved. For example, setting a 7 or 8-layer structure corresponds exactly to the INT8 or signed INT7 data format, which meets the quantization requirements of mainstream AI algorithms and improves storage density and computational accuracy.

[0049] Embodiments of the present invention are described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processing unit of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processing unit of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0050] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0051] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0052] Other embodiments of the invention will readily occur to those skilled in the art upon consideration of the specification and practice of the invention disclosed herein. This invention is intended to cover any variations, uses, or adaptations of the invention that follow the general principles of the invention and include common knowledge or customary techniques in the art not disclosed herein. The specification and examples are to be considered exemplary only, and the true scope and spirit of the invention are indicated by the claims.

[0053] It should be understood that the present invention is not limited to the precise structure described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from its scope. The scope of the invention is limited only by the appended claims.

Claims

1. An in-situ persistent three-dimensional storage array, characterized in that, It includes multilayer horizontal conductors (101), through silicon vias (TSVs) (102), and RRAM memory cells (103). Each of the horizontal conductors (101) in each layer is provided with a through silicon via (TSV) (102), and the horizontal conductors (101) in each layer are the same size and the TSVs (102) are located in the same position to form a vertical through TSV; an RRAM memory cell (103) is provided at the intersection of each horizontal conductor (101) and the vertical through TSV, and all RRAM memory cells (103) constitute a logic page.

2. The in-situ persistent three-dimensional storage array according to claim 1, characterized in that, The horizontal conductors (101) of each layer are physically isolated in the vertical direction and separated by an insulating dielectric layer; the horizontal conductors (101) of each layer are independently led out to the edge of the array to connect to the peripheral control circuit; and in the vertical direction, the geometric dimensions, shape and projection position of the horizontal conductors (101) of each layer are exactly the same and are stacked in sequence, and each layer of horizontal conductors (101) forms an ohmic contact with the sidewall of the vertically penetrating through silicon via (TSV) (102).

3. The in-situ persistent three-dimensional storage array according to claim 1, characterized in that, One end of the RRAM memory cell (103) is electrically connected to the horizontal conductor (101), and the other end is electrically connected to the sidewall of the through silicon via (TSV) (102). The RRAM memory cell (103) is located at the connection node between the horizontal conductor (101) and the through silicon via (TSV) (102), and the RRAM memory cells (103) of each layer in the same logical page are aligned in the vertical direction and share the same through silicon via (TSV) (102) as a common output terminal.

4. The in-situ persistent three-dimensional storage array according to claim 1, characterized in that, All RRAM memory cells (103) in the logical page are activated simultaneously in a single read cycle, and the storage states of all RRAM memory cells (103) are read out in parallel along the vertical through-silicon via (TSV).

5. The in-situ persistent three-dimensional storage array according to claim 1, characterized in that, The bottom of the vertical through-silicon via (TSV) is directly connected to the multiplier input bus of the AI ​​accelerator via a wide I / O interface, and participates in the multiplication-accumulation operation directly in the form of current.

6. The in-situ persistent three-dimensional storage array according to claim 5, characterized in that, The resistance values ​​of each RRAM storage cell (103) in the logical page represent different bits of the AI ​​model weights, so as to perform parallel output of multi-bit weights.

7. The in-situ persistent three-dimensional storage array according to claim 6, characterized in that, The multiplier input bus of the AI ​​accelerator includes multiple parallel multiplier input channels, each corresponding to a different vertical through-silicon via (TSV). The current signal output from the bottom of each TSV is simultaneously injected into the corresponding multiplier input channel, completing the parallel multiplication and accumulation operation of all weight parameters and input feature vectors within the logic page in a single read cycle.

8. The in-situ persistent three-dimensional storage array according to claim 6, characterized in that, The RRAM storage unit (103) maintains its resistance value when power is off and can be read directly after power is on.

9. The in-situ persistent three-dimensional storage array according to claim 1, characterized in that, The number of RRAM storage cells (103) and corresponding horizontal wires (101) in each logical page is set to 7 or 8.

10. The in-situ persistent three-dimensional storage array according to claim 1, characterized in that, The RRAM storage cell (103) uses an HfOx resistive switching layer.

Images