Multi-level ultra-low power inference engine accelerator

Abstract

The invention is entitled "Multi-Level Ultra-Low Power Inference Engine Accelerator". The invention provides a non-volatile memory structure for performing in-memory compute inference of neural networks. A memory array is formed according to a cross-point architecture, with a memory cell present at each cross-point junction. A multi-level memory cell (MLC) is formed from a plurality of ultra-thin dielectric layers separated by a metal layer, with programming of the memory cell accomplished by selectively breaking down one or more of the dielectric layers by selecting a write voltage level. In an alternative set of embodiments, the memory cell is formed as a resistive fuse.

Description

Background Technology

[0001] Artificial neural networks are increasingly used in artificial intelligence and machine learning applications. In an artificial neural network, a set of inputs is propagated through one or more intermediate or hidden layers to generate an output. The layers that connect the inputs to the outputs are connected by multiple sets of weights, which are generated during the training or learning phase by determining a set of mathematical transformations to convert the inputs into the outputs, moving through the layers, and calculating the probability of each output. Once the weights are established, they can be used during the inference phase to determine the outputs based on a set of inputs. While such neural networks can provide highly accurate results, they are extremely computationally intensive, and the data transfer involved in multiple processing units—reading connection weights from different layers in memory and transferring these weights to some processing unit—can be quite intensive. Attached Figure Description

[0002] Components with similar numbers refer to common parts in different drawings.

[0003] Figure 1 It is a block diagram of one implementation of a memory system connected to a host computer.

[0004] Figure 2 This is a block diagram of one implementation of the front-end processor circuitry. In some implementations, the front-end processor circuitry is part of the controller.

[0005] Figure 3 This is a block diagram of one implementation of the back-end processor circuitry. In some implementations, the back-end processor circuitry is part of the controller.

[0006] Figure 4 This is a block diagram of one implementation scheme of a memory package.

[0007] Figure 5 This is a block diagram of one implementation scheme for a memory die.

[0008] Figure 6A and Figure 6B An example of a control circuit coupled to a memory structure via wafer-to-wafer bonding is shown.

[0009] Figure 7 A simple example of a convolutional neural network (CNN) is shown.

[0010] Figure 8 A simple example of a fully connected layer in an artificial neural network is shown.

[0011] Figure 9A It is a flowchart describing one implementation of the process used to train a neural network to generate a set of weights.

[0012] Figure 9B This is a flowchart describing an implementation of a process for reasoning using a neural network.

[0013] Figure 10 This is a schematic diagram of the convolution operation in a convolutional neural network.

[0014] Figure 11 This is a schematic diagram illustrating the application of matrix multiplication in the fully connected layer of a neural network.

[0015] Figures 12A to 12C An implementation of a portion of a memory array with a cross-point architecture is described.

[0016] Figure 13 It is a top view of the crosspoint array and some control circuitry used to operate the array.

[0017] Figure 14 This is a schematic diagram of a first implementation scheme for a memory cell design in a multi-stage ultra-low power inference engine.

[0018] Figure 15 This is a schematic diagram of another set of implementations using a memory cell design with a grouped dielectric layer that can be used in a multi-stage ultra-low power inference engine.

[0019] Figure 16 A semi-selection scheme for choosing a memory cell to be programmed is shown.

[0020] Figure 17 This is a schematic diagram of an implementation scheme for writing a sequence of cross-point memory having memory cells formed by multiple dielectric layers.

[0021] Figures 18A to 18C The example demonstrates a write mechanism utilizing dielectric breakdown in a two-level layer.

[0022] Figure 19A and Figure 19B The example demonstrates a write mechanism utilizing dielectric breakdown in a three-layered layer.

[0023] Figure 20 This is a flowchart illustrating an implementation of a cross-point architecture that uses memory cells with an ultra-thin dielectric layer for the inference engine.

[0024] Figure 21 This is a schematic diagram of an antifuse based on a single FET and an implementation scheme for biasing to program the antifuse.

[0025] Figure 22 It is a combination of Figure 21A top view of a portion of an antifuse-based memory cell cross-point array structure.

[0026] Figure 23 This is a flowchart illustrating an implementation of a crosspoint architecture programming scheme that uses memory cells with FET-based antifuse for the inference engine. Detailed Implementation

[0027] When a neural network performs inference or training operations, it performs a large amount of computation involving a significant amount of data, especially in the case of deep neural networks or DNNs, which involve a very large number of computational layers, such as convolutional and fully connected layers, used to propagate the input. To avoid large amounts of data being moved in and out of the memory device, the weights of the neural network layers are stored in a non-volatile memory array of the memory device, and the computation of each layer is performed on that device. To further improve performance, in terms of increasing speed and reducing power consumption, an implementation of a multilevel memory cell (MLC) is provided below, which can be used to perform intra-array vector multiplication between weights and inputs during the inference operation of the neural network, and does require special on-chip programming. The vector multiplication device for the inference engine includes an array of crosspoints with memory cells at each crosspoint junction. The control circuitry is configured to provide an independent input voltage to each line in one line plane (i.e., word line) and read the current on each line in another line plane (i.e., bit line). Vector multiplication is performed in the crosspoint memory array as input voltage multiplied by matrix weights encoded by the memory cells. The memory cells in the crosspoint array can be programmed individually using the input voltage. In a first set of embodiments, a multi-level memory cell is formed from multiple ultrathin dielectric layers separated by metal layers, wherein programming of the memory cell is accomplished by selectively breaking down one or more dielectric layers by selecting a write voltage level. In a second set of embodiments, the memory cell is formed as an antifuse.

[0028] Figure 1 This is a block diagram of one embodiment of a memory system 100 connected to a host 120. The memory system 100 implements the techniques presented herein, wherein neural network input or other data is received from the host 120. According to the embodiment, input may be received from the host 120 and then provided to a memory package 104 for inference on weights previously programmed into a memory array of the memory package 104. Many different types of memory systems can be used with the techniques presented herein. Exemplary memory systems include solid-state drives (“SSDs”), memory cards, and embedded memory devices; however, other types of memory systems may also be used.

[0029] Figure 1The memory system 100 includes a controller 102, non-volatile memory 104 for storing data, and local memory (e.g., DRAM / ReRAM) 106. The controller 102 includes front-end processor (FEP) circuitry 110 and one or more back-end processor (BEP) circuits 112. In one embodiment, the FEP circuitry 110 is implemented on an ASIC. In one embodiment, each BEP circuit 112 is implemented on a separate ASIC. In other embodiments, a unified controller ASIC may combine both front-end and back-end functions. The ASICs for each of the BEP circuitry 112 and the FEP circuitry 110 are implemented on the same semiconductor, such that the controller 102 is fabricated as a system-on-a-chip (“SoC”). Both the FEP circuitry 110 and the BEP circuitry 112 include their own processors. In one embodiment, the FEP circuitry 110 and the BEP circuitry 112 are used in a master-slave configuration, where the FEP circuitry 110 is the master device and each BEP circuitry 112 is a slave device. For example, FEP circuit 110 implements a Flash Translation Layer (FTL) or Media Management Layer (MML) that performs memory management (e.g., garbage collection, wear leveling, etc.), logic-to-physical address translation, communication with the host, DRAM (Local Volatile Memory) management, and overall operation management of the SSD (or other non-volatile memory system). BEP circuit 112 manages memory operations within the memory package / die based on requests from FEP circuit 110. For example, BEP circuit 112 can perform read, erase, and program processes. Additionally, BEP circuit 112 can perform buffer management, set specific voltage levels required by FEP circuit 110, perform error correction (ECC), control the switching mode interface to the memory package, etc. In one embodiment, each BEP circuit 112 is responsible for its own set of memory packages.

[0030] In one embodiment, the non-volatile memory 104 includes multiple memory packages. Each memory package includes one or more memory dies. Therefore, the controller 102 is connected to one or more non-volatile memory dies. In one embodiment, each memory die in the memory package 104 utilizes NAND flash memory (including two-dimensional NAND flash memory and / or three-dimensional NAND flash memory). In other embodiments, the memory package may include other types of memory, such as storage-class memory (SCM) or phase-change memory (PCM) based on resistive random access memory (such as ReRAM, MRAM, FeRAM, or RRAM).

[0031] Controller 102 communicates with host 120 via interface 130, which implements protocols such as NVM Express (NVMe) over PCI Express (PCIe). To work with memory system 100, host 120 includes host processor 122, host memory 124, and PCIe interface 126 connected along bus 128. Host memory 124 is the host's physical memory and can be DRAM, SRAM, non-volatile memory, or another type of storage device. Host 120 is external to and separate from memory system 100. In one embodiment, memory system 100 is embedded within host 120.

[0032] Figure 2 This is a block diagram of one implementation scheme of FEP circuit 110. Figure 2 A PCIe interface 150 communicating with a host 120 and a host processor 152 communicating with the PCIe interface are shown. The host processor 152 can be any type of processor known in the art suitable for implementation. The host processor 152 communicates with a network on-chip (NOC) 154. An NOC is a communication subsystem on an integrated circuit, typically between cores in a SoC. NOCs can span synchronous and asynchronous clock domains or use non-clocked asynchronous logic. NOC technology applies network theory and methods to on-chip communication and delivers significant improvements compared to conventional bus and cross-switch interconnects. Compared to other designs, NOCs improve the scalability of SoCs and the power efficiency of complex SoCs. The wires and links of a NOC are shared by many signals. Because all links in a NOC can operate simultaneously on different data packets, a high degree of parallelism is achieved. Therefore, as the complexity of integrated subsystems increases, NOCs offer enhanced performance (such as throughput) and scalability compared to previous communication architectures (e.g., dedicated point-to-point signal lines, shared buses, or segmented buses with bridges). Connected to and communicating with the NOC 154 are the memory processor 156, SRAM 160, and DRAM controller 162. The DRAM controller 162 operates and communicates with the DRAM (e.g., DRAM 106). SRAM 160 is local RAM used by the memory processor 156. The memory processor 156 runs the FEP circuitry and performs various memory operations. Two PCIe interfaces, 164 and 166, also communicate with the NOC. Figure 2 In one implementation, the SSD controller will include two BEP circuits 112; therefore, there are two PCIe interfaces 164 / 166. Each PCIe interface communicates with one of the BEP circuits 112. In other implementations, there may be more or fewer than two BEP circuits 112; therefore, there may be more than two PCIe interfaces.

[0033] FEP circuit 110 may also include a flash translation layer (FTL), or more generally a media management layer (MML) 158, which performs memory management (e.g., garbage collection, wear leveling, load balancing, etc.), logic-to-physical address translation, communication with the host, DRAM (local volatile memory) management, and overall operation management of SSDs or other non-volatile storage systems. The media management layer (MML) 158 may be integrated as part of memory management that can handle memory errors and interact with the host interface. Specifically, the MML may be a module within FEP circuit 110 and may be responsible for the internals of memory management. Specifically, MML 158 may include algorithms in the memory device firmware that translate writes from the host into writes to the die's memory structure (e.g., the following...). Figure 5 Writes to memory (326) may be necessary with MML 158 because: 1) the memory may have limited endurance; 2) the memory structure may only have multiple pages written to it; and / or 3) the memory structure may not need to be written to unless it is erased as a block. MML 158 understands these potential limitations of the memory structure, which may not be visible to the host. Therefore, MML 158 attempts to translate writes from the host into writes to the memory structure.

[0034] Figure 3 This is a block diagram of one implementation scheme of BEP circuit 112. Figure 3 This is shown for communicating with FEP circuit 110 (e.g., with...). Figure 2The PCIe interface 200 communicates with one of the PCIe interfaces 164 and 166. The PCIe interface 200 communicates with two NOCs 202 and 204. In one embodiment, the two NOCs can be combined into a single large NOC. Each NOC (202 / 204) is connected to SRAM (230 / 260), buffers (232 / 262), a processor (220 / 250), and a data path controller (222 / 252) via an XOR engine (224 / 254) and an ECC engine (226 / 256). The ECC engine 226 / 256 is used to perform error correction, as is known in the art. The XOR engine 224 / 254 is used to perform XOR on the data, enabling data to be combined and stored in a recoverable manner in the event of a programming error. The data path controller 222 is connected to the interface module for communication with the memory package via four channels. Therefore, the top NOC 202 is associated with interface 228 for four channels for communicating with the memory package, and the bottom NOC 204 is associated with interface 258 for four additional channels for communicating with the memory package. Each interface 228 / 258 includes four switching mode interfaces (TM interfaces), four buffers, and four schedulers. One scheduler, buffer, and TM interface exist for each of the channels. The processor can be any standard processor known in the art. The data path controllers 222 / 252 can be a processor, FPGA, microprocessor, or other type of controller. The XOR engines 224 / 254 and ECC engines 226 / 256 are dedicated hardware circuits referred to as hardware accelerators. In other embodiments, the XOR engines 224 / 254 and ECC engines 226 / 256 can be implemented in software. The schedulers, buffers, and TM interfaces are hardware circuits.

[0035] Figure 4 This is a block diagram of one embodiment of a memory package 104 including multiple memory dies 292 connected to a memory bus (data lines and chip enable lines) 294. The memory bus 294 is connected to a switching mode interface 296 for communication with the TM interface of the BEP circuit 112 (see, for example...). Figure 3 In some implementations, the memory package may include a small controller connected to the memory bus and the TM interface. The memory package may have one or more memory dies. In one implementation, each memory package includes eight or 16 memory dies; however, other numbers of memory dies may also be implemented. The techniques described herein are not limited to any particular number of memory dies.

[0036] Figure 5 This is a functional block diagram of one implementation scheme of the memory die 300. Figure 5The components depicted are electronic circuits. In one embodiment, each memory die 300 includes a memory structure 326, control circuitry 310, and read / write circuitry 328. The memory structure 326 is addressable via word lines through a row decoder 324 and via bit lines through a column decoder 332. The read / write circuitry 328 includes a plurality of sensing blocks 350 that allow memory cell pages to be read or programmed in parallel; these sensing blocks include SB1, SB2, ..., SBp (sensing circuitry). Commands and data are transmitted between the controller and the memory die 300 via line 318. In one embodiment, the memory die 300 includes a set of input and / or output (I / O) pins connected to line 318.

[0037] Control circuitry 310 cooperates with read / write circuitry 328 to perform memory operations (e.g., write, read, etc.) on memory structure 326, and includes state machine 312, on-chip address decoder 314, and power control circuitry 316. State machine 312 provides die-level control of memory operations. In one embodiment, state machine 312 may be software-programmable. In other embodiments, state machine 312 is not implemented in software and is entirely implemented in hardware (e.g., electronic circuitry). In yet another embodiment, state machine 312 is replaced by a microcontroller.

[0038] The on-chip address decoder 314 provides an address interface between the address used by controller 102 and the hardware address used by decoders 324 and 332. The power control module 316 controls the power and voltage supplied to the word lines and bit lines during memory operations. The power control module 316 may include a charge pump for generating voltage. The sensing block includes bit line drivers.

[0039] For the purposes of this document, the phrase “one or more control circuits” may include a controller, state machine, microcontroller and / or control circuit 310, or other similar circuitry for controlling non-volatile memory.

[0040] In one embodiment, memory structure 326 includes a three-dimensional memory array of non-volatile memory cells, wherein multiple memory stages are formed over a single substrate, such as a wafer. The memory structure may include any type of non-volatile memory, which is integrally formed in one or more physical stages of memory cells having active regions disposed over a silicon (or other type) substrate. In one example, the non-volatile memory cells include vertical NAND strings with charge-trapping material, such as that described, for example, in U.S. Patent 9,721,662, the entire contents of which are incorporated herein by reference.

[0041] In another embodiment, memory structure 326 includes a two-dimensional memory array of non-volatile memory cells. In one example, the non-volatile memory cells are NAND flash memory cells utilizing floating gates, such as those described, for example, in U.S. Patent 9,082,502, the entire contents of which are incorporated herein by reference. Other types of memory cells (e.g., NOR flash memory) may also be used.

[0042] The exact type of memory array architecture or memory cell included in memory structure 326 is not limited to the examples described above. Many different types of memory array architectures or memory technologies can be used to form memory structure 326. Implementing the new embodiments claimed herein does not require a specific non-volatile memory technology. Other examples of suitable technologies for the memory cells of memory structure 326 include ReRAM (Resistive Random Access Memory), magnetoresistive memory (e.g., MRAM, spin-torque MRAM, spin-orbit torque MRAM), FeRAM, phase-change memory (e.g., PCM), etc. Examples of suitable technologies for the memory cell architecture of memory structure 326 include two-dimensional arrays, three-dimensional arrays, cross-point arrays, stacked two-dimensional arrays, vertical bitline arrays, etc.

[0043] One example of a ReRAM crosspoint memory includes reversible resistive switching elements arranged in a crosspoint array accessed by X-lines and Y-lines (e.g., word lines and bit lines). In another embodiment, the memory cell may include a conductive bridge memory element. A conductive bridge memory element may also be referred to as a programmable metallized cell. Based on the physical repositioning of ions within a solid electrolyte, the conductive bridge memory element can be used as a state-changing element. In some cases, the conductive bridge memory element may include two solid metal electrodes, one relatively inert (e.g., tungsten) and the other electrochemically active (e.g., silver or copper), with a thin film of solid electrolyte between the two electrodes. As temperature increases, ion mobility also increases, leading to a decrease in the programming threshold of the conductive bridge memory cell. Therefore, the conductive bridge memory element can have a wide range of programming thresholds across the entire temperature range.

[0044] Magnetoresistive random access memory (MRAM) uses magnetic storage elements to store data. These elements are formed from two ferromagnetic thin films, also called layers, each ferromagnetic and separated by a thin insulating layer. One of these layers is a permanent magnet set to a specific polarity; the magnetization of the other layer can be changed to match the magnetization of an external magnetic field to store the memory. The memory device is constructed from a grid of such memory cells. In one embodiment for programming, each memory cell is located between a pair of write lines arranged at right angles to each other, parallel to the cell, one above and one below. When current passes through them, an induced magnetic field is generated.

[0045] Phase-change memories (PCMs) utilize the unique properties of chalcogenide glasses. One embodiment uses a GeTe-Sb₂Te₃ superlattice to achieve a non-thermal phase transition by changing the coordination state of germanium atoms using only a laser pulse (or a light pulse from another source). Therefore, the programming dose is the laser pulse. Memory cells can be suppressed by preventing them from receiving light. In other PCM embodiments, memory cells are programmed by current pulses. It should be noted that the use of "pulse" in this document does not require a rectangular pulse, but includes (continuous or discontinuous) vibrations or pulse trains of sound, current, voltage, light, or other waves.

[0046] Those skilled in the art will recognize that the techniques described herein are not limited to a single particular memory structure, memory configuration, or material composition, but encompass many related memory structures within the technical essence and scope as described herein and as understood by those skilled in the art.

[0047] Can Figure 5 The components are divided into two parts: the memory structure 326 containing the memory cells; and the peripheral circuitry, including all other components. A key characteristic of the memory circuitry is its capacity, which can be increased by increasing the area of ​​the memory die 300 reserved for the memory structure 326; however, this reduces the area of ​​the memory die 300 available for the peripheral circuitry. This can impose significant limitations on these peripheral components. For example, the need to mount sense amplifier circuitry within the available area is a major constraint on sense amplifier design architecture. The reduced available area relative to the on-die control circuitry 310 limits the available functionality that can be implemented on the chip. Therefore, a fundamental trade-off must be made between the amount of dedicated area for the memory structure 326 and the amount of dedicated area for the peripheral circuitry in the design of the memory die 300.

[0048] Another area where memory structure 326 and peripheral circuitry often diverge is the processing involved in forming these areas, as these areas typically involve different processing techniques and trade-offs in assigning different standards to individual dies. For example, when memory structure 326 is NAND flash memory, it is an NMOS structure, while the peripheral circuitry is typically CMOS-based. For instance, the sense amplifier circuitry in sense block 350, the charge pump in power control block 316, the logic elements in state machine 312, and other peripheral circuitry typically employ PMOS devices. The processing operations used to manufacture CMOS dies will differ in many ways from those optimized for NMOS flash NAND memory or other memory cell technologies.

[0049] To mitigate these limitations, the implementation scheme described below can... Figure 5The components are separated onto separately formed dies, and then these dies are bonded together. More specifically, the memory structure 326 can be formed on a single die, and some or all of the peripheral circuitry elements (including one or more control circuits) can be formed on separate dies. For example, a memory die can be formed solely of memory elements, such as flash NAND memory, PCM memory, ReRAM memory, or other memory cell arrays of other memory types. Some or all of the peripheral circuitry (even including elements such as decoders and sense amplifiers) can then be moved to separate dies. This allows each die in the memory die to be optimized individually according to its technology. For example, a NAND memory die can be optimized for an NMOS-based memory array structure without worrying about CMOS elements now moved to separate peripheral circuitry dies that can be optimized for CMOS processing. This provides more space for peripheral elements, and additional capabilities that might not be easily bonded can now be combined if peripheral elements are confined to the edges of the same die that houses the memory cell array. Two dies can then be bonded together in a bonded multi-die memory circuit, with an array on one die connected to peripheral elements on another memory circuit. For example, although the following description will focus on a combined memory circuit with one memory die and one peripheral circuit die, other implementations may use more dies, such as two memory dies and one peripheral circuit die.

[0050] Figure 6A and Figure 6B It shows Figure 5 An alternative arrangement can be achieved using wafer-to-wafer bonding to provide a bonding die pair 604. Figure 6A Examples of peripheral circuitry are shown, including control circuitry formed in the peripheral circuitry or control die 608, which is coupled to the memory structure 326 formed in the memory die 610. Common components such as... Figure 5The numbering is as described. It can be seen that control circuitry 310, read / write circuitry 328, and row decoder 324 (which can be formed using a CMOS process) are located within control die 608. Additional components, such as functionality from controller 102, can also be moved to control die 608. Control circuitry 310, read / write circuitry 328, row decoder 324, and column decoder 332 can be formed using conventional processes (e.g., CMOS processes), making it possible to add components and functions more commonly found on memory controller 102 with few or no additional process steps (i.e., the same process steps used to manufacture controller 102 can also be used to manufacture control circuitry 310, read / write circuitry 328, and row decoder 324). Therefore, while removing such circuitry from a die (such as memory die 300) reduces the number of steps required to manufacture such a die, adding such circuitry to a die (such as control die 608) may not require any additional process steps.

[0051] Figure 6A A read / write circuit 328 on a control die 608 is shown, coupled to a memory structure 326 on a memory die 610 via an electrical path 612. For example, electrical path 612 can provide an electrical connection between the read / write circuit 328 and a bit line of the memory structure 326. The electrical path can extend from the read / write circuit 328 in the control die 608 through pads on the control die 608 that bond to corresponding pads on the memory die 610 that are connected to bit lines of the memory structure 326. Each bit line of the memory structure 326 may have a corresponding electrical path in electrical path 612, including a pair of bonded pads connected to the read / write circuit 328. Similarly, a row decoder circuit 324 is coupled to the memory structure 326 via electrical path 614. Each electrical path in electrical path 614 may correspond to a word line, a dummy word line, or a selected gate line. Additional electrical paths may also be provided between the control die 608 and the memory die 610.

[0052] Figure 6B This is a block diagram illustrating further details of the arrangement of an integrated memory assembly with die pair 604. Memory die 610 includes a plane 620 or array of memory cells. Memory die 610 may have additional planes or arrays. A representative bit line (BL) and a representative word line (WL) are shown for each plane or array 620. Each plane or array 620 may have thousands or tens of thousands of such bit lines. In one embodiment, the array or plane represents a group of memory cells sharing a common set of uninterrupted word lines and uninterrupted bit line connections.

[0053] The control die 608 includes a plurality of sense amplifiers (SA) 350. In some embodiments, each sense amplifier 350 is connected to a bit line or may be connected to multiple bit lines. The sense amplifier includes a bit line driver. Therefore, the sense amplifier can provide a voltage to the bit line to which it is connected. The sense amplifier is configured to sense the condition of the bit line. In one embodiment, the sense amplifier is configured to sense a current flowing in the bit line. In one embodiment, the sense amplifier is configured to sense a voltage on the bit line.

[0054] The control die 608 includes a plurality of word line drivers 660(1)-660(n). The word line drivers 660 are configured to provide voltage to the word lines. In this example, each array or planar memory cell has “n” word lines. In one embodiment, if the memory operation is programming or reading, one word line within the selected block is selected for the memory operation. In one embodiment, if the memory operation is erasing, all word lines within the selected block are selected for erasing. The word line drivers 660 (e.g., part of the power controller 316) provide voltage to the word lines in the memory die 610. As described above relative to… Figure 6A The control die 608 discussed may also include a charge pump, a voltage generator, etc. Figure 6B (Not shown in the text), it can be used to provide voltage for word line driver 660 and / or bit line driver.

[0055] The memory die 610 has a plurality of bonding pads 670a, 670b on its first main surface 682. There may be “n” bonding pads 670a to receive voltage from corresponding “n” word line drivers 660(1)-660(n). Each bit line associated with plane 620 may have one bonding pad 670b. Reference numeral 670 will be used generally to refer to the bonding pads on the main surface 682.

[0056] In some implementations, each data bit and each parity bit of the codeword is transmitted via different bonding pads 670b and 674b. The bits of the codeword can be transmitted in parallel via bonding pads 670b and 674b. This provides a very efficient data transfer compared to, for example, transferring data between memory controller 102 and integrated memory component 604. For example, the data bus between memory controller 102 and integrated memory component 604 may be provided for, for example, eight bits, sixteen bits, or possibly 32 bits to be transmitted in parallel. However, the data bus between memory controller 102 and integrated memory component 604 is not limited to these examples.

[0057] The control die 608 has a plurality of bonding pads 674a, 674b on its first main surface 684. There may be “n” bonding pads 674a to deliver voltage from the corresponding “n” word line drivers 660(1)-660(n) to the memory die 610. Each bit line associated with plane 620 may have one bonding pad 674b. Reference numeral 674 will be used generally to refer to the bonding pads on the main surface 682. Note that bonding pad pairs 670a / 674a and bonding pad pairs 670b / 674b may exist. In some embodiments, bonding pads 670 and / or 674 are flip-chip bonding pads.

[0058] In one embodiment, the pattern of bonding pad 670 matches the pattern of bonding pad 674. Bonding pad 670 is bonded (e.g., flip-chip bonding) to bonding pad 674. Thus, bonding pads 670 and 674 electrically and physically couple memory die 610 to control die 608.

[0059] Furthermore, bonding pads 670 and 674 allow internal signal transmission between memory die 610 and control die 608. Therefore, memory die 610 and control die 608 are bonded together using bonding pads. Although Figure 6A One embodiment depicts a control die 608 coupled to a memory die 610, but in another embodiment, a control die 608 is coupled to multiple memory dies 610.

[0060] In this document, "internal signal transmission" refers to the signal transmission between control die 608 and memory die 610. Internal signal transmission allows circuitry on control die 608 to control memory operations in memory die 610. Therefore, bonding pads 670 and 674 can be used for memory operation signal transmission. In this document, "memory operation signal transmission" refers to any signal related to memory operations in memory die 610. Memory operation signal transmission may include, but is not limited to, providing voltage, providing current, receiving voltage, receiving current, sensing voltage, and / or sensing current.

[0061] The bonding pads 670 and 674 can be formed of, for example, copper, aluminum, or their alloys. A pad may be present between the bonding pads 670 and 674 and the main surfaces (682, 684). The pad may be formed of, for example, titanium / titanium nitride stacks. The bonding pads 670 and 674 and the pads can be applied by vapor deposition and / or electroplating techniques. The bonding pads and pads together can have a thickness of 720 nm, but in other embodiments this thickness can be greater or less.

[0062] Metal interconnects and / or vias can be used to electrically connect various components in the die to bonding pads 670, 674. Several conductive paths that can be implemented using metal interconnects and / or vias are described. For example, a sense amplifier 350 can be electrically connected to bonding pad 674b via path 664. Relative to Figure 6A Electrical path 612 may correspond to path 664, bonding pad 674b, and bonding pad 670b. There may be thousands of such sense amplifiers, paths, and bonding pads. Note that the BL is not necessarily directly connected to bonding pad 670b. The word line driver 660 can be electrically connected to bonding pad 674a via path 662. Relative to Figure 6A Electrical path 614 may correspond to path 662, bonding pad 674a, and bonding pad 670a. Note that path 662 may include a separate conductive path for each word line driver 660(1)-660(n). Similarly, a separate bonding pad 674a may exist for each word line driver 660(1)-660(n). Word lines in block 2 of memory die 610 may be electrically connected to bonding pad 670a via path 664. Figure 6B In the block, for the corresponding "n" word lines, there are "n" paths 664. For each path 664, there can be a separate pair of bonding pads 670a and 674a.

[0063] Compared to Figure 5 , Figure 6A The on-die control circuitry may also include additional functions within its logic elements, including more general capabilities than those commonly found in memory controller 102, some CPU capabilities, and application-specific functions. For example, these activation functions may include addition operations and threshold determination operations used in the accumulation portion of multiplication and accumulation (MAC) operations, as well as more advanced operations (such as sigmoid functions or tanh functions).

[0064] In the following text, state machine 312 and / or controller 102 (or equivalent functional circuit) are... Figure 6A All or subsets of other circuits shown on the control die 608 in the circuit and Figure 5 Similar elements combined in this document can be considered as part of one or more control circuits that perform the functions described herein. Control circuitry may consist solely of hardware or a combination of hardware and software (including firmware). For example, a controller programmed by firmware to perform the functions described herein is an example of control circuitry. Control circuitry may include processors, FGAs, ASICs, integrated circuits, or other types of circuitry.

[0065] Now let's turn to data types that can be stored in non-volatile memory devices. In the following discussion, a specific example of the data types we're interested in is the weights used in artificial neural networks such as convolutional neural networks (CNNs). The name "convolutional neural network" refers to the fact that the network employs a mathematical operation called convolution, a specialized linear operation. A convolutional network is a neural network in which, in at least one of its layers, convolutions replace general matrix multiplications. A CNN consists of an input layer, an output layer, and multiple intermediate hidden layers. The hidden layers of a CNN are typically a series of convolutional layers that "convolve" multiplications or other dot products.

[0066] Each neuron in a neural network computes an output value by applying a specific function to the input value from the receptive domain in the previous layer. The function applied to the input value is determined by weight vectors and biases. In a neural network, learning occurs by iteratively adjusting these biases and weights. The weight vectors and biases are called filters and represent specific features of the input (e.g., a specific shape). A distinguishing feature of CNNs is that many neurons can share the same filters.

[0067] Figure 7 This is a schematic diagram of an example of a CNN. Figure 7 The diagram shows an initial input image with an array of pixel values, followed by multiple convolutional layers, then multiple fully connected layers, with the last layer providing the output. Each neuron in the first convolutional layer (Convolutional Layer 1) takes input data from an n×n pixel sub-region of the input image. The weights learned by the neuron (collectively referred to as its convolutional filter) determine the neuron's single-valued output in response to that input. In this convolutional layer, the neuron's filter is applied to the input image by sliding the input region along the x and y dimensions of the image to generate the values ​​for the convolutional layer. In practice, equivalent convolution is typically achieved by the neuron making static copies of the same input region for different regions. This process is repeated through each convolutional layer (Convolutional Layer 1 to Convolutional Layer N) using the learned weights from each layer, and then propagated through fully connected layers (L1 to LM) using the learned weights from each layer.

[0068] Figure 8 Several fully connected layers of the neural network are shown in more detail. Figure 8In the diagram, the three-layer artificial neural network is represented as interconnected groups of nodes or artificial neurons (circled), and a set of connections from the output of one artificial neuron to the input of another. This example shows three input nodes (I1, I2, I3) and two output nodes (O1, O2), as well as an intermediate layer with four hidden or intermediate nodes (H1, H2, H3, H4). The nodes or artificial neurons / synapses of an artificial neural network are implemented as mathematical functions by logical elements of a host or other processing system. These mathematical functions receive one or more inputs and sum them to produce an output. Typically, each input is weighted individually, and the sum is passed through the node's mathematical function to provide the node's output.

[0069] In common artificial neural network implementations, the signals at the connections between nodes (artificial neurons / synapses) are real numbers, and the output of each artificial neuron is calculated as a nonlinear function of the sum of its inputs. Nodes and their connections typically have weights that adjust as the learning process progresses. Weights increase or decrease the signal strength at the connection. Nodes may have thresholds such that a signal is only sent when the aggregated signal exceeds that threshold. Typically, nodes are clustered into layers. Different layers can perform different types of transformations on their inputs. The signal may travel from the first layer (input layer) to the last layer (output layer) after multiple traversals of the layers. Although Figure 7 Only a single intermediate or hidden layer is shown, but complex deep neural networks (DNNs) can have many such intermediate layers.

[0070] Supervised artificial neural networks are "trained" by feeding inputs and then examining and correcting the outputs. For example, a neural network trained to identify dog ​​breeds will process a set of images and calculate the probability that the dogs in the images are of a particular breed. Users can view the results and select which probabilities the network should display (above a certain threshold, etc.), and the network will return suggested labels. Each such mathematical transformation is considered a layer, and complex neural networks have many layers. Due to the depth provided by the large number of intermediate or hidden layers, neural networks can model complex nonlinear relationships during training.

[0071] Figure 9A This is a flowchart describing one implementation of a process for training a neural network to generate a set of weights. This training process is often performed in the cloud to enable additional or more powerful processing to be used. At step 901, input is received, such as a set of images (e.g., Figure 7 The image input in the image is used. At step 903, the current filter or weight reassembly is used to connect the input to the next layer (e.g., ...). Figure 7 The input is propagated through the convolutional layer 1 in the middle. Then in step 905, the input is propagated through the next layer (e.g., ...). Figure 7The output of the neural network is received at convolutional layer 2, such that the value received as the output from one layer is used as the input to the next layer. The input from the first layer propagates through all intermediate or hidden layers in this way until they reach the output. In the dog breed example in the previous paragraph, the input would be image data of many dogs, and the intermediate layers use the current weight values ​​to calculate the probability that the dog in the image is a certain breed, and return the proposed dog breed label at step 905. The user can then view the results at step 907 to select which probabilities the neural network should return and determine whether the current weight reassembly provides a sufficiently accurate label, and if so, training is complete (step 911). If the results are not accurate enough, the neural network adjusts the weights at step 909 based on the probabilities selected by the user, and then loops back to step 903 to run the input data again using the adjusted weights. Once the weight reassembly of the neural network has been determined, the weights can be used for "inference," which is the process of using the determined weights to generate an output result based on the data input to the neural network. Once the weights are determined in step 911, they can be stored in non-volatile memory for later use. The process of storing these weights in non-volatile memory is discussed in further detail below.

[0072] Figure 9B This is a flowchart describing a supervised learning process using a neural network for the inference phase, where the estimated accuracy is used to predict the "meaning" of the input data. Depending on the situation, the neural network may be in the cloud and the inference may be performed by a processor on an edge device (e.g., a smartphone, automotive manufacturing, hardware accelerator). At step 921, input is received, such as an image of a dog in the example used above. If the previously determined weights are not present on the device running the neural network application, they are loaded at step 922. For example, on the host processor executing the neural network, the weights may be read from an SSD storing the weights and loaded into RAM on the host device. At step 923, the input data is then propagated through the layers of the neural network. Step 923 will be similar to... Figure 9B Step 903, but now using the weights established at the end of the training process in step 911. After propagating the input through the intermediate layers, the output is then provided in step 925.

[0073] Figure 10This is a schematic diagram of a convolution operation between an input image and a filter or weighted array. In this example, the input image is a 6×6 array of pixel values, and the filter is a 3×3 array of weights. The convolution operation is performed by multiplying the 3×3 filter by a matrix of 3×3 blocks of the input image. For example, multiplying the top-left 3×3 block of the image with the filter yields the top-left value of the output matrix. The filter can then slide one pixel across the image to generate the next entry for the output, and so on to generate the top 4 elements for the output. This process is repeated by sliding the filter down one pixel at a time to generate a 4×4 output matrix. A similar operation is performed for each layer. In practical CNNs, the size of the dataset and the number of convolutions performed mean that a very large number of such operations are performed, thus involving a very large amount of data.

[0074] Figure 11 This is a schematic diagram illustrating the application of matrix multiplication in the fully connected layers of a neural network. Matrix multiplication, or MatMul, is a commonly used method in the training and inference phases of neural networks and is also used in core methods of machine learning. Figure 11 The top is similar to Figure 8 The algorithm presents a single hidden layer between the input and output layers. Input data is represented as a vector with a length corresponding to the number of input nodes. Weights are represented by a weight matrix, where the number of columns corresponds to the number of intermediate nodes in the hidden layer, and the number of rows corresponds to the number of input nodes. The output is determined by matrix multiplication of the input vector and the weight matrix, where each element of the output vector is the dot product of a component of the input data and a column of the weight matrix.

[0075] A common technique for performing matrix multiplication is to use a multiplier-accumulator (MAC, or MAC unit). However, this has many problems. See again. Figure 9B In the inference phase, neural network weights are loaded at step 922, and matrix multiplication is performed via propagation at step 923. However, due to the potentially very large amount of data involved, using a multiplier-accumulator for inference introduces several problems related to weight loading. One of these problems is the high energy dissipation caused by the need to use a large MAC array with the required bit width. Another problem is the high energy dissipation due to the finite size of the MAC array, resulting in high data movement between logic and memory, and energy dissipation that can be much higher than the energy dissipation used in the logic computation itself.

[0076] To help avoid these limitations, alternative memory technologies can be used to replace the use of multiplier-accumulator arrays. For example, by leveraging the characteristics of NAND memory and storage-class memory (SCM), such as memory based on ReRAM, PCM, FeRAM, or MRAM, matrix multiplication can be computed within the memory array. This allows neural network input to be provided via read commands and allows preloading of neural weights for inference. By using in-memory computing, this eliminates the need for the logic required to perform matrix multiplication in the MAC array and the need to move data between the memory and the MAC array.

[0077] The following presents an implementation of an in-memory computed DNN that performs multiplication between the input values ​​and weights of inference operations within an array of memory cells by storing the weights of the neural network layers within memory cells and applying the layer inputs as voltage levels to the word lines of the array. The resulting current on the bit lines corresponds to the product of the input and weights, where multiple such multiplications can be performed simultaneously. To reduce the power consumption of such inference engines, the following discussion presents an implementation based on multi-level memory cells formed by multiple ultrathin dielectric layers separated by metal layers, which are part of an array with a cross-point structure. The memory cell implementation has well-separated low-resistance and high-resistance states, while still maintaining high resistance even for the low-resistance state, thus allowing for very low power consumption during inference. To store the weights of the DNN, the memory cells can be individually programmed using the input voltage by selectively breaking down one or more dielectric layers in the dielectric layers by selecting a programming voltage.

[0078] The use of a cross-point architecture allows for arrays with a smaller footprint, where vector multiplication is implemented via a voltage-to-current conversion process, and matrix weights are encoded in the resistance values ​​of the cells in the array. The memory cells can have a memory window with a coefficient of 2 or greater between the lowest and highest resistance states, while still having a lowest resistance state where the resistance is very high, such as R... LRS >10 MOhm. Compared to other memory cell technologies, the memory cell described herein allows for multi-level cells (MLCs) with less stringent programming requirements and can be programmed with relatively low-power on-chip circuitry. Furthermore, the programming method allows for device selection without requiring dedicated selection transistors for each bit, thus reducing the footprint per bit. Since the memory device is used for inference, it can be pre-programmed with pre-determined DNN weight values ​​before being supplied to the customer.

[0079] More specifically, an implementation of a crosspoint inference engine with multi-level memory and electric field programming is presented. The design is for an ultra-low-power, low-cost multi-level cell (MLC) vector multiplication device that does not require special on-chip programming. The inference engine or vector multiplication device consists of a crosspoint array having cells at each crosspoint junction and control (or peripheral) circuitry configured to supply an independent input voltage to each line in one line plane (i.e., word lines) and read the current on each line in another line plane (i.e., bit lines). Vector multiplication involves multiplying the input voltage by matrix weights encoded by the memory cells. The control circuitry and the memory die holding one or more such arrays can be part of the same die or a die pair of the control die and the memory die, as described in the reference above. Figure 6A and Figure 6B The embodiments presented herein allow for the individual programming of a crosspoint array memory cell to one of a plurality of resistive states using only an input voltage. In a first set of embodiments described below, the MLC memory cell is formed of a plurality of ultrathin dielectric layers separated by metal layers, wherein programming of the memory cell is accomplished by selecting a programming voltage level to selectively break down one or more dielectric layers, thereby setting the resistance of the memory cell. In another set of embodiments, the memory cell is antifuse-based, wherein programming of the antifuse-based memory cell is performed by a current flowing through each cell and setting a compliance limit for that current.

[0080] Figure 12A An implementation of a portion of the memory array 326 forming the cross-point architecture is depicted in a perspective view. Figure 12A The memory array 326 is Figure 5 An example of a specific implementation of the memory array 326. Bit lines BL1-BL5 are arranged along a first direction (represented as entering a page), and word lines WL1-WL5 are arranged along a second direction perpendicular to the first direction. Figure 12A This is an example of a horizontal crossover structure, where word lines WL1-WL5 and BL1-BL5 both run horizontally relative to the substrate, while memory cells (two of which are shown at 1201) are oriented such that current flows vertically. In a memory array with additional layers of memory cells, corresponding additional layers for bit lines and word lines will exist.

[0081] like Figure 12AThe depicted memory array 326 includes a plurality of memory cells 1201. Memory cells 1201 may include rewritable memory cells, such as those implemented using ReRAM, MRAM, PCM, or other materials with programmable resistors. The following discussion will focus on the specific structure of non-volatile memory cells, although much of the discussion can be applied more generally. The current in the memory cells of the first memory level can be as shown by arrow I. cell As shown, it flows upwards.

[0082] Figure 12B and Figure 12C Show each Figure 12A Side and top views of the intersection structure. Figure 12B The side view shows a bottom conductor or word line WL1 and a top conductor or bit line BL1-BL2. n At the intersection of each top and bottom wire is a multi-level memory cell 1201. Figure 12C This shows m bottom wires WL1-WL m and n top wires BL1-BL n A top view of the intersection structure. In a multi-level implementation, the memory cell at each intersection can be programmed to one of two or more resistance states, such as levels 2-5. Further details of the memory cell design implementation and its programming techniques are given below.

[0083] Figures 12A to 12C The array structure can be used to perform array multiplication for inference operations, where the matrix weights of the neural network layers are stored in memory cells as their resistances. The voltage vector... V i in The input applied to the word line generates a current at the bit line's output, which is a vector multiplication of the input voltage and weights encoded by the memory cell resistors. These weights are encoded as conductivity. b i,j， It is the word line WL i and bit line BL j The resistance of the memory cell at the intersection R i,j The reciprocal: . (Formula 1) If the input to the layers of a neural network is encoded as applied to the corresponding word line WL i voltage V i in Then each bit line BL j The current on it will be the following product: , (Formula 2) Among them, targetingi Summation. According to this embodiment, the weights can be binary values ​​stored in binary memory cells, or they can be multi-level values ​​stored in MLC memory cells, as described below. Similarly, according to this embodiment, the input can be a binary value or a value with a higher number of values. By performing sensing operations simultaneously on multiple selected word lines, the crosspoint structure will automatically calculate the input-weight product of this layer.

[0084] It should be noted that, according to this arrangement, multiple memory cells along the bit lines are simultaneously selected for sensing along with the corresponding input voltage applied to their respective word lines. Current I j out This corresponds to the combined result of all these memory cells, and therefore the process differs from a regular read operation that determines the state of a single memory cell. Based on this type of reasoning operation... I j out The combination result will be determined, and the contribution of each memory cell will be unknown.

[0085] Figure 13 This is a top view of the crosspoint array and some control circuitry used to operate the array, showing a 3×3 segment of the array. At each word line WL... i and bit line BL j The intersection is the memory cell M. i,j 1301 i,j The input IN of a neural network layer i In digital-to-analog converter 1311 i The input value is converted to a voltage level to the corresponding voltage level. Input IN i It can be the initial input of the network or the output of the previous layer. In a binary implementation, the 1 input can be converted to a high read voltage V. read A zero input can be converted to a low read voltage, such as 0V or ground. In this case, the DAC 1311... i It can be a single-bit DAC. For multi-bit inputs, the DAC 1311... i It will have a corresponding number of bits. Each bit line BL j The analog current I on j out In the corresponding analog-to-digital converter ADC 1311 j The input is converted into a digital value to generate a digital output OUT. j Output value OUT j You can then proceed to digital computation block 1315, which performs operations such as accumulating multiplication and accumulation operations, applying activation, and other such operations as part of propagating the input through the neural network. See again Figure 5and Figure 6A DAC1311 i ADC j 1311 j The digital computing block 1315 can be considered as part of the row decoder 324, column decoder 332, and read / write circuitry 328, together with the control circuitry of 310. The DAC and / or ADC can be multi-bit devices capable of generating or outputting 2... n (n>1) level quantized signal.

[0086] Figure 14 This is a schematic diagram of a first embodiment of a memory cell design that can be used in a multi-stage ultra-low-power inference engine. The array uses a cross-point architecture, where each cell has multiple dielectric layers, each with a different resistance R. i and breakdown voltage V BD,i .exist Figure 14 In the diagram, the memory cell is located at the intersection between word line 1401 and bit line 1411. This memory cell has N dielectric layers, three of which are shown as follows: having R1 and V... BD,1 The dielectric 11407 has R i and V BD,i The dielectric i 1405, and having R N and V BD,N The dielectric layer is N 1403. Between these dielectric layers are conductive layers, such as 1404 and 1406. A metal layer 1402 may also be included between word line 1401 and the adjacent dielectric layer (here, dielectric N 1403), and a metal layer 1408 may also be included between bit line 1411 and the adjacent dielectric layer (here, dielectric 1 1407). In this example, the dielectric layers will be arranged such that their resistance levels gradually decrease from R1 to R... N However, other sequences can be used. The dielectric layers are formed such that the difference in resistance levels between the different layers is large enough to clearly define the different programming states.

[0087] For a given dielectric layer, the breakdown voltage V of the dielectric layer BD The dielectric layer changes from a high-resistance state to an electrical short circuit R. short << R i The voltage at that time. In an exemplary embodiment, the dielectric layer is formed such that the initial resistance (R1, …, R) is such that the voltage at that time ... N ) and breakdown strength (V BD,1 , …, V BD,N The layer is adjusted to have a lower Vt, resulting in a layer with higher resistance. BDResistance and breakdown strength can be controlled by adjusting the thickness, defect density, and material composition of each dielectric layer. Thus, the resistance of a memory cell can be set by gradually increasing the programming stress to break down more dielectric layers and reduce the overall cell resistance.

[0088] For example, consider the programming of a cell with three dielectric layers, configured such that R1 >> R2 >> R3 and V BD,1 << V BD,2 << V BD,3 In this example, almost all the voltage drops across the non-short-circuit dielectric layer with the highest resistance (or lowest subscript number). Initially, with all dielectric layers intact, the cell begins in the highest resistance state: R cell = R1 + R2 + R3 ≈ R1. To program a memory cell to have resistor R3, for example, the programming or write voltage V... wr Placed on memory cells to penetrate layers 1 and 2, but not layer 3: V BD,1 < V BD,2 < V wr < V BD,3 This keeps dielectric layer 3 intact, allowing R... cell = R3.

[0089] Figure 15 This is a schematic diagram of another set of implementations using a memory cell design with a grouped dielectric layer, which can be used in multi-stage ultra-low power inference engines. For example... Figure 14 As shown, the memory cell is located at the intersection between word line 1501 and bit line 1511 and is formed by multiple alternating dielectric and conductive layers. However, the dielectric layers are now grouped into levels that have different numbers of layers but have the same or very similar layer characteristics within each level. In this example, for level i, there will be layer i, but other embodiments may arrange them differently. Figure 15 An example of N levels is shown, each consisting of N layers, with only levels 1 and 2, and the last layer of level N explicitly shown. Dielectric level T1 has a layer L1 1527-1 separated from bit line 1511 by metal layer 1530, and separated from level 2 dielectric layers 1525-1 and 1525-2 by metal layer 1528. The second dielectric level is formed by a pair of layers L1 1525-1 and L2 1525-2, separated by inter-level metal layer 1526, and wherein level 2 is separated from the first layer of level 3 by metal layer 1524. The first layer of level 3 and other intermediary dielectric and metal layers are not shown, except for the Nth layer 1523-N of level N, which is then separated from word line 1501 by metal layer 1522.

[0090] exist Figure 15 In an exemplary implementation, the hierarchy is ordered such that lower-numbered levels have higher resistance, and higher-numbered levels may contain more layers than lower-numbered levels. For example, level T i The resistance is , (Formula 3) Where the summation is performed for n, R Ti > R T(i+1) Even if Ti has fewer layers than the level T(i+1). In one set of implementations, the breakdown voltage V of all layers is... BD They can be configured to have the same nominal value. Thus, the applied voltage n V BD It will break through all levels up to level n.

[0091] Considering the process of programming units in an example with a total of (1 + 2 + 3 =) 6 layers across three levels, these layers can be designed and formed such that R T1 >> R T2 >> R T3 And for each layer, V BD Similar. (Resistance is typically exponentially sensitive to thickness, while V...) BD It is only linearly related to thickness. (As in...) Figure 14 In the implementation scenario, a large portion of the total resistance of the memory cell will come from the entire level with the highest resistance (as shown here by the smallest subscript number), and most of the voltage applied to the memory cell will drop across this highest resistance non-short-circuit level. To program the cell to have resistance R... T3 For example, applying a programming voltage V wr To penetrate each layer in level T1 and level T2. In this case, take V. wr Make 2 V BD < V wr < 3 V BD .

[0092] Considering writing data (i.e., weight values) to the array, the implementation presented here uses the applied voltage to selectively write memory cells (such as relative to the array) Figure 14 and Figure 15The aforementioned (those) are programmed as one of a plurality of resistance values. For example, an implementation with N layers / levels can be programmed with N states, but for low-power inference with concurrent sensing along multiple word lines, it is possible to avoid all dielectric short-circuit states. However, in some implementations, additional resistors can be formed in series with the cells to avoid excessive current. A half-select scheme is used to perform electrical selection of the memory cell to be written, such as relative to... Figure 16 As shown.

[0093] Figure 16 A semi-selection scheme for selecting memory cells to be programmed is illustrated. Voltages are applied to the top line (bit line) and the bottom line (word line) such that the cell intended to be written along the bit line has a write voltage V applied to the bottom line and the top line. wr V wr The value is selected as the programming voltage to provide the target data state. Cells not written to or experiencing only half the voltage drop (V) wr / 2) or no voltage drop occurs. In Figure 16 In the example, the option to write to the bit line BL is selected. (N-1) With word lines WL1 and WL (M-1) The two units at the intersection, where V wr Apply 0V to the selected word line and 0V to the selected bit line. Other word lines and bit lines are set to ½V. wr This will V wr The differential voltage is placed on the selected memory cell, and 0V or ½V is applied. wr Placed on an unselected memory location. If V wr.max It is the maximum programming voltage and V wr.min If this is the minimum programming voltage, then these values ​​should be configured such that ½V wr.max < V wr.min Otherwise, the half-select voltage will break down a portion of the unselected cell. Programming can be performed bit-by-bit sequentially, as discussed in more detail below. (In the half-select process described herein, a low voltage level is obtained at 0V, and ½V is used.) wr The half-select value. More generally, these can be another low voltage value, and the half-select voltage level can be V. wr Other voltage levels between low and high voltage, provided the half-select process does not cause unwanted programming or reading values. By applying an appropriate programming voltage V wr This allows for selective breakdown of memory cells, portions of or all of the dielectric layers that were not selected to remain in the highest resistance state, thereby completing memory cell programming in a cross-point inference engine with multiple dielectric layers. The number of levels in each cell is controlled by the maximum half-select voltage (½V). wr.max ) and minimum selection voltage (V wr.minThe distance between ) makes ½V wr.max < V wr.min This provides a margin for error, the ability to control the resistance distribution of the dielectric layer, and the limitations of any design rules restricting the thickness of multi-level memory cells. Compared to other memory technologies, this memory cell structure allows for virtually any combination of resistance states and can form simple dielectric layers from any material commonly available in semiconductor manufacturing plants (SiO2, HfO2, MgO, etc.). While memory cells can only be programmed once, in applications as inference engines, the weights of layers in a neural network can be written before the device is delivered to the end user, such as at the manufacturing facility at the end of the testing process the device typically undergoes, or performed by a third party such as an original equipment manufacturer (OEM) that programs the weights of its DNN model into a memory array, which is used, for example, by a processor implementing a DNN in applications such as autonomous vehicles.

[0094] Figure 17 This is a schematic diagram of an implementation for writing sequences of crosspoint memory having memory cells formed by multiple dielectric layers. Programming operations that write network weights or other data from a neural network to a crosspoint array typically require accessing all or most word lines, all or most bit lines, and programming all data states, which would require multiple V for non-binary implementations. wr Values. Because all these different combinations cannot be realized simultaneously outside of a very unusual set of data values, write operations will involve sequential operations. As relative to... Figure 16 The memory cells along one or more word lines shared by a single word line can be programmed, allowing the word lines to be programmed individually or simultaneously to increase parallelism. The number of word lines that can be programmed simultaneously can be all the word lines of the array, or a smaller number if power / current considerations limit this number. In a multi-level cell implementation, a corresponding number of different Vc values ​​are used in the programming operation. wr Value. For a given bit line, multiple V values ​​can be applied simultaneously. wr Multiple data states can be programmed simultaneously; however, if not properly biased in a semi-selective implementation, simultaneous programming of multiple data states can result in programming of unselected memory cells. The exemplary embodiment here writes selected memory cells on a single bit line for all bit lines programmed to the same state. According to this embodiment, the programming sequence can be performed bit-line by bit for a given data state and then repeated to cover all data states, or for a given bit line, all data states can be programmed first, and then the process can move to the next bit line.

[0095] Figure 17 This demonstrates the sequential programming of each bit line in the array, starting with BL0 and proceeding sequentially to BL. n.like Figure 17 As noted, memory cells selected for programming on the selected bit lines are marked as S, and memory cells not selected for programming are marked as NS. This is achieved by grounding the corresponding bit lines or setting them to low voltage, while setting all other bit lines to V. wr Use the / 2 or middle "half-select" write voltage to select the bit line to be programmed. By using V... wr Apply to the corresponding word line, while setting all other word lines to V. wr / 2, to select the memory cell to be programmed on the bit line. In a multi-level cell implementation, for the selected bit line, V wr The value can advance through all corresponding programmed voltage levels. For example, in Figure 17 On the left, bit line BL1 and word lines WL2 and WL m The memory cell on is selected for writing to one of the programming states, and... Figure 17 On the right side, bit line BL n The memory cell on word line WL1 is selected for one of the write programming states.

[0096] In the implementation where all data states are written to the memory cell on the positioning line before moving to the next bit line in the sequence, the unselected word line and bit line are at ½V. wr The location is half-selected, and therefore will also be affected by the programming voltage V. wr The increments are gradually increased to accommodate different data states to be programmed. In an alternative implementation, for a V... wr After looping through all bit lines, proceed to the next V in the programmed voltage sequence. wr The loop passes through all bit lines. For a given loop that passes through bit lines, the non-selected word lines and bit lines will remain at the same ½V. wr When the bit line is selected / deselected, the bit line is at ½V. wr Switching between V and 0V, and when the programming flow loops through V wr When a given bit line is selected / deselected, the word line moves at ½V. wr With V wr Switch between them.

[0097] See again Figure 5 and Figure 6A The control circuitry involved in reading and writing memory cells may be part of the row decoder 324, column decoder 332, and read / write circuitry 328, together with the control circuitry of 310 (such as state machine 312). During a write operation, the driver / bias circuitry within the row decoder 324 and column decoder 332 may bias the word lines and bit lines, as relative to... Figure 19A As shown, this allows memory cells to be selected individually for programming.

[0098] Regarding data reading in inference operations, such as... Figure 12C Unlike determining the data state of each memory cell, the bias circuit within the line decoder 324 converts the input vectors of the network layers into a set of V... in The set of values ​​is simultaneously applied to the word lines of the memory cells of the selected group of bit lines. The column decoder 332 and read / write circuitry 328 (including the sense amplifier 350) determine the total I along each word line. out As a multi-bit sensing operation, the input vector is multiplied by the weight values ​​stored along the corresponding bit lines. Since the memory cells described herein can have high resistance even in their lowest resistance state, the current value remains relatively low and the inference process is low-power, even when sensing multiple memory cells simultaneously. The control circuitry on the memory device can then accumulate the multiplication results as part of a multiplication and accumulation operation to determine the output of a layer in the neural network, which can be the input to a subsequent layer or the output of the network itself.

[0099] Figures 18A to 18C The example demonstrates a write mechanism utilizing dielectric breakdown in a two-level layer. Figure 18A This is a schematic diagram of a two-level memory cell. The first level is a single-layer dielectric 11807 formed between conductive layers 1806 and 1808, which has a breakdown voltage V. BD And resistance R1. The second layer between conductive layers 1802 and 1806 is formed by two layers: dielectric 2A 1805 and dielectric 2B 1803, each with a breakdown voltage V. BD And resistor R2, and separated by conductive layer 1804. The programming voltage V applied to the memory cell at the corresponding word line and bit line intersection point. wr Represented by voltage source 1801, the memory cell can be in the following states: Level 0, corresponding to the highest resistance state where no dielectric layer is broken down; Level 1, where dielectric 1 1807 is broken down while dielectrics 2A 1805 and 2B 1803 remain intact; and Level 2, where dielectrics 2A 1805 and 2B 180 are also broken down. In a binary implementation, Level 0 and Level 1 are used, corresponding to the high-resistance and low-resistance states, respectively. When operating as a tri-state device, Level 0 will be the high-resistance state, Level 1 will be the medium-resistance state, and Level 2 will be the low-resistance state. When all dielectric layers are short-circuited at Level 2, Level 2 can have very low resistance and will consume more power. When operating as a tri-state memory cell, some implementations may add a series resistor to the memory cell to allow for low-power inference.

[0100] Figure 18B It is relative to the breakdown voltage V BDThe voltage at each level is normalized to the voltage applied across the entire memory cell from voltage source 1801 (also regarding V). BD A graph showing the relationship between (normalized) and (unified) parameters. Figure 18B In the example, R1 = 10R2, causing the memory cell to act as a voltage divider, and the relative voltage drop across each layer will vary with the ratio of its resistance to the total cell resistance. This is reflected in... Figure 18B In the graph, 1821 is the voltage drop across dielectric 1 1807, and 1823 is the voltage drop across each of dielectrics 2A 1805 and 2B 1803.

[0101] Observe the voltage applied across the entire memory cell (x-axis), initially the applied voltage is V. BD (1 on the normalized scale), at 1.0V BD At this point, the voltage drop across dielectric 11807 is less than V. BD This is because a portion of the voltage drops across layer 2. As the applied voltage increases, the voltage across all dielectric layers increases until the voltage across the memory cell is ~1.2V. BD Dielectric 1 1807 will break down. Once dielectric 1 1807 breaks down, there is no voltage drop across dielectric 1 1807, causing the applied voltage to drop across dielectrics 2A 1805 and 2B 1803, with half of the applied voltage dropping across each of dielectrics 2A 1805 and 2B 1803 because they have the same resistance R2.

[0102] In order to break down the dielectrics of the second-level dielectrics 2A 1805 and 2B 1803, the applied voltage will need to exceed 2V. BD Therefore, in order to program a level 1 data value into a memory cell, the programming voltage V... wr It should be ~1.2V BD <V wr < 2V BD If V bd.effi It is a hierarchy i The effective breakdown voltage, taking into account the voltage divider, is the initial breakdown voltage V on dielectric 1 1807. bd.eff.1 = V bd (R1+2 R2) / R1. After dielectric 1 1807 breaks down, all voltage drops will cross the series pair of dielectrics 2A 1805 and 2B 1803, making V bd.eff.2 = 2 V bd The condition for sufficient margin between the states of level 0 and level 1 is V.bd.eff.1 < V bd.eff.2 This makes R1 > 2 R2.

[0103] When data is programmed into, for example Figure 18A When the memory cell array shown is used, Figure 18C The application to the corresponding text above is shown. Figure 17 The write voltage V of the word line of the memory cell to be programmed wr Waveform of the step sequence. Figure 18C The waveform is the V applied to the selected word line during the write operation. wr The implementation scheme for the level is as follows: Initially, all memory cells are at level 0, and memory cells with a target state at level 0 will not be programmed from this initial state. To write to a level 1 state, the programming voltage V is... wr Apply to the selected word line so that V wr > V bd.eff.1 And V wr < 2V BD If dielectrics 2A1805 and 2B1803 also break down at level 2, then for the memory cell selected to be written to level 2, V wr It then rose to V. wr > 2V BD .

[0104] Figure 19A and Figure 19B The example of a three-level memory cell demonstrates a write mechanism utilizing dielectric breakdown. Figure 19A This is a schematic diagram of a three-level memory cell, where the first level is a single-layer dielectric 11913 between conductive layers 1914 and 1912, which has a breakdown voltage V. BD And resistance R1. The second layer is formed between conductive layers 1908 and 1912, and has two layers: dielectric 2A 1911 and dielectric 2B 1909, each with a breakdown voltage V. BD And resistor R2, and separated by conductive layer 1908. The third layer between conductive layers 1908 and 1902 is formed by three layers: dielectric 3A 1907, dielectric 3B 1905, and dielectric 3C 1903, wherein there are inter-dielectric conductive layers 1906 and 1904. The programming voltage V applied to the memory cell at the intersection of the corresponding word line and bit line. wrRepresented by voltage source 1901, the memory cell can be in the following states: Level 0, corresponding to the highest resistance state, where no dielectric layer is broken down; Level 1, where dielectric 1 1913 is broken down while dielectrics 2A 1911, 2B 1909, and 3A 1907, 3B 1905, and 3C 1903 remain intact; Level 2, where dielectrics 2A 1911 and 2B 1909 are also broken down; and Level 3, where dielectrics 1907, 1905, and 1903 of layer 3 are broken down. In a ternary implementation, Levels 0, 1, and 2 will be used, corresponding to the high resistance state, medium resistance state, and low resistance state, respectively. When operating as a four-state device, Level 0 will be the high resistance state, Levels 1 and 2 will be the medium resistance states, and Level 3 will be the low resistance state. When all dielectric layers are short-circuited at level 3, level 3 can have very low resistance and will consume more power, so some implementations may add series resistors to the memory cells to allow for low-power inference when operating as tri-state memory cells. For a three-level, three-tier system, the edge condition is: for factor i, V bd.eff.1 < V bd.eff.2 or V bd.eff.1 =i V bd.eff.2 This ensures that 0.5 < i < 1 so that ½V is satisfied. wr.max < V wr.min Similarly, for the factor n, V bd.eff.2 < V bd.eff.3 or V bd.eff.2 = n V bd.eff.3 , where n < 1. Let V bd With R3 as the unit, this gives the condition R2 = 3 / (3 (n-2) and R1 = 3n R2 / (3 n i -1).

[0105] When data is programmed into, for example Figure 19A When the memory cell array shown is used, Figure 19B The following is an example of what is applied to the text corresponding to the above. Figure 17 The V of the word line of the memory cell to be programmed wr Waveform of the step sequence. Figure 19B The waveform is the V applied to the selected word line during the write operation. wr The implementation scheme for the level is as follows: Initially, all memory cells are at level 0, and memory cells with a target state of level 0 will not be programmed from this initial state. To write to a level 1 state, the programming voltage V is... wrApply to the selected word line so that V wr > V bd.eff.1 And V wr < V bd.eff.2 This is to break down the first-level dielectric, rather than the second or third-level dielectric. To write the level 2 state, the programming voltage V is... wr Apply to the selected word line so that V wr > V bd.eff.2 And V wr < 3V BD This is to break down the second-level dielectric, rather than the third-level dielectric. If dielectrics 3A 1907, 3B 1905, and 3C 1903 are also to break down at level 3, then for the memory cell selected to be written to level 3, V wr It then rose to V. wr > 3V BD .

[0106] Figure 20 This is a flowchart illustrating an implementation of a cross-point architecture that uses memory cells with an ultra-thin dielectric layer for the inference engine. Figure 20 The process includes implementations of methods for programming the cross-point memory array (steps 2001-2007) and subsequent use of the memory array for inference (2011-2019). While the description herein is in the context of inference engines for neural networks, and the structures and techniques described may have specific utility in such applications, they are applicable more generally because low-power non-volatile memories with high retention can be used in many contexts.

[0107] At step 2001, the array is biased against selected bit lines in the bit lines, wherein the programming sequence may start from BL0, as relative to Figure 17 As shown. For example, the selected positioning line can be offset to ground, where the current V wr The other bit lines and non-selected word lines of the value are selected as half, i.e., ½V. wr In step 2003, the programming voltage V is... wr Apply to the corresponding programmable to V wr The word lines of the memory cells corresponding to the data states. The biasing of the word lines and bit lines at steps 2001 and 2003 can be performed by one or more control circuits, including biasing circuits within the row decoder 324 and column decoder 332, as well as the read / write circuit 328.

[0108] Repeat steps 2001 and 2003 for each bit line. For a multi-level cell implementation, repeat steps 2001 and 2003 for each V. wr The value repeats this process. Figure 20In the implementation plan, V wr The loop is within the bit line loop, but other implementations can reverse this order. At step 2004, it is determined whether there are additional programming voltages V for the selected bit line. wr If it exists, then V will be set at step 2005. wr The value is incremented to the next programmed value, such as stepping to V corresponding to shorting the next dielectric layer or the next layer below the dielectric layer. wr The value is then looped back to step 2001 to re-bias the currently unselected word line to half-select accordingly. If no more V lines to write exist at step 2004... wr If the value is not found, the process proceeds to step 2006 to determine if there are more bit lines to write. If so, the bit lines are incremented at step 2007, and the loop returns to step 2001. If not, the programming of the memory array is complete.

[0109] In applications such as inference engines, the programming of the array is typically completed before it is provided to the end user. For example, a memory array may be programmed by the manufacturer before shipping, such as at the end of a testing process usually performed on new equipment. Alternatively, the memory array may be programmed by an intermediary before being provided to the end user: for example, an unprogrammed memory array (initialized or uninitialized) may be provided to an original equipment manufacturer (OEM), who can then program its data (such as the weights of a DNN model) and then provide the product incorporating the programmed array to the OEM's customer. This would be, for example, the case of autonomous vehicles that rely on neural networks, where the manufacturer programs the weights of its neural network model into the array before providing the vehicle to the end operator.

[0110] Since memory arrays are typically programmed only once before being provided to the end user, in some implementations, the programming voltage V... wr It may not be generated on the memory device itself, but rather provided externally during programming. In this case, one or more control circuits on the memory device (such as power controller 316) do not need to have a power generation function. wr It has the ability to avoid circuit elements (such as charge pumps) and circuits that handle higher voltages, thus saving the area and power consumption of these circuits.

[0111] Steps 2011-2019 consider when to use a memory array as the inference engine network in a neural network, where weights are stored as values ​​in the array's memory cells, and the input will be an input value vector. (See above regarding...) Figure 13 As shown, the input vector IN i By DAC 1311 i Convert to be applied to the corresponding word line WL ivoltage level V i in The input value can be a binary value or a multilevel, depending on the implementation. In the word line WL i and BL j The current flowing between them corresponds to the input vector component. i in With stored in memory unit 1301 i,j The matrix weights b i,j The product of and . Total current I j out This will be equal to the voltage level V. i in The input vector represented by the j-th matrix column is stored in memory 1301. i,j The weights in the BL represent the weights used for the corresponding BL. j The sum of the vector products of the values ​​of I. j This represents the output vector of matrix multiplication in memory, as described above by Equations 1 and 2.

[0112] Step 2015 determines each bit line BL corresponding to the product of Equation 2. j Current I on j out The current is supplied by ADC1313 j Convert to OUT j These partial products of the individual columns can then be accumulated in digital computation block 1315 to generate the layer's output in step 2017. Step 2019 corresponds to additional propagation from the output of step 2017. According to this embodiment, step 2017 may include applying an activation function, repeating earlier steps of the inference operation for additional values, providing the layer's output, or some combination of these operations. The layer's output can then be used as input to subsequent layers of the network or as the network's final output, depending on the layer's location within the network.

[0113] In another set of embodiments, the inference engine with a cross-point architecture may be based on antifuse-type memory cells, wherein each memory cell includes an antifuse at each cross-point. Similar to embodiments based on thin dielectric layers, the antifuse implementation is a write-once, read-many-type array, where programming is performed relative to the binary implementation. Figure 20 The process described is similar to the one described above. Antifuse-based implementations can achieve very high resistance levels (e.g., in the gigaΩ or teraΩ range), and low resistance levels can be further improved by using large series resistors. Antifuse technology can be implemented on a small scale (i.e., 10nm) and multi-level resistance states can be implemented.

[0114] An antifuse is a one-time programmable device that is typically off in an unprogrammed state and typically on in a programmed state. However, in some implementations, multiple resistance levels can be defined after programming by modifying the antifuse's bias, resulting in compliant current. A common design for an antifuse consists of a select transistor connected in series between a power supply level and a bit line. This select transistor has a control gate connected to the read word line, and the antifuse transistor has a gate connected to the programming word line. An antifuse is programmed by applying a high voltage to the programming word line to break down the oxide in the antifuse transistor and short-circuit it. To read the antifuse, a voltage is applied to the read word line when the programming word line is low to turn on the select transistor, allowing current to flow through the programmed antifuse, whereas an unprogrammed antifuse will not conduct current.

[0115] The following discussion presents an implementation of a crosspoint memory using a single normally closed field-effect transistor (FET) antifuse cell. The antifuse cell is based on a FET with a metal oxide gate (e.g., a high-k metal gate) or a semiconductor oxide gate (e.g., SiO2). The FET channel is non-conductive in its initial state and becomes short-circuited after the gate oxide has been programmed (i.e., electrically broken down). Specific details of the material implementation vary depending on the lithography node chosen for the design. The programming of the antifuse can be determined according to… Figure 21 As shown.

[0116] Figure 21 This is a schematic diagram of an antifuse based on a single FET and an implementation scheme for biasing to program the antifuse. FET transistor 2101 is connected between bit line BL 2105 and word line WL 2103. A third line, or programming word line WLP 2107, is connected to the control gate of FET 2101. Figure 21 This illustrates the bias for programming the FET 2101, where the programming word line WLP2107 is raised to the write voltage V. wr At the same time, ground the word line WL 2103 and set the bit line BL 2105 to ½V. wr To perform antifuse programming. Select the programming voltage V. wr This is sufficient to cause oxide breakdown in the antifuse transistor, and ½V wr This does not cause oxide breakdown, thus allowing cell selection in the crosspoint array. To increase the resistance in the low-resistance state, each antifuse FET 2101 may have a resistor connected in series between the bit line BL 2105 and the word line WL 2103. (In the half-select process described herein, a low voltage level is obtained at 0V, and ½V is used.) wr The half-select value; but more generally, these can be another low voltage value, and the half-select voltage level can be V. wrOther voltage levels between low and high voltage, provided the half-select process does not cause unwanted programming or reading values. Figure 22 It is a combination of Figure 21 A top view of a portion of an antifuse-based memory cell cross-point array structure. Figure 22 Similar to Figure 12C or Figure 17 The diagram shows the antifuse memory cell at each intersection, and now also includes programming lines WLP1-WLP4. For example, memory cell 2201 is connected between word line WL4 and bit line BL3, and its control gate is connected to WLP3. Although not shown for simplicity, resistors can be connected in series with each antifuse memory cell to increase the value of the low resistance state.

[0117] Figure 22 The diagram shows a portion of a crosspoint array biased in one embodiment to program memory cell 2201, wherein a half-select scheme is used in conjunction with a programming voltage V applied to the programming word line for programming. wr To complete the electrical selection, in order to write the selected weight value to the antifuse 2201, the corresponding programming word line WLP3 is set to V. wr Set the corresponding word line WL4 to 0V and the corresponding bit line BL3 to ½V. wr Also set the bit lines in the unselected positioning lines to ½V. wr The same applies to unselected word lines. Setting the programming word line for an unselected memory cell to 0V allows programming to be performed sequentially, one word line after another, similar to the above description relative to... Figure 17 The order is now used for programming word lines, not bit lines (now half-selected for both selected and unselected memory cells). This is because the gate-source or gate-drain voltage remains at ½V. wr Therefore, unselected antifuse cells that share a programming word line with the programmed selected antifuse memory cell (WLP3 in this example) will not experience oxide breakdown. Once the array is programmed, programming word lines are no longer needed for read operations, which is similar to what was described above for memory cells based on thin dielectric layers.

[0118] Figure 23 This is a flowchart illustrating an implementation of a crosspoint architecture programming scheme that uses memory cells with FET-based antifuse for the inference engine. Figure 23 The process includes an implementation scheme (steps 2301-2307) for programming a crosspoint memory array, and the subsequent use of the memory array for inference can be as described above relative to... Figure 20The steps described in 2011-2019 are as follows. Although the description herein is in the context of inference engines for neural networks, and the structures and techniques described may have specific utility in such applications, they can be applied more generally because low-power non-volatile memory with high retention can be used in many contexts.

[0119] At step 2301, the array is biased for the selected programming word line, wherein the programming sequence may begin with WLP0, similar to the one about Figure 17 The sequence was shown for bit lines, but now it's for programming word lines. For example, all bit lines and unselected word lines can be half-selected and in ½V. wr The word line corresponding to the selected memory cell is biased to ground. At step 2303, the programming voltage V is... wr The bias is applied to the selected programming word line. The biasing of the word line, programming word line, and bit line at steps 2301 and 2303 can be performed by one or more control circuits, including biasing circuits within the row decoder 324 and column decoder 332, as well as read / write circuitry 328.

[0120] Steps 2301 and 2303 are repeated for each programming word line. At step 2305, it is determined whether there are more programming bit lines to be written. If so, the bit lines are incremented at step 2307, and the loop returns to step 2301; otherwise, the programming of the memory array is complete. Once the memory array has been programmed using the weights of the neural network, it can be used as described above relative to... Figure 20 The inference engine described in steps 2011-2019.

[0121] Similar to the implementation described above, in applications serving as inference engines, the programming of the array is typically completed before it is provided to the end user. For example, the memory array may be programmed by the manufacturer before shipment, such as at the end of a testing process usually performed on new equipment. Alternatively, the memory array may be programmed by an intermediary before being provided to the end user: for example, an unprogrammed memory array (initialized or uninitialized) may be provided to an original equipment manufacturer (OEM), who can then program its data (such as the weights of a DNN model) and then provide the product incorporating the programmed array to the OEM's customer. This would be, for example, the case of autonomous vehicles that rely on neural networks, where the manufacturer programs the weights of its neural network model into the array before providing the vehicle to the end operator.

[0122] Since memory arrays are typically programmed only once before being provided to the end user, in some implementations, the programming voltage V... wrIt may not be generated on the memory device itself, but rather provided externally during programming. In this case, one or more control circuits on the memory device (such as power controller 316) do not need to have a power generation function. wr It has the ability to avoid circuit elements (such as charge pumps) and circuits that handle higher voltages, thus saving the area and power consumption of these circuits.

[0123] According to a first aspect, an apparatus includes control circuitry configured to be connected to a plurality of multilevel memory cells connected along one or more bit lines, each memory cell being formed of a plurality of dielectric layers separated by conductive layers. The control circuitry is configured to: individually program a selected memory cell into one of a plurality of data states by applying a write voltage of one of a plurality of corresponding write voltages to break down one or more dielectric layers in the dielectric layer of the selected memory cell; and simultaneously sense the first plurality of memory cells in response to a set of corresponding first plurality of input voltages applied to the first plurality of memory cells connected along the first bit line.

[0124] In another aspect, a method includes biasing an array of memory cells to simultaneously program one or more first selected memory cells into a first data state of a plurality of data states. The array is formed according to a cross-point architecture, wherein each memory cell is connected between a corresponding bit line and a corresponding word line, and each memory cell is formed by a plurality of dielectric layers separated by conductive layers. The biasing includes: setting the bit lines of the array corresponding to the one or more first selected memory cells to a low voltage value; setting the bit lines of the array not corresponding to the one or more first selected memory cells to a first intermediate voltage level; and setting the word lines of the array not corresponding to the one or more first selected memory cells to a first intermediate voltage level. While biasing the array of memory cells to simultaneously program one or more first selected memory cells into the first data state of a plurality of data states, a first write voltage is applied to the word lines of the array corresponding to the one or more first selected memory cells, the first write voltage being selected to break down one or more dielectric layers in the dielectric layers of the first selected memory cells to program the first selected memory cells into the first data state of a plurality of data states, wherein the first intermediate voltage level is between the low voltage value and the first write voltage.

[0125] In another aspect, a non-volatile memory device includes a memory array and one or more control circuits connected to the memory array. The memory array includes: a plurality of word lines; a plurality of bit lines; a plurality of programming lines; and a plurality of antifuse memory cells. The memory array has a cross-point architecture, each antifuse memory cell including a field-effect transistor (FET) connected between a corresponding bit line in the bit lines and a corresponding word line in the word lines, and having a control gate connected to a corresponding programming line in the programming lines. Each antifuse memory cell can be programmed from a high-resistance state to a low-resistance state by breaking down the gate oxide of the memory cell. The one or more control circuits are configured to: program a selected antifuse memory cell in the antifuse memory cell by biasing the corresponding word line and the corresponding bit line to establish a voltage difference across the selected memory cell while applying a write voltage to the corresponding programming line to break down the gate oxide of the selected memory cell; and to sense a first plurality of memory cells connected along a shared bit line in response to a set of corresponding first plurality of input voltages applied to the corresponding plurality of word lines when the programming line is set to ground.

[0126] For the purposes of this document, the terms “implementation scheme,” “one implementation scheme,” “some implementation schemes,” or “another implementation scheme” used in the specification may be used to describe different implementation schemes or the same implementation scheme.

[0127] For the purposes of this document, a connection may be a direct connection or an indirect connection (e.g., via one or more other components). In some cases, when an element is mentioned as being connected or coupled to another element, the element may be directly connected to the other element or indirectly connected to the other element via an intermediary element. When an element is mentioned as being directly connected to another element, there is no intermediary element between the two elements. If two devices are directly or indirectly connected, the two devices are “communicating”, enabling them to communicate electronic signals between them.

[0128] For the purposes of this document, the term "based on" may be understood as "at least partially based on".

[0129] For the purposes of this document, the use of numerical terms such as “first” object, “second” object, and “third” object without additional context may not imply an ordering of objects, but may be used for identification purposes to distinguish different objects.

[0130] For the purposes of this document, the term "group" of objects may refer to a "group" of one or more objects.

[0131] The detailed description above has been provided for purposes of illustration and description. It is not intended to be exhaustive or to limit the precise forms disclosed in the invention. Many modifications and variations are possible based on the teachings above. The described embodiments were chosen to best explain the principles of the proposed technology and its practical application, thereby enabling others skilled in the art to best utilize it in various embodiments and various modifications suitable for the specific intended use. The scope of the invention is intended to be defined by the appended claims.

Claims

1. An apparatus for programming a memory cell, comprising: A control circuit configured to be connected to a plurality of multilevel memory cells connected along one or more bit lines, and each memory cell being formed of a plurality of dielectric layers separated by conductive layers, the control circuit being configured to: By applying a selected write voltage from a plurality of write voltages to break down one or more corresponding dielectric layers among the plurality of dielectric layers written to the selected memory cell without limiting the current during programming, the selected memory cell is individually programmed into one of a plurality of data states. as well as In response to a set of corresponding first plurality of input voltages applied to a first plurality of memory cells connected along the first bit line, the first plurality of memory cells are sensed simultaneously.

2. The apparatus according to claim 1, wherein the control circuit comprises: An analog-to-digital converter configured to receive the value of a current in the first bit line in response to the first set of corresponding first plurality of input voltages applied to the first plurality of memory cells, and to determine a plurality of bit output values ​​from the current.

3. The apparatus of claim 1, wherein the control circuit is formed on a control die, the apparatus further comprising: A memory die, the memory die including the memory cell, the memory die being separately formed from and coupled to the control die.

4. The apparatus of claim 3, wherein the memory die comprises a memory cell array, the memory cell array comprising: The memory unit; Multiple bit lines, wherein the multiple bit lines include the first bit line; and Multiple word lines, the array being formed according to a crosspoint architecture, in which each memory cell is connected between a corresponding bit line in the bit lines and a corresponding word line in the word lines.

5. The apparatus of claim 4, wherein each of the memory cells comprises: Multiple dielectric layers, each of the dielectric layers being formed to have one of a plurality of different effective breakdown voltages and one of a plurality of different resistance values, such that the higher the resistance value of the dielectric layer, the lower the effective breakdown voltage of the dielectric layer; and One or more conductive layers, which alternate with the dielectric layer.

6. The apparatus of claim 4, wherein each of the memory cells comprises: Multiple layers, each layer having one or more dielectric layers, each dielectric layer being formed to have a common effective breakdown voltage and a resistance value among a plurality of resistance values, dielectric layers of the same layer being formed to have the same resistance value and dielectric layers of different layers being formed to have different resistance values, and such that the more layers in the layer, the lower the resistance value of the dielectric layer of the layer. and Multiple conductive layers, which alternate with the dielectric layer.

7. The apparatus according to claim 4, wherein, In order to program one or more selected memory cells along the selected positioning line to a first data state, the control circuit is configured to simultaneously: The selected positioning line is biased to a low voltage level; The word line corresponding to the writing to the selected memory cell is biased to the write voltage corresponding to the first data state; as well as The word lines that do not correspond to the write to the selected memory cell and the bit lines other than the selected bit lines are biased to a voltage level between the low voltage level and the write voltage corresponding to the first data state.

8. The apparatus of claim 4, wherein the control circuit comprises: One or more analog-to-digital converters are connected to the bit lines and configured to receive the value of the current in each bit line in response to the set of corresponding input voltages applied to the word lines, and to determine a multi-bit output value from each current.

9. The apparatus of claim 8, wherein the control circuit comprises: A digital computing circuit connected to the one or more analog-to-digital converters and configured to accumulate the multi-bit output values.

10. The apparatus of claim 9, wherein the memory unit is configured to store the weights of layers of a neural network, and wherein the control circuitry is further configured to: Receive the input vector of the layer of the neural network, the input vector having multiple components; Convert the components of the input vector into a set of voltage levels; and Each voltage level in the set of voltage levels is applied to the corresponding set of word lines.

11. The apparatus of claim 10, wherein the control circuit is further configured to: In response to applying each of the set of voltage levels to the corresponding set of word lines, the multi-bit output from the plurality of bit lines is accumulated.

12. The apparatus of claim 1, wherein the control circuit is further configured to: It receives multiple weight values ​​from layers of a neural network; The weight values ​​are programmed into the memory unit as data states; After the weight values ​​are programmed into the memory cell, the input vector of the layer of the neural network is received; Convert the input vector into the first plurality of input voltages; as well as The array multiplication of the input vector and the weight value is performed by applying the first plurality of input voltages to the memory cell.

13. A method for programming a memory cell, comprising: A biased memory cell array is configured to simultaneously program one or more first selected memory cells to a first data state among multiple data states. The array is formed according to a cross-point architecture, in which the memory cells are connected between corresponding bit lines and corresponding word lines, and each memory cell is formed by multiple dielectric layers separated by conductive layers. The biasing includes: Set the bit lines of the array corresponding to the one or more first selected memory cells to a low voltage value; Set the bit lines of the array that do not correspond to the one or more first selected memory cells to a first intermediate voltage level; and Set the word lines of the array that do not correspond to the one or more first selected memory cells to the first intermediate voltage level; and When the memory cell array is biased to simultaneously program one or more first selected memory cells to the first data state of the plurality of data states, a first write voltage is applied to the word line of the array corresponding to the one or more first selected memory cells without limiting current during programming. The first write voltage is selected to break down one or more dielectric layers of the first selected memory cells to program the first selected memory cells to the first data state of the plurality of data states, wherein the first intermediate voltage level is between the low voltage value and the first write voltage.

14. The method of claim 13, further comprising: After biasing the memory cell array to simultaneously program the first selected memory cell, biasing the array to simultaneously program one or more second selected memory cells corresponding to the first selected memory cell connected to the array into a second data state of a plurality of data states, wherein biasing the array to simultaneously program one or more second selected memory cells includes: Set the bit lines of the array corresponding to the one or more first selected memory cells to a low voltage value; Set the bit lines of the array that do not correspond to the one or more second selected memory cells to a second intermediate voltage level; and Set the word lines of the array that do not correspond to the one or more second selected memory cells to the second intermediate voltage level; and When the memory cell array is biased to simultaneously program one or more second selected memory cells to the second data state of the plurality of data states, a second write voltage is applied to the word line of the array corresponding to the second selected memory cell. The second write voltage is selected to break down one or more dielectric layers of the dielectric layer of the second selected memory cell to program the second selected memory cell to the second data state of the plurality of data states, wherein the second intermediate voltage level is between the low voltage value and the second write voltage.

15. The method of claim 13, wherein the memory cells of the array are programmed to store weights of layers of a neural network, and the method further comprises: Receive the input vector of the layer of the neural network; as well as The in-array multiplication of the input vector with the weights of the neural network is performed by the following operation: Convert the input vector into a set of voltage levels; as well as The set of voltage levels is applied to the word lines of the array.

Images