CONFIGURATED PRECISE NEURAL NETWORK WITH DIFFERENTIAL BINARY, NON-FLAMMATORY STORAGE CELL STRUCTURE

Abstract

A non-volatile memory circuit, comprising: an array of one or more memory units connected along one or more first input lines and each configured to store an N-bit weight of a neural network, where N is an integer greater than 1, wherein each of the memory units comprises N non-volatile binary memory elements connected to a corresponding output line and configured to store one bit of the N-bit weight; and one or more control circuits connected to the array of storage units; the one or more control circuits are configured to: to apply a first input of a neural network to one or more first input lines in order to generate an output voltage level on each of the output lines connected to one of the first storage units in response; to individually weight the output voltage levels generated on each of the output lines in response to the first input, according to a meaning of the weight bit stored in the binary memory element of the first memory unit corresponding to the output line; and to determine a multi-bit value for the response of the weight stored in the first memory unit to the first input from a combination of the individually weighted output voltages, in order to perform an in-array multiplication of the first input with the weight stored in the first memory unit.

Description

REFERENCE TO RELATED APPLICATIONS

[0001] This application is a continuation of U.S. patent application No. 16 / 405,178, filed May 7, 2019, which claims priority to U.S. provisional application No. 62 / 702,713, filed July 24, 2018, and which relates to U.S. patent applications Nos. 16 / 368,441 and 16 / 368,347, both filed March 28, 2019, all of which are incorporated herein by reference. BACKGROUND

[0002] Artificial neural networks are increasingly being used in artificial intelligence and machine learning applications. In an artificial neural network, a set of inputs is propagated through one or more intermediate or hidden layers to produce an output. The layers connecting the input to the output are connected by sets of weights, which are generated in a training or learning phase by determining a set of mathematical manipulations to transform the input into the output, calculating the probability of each output as the network moves through the layers. Once the weights are determined, they can be used in the inference phase to determine the output from a set of inputs.Although such neural networks can produce highly accurate results, they are extremely computationally intensive, and the data transfers involved in reading the weights connecting the different layers from memory and transferring them to a processing unit's processing units can be quite intensive.

[0003] The publication US 2016 / 0 141 337 A1 relates to devices and methods for non-volatile data storage.

[0004] The publication US 2011 / 0 119 215 A1 relates to neural networks and in particular to hardware-implemented analog-digital neural networks implemented in both two and three dimensions.

[0005] The document US 2018 / 0 018 560 A1 relates to storage devices, and in particular to systems, devices and methods for using data stored in the memory.

[0006] The document US 9 887 240 B2 relates to devices and methods for non-volatile data storage.

[0007] The publication DE 20 2016 107 439 U1 refers to the data processing of neural network inferences in hardware. SHORT DESCRIPTION OF THE DRAWING

[0008] Like-numbered elements refer to common components in the different figures. Fig. 1 is a block diagram illustrating one embodiment of a storage system coupled to the host. Fig. Figure 2 is a block diagram of one embodiment of a front-end processor circuit. In some embodiments, the front-end processor circuit is part of a controller. Fig. Figure 3 is a block diagram of one embodiment of a back-end processor circuit. In some embodiments, the back-end processor circuit is part of a controller. Fig. 4 is a block diagram of one embodiment of a memory package. Fig. 5 is a block diagram of one embodiment of a memory chip. Fig. Figure 6 shows a simple example of an artificial neural network. Fig. Figure 7A is a flowchart describing one embodiment of a process for training a neural network to generate a set of weights. Fig. Figure 7B is a flowchart describing one embodiment of an inference process using a neural network. Fig. Figure 8 is a schematic representation of the use of matrix multiplication in a neural network. Fig. Figure 9 is a table illustrating the output of a binary neural network in response to different input-weight combinations. Fig. 10 illustrates an embodiment in which two differential memory cells are used to form a unit synapse for storing a weight bit. Fig. 11 illustrates the encoding of the input values, weight values ​​and output values ​​as word line voltages, resistance values ​​and bit line voltages, respectively. Fig. 12A-12D correspond to the four cases shown in the table of Fig. 11 are shown. Fig. 13A-13C illustrate some embodiments for the resistive memory cells of the unit synapse. Fig. 14 shows the integration of the unit synapses of Fig. 10 into an array. Fig. 15 refers to matrix multiplication for a neural network closer to the structure of Fig. 14. Fig. 16 is a flowchart for one embodiment of matrix multiplication using a binary neural network in inference as used in the array architecture of the Fig. 14 is shown. Fig. Figure 17 illustrates an equivalent voltage divider for simultaneous multiple input in an array-matrix multiplication operation. Fig. Figure 18 illustrates the relationship between the output voltage as seen on the bit line and the sum code determined when multiple input weight products are simultaneously detected. Fig. 19 is a flowchart for one embodiment of a dot product calculation, similar to Fig. 16, which, however, includes the parallelism of creating multiple inputs simultaneously. Fig. 20 illustrates one embodiment of a portion of a monolithic three-dimensional memory array forming a differential cross-point (DX) architecture. Fig. Figure 21 provides an embodiment that uses an extended three-dimensional structure for storing neural network weights. Fig. Figure 22 is a schematic diagram of storing a multi-bit weight in a non-volatile memory array for use in an in-array inference operation. Fig. 23 illustrates the integration of the unit synapses of Fig. 10 into an array for a multi-bit weight embodiment. Fig. 24 is a flowchart for one embodiment of a matrix multiplication or dot product calculation using multi-bit weights in inference based on the array architecture of Fig. 23. Fig. 25 is a flowchart for one embodiment of a dot product calculation, similar to Fig. 24, which, however, includes the parallelism of applying multiple inputs to the multi-bit weights simultaneously. Fig. Figure 26 illustrates one embodiment for a binary weighted voltage summing circuit that may be used for weighting and combining the bitwise output of the in-array matrix multiplication operation. Fig. Figure 27 is a schematic diagram of one embodiment of a current-based weighting and summing circuit that may be used for weighting and combining the bitwise output of the in-array matrix multiplication. DETAILED DESCRIPTION

[0009] To reduce the computational complexity and relax the memory requirements of neural networks, binary neural networks (BNNs) were introduced. In BNNs, the weights and inputs of the neural network are truncated to binary values ​​(-1, +1), and binary arithmetic simplifies multiplication and addition to XNOR and bit counting operations. The following disclosure presents techniques for utilizing the structure of non-volatile memory with a crosspoint structure to store the weights of the neural networks and perform the multiplication and accumulation operations within the memory array.

[0010] Each weight of a neural network is stored in a unit synapse formed from a pair of resistive non-volatile memory cells in a differential memory cell array, with each of the two memory cells of the pair connected between a corresponding wordline pair and a common bitline. For example, to store a binary weight on a unit synapse, one of the pair of memory cells is programmed to a high-resistance state and the other to a low-resistance state, so that the unit synapse acts as a voltage divider when a voltage pattern is applied to the wordline pair. Depending on which memory cell of the unit synapse is in the high-resistance state and which memory cell is in the low-resistance state, the unit synapse corresponds to a weight value of -1 or a weight value of +1.The binary input is then applied as a voltage pattern to the corresponding wordline pair, with one of the wordline pair at a high voltage level and the other of the wordline pair at a low voltage level, such as ground. Depending on which wordline of the wordline pair is at which value, the input will be either a -1 or a +1. By applying the input to the wordline pair, the unit synapse will act as a voltage divider, setting either a high voltage level or a low voltage level on the divided bitline, depending on whether the input and weight match or not. The result can be determined by a multi-level sense amplifier or an analog-to-digital converter (ADC) connected to the corresponding bitline.By applying the input / unit synapse pairs along a bit line and accumulating the results of the sense amplifier, the multiplication and accumulation operations of propagating an input through a layer of a neural network can be performed. Since the word lines of the array span multiple bit lines, the operation can be performed simultaneously for the binary weights of multiple unit synapses.

[0011] The approach can be extended to embodiments with multi-bit weight values ​​for greater precision. To store an N-bit weight value, each of the N bits can be stored in a differential memory cell connected along a common wordline pair, but each on a separate bitline. In an inference operation, the input is applied to the weight via the wordline pair, and the bitwise output is generated on the bitlines of the weight's differential memory cells. The outputs are then weighted according to their significance, combined, and the combined output is determined by a multi-bit sense amplifier or analog-to-digital converter.

[0012] Fig. 1 is a block diagram illustrating one embodiment of a storage system 100 connected to host 120. Storage system 100 may implement the technology proposed herein, receiving neural network inputs or other data from host 120. Depending on the embodiment, the inputs may be received from host 120 and then provided to memory packs 104 to infer the weights previously programmed into the memory arrays of memory packs 104. Many different types of storage systems may be used with the technology proposed herein. Example storage systems include solid-state drives ("SSDs"), memory cards, and embedded storage devices; however, other types of storage systems may also be used.

[0013] The storage system 100 from Fig. 1 includes a controller 102, a non-volatile memory 104 for storing data, and local memory (e.g., DRAM / ReRAM) 106. The controller 102 includes a front-end processor (FEP) circuit 110 and one or more back-end processor (BEP) circuits 112. In one embodiment, the FEP circuit 110 is implemented on an ASIC. In one embodiment, each BEP circuit 112 is implemented on a separate ASIC. In other embodiments, a unified ASIC controller may combine both the front-end and back-end functions. The ASICs for each of the BEP circuits 112 and the FEP circuit 110 are implemented on the same semiconductor, so the controller 102 is manufactured as a system on a chip ("SoC"). The FEP circuit 110 and the BEP circuit 112 both contain their own processors.In one embodiment, FEP circuitry 110 and BEP circuitry 112 operate in a master-slave configuration, with FEP circuitry 110 being the master and each BEP circuitry 112 being a slave. For example, FEP circuitry 110 implements a flash translation layer (FTL) or a media management layer (MML) that performs memory management (e.g., garbage collection, wear leveling, etc.), logical-to-physical address translation, host communication, DRAM (local volatile memory) management, and overall SSD (or other non-volatile memory) operation. BEP circuitry 112 manages memory operations within the memory packages / chip at the request of FEP circuitry 110. For example, BEP circuitry 112 may perform the read, erase, and program processes.Additionally, the BEP circuit 112 may perform buffer management, set specific voltage levels required by the FEP circuit 110, perform error correction (ECC), control the toggle mode interfaces to the memory packs, etc. In one embodiment, each BEP circuit 112 is responsible for its own set of memory packs.

[0014] In one embodiment, non-volatile memory 104 includes a plurality of memory packages. Each memory package contains one or more memory chips. Therefore, controller 102 is connected to one or more non-volatile memory chips. In one embodiment, each memory chip in memory packages 104 uses NAND flash memory (including two-dimensional NAND flash memory and / or three-dimensional NAND flash memory). In other embodiments, the memory package may include other types of memory.

[0015] The controller 102 communicates with the host 120 via an interface 130 that implements NVM Express (NVMe) over PCI Express (PCIe). To interoperate with the storage system 100, the host 120 includes a host processor 122, a host memory 124, and a PCIe interface 126 connected along bus 128. The host memory 124 is the physical memory of the host and may be DRAM, SRAM, non-volatile memory, or another memory type. The host 120 is external and separate from the storage system 100. In one embodiment, the storage system 100 is embedded within the host 120.

[0016] Fig. 2 is a block diagram of one embodiment of the FEP circuit 110. Fig. Figure 2 shows a PCIe interface 150 for communicating with host 120 and a host processor 152 in communication with this PCIe interface. Host processor 152 may be any processor type known in the art that is suitable for implementation. Host processor 152 communicates with a Network-On-Chip (NOC) 154. A NOC is a communications subsystem on an integrated circuit, typically between cores in a SoC. NOCs can span synchronous and asynchronous clock domains or use unclocked asynchronous logic. NOC technology applies network theory and techniques to on-chip communications, providing significant improvements over traditional bus and crossbar interconnects. NOC improves the scalability of SoCs and the power efficiency of complex SoCs compared to other designs. The NOC's wires and interconnects are shared by many signals.High parallelism is achieved because all connections in the NOC can work on different data packets simultaneously. Therefore, as the complexity of integrated subsystems grows, a NOC provides improved performance (such as throughput) and scalability compared to previous communication architectures (e.g., dedicated point-to-point signal wires, shared buses, or segmented buses with bridges). Connected to and in communication with NOC 154 are storage processor 156, SRAM 160, and a DRAM controller 162. DRAM controller 162 is used to work with and communicate with DRAM (e.g., DRAM 106). SRAM 160 is local RAM used by storage processor 156. Storage processor 156 is used to operate the FEP circuitry and perform the various memory operations. Two PCIe interfaces 164 and 166 are also in communication with the NOC. In the embodiment of . Fig. 2, the SSD controller includes two BEP circuits 112; therefore, there are two PCIe interfaces 164 / 166. Each PCIe interface communicates with one of the BEP circuits 112. In other embodiments, there may be more or fewer than two BEP circuits 112; therefore, there may be more than two PCIe interfaces.

[0017] The FEP circuitry 110 may also include a flash translation layer (FTL) or, more generally, a media management layer (MML) 158 that performs memory management (e.g., garbage collection, wear leveling, load balancing, etc.), logical-to-physical address translation, communication with the host, DRAM (local volatile memory) management, and management of the overall operation of the SSD or other non-volatile storage system. The media management layer MML 158 may be integrated as part of the memory management, which can handle memory errors and interact with the host. In particular, MML may be a module in the FEP circuitry 110 and be responsible for the internals of memory management. In particular, the MML 158 may include an algorithm in the memory device firmware that translates writes from the host into writes to the memory structure (e.g., 326 of Fig. 5 below) of a chip. MML 158 may be required because: 1) the memory may have a limited lifetime; 2) the memory structure can only be written to in multiples of pages; and / or 3) the memory structure cannot be written to unless erased as a block. MML 158 understands these potential limitations of the memory structure, which may not be visible to the host. Accordingly, MML 158 attempts to translate writes from the host into writes to the memory structure.

[0018] Fig. 3 is a block diagram of one embodiment of a BEP circuit 112. Fig. Figure 3 shows a PCIe interface 200 for communicating with the FEP circuit 110 (e.g., in conjunction with one of the PCIe interfaces 164 and 166 of Fig. 2). The PCIe interface 200 is connected to two NOCs, 202 and 204. In one embodiment, the two NOCs can be combined into one large NOC. Each NOC (202 / 204) is connected to the SRAM (230 / 260), a buffer (232 / 262), a processor (220 / 250), and a data path controller (222 / 252) via an XOR engine (224 / 254) and an ECC engine (226 / 256). The ECC engines 226 / 256 are used to perform error correction, as is known in the art. The XOR engines 224 / 254 are used to XOR the data so that data can be combined and stored in a manner that can be recovered in the event of a programming error. The data path controller 222 is connected to an interface module to communicate with memory packets over four channels.Thus, the upper NOC 202 is associated with an interface 228 for four channels for communicating with memory packets, and the lower NOC 204 is associated with an interface 258 for four additional channels for communicating with memory packets. Each interface 228 / 258 includes four toggle mode (TM) interfaces, four buffers, and four schedulers. For each channel, there is one scheduler, one buffer, and one TM interface. The processor may be any standard processor known in the art. The datapath controllers 222 / 252 may be a processor, an FPGA, a microprocessor, or another type of controller. The XOR engines 224 / 254 and ECC engines 226 / 256 are dedicated hardware circuits known as hardware accelerators. In other embodiments, the XOR engines 224 / 254 and ECC engines 226 / 256 may be implemented in software. The schedulers, buffers and TM interfaces are hardware circuits.

[0019] Fig. Figure 4 is a block diagram of one embodiment of a memory package 104 including a plurality of memory chips 292 connected to a memory bus 294 (data lines and chip enable lines). The memory bus 294 is connected to a toggle mode interface 296 for communication with the TM interface of a BEP circuit 112 (see, e.g., Fig. 3). In some embodiments, the memory package may include a small controller connected to the memory bus and the TM interface. The memory package may include one or more memory chips. In one embodiment, each memory package includes eight or 16 memory chips; however, other numbers of memory chips may also be implemented. The technology described herein is not limited to a specific number of memory chips.

[0020] Fig. 5 is a functional block diagram of an embodiment of a memory chip 300. The Fig. The components illustrated in Figure 5 are electrical circuits. In one embodiment, each memory chip 300 includes a memory structure 326, a control circuit 310, and read / write circuits 328. The memory structure 326 is addressable by word lines via a row decoder 324 and by bit lines via a column decoder 332. The read / write circuits 328 include multiple read blocks 350 including SB1, SB2, ..., SBp (sense circuit) and enable parallel reading or programming of a page of memory cells. Commands and data are transferred between the controller and the memory chip 300 via lines 318. In one embodiment, the memory chip 300 includes a set of input and / or output (I / O) pins connected to the lines 318.

[0021] The control circuit 310 cooperates with the read / write circuits 328 to perform memory operations (e.g., write, read, and others) on the memory structure 326 and includes a state machine 312, an on-chip address decoder 314, and a power control circuit 316. The state machine 312 provides chip-level control of memory operations. In one embodiment, the state machine 312 is programmable by software. In other embodiments, the state machine 312 uses no software and is implemented entirely in hardware (e.g., electrical circuits). In another embodiment, the state machine 312 is replaced by a microcontroller. In one embodiment, the control circuit 310 includes buffers, such as registers, ROM fuses, and other storage devices for storing default values ​​such as base voltages and other parameters.

[0022] The on-chip address decoder 314 provides an address interface between addresses used by a controller 102 and the hardware address used by decoders 324 and 332. The power control module 316 controls the power and voltages supplied to the word lines and bit lines during memory operations. The power control module 316 may include charge pumps for generating voltages. The sense blocks include bit line drivers.

[0023] For the purposes of this document, the term “one or more control circuits” may refer to a controller, a state machine, a microcontroller and / or the control circuit 310, or other analog circuits used to control non-volatile memory.

[0024] In one embodiment, memory structure 326 comprises a three-dimensional array of non-volatile memory cells having multiple memory levels formed over a single substrate, such as a wafer. The memory structure may comprise any type of non-volatile memory formed monolithically in one or more physical levels of memory cells having an active area disposed over a silicon (or other type of) substrate. In one example, the non-volatile memory cells comprise vertical NAND strings with charge-trapping material, such as described in U.S. Patent 9,721,662, which is incorporated herein by reference in its entirety.

[0025] In another embodiment, memory structure 326 comprises a two-dimensional memory array of non-volatile memory cells. In one example, the non-volatile memory cells are NAND flash memory cells using floating gates, such as those described in U.S. Patent 9,082,502, which is incorporated herein by reference in its entirety. Other types of memory cells (e.g., NOR-type flash memory) may also be used.

[0026] The exact type of memory array architecture or memory cell included in memory structure 326 is not limited to the above examples. Many different types of memory array architectures or memory technologies may be used to form memory structure 326. No specific non-volatile memory technology is required for the purposes of the new claimed embodiments proposed herein. Other examples of suitable technologies for memory cells of memory structure 326 include ReRAM (resistive random access memory), magnetoresistive memory (e.g., MRAM, spin-transfer torque MRAM, spin-orbit torque MRAM), phase-change memory (e.g., PCM), and the like.Examples of suitable technologies for the memory cell architectures of memory structure 126 include two-dimensional arrays, three-dimensional arrays, cross-point arrays, stacked two-dimensional arrays, vertical bitline arrays, and the like.

[0027] An example of a ReRAM or PCM cross-point memory includes reversible resistive switching elements arranged in cross-point arrays accessed by X-lines and Y-lines (e.g., wordlines and bitlines). In another embodiment, the memory cells may include conductive bridge memory elements. A conductive bridge memory element may also be referred to as a programmable metallization cell. A conductive bridge memory element may be used as a state-change element based on the physical relocation of ions within a solid electrolyte. In some cases, a conductive bridge memory element may include two solid metal electrodes, one relatively inert (e.g., tungsten) and one electrochemically active (e.g., silver or copper), with a thin film of the solid electrolyte between the two electrodes.As temperature increases, ion mobility also increases, so the programming threshold for the conductive bridge memory cell decreases. Thus, the conductive bridge memory element can exhibit a wide range of programming thresholds as a function of temperature.

[0028] Magnetoresistive memory (MRAM) stores data using magnetic storage elements. The elements are formed from two ferromagnetic plates, each capable of holding a magnetization separated by a thin insulating layer. One of the two plates is a permanent magnet set to a specific polarity; the magnetization of the other plate can be changed to match that of an external field to store memory. A memory device is constructed from a grid of such memory cells. In one embodiment for programming, each memory cell lies between a pair of write lines arranged at right angles to each other, parallel to the cell, one above and one below the cell. When current flows through them, an induced magnetic field is generated.

[0029] Phase-change memory (PCM) exploits the unique behavior of chalcogenide glass. One embodiment uses a GeTe-Sb2Te3 superlattice to achieve non-thermal phase changes simply by changing the coordination state of the germanium atoms with a laser pulse (or light pulse from another source). Therefore, the programming doses are laser pulses. The memory cells can be locked by preventing the memory cells from receiving the light. In other PCM embodiments, the memory cells are programmed by current pulses. It should be noted that the use of "pulse" in this document does not require a rectangular pulse, but rather implies a (continuous or discontinuous) oscillation or burst of sound, current, voltage, light, or other wave.

[0030] One of ordinary skill in the art will recognize that the technology described herein is not limited to a single specific memory structure, but covers many relevant memory structures according to the spirit and scope of the technology as described herein and as known to one of ordinary skill in the art.

[0031] Turning now to the data types that can be stored on non-volatile memory devices, a specific example of the data type of interest in the following discussion is the weight used in deep neural networks. An artificial neural network is formed from one or more intermediate layers between an input layer and an output layer. The neural network finds a mathematical manipulation to convert the input into the output, moving through the layers and calculating the probability of each output. Fig. Figure 6 shows a simple example of an artificial neural network.

[0032] In Fig. Figure 6 depicts an artificial neural network as a connected group of nodes, or artificial neurons, represented by the circles, and a series of connections from the output of one artificial neuron to the input of another. The example shows three input nodes (I1, I2, I3) and two output nodes (O1, O2), with an intermediate layer of four hidden or intermediate nodes (H1, H2, H3, H4). The nodes, or artificial neurons / synapses, of the artificial neural network are implemented by logic elements from a host or other processing system as a mathematical function that receives one or more inputs and sums them to produce an output. Typically, each input is weighted separately, and the sum is passed through the node's mathematical function to provide the node's output.

[0033] In common implementations of artificial neural networks, the signal at a connection between nodes (artificial neurons / synapses) is a real number, and the output of each artificial neuron is calculated as a nonlinear function of the sum of its inputs. Nodes and their connections typically have a weight that adjusts during the learning process. The weight increases or decreases the strength of the signal at a connection. Nodes can have a threshold so that the signal is only transmitted when the sum signal crosses this threshold. Typically, the nodes are aggregated into layers. Different layers can perform different types of transformations on their inputs. The signals travel from the first layer (the input layer) to the last layer (the output layer), possibly after traversing the layers several times. Although Fig. 6 shows only a single intermediate or hidden layer, a complex deep neural network (DNN) can have many such intermediate layers.

[0034] A supervised artificial neural network is "trained" by providing inputs and then checking and correcting the outputs. For example, a neural network trained to recognize dog breeds processes a set of images and calculates the probability that the dog in an image is of a particular breed. A user can review the results, select which probabilities the network should display (above a certain threshold, etc.), and return the suggested label. Each mathematical manipulation as such is considered a layer, and complex neural networks have many layers. Due to the depth provided by a large number of intermediate or hidden layers, neural networks can model complex nonlinear relationships when trained.

[0035] Fig. Figure 7A is a flowchart describing one embodiment of a process for training a neural network to generate a set of weights. The training process is often performed in the cloud, allowing for additional or more powerful access processing. In step 701, the input, such as a set of images, is received at the input node (e.g., I1, I2, I3 in Fig. 6). In step 703, the input is propagated through the synapses connecting the input nodes to the node of the next layer (e.g., H1, H2, H3, H4 in Fig. 6) using the current set of weights. The output of the neural network is then applied to the next layer of nodes (e.g., H1, H2, H3, H4 in Fig. 6) in step 705, so that the values ​​received as output from one layer serve as input to the next layer. The inputs from the first layer are propagated in this way through all intermediate or hidden layers until they reach the output. In the dog breed example from the previous paragraph, the input would be the image data of a number of dogs, and the intermediate layers use the current weight values ​​to calculate the probability that the dog in an image belongs to a particular breed, with the suggested dog breed label being returned in step 705. A user can then review the results in step 707 to select which probabilities the neural network should return and to decide whether the current set of weights provides a sufficiently accurate label. If so, training is complete (step 711).If the result is not accurate enough, the neural network adjusts the weights in step 709 based on the probabilities selected by the user and then returns to step 703 to rerun the input data with the adjusted weights. Once the neural network's set of weights has been determined, they can be used for "inference." This is the process of using the determined weights to generate an output result from input data to the neural network. Once the weights have been determined in step 711, they can be stored in non-volatile memory for later use. The storage of these weights in non-volatile memory is discussed in more detail below.

[0036] Fig. Figure 7B is a flowchart describing a process for the inference phase of supervised learning using a neural network to predict the "meaning" of the input data using an estimated accuracy. Depending on the case, the neural network can be derived either in the cloud or by the processor of an edge device (e.g., smartphone, automotive process, hardware accelerator). In step 721, the input is received, for example, the image of a dog in the example used above. If the previously determined weights are not present on the device running the neural network application, they are loaded in step 722. For example, on a host processor running the neural network, the weight could be read from an SSD, stored in it, and loaded into RAM on the host device.In step 723, the input data is then passed through the layers of the neural network. Step 723 is similar to step 703 of . Fig. 7B, but now using the weights established at the end of the training process in step 711. After propagating the input through the intermediate layers, the output is then provided in step 725.

[0037] A subset of neural networks is a feedforward network, in which data flows from the input layer through the intermediate layers and to the output layer without returning to the beginning. Initially, in the training phase of supervised learning, the neural network creates Fig. 7A, a network creates a map of virtual neurons and assigns random numerical values, or "weights," to the connections between them. The weights and inputs are multiplied, returning an output between 0 and 1. If the network fails to accurately recognize a particular pattern, an algorithm adjusts the weights. In this way, the algorithm can make certain parameters more influential (by increasing the corresponding weight) or less influential (by decreasing the weight), adjusting the weights accordingly until it determines a set of weights that provides sufficiently correct mathematical manipulation to fully process the data.

[0038] Fig. Figure 8 is a schematic representation of the use of matrix multiplication in a neural network. Matrix multiplication, or MatMul, is a commonly used approach in both the training and inference phases of neural networks and is used in kernel methods for machine learning. Fig. 8 is similar to Fig. 6, where only a single hidden layer is shown between the input and output layers. The input data is represented as a vector of length equal to the number of input nodes. The weights are represented in a weight matrix, where the number of columns corresponds to the number of intermediate nodes in the hidden layer and the number of rows corresponds to the number of input nodes. The output is determined by a matrix multiplication of the input vector and the weight matrix, where each element of the output vector is a dot product of the input data vector and a column of the weight matrix.

[0039] A common technique for performing matrix multiplications is to use a multiplier-accumulator (MAC or MAC unit). However, this has several problems. Referring again to Fig. 7B, the inference phase loads the neural network weights in step 722 before performing the matrix multiplications by propagation in step 723. However, since the amount of data involved can be extremely large, using a multiplier-accumulator for inference has several problems related to loading weights. One of these is high power dissipation, since large MAC arrays with the required resolution must be used. Another problem is high power dissipation due to the limited size of the MAC arrays, which results in high amounts of data movement between logic and memory and far higher power dissipation than that used in the logic calculations themselves.

[0040] To help overcome these limitations, the use of a multiplier-accumulator array can be replaced with other memory technologies. For example, matrix multiplication can be computed in a memory array by leveraging the properties of storage-class memory (SCM), such as those based on ReRAM, PCM, or MRAM-based memory cells. This allows the neural network inputs to be provided via read commands and the neural weights to be pre-loaded for inference. Using in-memory computing can eliminate the need for logic to perform matrix multiplication in the MAC array and the need to move data between memory and the MAC array.

[0041] The following considers embodiments based on a non-volatile memory array structure in which pairs of resistive memory cells, such as ReRAM, MRAM, or PCM memory cells, are each connected to a common bit line but to different word lines. The differential pairs of memory cells form unit synapses, each of which stores a neural network weight. By applying the input of a layer of neural network inputs as differential voltage inputs to a word line pair, a multiplication of the input value by all weights stored in the corresponding unit synapses of the differential memory cell pairs connected along the word line pair can be determined simultaneously within the array.By accumulating the results for the various wordline pairs of the array, matrix multiplication of an input vector and a weight matrix in the array can be performed. The following discussion primarily considers embodiments of a binary neural network, which are discussed first, but can also be extended to more general embodiments.

[0042] One technique that can be used to reduce the computational complexity of the inference process is the use of a binary neural network (BNN), which operates with binary weights and activations. A BNN (also called an XNOR network) computes matrix-vector multiplication with "binary" inputs {-1, 1} and "binary" weights {-1, 1}. Fig. Figure 9 is a table illustrating the output of a binary neural network in response to various input-weight combinations. As shown in the rightmost column, the output is 1 when the input and weight match; and when the input and weight differ, the output is -1.

[0043] When a binary weight is stored in a binary memory cell format, a non-programmed weight (-1) will have a low output for either a low read level (such as mass) or a high read level if the weights -1 and 1 are stored as non-programmed and programmed memory cells, respectively. Due to this, only the +1 weight entries in the table of Fig. 9 correctly. To generate the correct answer for the -1 weight levels, they must be stored separately and complementarily as negative weight levels. In previous approaches to storing binary weights, the weights and negative weights were stored either in separate arrays or along different bit lines of an array. This requires two separate read operations, one for -1 weights and one for +1 weights, with the two read results combined to determine the complete result, which is added to the table of Fig. 9. To improve this situation, the Fig. 10-14 illustrates an embodiment for implementing a neural network with binary inputs and binary weights in a non-volatile memory array that stores weights in a differential memory cell structure using a pair of memory cells to store the weight to allow the weight value to be read in a single read operation.

[0044] Fig. Figure 10 illustrates an embodiment in which two differential memory cells are used to form a unit synapse for storing a weight bit W. In the embodiment shown, the two memory cells R A and R BThe unit synapse consists of resistive memory cells, such as ReRAM, MRAM, or PCM-based memory cells, where complementary resistance levels are each connected between a corresponding word line WL, WLB and a common bit line. The input or neuron value IN is applied to the word line pair or the differential word line of WL, WLB. The output O is then the product of the input IN and the weight W, which corresponds to the level on the bit line BL.

[0045] Fig. Figure 11 illustrates the encoding of the input values, weight values, and output values ​​as word line voltages, resistance values, and bit line voltages, respectively. For the input values ​​IN, an input +1 corresponds to a high voltage level V (a few volts or a few tenths of a volt, for example) applied to WL and a low voltage level (such as ground or 0) applied to WLB. An input of IN=-1 corresponds to a low voltage level 0 applied to WL and a high voltage level V applied to WLB. Consequently, as shown in the first three columns of Fig. 11, a +1 neuron is presented as (V, 0) on (WL, WLB) and a -1 neuron as (0, V).

[0046] For the weight values ​​W, a +1 weight corresponds to a low resistance state (LRS) for R A and a high resistance state (HRS) for R B . A weight value of W=+1 corresponds to a high resistance state (HRS) for R Aand a low resistance state (LRS) for R B , as in the 4th, 5th, and 6th columns of Fig. 11. When an input voltage pattern is applied to the word line pair (WL, WLB), the unit synapse acts as a voltage divider whose output is the voltage V O on the bit line BL with an output value of O=IN*W, as described with reference to the Fig. 12A-12D can be shown.

[0047] The Fig. 12A-12D correspond to the four cases shown in the four rows of Fig. 11 are shown. In Fig. 12A, an input of IN=-1, corresponding to (WL, WLB)=(0, V), is applied to a weight of W=+1, corresponding to (R A , R B )=(LRS, HRS). The resulting voltage on the bit line is then: VO=VBL=V(RL / (RL+RH))=VL, where V L corresponds to an output of O=-1. In Fig. 12B, an input of IN=+1 corresponding to (WL, WLB)=(V, 0) is applied to a weight of W=+1, with a resulting voltage on the bit line of: VO=VBL=V(RH / (RL+RH))=VH, where V H corresponds to an output of O=+1. The Fig. 12C and Fig. 12D similarly represent the respective cases IN=-1 and IN=+1 for the cases W=-1, with respective outputs on the bit line BL of V O = V H (O=+1) and V O = V L (O=-1).

[0048] As through the Fig. As shown in Figures 12A-12D, the differential pair of memory cells with complementary resistance values ​​of a unit synapse forms a voltage divider, so that the bitline BL voltage corresponds to the output values ​​(+1, -1). The differential representation of wordline voltage patterns and resistance states perfectly matches the truth table of Fig. 11 to produce O = IN*W in a single in-array scan operation.

[0049] The resistive memory cells of the unit synapse can be implemented in a number of different forms, some of which are described in the Fig. 13A-13C. Fig. Figure 13A illustrates a 1 transistor 1 resistor (1T1R) embodiment for a resistive memory cell in which a resistor and a transistor are connected in series. The resistor portion in Fig. 13A (and Fig. 13B and Fig. 13C) can be implemented using ReRAM, MRAM, PCM, or another programmable resistance material. When the transistor is off, the memory cell will not conduct; and when the transistor is on, the resistance value of the memory cell is based on the resistance level programmed into the programmable resistance material. Fig. 13B illustrates an embodiment where the memory cell includes 1 switch in series with 1 resistor (1S1R), where, for example, the resistor may be implemented as a phase change material and the switch is an Ovonic threshold switch (OTS). Fig. Figure 13C illustrates an embodiment in which the cell is a single resistance (1R) embodiment.

[0050] The inclusion of a transistor or switch in parallel with the resistor, as in the 1T1R and 1S1R embodiments, which allows the memory cell to turn off, can be useful in situations such as facilitating the programming of a memory array. Although the structures presented here can be used in the training phase of operating a neural network, which involves programming the memory cells, the focus below is on reasoning, where the weights were previously written to the memory array, and is based on the 1R embodiments for the memory cells of the unit synapses.

[0051] Fig. 14 shows the integration of the unit synapses of Fig. 10 into an array, such as in the memory structure 326 of Fig. 5. Fig. Figure 14 shows a small portion (four wordline pairs and two bitlines) of what can be much larger. More specifically, four wordline pairs are shown (WL1, WLB1; WL2, WLB2; WL3, WLB3; WL4, WLB4) spanning N bitlines, of which only BL1 and BL4 are shown, and where the number of wordline pairs can extend to a much larger number of such pairs spanning the bitlines of the array. Although the wordlines of a (WL, WLB) pair in the embodiment of Fig. 14 are shown as adjacent, this need not be the case in other embodiments. The array may be designed in a two-dimensional structure, as shown in Fig. 14, or use a three-dimensional architecture, some examples of which are given below with reference to the Fig. 20 and Fig. 21. Depending on the embodiment, a single integrated memory circuit may have one or more such arrays formed thereon.

[0052] At the intersection of each word line pair (WL i , WBL i ) with a bit line BL j there is a unit synapse that has a weight W i,j stores, whereby a resistive memory cell of the unit synapse between WL i and BL j and the other resistance memory cell of the unit synapse is connected between WLB i and BL jswitched to provide the differential memory cell structure. When a binary weight is stored, one of the resistors is programmed to a high resistance state and the other to a low resistance state. A wordline decoder 1401 applies the differential voltage pattern to the wordline pairs based on the received inputs, which are translated into either the +1 pattern of (V, 0) or the -1 pattern of (0, V). As the wordline pairs span the bitlines of the array, the input applied to (WL1, WLB1) along that wordline pair is applied simultaneously to all weights W 1,j This allows the simultaneous multiplication of the input Input with all weights W 1,j along the word line pair (WL1, WLB1), with the output for each multiplication corresponding to the voltage level on the corresponding bit line BL j corresponds.

[0053] The output level on each bit line BL j can be controlled by a sampling amplifier SA / ADC 1403 j In the case where a single binary input is applied to binary weights (as discussed so far), the sampling amplifiers SA / ADC 1403 j only between a V H state, corresponding to a +1 output, and a V L state, corresponding to a -1 output. For embodiments (as described below) where multiple inputs are applied simultaneously, multi-bit weights are used, or both, the sense amplifiers 1403 j can be multi-bit sampling amplifiers or ADCs capable of distinguishing a multi-bit output. For multi-bit sampling, the different levels can be detected simultaneously or sequentially, depending on the embodiment, as described below with reference to the Fig. 16 and Fig. 19 is discussed.

[0054] Applying the inputs to the array of weights causes the multiplication operations within the array, the results of which are processed by the sampling amplifiers SA / ADC 1403 j along the bit lines BL j To perform the accumulation phase of a multiplication and accumulation operation, each of the sense amplifiers 1403 j with a corresponding digital summing circuit DSC j 1405 j connected to the accumulated output O j provide the information provided above with reference to Fig. 8. The digital summing circuit may be implemented on the memory circuit or, in other embodiments, outside the memory circuit (such as on the controller 102, Fig. 1), so that the summing circuit can accumulate the sum from different memory arrays. Fig. 15 refers to matrix multiplication for a neural network closer to the structure of Fig. 14.

[0055] Fig. Figure 15 shows an example of a neural network and its implementation by an array of differential memory cells. In the example described above with reference to Fig. In the process described in Figure 14, the response to an input of a unit synapse along each bitline is determined based on the voltage level on the bitline. The contribution of each of the synapses along a bitline can be determined and accumulated sequentially by the sense amplifiers for the various inputs. In some embodiments, multiple inputs can be applied simultaneously.

[0056] The example of Fig. Figure 15 shows three fully connected layers, each with four nodes (corresponding to the neurons of the layer), so that the weight matrix between the layers is a 4×4 matrix. Fig. 25 are the inputs from the nodes as IN l,i where I is the layer index and i is the neuron index. In the example of Fig. 15 shows three layers, I = (0, 1, 2), each with four nodes, i = (1, 2, 3, 4). The weight matrices W l,i,j , whose entries correspond to the synapses and which connect the layers, are then 4x4 matrices, where the matrix multiplication to form the scalar products from the inputs of one layer to the next: INl+1,j=Ol,j=∑iINl,i*Wl,i,j, is, where the sum runs over the index i. The inputs of one layer are applied as voltage patterns on the word line pairs to the unit synapses to generate dot product values, which in turn are the inputs of the next layer.

[0057] As discussed above, Fig. 14 is a schematic representation of how these weight matrices are stored in the unit synapses of the array for the in-array calculations of the matrix multiplication. The weight matrix between a pair of layers is stored in a number of unit synapses along a number of bitlines, where the number of unit synapses per bitline and the number of bitlines corresponds to the size of the weight matrix. In the example of Fig. 14, a 4x4 weight matrix would correspond to 4 unit synapses along each of the 4 bit lines. As in Fig. 14, these 4 adjacent unit synapses may be on bit lines BL1-BL4 (where only BL1 and BL4 are shown), but these may be distributed differently across the array depending on the embodiment.

[0058] Regarding the representation of Fig. 15, a weight matrix is ​​stored in transposed form on the array. For example, the weights from the various inputs of the first layer of Fig. 15 are stored in the upper node 1501 of the second layer along the first bit line BL1; and the weights in the lower nodes 1503 are stored along the fourth bit line BL4.

[0059] To calculate the various dot products of matrix multiplication (the sum over i in Equation 1 above), the data inputs can be provided in a sequence of read commands. To calculate the output of a single layer, the pages of weights are then read sequentially by the sense amplifiers over, in this example, four cycles: Cycle 1: reaches IN 0,1 * W 0,1,1 ; Cycle 2: reaches IN 0,2 * W 0,1,2 ; Cycle 3: reaches IN 0,3 * W 0,1,3 ; and Cycle 4: reaches IN 0,4 * W 0,1,4 ,where each of the cycles of a loop in the sequence of Fig. 16 below, and different sampling sequences can be used in different embodiments. The results of the cycles are processed by the sampling amplifier SA / ADC 1403 j on each bit line BL j recorded and in the DSC 1405 j accumulated. The output O l,j of each bit line is then the input IN I+1,j the next layer of the network.

[0060] Fig. 16 is a flowchart for one embodiment of a matrix multiplication or dot product calculation using a binary neural network in inference, as schematically illustrated for the layers of the neural network of Fig. 15 and the array architecture of Fig. 14. Beginning at step 1601, and referring to Fig. 14, the memory array receives an input Input i at the word line decoder 1401 and translates it into the pattern of voltage values ​​corresponding to the -1 or +1 input value, as shown in the table of Fig. 11. Step 1603 applies the voltage levels to the selected word line pair (WL i , WLB i ) corresponding to the unit synapses of the selected weights. Since the word lines span the bit lines of the array, the process can be Fig. 16 simultaneously for each of the unit synapses located along the word line pair (WL i , WLB i ). In addition, in the array structure, the unit synapses on unselected wordline pairs along the bitlines of the array are biased so as not to contribute to the output, such as floating both wordlines of unselected wordline pairs, in step 1605. Although in Fig. 16 as an ordered set of separate steps, steps 1603 and 1605 are typically performed concurrently by wordline decoder 1401.

[0061] Step 1607 determines the response of memory cells of the selected unit synapse. As shown in the table of Fig. 11, the voltage level on the bit line of the unit synapses corresponds to the output logic value of the unit synapse in response to the input and can be determined by the sense amplifier SA / ADC 1403 j on the corresponding bit line BL j Based on the state of the unit synapse, the value of the count of the corresponding DSC 1405 is determined in step 1609. j either incremented or decremented based on the output O j , as above with reference to the table of Fig. 11 discussed.

[0062] Step 1611 determines whether there are more input weight pairs to be added to the matrix multiplication or the dot product, and if so, flow returns to step 1601. Once the contributions of all input weight pairs to the dot products have been determined, the dot product may be provided in step 1613. The set of dot products determined in step 1613 may then serve as the input to a subsequent neural network layer or as the output to the final layer of the inference process.

[0063] The process of Fig. 16 applied one input to the weight matrix and summed the outputs of the individual inputs to complete the matrix multiplication. For embodiments of Fig. 14, where multi-bit sampling amplifiers for SA / ADC 1403 jare available, multiple inputs can be applied simultaneously to the wordline pairs. The voltage level on a bitline then reflects the summed outputs from the input weight products, increasing the degree of parallelism and the resulting performance. The number of inputs that can be successfully applied simultaneously depends on the resolution of the analog-to-digital conversion implemented by an SA / ADC 1403. j where, if sufficient resolution is available, an entire column of all unit synapses along a bit line can be calculated in one operation. The sampling operation with a plurality of inputs can be described with reference to Fig. 17 will be shown.

[0064] Fig. Figure 17 illustrates an equivalent voltage divider for simultaneous multiple inputs in an array-matrix multiplication operation. A first equivalent resistance R eq1is connected between the high input voltage level V and the shared bit line, and a second equivalent resistance R eq2 is connected between the low input voltage level 0 and the common bit line. The V out -Level on the bit line will be: Vout=V⋅Req1Req1+Req2.

[0065] If N is the number of inputs, such as the entire length or a subset of a bit line, let A be the number of +1 outputs and B be the number of -1 outputs, then N=A+B and: Req1=1A⋅1RH+B⋅1RL, Req2=1A⋅1RL+B⋅1RH

[0066] Using these expressions for R eq1 and R eq2 surrendered: Vout=V⋅1A⋅1RH+B⋅1RL1A⋅1RL+B⋅1RH+1A⋅1RH+B⋅1RL=V⋅A⋅(RH−RL)+N⋅RLN⋅(RH+RL)

[0067] The equation shows that a sequence of weight bits follows the voltage divider rule and the output voltage Vout linearly corresponds to the output sum code (AB), as in Fig. 18. The area of ​​the V out -level depends only weakly on the absolute value of the R H and R L -values. In one embodiment, the values ​​of V = 0.5 V, R L = 10 K, and R H = 100 K can be used, and a 3-bit ADC for the multi-state sampling amplifiers allows a step size of 50 mV.

[0068] Fig. 18 shows the relationship between the output voltage V out , as seen on the bit line, and the sum code (AB), which corresponds to the number of +1 outputs minus the number of -1 outputs determined by simultaneously detecting multiple input weight products. As in Fig. As shown in Figure 18, the output voltage increases linearly with the difference (AB), so that a multi-bit sampling amplifier can translate the output voltage into the corresponding matrix multiplication value.

[0069] Fig. 19 is a flowchart for one embodiment of a dot product calculation similar to that shown in Fig. 16, but which includes the parallelism of applying multiple inputs simultaneously. With respect to step 1601, the parallel sensing of weights on each bit line in step 1901 can now apply multiple inputs simultaneously in each loop. At step 1903, the output of the sense amplifier is now a multi-bit value instead of the binary value of step 1603 and corresponds to the number of unit synapses providing high outputs minus those providing low outputs along a bit line. The multi-bit value is then accumulated in step 1905. The flow then determines if there are more inputs in step 1907 and, if so, returns to step 1901. Once all input weight combinations have been determined, the flow proceeds from step 1907 to 1909 and provides the accumulated count as the result of the matrix multiplication.

[0070] The reading process for a unit synapse formed from a differential pair of memory cells (whether only a single word line pair as in Fig. 16 is selected or several word line pairs are selected simultaneously as in Fig. 19) differs from a standard read operation. In a standard read operation, where the goal is to determine the data state stored in a memory cell, the determination is made by a sense amplifier based on a current or voltage level along a bit line, based on the state of a single selected memory cell. When multiple cells along a common bit line are sensed at the same time, it is not possible to determine the resistance value of the individual memory cells and establish their corresponding data states. However, for the output of the summing circuit from the matrix multiplication, only the sum of the cumulative sum of the outputs of the unit synapses is desired, not those of the individual synapses.As a result, the response of multiple unit synapses on wordline pairs can be determined simultaneously in response to a corresponding set of inputs, increasing parallelism when the sense amplifier can determine the number of conducting synapses. By integrating multiple sense amplifiers, the embodiment of [the text] enables [the text]. Fig. 14, that multiple unit synapses are sampled in parallel along a common bit line.

[0071] In addition to applying multiple inputs simultaneously to an array of weights, parallelism can be further increased by operating multiple arrays in parallel. The multiple arrays can be formed on the same memory chip or distributed across multiple memory chips. Depending on the implementation, the weights corresponding to different layers of a neural network can be stored in the same array or in different arrays. Different arrays for different layers can enable pipelining, with the outputs of one layer / array serving as inputs to the next layer / array.

[0072] With further reference to Fig. 14, this represented the differential memory array for storing the weights in a two-dimensional structure. The same differential topology can also use a three-dimensional structure, such as a differential crosspoint (DX) memristor array, as in Fig. 20, or an extended three-dimensional architecture, as in Fig. 21 shown.

[0073] Fig. 20 shows an embodiment of a portion of a monolithic three-dimensional memory array 2001 forming a differential crosspoint (DX) architecture including a second memory plane 2020 positioned above a first memory plane 2018. The memory array 2001 is an example of an implementation for the memory array 326 in Fig. 5. The bit lines BL1-BL5 are arranged in a first direction (shown as extending into the page) and the word lines WL1-WL4 and WLB1-WLB4 are arranged in a second direction perpendicular to the first direction. Fig. Figure 20 is an example of a horizontal crosspoint structure in which the word lines WL1-WL4 and WLB1-WLB4 and BL1-BL5 both extend horizontally relative to the substrate, while the memory cells 2000 are aligned so that current flows vertically. As illustrated, the upper conductors of the first memory level 2018 can be used as the lower conductors of the second memory level 2020, which is positioned above the first memory level. In a memory array with additional layers of memory cells, there would be corresponding additional layers of bit lines and word lines.

[0074] As in Fig. 20, the memory array 2001 includes a plurality of memory cells 2000. The memory cells 2000 may include rewritable memory cells, such as may be implemented using ReRAM, MRAM, PCM, or other material with a programmable resistor. With respect to the first memory level 2018, a first portion of the memory cells 2000 is located between and connected to the bit lines BL1-BL5 and the word lines WL1-WL4. With respect to the second memory level 2020, a second portion of the memory cells 2000 is located between and connected to the bit lines BL1-BL5 and the word lines WLB1-WLB4. As with respect to the Fig. 13A-13C, in one embodiment, each memory cell includes a select switch or control element (e.g., an Ovonic threshold switch or a diode) and a memory element (i.e., a phase-change memory element). In one example, the current in the memory cells of the first memory level 2018 may flow upward, as indicated by arrow A1, while the current flowing in the memory cells of the second memory level 2020 may flow downward, as indicated by arrow A2, or vice versa. In another embodiment, each memory cell includes a state-change element and does not include a select switch element. The absence of a select switch (or other control element) in a memory cell may reduce the process complexity and cost associated with manufacturing a memory array.

[0075] In the intersection point structure of Fig. 20, a differential memory cell of a unit synapse can be formed by a memory cell from layer 2020 connected between one of the word lines WL1-WL4 and one of the bit lines BL1-BL5, and a memory cell from layer 2018 connected between WLB1-WLB4 and the same of the bit lines BL1-BL5. For example, a unit synapse can be formed by a combination of the memory cell 2000 A , which is connected between WL1 and BL1 and has a resistance level R A and the memory cell 2000 B , which is connected between WLB1 and BL1 and has a resistance level R B as in Fig. 10. Although this example is formed from directly opposite memory cells in the cross-point structure, more generally, other memory cell pairs along a shared bit line may be used.

[0076] Fig. Figure 21 illustrates an embodiment of a portion of a monolithic three-dimensional memory array 2116 including a first memory level 2112 positioned beneath a second memory level 2110. The architecture of the Fig. Figure 21 provides another example of an embodiment that can be used to store neural network weights, in this case using an extended three-dimensional structure. Memory array 2116 is an example of an implementation for memory array 326 in Fig. 5. As shown, the local bit lines LBL 11 to LBL 33 arranged in a first direction (ie a vertical direction), and the word lines WL 10 to WL 23are arranged in a second direction perpendicular to the first direction. This arrangement of vertical bit lines in a monolithic three-dimensional memory array is one embodiment of a memory array with vertical bit lines. As shown, a specific memory cell is arranged between the intersection of each local bit line and each word line (e.g., memory cell M 111 between the local bit line LBL 11 and the word line WL 10arranged). This structure can be used with a number of different memory cell structures. In one example, the particular memory cell may include a floating gate device or a charge trapping device (e.g., using a silicon nitride material). In another example, the particular memory cell may include a reversible resistive switching material, a metal oxide, a phase-change memory material, a ReRAM material, an MRAM material, or a PCM material. The global bitlines GBL1 to GBL3 are arranged in a third direction that is perpendicular to both the first direction and the second direction. A set of the bitline select devices (e.g., Q 11 -Q 31 ), such as a vertical thin-film transistor (VTFT), can be used to select a set of local bit lines (e.g., LBL 11 -LBL 31) are used. As shown, the bit line select devices Q 11 -Q 31 used to control the local bit lines LBL 11 -LBL 31 and the local bit lines LBL 11 -LBL 31 to the global bit lines GBL1-GBL3 using a row select line SG1. Analogously, the bit line select devices Q 12 to Q 32 used to control the local bit lines LBL 12 to LBL 32 to selectively connect the global bit lines GBL1 to GBL3 using a row selection line SG2, and the bit line selection devices Q 13 to Q 33 are used to control the local bit lines LBL 13 to LBL 33 to selectively connect the global bit lines GBL1 to GBL3 using a row select line SG3.

[0077] In the embodiment of Fig. 21, a unit synapse can be formed from a differential pair of memory cells connected between a shared bit line and one of a respective pair of word lines. For example, a unit synapse can be formed from the differential pair of memory cells (top left) M 211 , which is between WL 20 and LBL 11 are connected, and M 212 , which is between WL 21 and LBL 11 are connected.

[0078] The discussion so far has considered binary neural networks where both the input and the weights are binary values, but the use of the differential memory cell structure can be applied more generally. For example, multiple differential memory cells can be used to store multi-bit weight values. In other embodiments, the technique can be extended by using multi-level cells, multi-level input voltages, or both.

[0079] In a multi-level cell embodiment, each of the memory cells of the unit synapse can be Fig. 10 Store data in a multi-level format and store more than two values ​​for the resistance levels R A and R B For example, in a differential 2-bit multilevel cell embodiment, R A one of the values ​​{-1, -0.5, 0.5, 1} and R Bbe the complementary value of {1, 0.5, -0.5, -1}. Other embodiments may store more bits per unit synapse of the differential memory cell pair, or even store data as a continuous range of resistance levels. Using a multi-stage cell embodiment may achieve a neural network with higher accuracy, but at the cost of increased complexity with read tolerance limited by the read voltage and ADC resolution.

[0080] The use of multiple differential memory cells to store multi-bit weight values ​​is described with reference to the Fig. 22 - 27 in detail. To store an N-bit weight value in one set of embodiments, N copies of the differential memory cell structure of Fig. 10 connected along the same wordline pair, each storing one bit of a multi-bit weight value and each connected to a different bitline. When a differential input is applied to the wordline pair, the output on the bitline of each of the differential memory cells yields the corresponding bit from the matrix multiplication. The different outputs can then be weighted and combined according to the significance of the bit they represent. As with the binary case described above, the results of multiple inputs can be accumulated to perform a matrix multiplication operation.Although the following discussion is presented primarily in the context of a differential memory cell structure using resistive memory cells, it may be applied more generally, such as in embodiments based on programmable threshold devices having a NAND structure of two or more memory cells. Fig. Figure 22 illustrates an example of a more general arrangement.

[0081] Fig. Figure 22 is a schematic diagram of storing a multi-bit weight in a non-volatile array for use in an in-array inference operation. More specifically, Fig. 22 an input, such as a differential word line pair, applied to a set of multi-bit weights as part of a matrix multiplication, wherein two weights (W1 and W j) of N bits. By arranging several binary unit synapses, formed from non-volatile memory cells, along a row of an array, the weights can be represented with greater precision. The N bits of W1, for example, are denoted here as (W N 1 , ...., W 2 1, W 11) in order of the most significant bit (superscript bit N) to the least significant bit (superscript bit 1) and shown arranged along the bit lines in this order, but other embodiments may arrange the weights and the individual bits of the weights differently, such as interleaving bits from different weights. In the following, a memory unit refers to a group or set of multiple memory elements storing an N-bit weight, where each of the bits is stored in a binary memory element. Furthermore, the input lines for the weights correspond to embodiments based on a differential memory cell, as shown in Fig. 10, a pair of word lines to which a differential input is applied.

[0082] When the input In is applied to the series of weights, the outputs (O N 1, ..., O 21, O 1 1) each of the bit lines (BL N 1, ..., BL 2 1, BL 1 1) Weighted according to the significance of the bit stored in the weighting and summing circuit 22071. Depending on the implementation, the sensing can be voltage-based or current-based. The weighted and combined output O1 can then be captured by the multi-bit sense amplifier or the analog-to-digital converter SA / ADC 22031. The counter-based digital summing circuit DSC1 14051 can accumulate the input weight multiplication operations as in the binary case using a multi-state sense amplifier.

[0083] To implement N-bit precision weights with binary non-volatile memory cells, the changes from the binary weight example only need to increase the number of columns / bit lines by a factor of N and to use the binary weighted summation element (2207 from Fig. 22) at the end of the bit lines. The other peripheral circuits (i.e., write driver / decoder, read-related circuits, ADC) can remain unchanged. The main discussion below is based on the differential memory cell array and voltage-based output values, as for the binary array of Fig. 14, but can be applied to the more general situation arising in relation to Fig. 22 is illustrated.

[0084] Fig. 23 illustrates the integration of the unit synapses of Fig. 10 into an array for a multi-bit weight embodiment. Fig. Figure 23 shows a 3-bit embodiment and largely repeats the elements of Fig. 14, but groups the binary unit synapses and other elements of Fig. 14 into sets of three for storing 3-bit weight values ​​and adds a weighting and summation section.

[0085] More specifically, a wordline decoder 2301 receives a set of one or more inputs and converts them into the voltage patterns to be applied to the differential wordline pairs, such as (WL1, WLB1). This can be a binary input or, more generally, another differential input. Only a single 3-bit weight is applied for each differential wordline pair in Fig. 23, but a typical embodiment includes more such weights. Regarding the weight W 11 is its most significant bit W 3 11 along BL 3 1, its second bit W 2 11 is along BL 2 1, and its least significant bit W 1 11 is along BL 1 1. If the input, on the weight W 11 is applied as part of a matrix multiplication operation, the bit lines BL 3 1, BL 21, BL 1 1 the output voltage levels V out 3 1, V out 2 1, and V out 1 1. The output voltage levels V out 3 1, V out 2 , and V out 1 1 are then weighted according to their significance level and combined by 22071 to produce the multi-bit output O1 corresponding to In1 * W 11 The sense amplifier or ADC SA / ADC 23031 senses the level of the multi-bit output O1, with the results then being accumulated in DSC 23051. Multi-bit sampling and accumulation can be implemented similarly to sampling multiple synapses along the same bit line, as described with reference to Fig. 18. Although the Fig. 23 shows only one multi-bit weight per word line pair, the read operation may be performed simultaneously for some or all of the weights connected along a word line pair.

[0086] Fig. 24 is a flowchart for one embodiment of a matrix multiplication or dot product calculation using multi-bit weights during reasoning based on the array architecture of Fig. 23, which makes the approach of Fig. 16 to include multi-bit weights. Beginning at step 2401, the memory array receives an input signal at wordline decoder 2301 and converts it into the pattern of voltage values. For a binary input, this may again correspond to a -1 or +1 input value, as shown in the table of Fig. 11. Step 2403 applies the voltage levels to the selected word line pair (WL i , WLB i ) corresponding to the unit synapses of the selected multi-bit weights. Since the word lines span the bit lines of the array, the process can be Fig. 24 simultaneously for each of the weights along the word line pair (WL i , WLB i ). In addition, in the array structure, the unit synapses on unselected wordline pairs along the bitlines of the array are biased so as not to contribute to the output, such as floating both wordlines of unselected wordline pairs, in step 2405. Although in Fig. 24 as an ordered set of separate steps, steps 2403 and 2405 are typically executed concurrently by wordline decoder 2301.

[0087] The outputs of each of the bits of the weight in response to the input are generated in step 2407. This corresponds to the output voltage levels V out 3 1, V out 2 1 and V out 1 1 on the respective bit lines BL 3 1, BL 2 1 and BL 11 in the stress-based determination of Fig. 23. The contributions of the individual bits are then weighted and combined in elements 23071 and then provided to SA 23031 to determine the multi-bit output of the weight in response to the input at step 2409. Based on the state of the weight, in step 2411, the value of the count of the corresponding DSC 2305 j either incremented or decremented based on the multi-bit output O j .

[0088] Step 2413 determines whether there are more input weight pairs to be added to the matrix multiplication or the dot product, and if so, flow returns to step 2401. Once the contributions of all input weight pairs to the dot products have been determined, the dot product may be provided in step 2415. The set of dot products determined in step 2415 may then serve as the input to a subsequent neural network layer or as the output to the final layer of the inference process.

[0089] As regards the Fig. 23 and Fig. 24, allows combining multiple binary unit synapses of Fig. 10, the memory array stores the weights with higher precision, whether for the illustrated 3-bit embodiment, for a 2-bit embodiment, or for embodiments with higher bit counts. Regarding the input values, this can again be a binary input, as shown in the table in Fig. shown, or a more general differential input. Although the embodiment of Fig. 23 is based on the differential memory cell synapse structure, the process of Fig. 24 similarly applicable to the more general situation arising in Fig. 22, where other synapse cell structures with a voltage-based or current-based output of the input vector weight matrix multiplication are shown.

[0090] As with the binary weight example, multiple inputs can be applied to the array simultaneously to increase the degree of parallelism, with the combined outputs on the bit lines being a bitwise addition of the results of the input vector-weight matrix multiplication for the applied inputs. The flow of Fig. 25 extends the flow of Fig. 24 to the embodiment with multiple simultaneous inputs and also corresponds to the multi-bit weight extension of the flow of Fig. 19.

[0091] Fig. 25 is a flowchart for an embodiment of a dot product calculation similar to that shown in Fig. 24, but which includes the parallelism of applying multiple inputs to the multi-bit weights simultaneously. With respect to step 2401, the parallel sampling in step 2501 can now apply multiple inputs simultaneously in each loop. At step 2503, the bitwise outputs along the various word lines are determined. For example, referring back to Fig. 23, when Input1, Input2, Input3 and Input4 are applied simultaneously, the bit line BL 3 1 the contributions of the most significant bits W 4 11 , W 4 12 , W 4 13 and W 4 14 These bitwise contributions are then weighted, combined, and fed to the sense amplifiers in step 2505.

[0092] At step 2507, the output determined by the sense amplifier or ADC 23031 is now a multi-bit value and corresponds to the combined product of all currently applied inputs and the multi-bit weights. The multi-bit value is accumulated in step 2509. The flow then determines if there are more inputs in step 2511, and if so, returns to step 2501. Once all input / weight combinations have been determined, the flow continues from step 2511 to 2513 and provides the accumulated count as the result of the matrix multiplication.

[0093] With reference to the Fig. 22 and Fig. 23 contain a weighting and summing circuit 2207 j and 2307 j to weight the output levels on the different bit lines according to their importance and to combine the result. Fig. Figure 26 illustrates an embodiment of such a circuit for voltage-based summation, and Fig. Figure 27 shows such a circuit for current-based summation.

[0094] Fig. Figure 26 illustrates an embodiment for a binary weighting and summing circuit 23071 of Fig. 23, which is used to weight and combine the bitwise output of the in-array matrix multiplication operation of Fig. 23 can be used. For the 3-bit embodiment, the output voltage levels V out 3 1, V ou t 2 1 and V ou t 1 1 on the respective bit lines BL 3 1, BL 2 1, BL 11 is provided by a corresponding resistor 2615, 2613 and 2611 respectively. The relative values ​​for the resistors 2615, 2613 and 2611 are chosen to weight the inputs based on their relevance, where if 2611 has a value R, then 2613 is R / 2 and 2615 is R / 4. More generally, each bit of importance i is represented by a resistor with a value of R / 2 (i-1) provided.

[0095] The weighted output voltages are fed to the input of the summing amplifier 2601, which also provides a reference voltage V cm at the +input. A feedback loop with the base resistance R 2603 connects the output of the summing amplifier 2601 back to the -input and the output of the summing amplifier 2601 provides the weighted and summed output V out1 through the resistor R 2605. The resulting output of the weighting and summing circuit 2307 of Fig. 26 is: Vout1=Vcm−(4*Vout31+2*Vout21+1*Vout11).

[0096] With further reference to Fig. 23 is then V out1 of 23071 the input of the multi-bit sampling amplifier SA 23071.

[0097] The above discussion has been described primarily with reference to a voltage-based determination for the outputs, wherein an embodiment of a weighting and summing circuit 23071 in Fig. 26. Other embodiments may use a current-based determination for the outputs, wherein an embodiment of a current-based weighting and summing circuit 23071 is shown in Fig. 27 is shown.

[0098] Fig. Figure 27 is a schematic diagram of one embodiment of a current-based weighting and summing circuit that can be used to implement element 23071 of Fig. 23. The embodiment of the Fig. 27 includes a set of multiplier current mirrors 2740 1 1, 2740 2 1 and 2740 3 1, which are set up to represent the different bits of the weight W 11 corresponding outputs, and a summing current mirror 27401 to sum these results. The first, second, and third multiplying current mirrors 2740 1 1, 2740 2 1 and 2740 3 1 each receive a current from a corresponding one of the bit lines BL 1 1, BL 2 1 and BL 3 1 of Fig. 23.

[0099] The first multiplier current mirror 2740 1 1 has the transistors T1 and T2, the second current mirror 2740 2 has the transistors T3 and T4, the third current mirror 2740 3 1 comprises the transistors T5 and T6 and the summing current mirror 27401 comprises the transistors T7 and T8. In the embodiment of Fig. 27, the weighting of the various inputs, taking into account their relative importance, is implemented by the relative dimensioning of the pair transistors in each of the multiplier current mirrors. Starting with the NMOS transistor T1 with the width-to-length ratio W n / L, the other transistors of the multiplier current mirrors can dimension their transistors relative to T1. To achieve a 1:1 ratio in the first multiplier current mirror 2740 1 1, the transistor T2 has the same size as T1, W n / L. To achieve a current ratio of 2:1 for the second current mirror 2740 2 To provide the transistor T3 can be used again as W n / L and the transistor T4 as 2W n / L. Likewise, the third current mirror can be 2740 3 1 the transistor T5 as W n / L and the transistor T6 as 4W n / L to provide a current ratio of 4:1. More generally, when bits of higher importance are included, the ratio of the width-to-length ratios between the mirror transistors would be 2 (i-1) W n / L to W n / L for a bit of meaning i.

[0100] To perform the summation operation of the weighted currents from the multiplier current mirrors 2740 1 1, 2740 2 1 and 2740 3 1, the PMOS transistor T7 of the summing current mirror 27401 collectively supplies currents to T2, T4 and T6, so that the current through T7 is the sum of the weighted currents: 4 * I out 3 1 + 2 * I out 2 1 + 1 * I out 1 1. The PMOS transistor T7 is connected as a diode, and its control gate is connected to the control gate of the PMOS transistor T8 in a current mirror arrangement. The output current I outthrough transistor T8 is proportional to the current through transistor T7, where the proportionality is based on their relative size and can be chosen depending on the characteristics of the sampling amplifier 23051. For example, if both T7 and T8 are used as W p / L, then I out1 = 4 * I out 3 1 + 2 * I out 2 1 + 1 * I out 1 1.

[0101] Once the currents corresponding to the different bits of a weight are weighted and combined, the combined current level can then be fed to a current-based multi-bit sense amplifier SA 23051 to determine its contribution to the input vector weight matrix multiplication. For both current-based implementations, as in Fig. 27, as well as for voltage-based implementations, as in Fig.26, the multi-bit output can determine all bits in parallel, sequentially on a bit-by-bit basis, or by a combination of the two approaches.

[0102] According to a first set of aspects, a non-volatile memory circuit includes an array of one or more memory units connected along one or more first input lines and each configured to store an N-bit weight of a neural network, and one or more control circuits connected to the array of memory units. N is an integer greater than 1, and each of the memory units includes N non-volatile binary storage elements connected to a corresponding output line and configured to store one bit of the N-bit weight.The one or more control circuits are configured to: apply a first input of a neural network to the one or more first input lines to generate, in response thereto, an output voltage level on each of the output lines connected to a first one of the storage units; individually weight the output voltage levels generated in response to the first input on each of the output lines according to a meaning of the bit of the weight stored in the binary storage element of the first storage unit corresponding to the output line; and determine a multi-bit value for the response of the weight stored in the first storage unit to the first input from a combination of the individually weighted output voltages to thereby perform an in-array multiplication of the first input by the weight stored in the first storage unit.

[0103] Further aspects include a method including receiving a plurality of input values ​​for a neural network and translating each of the plurality of input values ​​into a corresponding first voltage pattern, each first voltage pattern being one of a plurality of voltage patterns comprising a pair of voltage values.The plurality of first voltage patterns are applied to one or more pairs of word lines, each pair of word lines being connected to a first set of N pairs of non-volatile memory cells storing an N-bit weight value of the neural network, each pair of memory cells of each first set storing one bit of the weight value and comprising a first memory cell connected between a first word line of the word line pair and a corresponding bit line and a second memory cell connected between a second word line of the word line pair and the corresponding bit line.The method further includes weighting an output level on each of the bit lines corresponding to the pair of memory cells of each first set in response to the applied first voltage patterns according to a meaning of the bit stored in the corresponding pair of memory cells, and combining the weighted output levels to determine a multi-bit value for one or more products of the one or more input values ​​with the one or more weight values.

[0104] In additional aspects, a device comprises an array of non-volatile memory cells and one or more control circuits connected to the array of non-volatile memory cells. The array of non-volatile memory cells includes a plurality of N first bit lines, a first word line pair, and a first group of N non-volatile memory cell pairs. A first memory cell of each pair of the first group is connected between a first word line of the first word line pair and a corresponding first bit line; and a second memory cell of each pair of the first group is connected between a second word line of the first word line pair and the corresponding first bit line.The one or more control circuits are configured to: receive a first input and apply a first voltage pattern corresponding to the first input to the first word line pair; individually weight an output level on each of the N first bit lines in response to applying the first voltage pattern to the first word line pair; combine the individually weighted output levels to form a weighted response of the first group of memory cells to the first input; and determine a multi-bit value for the weighted response of the first group of memory cells to the first input.

[0105] For the purposes of this document, reference in the specification to "one embodiment," "some embodiments," or "another embodiment" may be used to describe different embodiments or the same embodiment.

[0106] For the purposes of this document, a connection can be a direct connection or an indirect connection (e.g., via one or more other parts). In some cases, when one element is referred to as being connected or coupled to another element, the element may be directly connected to the other element or indirectly connected to the other element through intervening elements. When an element is referred to as being directly connected to another element, there are no intervening elements between the element and the other element. Two devices are "in communication" when they are connected, directly or indirectly, so that they can pass electronic signals between each other.

[0107] For the purposes of this document, the term “based on” can be read as “at least partly based on”.

[0108] For the purposes of this document, without additional context, the use of numeric expressions such as a "first" object, a "second" object, and a "third" object may not imply ordering of objects, but may instead be used for identification purposes to identify different objects.

[0109] For the purposes of this document, the term “set” of objects may refer to a “set” of one or more of the objects.

[0110] The foregoing detailed description has been presented for purposes of illustration and description. It is not intended to be exhaustive or to limit the precise form disclosed. Many modifications and variations are possible in light of the above teachings. The described embodiments were chosen to best explain the principles of the proposed technology and its practical application, and thus to enable others skilled in the art to best utilize it in various embodiments and with various modifications as are suited to the particular use contemplated. It is intended that the scope be defined by the claims appended hereto.

Claims

[1] A non-volatile memory circuit comprising: an array of one or more memory units connected along one or more first input lines and each configured to store an N-bit weight of a neural network, where N is an integer greater than 1, each of the memory units comprising N non-volatile binary storage elements connected to a respective output line and configured to store one bit of the N-bit weight; and one or more control circuits connected to the array of storage units; the one or more control circuits are configured to: applying a first input of a neural network to the one or more first input lines to generate, in response thereto, an output voltage level on each of the output lines connected to a first one of the memory units; individually weight the output voltage levels generated in response to the first input on each of the output lines according to a meaning of the bit of the weight stored in the binary storage element of the first storage unit corresponding to the output line; and to determine a multi-bit value for the response of the weight stored in the first memory unit to the first input from a combination of the individually weighted output voltages, thereby performing an in-array multiplication of the first input by the weight stored in the first memory unit. [2] The non-volatile memory circuit according to claim 1, wherein: the array further comprises a second storage unit connected along one or more second input lines and configured to store an N-bit weight of a neural network, wherein the second storage unit comprises N non-volatile binary storage elements connected to a corresponding output line and configured to store one bit of the N-bit weight, wherein for each of the binary storage elements with a given meaning, the second storage unit is connected to the same output line as the binary storage elements with an equal meaning of the first storage unit; and the one or more control circuits are further arranged to: applying a second input of the neural network to the one or more second input lines to generate, in response thereto, an output voltage level on each of the output lines connected to the second memory unit; individually weight the output voltage levels generated in response to the second input on each of the output lines according to the meaning of the bit of the weight stored in the binary storage element of the second storage unit corresponding to the output line; and determining a multi-bit value for the response of the weight stored in the second storage unit to the second input from a combination of the individually weighted output voltages generated in response to the second input, thereby performing an in-array multiplication of the second input by the weight stored in the second storage unit. [3] The non-volatile memory circuit of claim 2, wherein the one or more control circuits are further configured to: to accumulate the multi-bit value for the response of the weight stored in the first storage unit to the first input and the multi-bit value for the response of the weight stored in the second storage unit to the second input. [4] The non-volatile memory circuit of claim 3, wherein the one or more control circuits are further configured to: applying the second input of the neural network to the second input lines at the same time as applying the first input of the neural network to the first input lines. [5] The non-volatile memory circuit of claim 3, wherein the one or more control circuits are further configured to: applying the second input of the neural network to the second input lines sequentially with applying the first input of the neural network to the first input lines. [6] The non-volatile memory circuit of claim 3, wherein the one or more control circuits are further configured to: simultaneously with applying the first input to the first input lines to generate, in response, the output voltage level on each of the output lines connected to the first of the memory units, applying the first input to the first input lines to generate, in response, an output voltage level on each of the output lines connected to a second of the memory units; individually weight the output voltage levels generated in response to the first input on each of the output lines according to a meaning of the bit of the weight stored in the binary storage element of the second storage unit corresponding to the output line; and to determine a multi-bit value for the response of the weight stored in the second storage unit to the first input from a combination of the individually weighted output voltages, thereby performing an in-array multiplication of the second input by the weight stored in the first storage unit. [7] The non-volatile memory circuit according to claim 1, wherein: one or more first input lines are a pair of first word lines; each of the binary memory elements includes a pair of memory cells each connected between the corresponding output line and one of the first word lines; and the first input is applied to the pair of first word lines as one of a plurality of voltage patterns. [8] The non-volatile memory circuit according to claim 7, wherein the non-volatile memory cells include phase change memory (PCM) based memory cells. [9] The non-volatile memory circuit of claim 7, wherein the memory array is formed according to a cross-point architecture as part of a monolithic three-dimensional semiconductor memory device, the memory cells being arranged in multiple physical planes over a substrate. [10] Method comprising: receiving a plurality of input values ​​for a neural network; translating a plurality of input values ​​into a corresponding first voltage pattern, each first voltage pattern being one of a plurality of voltage patterns having a pair of voltage values; applying the plurality of first voltage patterns to one or more pairs of word lines, each pair of word lines being connected to a first set of N pairs of non-volatile memory cells storing an N-bit weight value of the neural network, each pair of memory cells of each first set storing one bit of the weight value and having a first memory cell connected between a first word line of the word line pair and a corresponding bit line, and a second memory cell connected between a second word line of the word line pair and the corresponding bit line; weighting an output level on each of the bit lines corresponding to the pair of memory cells of each first set in response to the applied first voltage patterns according to a meaning of the bit stored in the corresponding pair of memory cells; and combining the weighted output levels to produce a multi-bit value for one or to determine several products of the one or more input values ​​with the one or more weight values. [11] A method according to claim 10, wherein: applying the plurality of first voltage patterns to one or more pairs of word lines includes applying the plurality of first voltage patterns sequentially to a corresponding plurality of the word line pairs; and the weighting of the output levels on the bit lines occurs in response to the sequential application of the plurality of first voltage patterns. [12] A method according to claim 10, wherein: applying the plurality of first voltage patterns to one or more pairs of word lines includes applying the plurality of first voltage patterns simultaneously to a corresponding plurality of the word line pairs; and the weighting of the output levels on the bit lines occurs in response to the simultaneous application of the plurality of first voltage patterns. [13] The method of claim 10, wherein each pair of word lines is further connected to a second set of N pairs of non-volatile memory cells storing an N-bit weight value of the neural network, each pair of memory cells of each second set storing one bit of the weight value and comprising a first memory cell connected between a first word line of the word line pair and a corresponding bit line and a second memory cell connected between a second word line of the word line pair and the corresponding bit line, the method further comprising: weighting the output level on the bit lines corresponding to the pair of memory cells of each second set in response to the applied first voltage patterns according to a meaning of the bit stored in the corresponding pair of memory cells. [14] Device comprising: an array of non-volatile memory cells, including a plurality of N first bit lines, a first word line pair, and a first group of N non-volatile memory cell pairs, a first memory cell of each pair of the first group connected between a first word line of the first word line pair and a corresponding one of the first bit lines, and a second memory cell of each pair of the first group connected between a second word line of the first word line pair and the corresponding one of the first bit lines; and one or more control circuits connected to the array of non-volatile memory cells; the one or more control circuits are arranged to: receiving a first input and applying a first voltage pattern corresponding to the first input to the first word line pair; individually weighting an output level on each of the N first bit lines in response to applying the first voltage pattern to the first word line pair; combining the individually weighted output levels to form a weighted response of the first group of memory cells to the first input; and determining a multi-bit value for the weighted response of the first group of memory cells to the first input. [15] The apparatus of claim 14, wherein the output levels are current levels and the one or more control circuits include a current mirror configured to weight the output level on each of the N first bit lines. [16] The device of claim 15, wherein the current mirror is arranged to implement the weighting of the output level by relative dimensioning of transistors in the current mirror. [17] The apparatus of claim 14, wherein the output levels are voltage levels and the one or more control circuits include a voltage summing circuit for combining the individually weighted output levels. [18] The device of claim 14, wherein the array of non-volatile memory cells is formed according to a cross-point architecture as part of a monolithic three-dimensional semiconductor memory device, the memory cells being arranged in multiple physical planes over a substrate. [19] Device according to claim 14, wherein: the array of non-volatile memory cells further includes a second word line pair and a second group of N non-volatile memory cell pairs, a first memory cell of each pair of the second group connected between a first word line of the second word line pair and a corresponding one of the first bit lines, and a second memory cell of each pair of the second group connected between a second word line of the second word line pair and the corresponding one of the first bit lines; and the one or more control circuits are further arranged to: receive a second input and apply a second voltage pattern corresponding to the second input to the second word line pair; individually weight an output level on each of the N first bit lines in response to applying the second voltage pattern to the second word line pair; combining the individually weighted output levels to form a weighted response of the second group of memory cells to the second input; to determine a multi-bit value for the weighted response of the second group of memory cells to the second input; and to accumulate the multi-bit value for the weighted response of the second group of memory cells to the second input with the multi-bit value for the weighted response of the first group of memory cells to the first input. [20] The apparatus of claim 19, wherein the one or more control circuits are further configured to: applying the second voltage pattern to the second word line pair simultaneously with applying the first voltage pattern to the first word line pair; and individually weight the output level on each of the N first bit lines in response to the simultaneous application of the second voltage pattern to the second word line pair and the first voltage pattern to the first word line pair.

Images