Method and apparatus for programming simulated neural memory in deep learning artificial neural network

Abstract

Numerous embodiments of programming systems and methods for use with vector-matrix multiplication (VMM) arrays in artificial neural networks are disclosed. Accordingly, the selected cell can be programmed very precisely to maintain one of the N different values.

Description

[0001] This invention patent application is a divisional application of the invention patent application with international application number PCT / US2019 / 014357, international application date January 18, 2019, application number 201980018437.9 entering the Chinese national phase, and name “Methods and devices for programming simulated neural memories in deep learning artificial neural networks”.

[0002] Priority declaration

[0003] This application claims priority to U.S. Provisional Patent Application No. 62 / 642,878, filed on March 14, 2018, entitled “Method and Apparatus for Programming Analog Neuromorphic Memory in an Artificial Neural Network,” and U.S. Patent Application No. 15 / 990,395, filed on May 25, 2018, entitled “Method And Apparatus For Programming Analog Neural Memory In A Deep Learning Artificial Neural Network.” Technical Field

[0004] The present invention discloses a number of embodiments of programming apparatus and methods for use with vector-matrix multiplication (VMM) arrays in artificial neural networks. Background Art

[0005] Artificial neural networks simulate biological neural networks (the central nervous system of animals, especially the brain), and are used to estimate or approximate functions that may depend on a large number of inputs and are usually unknown. Artificial neural networks typically consist of layers of interconnected "neurons" that exchange messages with each other.

[0006] Figure 1 An artificial neural network is shown, where circles represent inputs or layers of neurons. Connections (called synapses) are represented by arrows and have numerical weights that can be adjusted based on experience. This allows the neural network to adapt to the input and be able to learn. Typically, a neural network includes multiple layers of inputs. There are typically one or more intermediate layers of neurons, and an output layer of neurons that provide the output of the neural network. Neurons at each level make decisions based on data received from the synapses, either individually or collectively.

[0007] One of the main challenges in developing artificial neural networks for high-performance information processing is the lack of adequate hardware technology. In fact, practical neural networks rely on a large number of synapses to achieve high connectivity between neurons, i.e. very high computational parallelism. In principle, such complexity can be achieved by digital supercomputers or clusters of dedicated graphics processing units. However, in addition to being high cost, these approaches are also mediocre in energy efficiency compared to biological networks, which consume less energy mainly due to the low-precision analog calculations they perform. CMOS analog circuits have been used for artificial neural networks, but given the large number of neurons and synapses, the synapses of most CMOS implementations are too large.

[0008] Applicant previously disclosed an artificial (simulated) neural network utilizing one or more non-volatile memory arrays as synapses in U.S. Patent Application 15 / 594,439, which is incorporated herein by reference. The non-volatile memory array operates as an simulated neuromorphic memory. The neural network device includes a first plurality of synapses configured to receive a first plurality of inputs and generate a first plurality of outputs therefrom, and a first plurality of neurons configured to receive the first plurality of outputs. The first plurality of synapses includes a plurality of memory cells, wherein each of the memory cells includes: a spaced-apart source region and a drain region formed in a semiconductor substrate, wherein a channel region extends between the source region and the drain region; a floating gate disposed over a first portion of the channel region and insulated from the first portion; and a non-floating gate disposed over a second portion of the channel region and insulated from the second portion. Each of the plurality of memory cells is configured to store a weight value corresponding to a plurality of electrons on the floating gate. The plurality of memory cells are configured to multiply the first plurality of inputs by the stored weight values ​​to generate a first plurality of outputs.

[0009] Each nonvolatile memory cell used in an analog neuromorphic memory system must be erased and programmed to hold a very specific and precise amount of charge in the floating gate. For example, each floating gate must hold one of N different values, where N is the number of different weights that can be indicated by each cell. Examples of N include 16, 32, and 64.

[0010] One challenge in a VMM system is being able to program a selected cell with the precision and granularity required for different values ​​of N. For example, if a selected cell can include one of 64 different values, extreme precision is required in the programming operation.

[0011] What are needed are improved programming systems and methods suitable for use with a VMM in an emulated neuromorphic memory system. Summary of the invention

[0012] The present invention discloses many embodiments of programming systems and methods for use with vector-matrix multiplication (VMM) arrays in artificial neural networks. Thus, selected cells can be programmed with great precision to hold one of N different values. BRIEF DESCRIPTION OF THE DRAWINGS

[0013] Figure 1 A schematic diagram showing an artificial neural network.

[0014] Figure 2 is a cross-sectional side view of a conventional 2-gate non-volatile memory cell.

[0015] Figure 3 is a cross-sectional side view of a conventional 4-gate non-volatile memory cell.

[0016] Figure 4 FIG. 4 is a side cross-sectional view of a conventional 3-gate non-volatile memory cell.

[0017] Figure 5 FIG. 4 is a cross-sectional side view of another conventional 2-gate non-volatile memory cell.

[0018] Figure 6 A schematic diagram showing different stages of an exemplary artificial neural network utilizing a non-volatile memory array.

[0019] Figure 7 is a block diagram showing a vector multiplier matrix.

[0020] Figure 8 is a block diagram showing the various stages of the vector multiplier matrix.

[0021] Fig. 9 Another implementation of a vector multiplier matrix is ​​shown.

[0022] Fig.10 Another implementation of a vector multiplier matrix is ​​shown.

[0023] Fig.11 Shows the Fig.10 The vector multiplier matrix performs the operation at the operating voltage.

[0024] Fig.12 Another implementation of a vector multiplier matrix is ​​shown.

[0025] Fig.13 Shows the Fig.12 The vector multiplier matrix performs the operation at the operating voltage.

[0026] Fig.14 Another implementation of a vector multiplier matrix is ​​shown.

[0027] Fig.15 Shows the Fig.14 The vector multiplier matrix performs the operation at the operating voltage.

[0028] Fig.16 Another implementation of a vector multiplier matrix is ​​shown.

[0029] Fig.17 Shows the Fig.16 The vector multiplier matrix performs the operation at the operating voltage.

[0030] Fig.18A and 18B A programming method for a vector multiplier matrix is ​​shown.

[0031] Fig.19 Shown for Fig.18A and 18B The programmed waveform.

[0032] Fig. 20 Shown for Fig.18A and 18B The programmed waveform.

[0033] Fig.21 Shown for Fig.18A and 18B The programmed waveform.

[0034] Fig. 22 A vector multiplier matrix system is shown.

[0035] Fig.23 A column driver is shown.

[0036] Fig.24 Several reference matrices are shown.

[0037] Fig.25 A single reference matrix is ​​shown.

[0038] Fig.26 A reference matrix is ​​shown.

[0039] Fig. 27 Another reference matrix is ​​shown.

[0040] Fig.28 A comparison circuit is shown.

[0041] Fig.29 Another comparison circuit is shown.

[0042] Fig.30 Another comparison circuit is shown.

[0043] Fig.31 A current-to-digital bit circuit is shown.

[0044] Fig.32 Shows Fig.31 The waveform of the circuit.

[0045] Fig.33 A current to slope circuit is shown.

[0046] Fig.34 Shows Fig.33 The waveform of the circuit.

[0047] Fig.35 A current to slope circuit is shown.

[0048] Fig.36 Shows Fig.35 The waveform of the circuit. DETAILED DESCRIPTION

[0049] The artificial neural network of the present invention utilizes a combination of CMOS technology and non-volatile memory arrays.

[0050] Non-volatile memory cell

[0051] Digital non-volatile memory is well known. For example, U.S. Patent No. 5,029,130 ​​("the '130 patent") discloses an array of split-gate non-volatile memory cells and is incorporated herein by reference for all purposes. Such memory cells are Figure 2 . Each memory cell 210 includes a source region 14 and a drain region 16 formed in a semiconductor substrate 12, with a channel region 18 therebetween. A floating gate 20 is formed over and insulated from (and controls the conductivity of) a first portion of the channel region 18, and is formed over a portion of the source region 16. A word line terminal 22 (which is typically coupled to a word line) has a first portion disposed over and insulated from (and controls the conductivity of) a second portion of the channel region 18, and a second portion extending upward and located over the floating gate 20. The floating gate 20 and the word line terminal 22 are insulated from the substrate 12 by a gate oxide. A bit line 24 is coupled to the drain region 16.

[0052] The storage cell 210 is erased (where electrons are removed from the floating gate) by placing a high positive voltage on the word line terminal 22, which causes the electrons on the floating gate 20 to tunnel through the intermediate insulator via Fowler-Nordheim tunneling from the floating gate 20 to the word line terminal 22.

[0053] The memory cell 210 is programmed (where electrons are placed on the floating gate) by placing a positive voltage on the word line terminal 22 and a positive voltage on the source 16. Electron current will flow from the source 16 to the drain 14. When the electrons reach the gap between the word line terminal 22 and the floating gate 20, the electrons will accelerate and become hot. Due to the electrostatic attraction from the floating gate 20, some of the heated electrons will be injected through the gate oxide 26 onto the floating gate 20.

[0054] The memory cell 210 is read by placing a positive read voltage on the drain 14 and the word line terminal 22 (which turns on the channel region below the word line terminal). If the floating gate 20 is positively charged (i.e., the electrons are erased and the anode is coupled to the drain 16), the portion of the channel region below the floating gate 20 is also turned on, and current will flow through the channel region 18, which is sensed as the erased state or the "1" state. If the floating gate 20 is negatively charged (i.e., programmed by electrons), the portion of the channel region below the floating gate 20 is mostly or completely turned off, and no current (or very little current) will flow through the channel region 18, which is sensed as the programmed state or the "0" state.

[0055] Table 1 shows typical voltage ranges that may be applied to the terminals of the memory cell 210 for performing read, erase, and program operations:

[0056] Table 1: Figure 2 Operation of the flash memory unit 210

[0057]

[0058] Other split-gate memory cell configurations are known. For example, Figure 3 A quad-gate memory cell 310 is depicted, which includes a source region 14, a drain region 16, a floating gate 20 over a first portion of a channel region 18, a select gate 28 (typically coupled to a word line) over a second portion of the channel region 18, a control gate 22 over the floating gate 20, and an erase gate 30 over the source region 14. This configuration is described in U.S. Pat. No. 6,747,310, which is incorporated herein by reference for all purposes. Here, except for the floating gate 20, all gates are non-floating, meaning that they are electrically connected or capable of being electrically connected to a voltage source. Programming is shown by heated electrons from the channel region 18 injecting themselves onto the floating gate 20. Erasing is shown by electrons tunneling from the floating gate 20 to the erase gate 30.

[0059] Table 2 shows typical voltage ranges that may be applied to the terminals of the memory cell 310 for performing read, erase, and program operations:

[0060] Table 2: Figure 3 Operation of the flash memory unit 310

[0061] WL / SG BL CG EG SL Read 1.0-2V 0.6-2V 0-2.6V 0-2.6V 0V Erase -0.5V / 0V 0V 0V / -8V 8-12V 0V programming 1V 1μA 8-11V 4.5-9V 4.5-5V

[0062] Figure 4 A split gate tri-gate memory cell 410 is depicted. The memory cell 410 is Figure 3 The memory cell 310 is the same as the memory cell 410 except that the memory cell 410 does not have a separate control gate. The erase operation (erasing through the erase gate) and the read operation are the same as Figure 3 The operation is similar to that of , except that there is no control gate bias. The programming operation is also done without a control gate bias, so the programming voltage on the source line is higher to compensate for the lack of control gate bias.

[0063] Table 3 shows typical voltage ranges that may be applied to the terminals of the memory cell 410 for performing read, erase, and program operations:

[0064] Table 3: Figure 4 Operation of the flash memory unit 410

[0065] WL / SG BL EG SL Read 0.7-2.2V 0.6-2V 0-2.6V 0V Erase -0.5V / 0V 0V 11.5V 0V programming 1V 2-3μA 4.5V 7-9V

[0066] Figure 5 A stacked gate memory cell 510 is depicted. The memory cell 510 is similar to Figure 2 Memory cell 210 of the embodiment of the present invention is shown, except that floating gate 20 extends over the entire channel region 18, and control gate 22 extends over floating gate 20, separated by an insulating layer. Erase, program and read operations operate in a similar manner as previously described for memory cell 210.

[0067] Table 4 shows typical voltage ranges that may be applied to the terminals of the memory cell 510 for performing read, erase, and program operations:

[0068] Table 4: Figure 5 Operation of the flash memory unit 510

[0069] CG BL SL P-sub Read 2-5V 0.6–2V 0V 0V Erase -8 to -10V / 0V FLT FLT 8-10V / 15-20V programming 8-12V 3-5V 0V 0V

[0070] In order to utilize a memory array comprising one of the above-described types of non-volatile memory cells in an artificial neural network, two modifications are made. First, the circuitry is configured so that each memory cell can be individually programmed, erased, and read without adversely affecting the memory states of other memory cells in the array, as further explained below. Second, continuous (analog) programming of the memory cells is provided.

[0071] Specifically, the memory state (i.e., the charge on the floating gate) of each memory cell in the array can be changed continuously from a fully erased state to a fully programmed state independently and with minimal disturbance to other memory cells. In another embodiment, the memory state (i.e., the charge on the floating gate) of each memory cell in the array can be changed continuously from a fully programmed state to a fully erased state, and vice versa, independently and with minimal disturbance to other memory cells. This means that the cell storage device is analog, or at least can store one of many discrete values ​​(such as 16 or 64 different values), which allows very precise and individual tuning of all cells in the memory array, and this makes the memory array ideal for storing and fine-tuning the synaptic weights of a neural network.

[0072] Neural Network Using Non-Volatile Memory Cell Array

[0073] Figure 6 A non-limiting example of a neural network utilizing a non-volatile memory array is conceptually illustrated. This example uses the non-volatile memory array neural network for a facial recognition application, but any other suitable application may also be implemented using a non-volatile memory array-based neural network.

[0074] For this example, S0 is the input, which is a 32x32 pixel RGB image with 5-bit precision (i.e., three 32x32 pixel arrays, one for each color R, G, and B, with 5-bit precision per pixel). The synapse CB1 from S0 to C1 has both different sets of weights and shared weights, and scans the input image with a 3x3 pixel overlapping filter (kernel), shifting the filter by 1 pixel (or more than 1 pixel as dictated by the model). Specifically, the values ​​of 9 pixels in the 3x3 portion of the image (i.e., called the filter or kernel) are provided to the synapse CB1, whereby these 9 input values ​​are multiplied by the appropriate weights, and after summing the outputs of this multiplication, a single output value is determined and provided by the first neuron of CB1 for the pixels of one of the layers C1 of the feature map. The 3x3 filter is then shifted one pixel to the right (i.e., a column of three pixels on the right is added, and a column of three pixels on the left is released), whereby the 9 pixel values ​​in this newly positioned filter are provided to the synapse CB1, whereby they are multiplied by the same weights and a second single output value is determined by the associated neuron. This process continues until the 3x3 filter has scanned all three colors and all bits (precision values) over the entire 32x32 pixel image. The process is then repeated using different sets of weights to generate different feature maps for C1 until all feature maps for layer C1 are calculated.

[0075] At C1, in this example, there are 16 feature maps, each with 30×30 pixels. Each pixel is a new feature pixel extracted from the product of the input and the kernel, so each feature map is a two-dimensional array, so in this example, the synapse CB1 is composed of a two-dimensional array of 16 layers (remember that the neuron layers and arrays referred to in this article are logical relationships, not necessarily physical relationships, that is, the arrays do not have to be oriented to physical two-dimensional arrays). Each of the 16 feature maps is generated by one of sixteen different sets of synaptic weights applied to the filter scan. The C1 feature maps can all relate to different aspects of the same image features, such as boundary recognition. For example, a first map (generated using a first set of weights, shared for all scans used to generate the first map) can identify circular edges, a second map (generated using a second set of weights different from the first set of weights) can identify rectangular edges, or the aspect ratio of certain features, and so on.

[0076] Before going from C1 to S1, an activation function P1 (pooling) is applied, which pools the values ​​from consecutive non-overlapping 2x2 regions in each feature map. The purpose of the pooling stage is to average neighboring positions (or a max function can also be used) to, for example, reduce the dependency of edge positions and reduce the data size before entering the next stage. At S1, there are 16 15x15 feature maps (i.e., 16 different arrays of 15x15 pixels each). The synapses and associated neurons from S1 to C2 in CB2 scan the maps in S1 using a 4x4 filter, where the filter is shifted by 1 pixel. At C2, there are 22 12x12 feature maps. Before going from C2 to S2, an activation function P2 (pooling) is applied, which pools the values ​​from consecutive non-overlapping 2x2 regions in each feature map. At S2, there are 22 6x6 feature maps. The activation function is applied to the synapse CB3 from S2 to C3, where each neuron in C3 is connected to each map in S2. At C3, there are 64 neurons. Synapse CB4 from C3 to output S3 fully connects S3 to C3. The output at S3 includes 10 neurons, where the highest output neuron determines the class. For example, this output may indicate recognition or classification of the content of the original image.

[0077] Each level of synapses is implemented using an array or portion of an array of non-volatile memory cells. Figure 7A block diagram of a vector-matrix multiplication (VMM) array that includes nonvolatile memory cells and is used as a synapse between an input layer and the next layer. Specifically, the VMM 32 includes a nonvolatile memory cell array 33, an erase gate and word line gate decoder 34, a control gate decoder 35, a bit line decoder 36, and a source line decoder 37, which decode the input of the memory array 33. In this example, the source line decoder 37 also decodes the output of the memory cell array. Alternatively, the bit line decoder 36 can decode the output of the memory array. The memory array has two purposes. First, it stores weights to be used by the VMM. Second, the memory array effectively multiplies the input with the weights stored in the memory array and each output line (source line or bit line) adds them to produce an output, which will serve as the input of the next layer or the input of the final layer. By performing multiplication and addition functions, the memory array eliminates the need for separate multiplication and addition logic circuits, and is also highly power-efficient due to in-situ memory calculations.

[0078] The output of the memory array is provided to a differential adder (such as an adding operational amplifier) ​​38, which sums the output of the memory cell array to create a single value for the convolution. The differential adder is to achieve the sum of positive weights and negative weights with the positive input. The summed output value is then provided to an activation function circuit 39, which corrects the output. The activation function may include a sigmoid, tanh, or ReLU function. The corrected output value becomes an element of the feature map of the next layer (e.g., C1 in the above description) and is then applied to the next synapse to produce the next feature map layer or the final layer. Therefore, in this example, the memory array constitutes a plurality of synapses (which receive their inputs from an existing neuron layer or from an input layer such as an image database), and the summing operational amplifier 38 and the activation function circuit 39 constitute a plurality of neurons.

[0079] Figure 8 It is a block diagram of each level of VMM. Figure 8 As shown, the input is converted from digital to analog by a digital-to-analog converter 31 and provided to an input VMM 32a. The output generated by the input VMM 32a is provided as the input of the next VMM (hidden level 1) 32b, which in turn generates an output provided as the input of the next VMM (hidden level 2) 32b, and so on. The layers of the VMM 32 serve as different layers of synapses and neurons of a convolutional neural network (CNN). Each VMM can be an independent non-volatile memory array, or multiple VMMs can utilize different portions of the same non-volatile memory array, or multiple VMMs can utilize overlapping portions of the same non-volatile memory array. Figure 8The example shown includes five layers (32a, 32b, 32c, 32d, 32e): one input layer (32a), two hidden layers (32b, 32c) and two fully connected layers (32d, 32e). Those of ordinary skill in the art will appreciate that this is merely exemplary and that, on the contrary, the system may include more than two hidden layers and more than two fully connected layers.

[0080] Vector-Matrix Multiplication (VMM) Array

[0081] Fig. 9 A neuron VMM 900 is shown, which is particularly suitable for Figure 3 Memory cells of the type shown, and used as synapses and components of neurons between the input layer and the next layer. VMM 900 includes a memory array 901 and a reference array 902 (at the top of the array) of non-volatile memory cells. Alternatively, another reference array can be placed at the bottom. In VMM 900, control gate lines (such as control gate lines 903) extend in the vertical direction (so reference array 902 is in the row direction orthogonal to the input control gate lines), and erase gate lines (such as erase gate lines 904) extend in the horizontal direction. Here, input is provided on the control gate lines, and output appears on the source lines. In one embodiment, only even rows are used, and in another embodiment, only odd rows are used. The current placed on the source lines performs the summing function of all currents from the memory cells connected to the source lines.

[0082] As described herein for neural networks, flash memory cells are preferably configured to operate in the subthreshold region.

[0083] The memory cell described in this article is weakly reverse biased:

[0084] Ids=Io*e (Vg-Vth) / kVt =w*Io*e (Vg) / kVt

[0085] w=e (-Vth) / kVt

[0086] For an I to V logarithmic converter that uses a memory cell to convert input current to input voltage:

[0087] Vg=k*Vt*log[Ids / wp*Io]

[0088] For a memory array used as a vector matrix multiplier VMM, the output current is:

[0089] Iout=wa*Io*e (Vg) / kVt ,Right now

[0090] Iout=(wa / wp)*Iin=W*Iin

[0091] W=e (Vthp-Vtha) / kVt

[0092] The word line or control gate may be used as the input of the memory cell for the input voltage.

[0093] Alternatively, the flash memory cell can be configured to operate in the linear region:

[0094] Ids=beta*(Vgs-Vth)*Vds; beta=u*Cox*W / L

[0095] Wα(Vgs-Vth)

[0096] For an I to V linear converter, a memory cell operating in the linear region can be used to linearly convert the input / output current to the input / output voltage.

[0097] Other implementations of the ESF vector-matrix multiplier are described in US Patent Application 15 / 826,345, which is incorporated herein by reference. A source line or a bit line can be used as a neuron output (current summing output).

[0098] Fig.10 A neuron VMM 1000 is shown, which is particularly suitable for Figure 2 1 and 10. The VMM 1000 includes a memory array 1003 of nonvolatile memory cells, a reference array 1001, and a reference array 1002. The reference arrays 1001 and 1002 in the column direction of the array are used to convert the current input flowing into the terminals BLR0-3 into the voltage input WL0-3. In fact, the reference memory cells are diodes connected through a multiplexer, in which the current input flows. The reference cells are tuned (e.g., programmed) to a target reference level. The target reference level is provided by a reference microarray matrix. The memory array 1003 is used for two purposes. First, it stores weights to be used by the VMM 1000. Second, the memory array 1003 effectively multiplies the input (current input provided in the terminals BLR0-3; the reference arrays 1001 and 1002 convert these current inputs into input voltages to be provided to the word lines WL0-3) by the weights stored in the memory array to produce an output, which will be the input to the next layer or the input to the final layer. By performing a multiplication function, the memory array eliminates the need for a separate multiplication logic circuit and is also power efficient. Here, the voltage input is provided on the word line and the output appears on the bit line during a read (inference) operation. The current placed on the bit line performs a summing function of all the currents from the memory cells connected to the bit line.

[0099] Fig.11The operating voltages for VMM 1000 are shown. The columns in the table indicate the voltages placed on the word line for a selected cell, the word line for an unselected cell, the bit line for a selected cell, the bit line for an unselected cell, the source line for a selected cell, and the source line for an unselected cell. The rows indicate read, erase, and program operations.

[0100] Fig.12 A neuron VMM 1200 is shown, which is particularly suitable for Figure 2 1 and 12. The VMM 1200 includes a memory array 1203 of nonvolatile memory cells, a reference array 1201, and a reference array 1202. The reference arrays 1201 and 1202 extending in the row direction of the array VMM 1200 are similar to the VMM 1000, except that in the VMM 1200, the word lines extend in the vertical direction. Here, the input is provided on the word lines, and the output appears on the source lines during a read operation. The current placed on the source line performs a summing function of all the currents from the memory cells connected to the source line.

[0101] Fig.13 The operating voltages for the VMM 1200 are shown. The columns in the table indicate the voltages placed on the word line for a selected cell, the word line for an unselected cell, the bit line for a selected cell, the bit line for an unselected cell, the source line for a selected cell, and the source line for an unselected cell. The rows indicate read, erase, and program operations.

[0102] Fig.14 A neuron VMM 1400 is shown, which is particularly suitable for Figure 3Memory cells of the type shown, and used as synapses and components of neurons between the input layer and the next layer. VMM 1400 includes a memory array 1403 of non-volatile memory cells, a reference array 1401, and a reference array 1402. Reference arrays 1401 and 1402 are used to convert current inputs flowing into terminals BLR0-3 into voltage inputs CG0-3. In fact, the reference memory cells are diodes connected through a common source and common gate multiplexer 1414, into which the current input flows. Multiplexer 1414 includes a multiplexer 1405 and a common source and common gate transistor 1404 to ensure that the voltage on the bit line of the reference cell being read is constant. The reference cell is tuned to a target reference level. Memory array 1403 is used for two purposes. First, it stores weights to be used by VMM 1400. Secondly, the memory array 1403 effectively multiplies the inputs (current inputs provided to terminals BLR0-3; reference arrays 1401 and 1402 convert these current inputs into input voltages to be provided to control gates CG0-3) by the weights stored in the memory array to produce an output, which will be the input to the next layer or the input to the final layer. By performing a multiplication function, the memory array eliminates the need for a separate multiplication logic circuit and is also highly power efficient. Here, the input is provided on the word line and the output appears on the bit line during a read operation. The current placed on the bit line performs a summing function of all the currents from the memory cells connected to the bit line.

[0103] VMM 1400 implements unidirectional tuning for the memory cells in memory array 1403. That is, each cell is erased and then partially programmed until the desired charge on the floating gate is reached. If too much charge is placed on the floating gate (so that the wrong value is stored in the cell), the cell must be erased and the sequence of partial programming operations must start over. As shown, two rows sharing the same erase gate need to be erased together (called a page erase), and thereafter, each cell is partially programmed until the desired charge on the floating gate is reached.

[0104] Fig.15 The operating voltages for the VMM 1400 are shown. The columns in the table indicate the voltages placed on the word line for a selected cell, the word line for an unselected cell, the bit line for a selected cell, the bit line for an unselected cell, the control gate for a selected cell, the control gate for an unselected cell in the same sector as the selected cell, the control gate for an unselected cell in a different sector than the selected cell, the erase gate for a selected cell, the erase gate for an unselected cell, the source line for a selected cell, the source line for an unselected cell. The rows indicate read, erase, and program operations.

[0105] Fig.16 A neuron VMM 1600 is shown, which is particularly suitable for Figure 31 and 16. The VMM 1600 includes a memory array 1603 of nonvolatile memory cells, a reference array 1601, and a reference array 1602. The EG lines extend vertically, while the CG and SL lines extend horizontally. The VMM 1600 is similar to the VMM 1400, except that the VMM 1600 implements bidirectional tuning, where each individual cell can be fully erased, partially programmed, and partially erased as needed to achieve the desired amount of charge on the floating gate. As shown, the reference arrays 1601 and 1602 convert the input current in the terminals BLR0-3 into control gate voltages CG0-3 to be applied to the memory cells in the row direction (through the action of the reference cells connected via the diodes of the multiplexer). The current output (neuron) is in the bit line, which sums all the currents from the memory cells connected to the bit line.

[0106] Fig.17 The operating voltages for the VMM 1600 are shown. The columns in the table indicate the voltages placed on the word line for a selected cell, the word line for an unselected cell, the bit line for a selected cell, the bit line for an unselected cell, the control gate for a selected cell, the control gate for an unselected cell in the same sector as the selected cell, the control gate for an unselected cell in a different sector than the selected cell, the erase gate for a selected cell, the erase gate for an unselected cell, the source line for a selected cell, the source line for an unselected cell. The rows indicate read, erase, and program operations.

[0107] Fig.18A and 18BA programming method 1800 is shown. First, the method starts (step 1801), which typically occurs in response to receiving a programming command. Next, a bulk programming operation programs all cells to a "0" state (step 1802). Then, a soft erase erases all cells to an intermediate weak erase level of approximately 3-5 μA (step 1803). This is in contrast to a deep erase that will bring all cells to a fully erased state for digital applications (e.g., a cell current of approximately 20-30 uA). Then, hard programming is performed on all unselected cells to remove charge from the cells (step 1804), bringing unused cells to a very deep programming state to ensure that the cells are truly turned off, meaning that these memory cells contribute insignificant current. Soft programming is then performed on selected cells using a coarse algorithm to remove some charge from the cells to n intermediate weak programming levels of approximately 0.7-1.5 μA (steps 1805, 1806, 1807). A coarse step programming cycle is performed, followed by a verification operation, in which the charge on the selected cell is compared to various thresholds in a coarse iterative manner (steps 1806 and 1807). The coarse step programming cycle includes a coarse voltage increment, and / or a coarse programming time, and / or a coarse programming current that results in a coarse cell current change from one programming step to the next programming step. Next, fine programming is performed (steps 1808-1813), in which all cells are programmed to a target level in the range of 100pA–20nA by a fine step programming algorithm, depending on the desired level. A fine step programming cycle is performed, followed by a verification operation (steps 1809 and 1810). The fine step programming cycle may include a combination of coarse and fine resolution of voltage increments, and / or programming times, and / or programming currents that result in a fine cell current change from one programming step to the next programming step. If the selected cell reaches the desired target, the programming operation is completed (step 1811). If not, the fine programming operation is repeated until the desired level is reached. However, if the number of attempts exceeds a threshold number (step 1812), the programming operation stops and the selected cell is considered a bad cell (step 1813).

[0108] Fig.19An exemplary waveform 1900 for performing a programming operation using pulse modulation is shown. Signal 1901 is a programming cycle enable signal, which consists of multiple programming and verification cycles. Signal 1902 is a separate pulse programming cycle enable signal (signal 1902 = logic "1" to enable programming). The verification cycle follows a separate pulse programming cycle (signal 1902 = logic "0" to enable verification). Signal 1903 is a separate pulse programming cycle enable signal for a specific bit line. Signal 1903 is a separate pulse programming cycle enable signal for another specific bit line. As shown, the width of signal 1903 is narrower than the width of signal 1904. This means that signal 1903 allows a smaller charge on the floating gate of the memory cell to be changed, resulting in a smaller current precision in programming. Within the programming cycle 1901, different programming pulse widths 1903 can be used to achieve the required programming precision of a specific memory cell.

[0109] Fig. 20 An exemplary waveform 2000 for performing a programming operation using high voltage level modulation is shown. Signal 2003 is a separate pulse programming cycle enable signal for a specific bit line. Signal 2004 is a separate pulse programming cycle enable signal for another specific bit line. The programming pulse widths of signals 2003 and 2004 are the same in this waveform. Signal 2005 is a high voltage increment such as a source line or control gate for programming. It increments from one programming pulse to the next programming pulse.

[0110] Fig.21 An exemplary waveform 2100 for performing a programming operation using high voltage level modulation is shown. Signal 2103 is a separate pulse programming cycle enable signal for a specific bit line. Signal 2104 is a separate pulse programming cycle enable signal for another specific bit line. Signal 2105 is a programming high voltage increment, such as for a source line or control gate for programming. It can be the same or increasing from one programming pulse to the next. The programming pulse width of signal 2103 is the same over multiple pulses with increasing high voltage. For the same high voltage increment, the programming pulse width of signal 2104 is different across multiple pulses, for example, narrower for the first pulse.

[0111] Fig. 22 A VMM system 2200 including a VMM matrix 2201 , a column decoder 2202 , and a column driver 2203 is shown.

[0112] Fig.23 It shows that Fig. 222300 for use as column driver 2203. Column driver 2300 includes latch 2301, inverter 2302, NOR gate 2303, PMOS transistor 2304, and NMOS transistors 2305 and 2306, configured as shown. Latch 2301 receives data signal DIN and enable signal EN. Node BLIO between PMOS transistor 2304 and NMOS transistor 2305 contains bit line input or output signal, which is transmitted through column decoder (such as Fig. 22 2202 in the column decoder 2204 is selectively connected to the bit line. Sense amplifier SA 2310 is coupled to the bit line through node BLIO to read the cell current of the selected memory cell. Sense amplifier SA 2310 is used to verify the desired current level of the selected memory cell, such as after an erase or programming operation. PMOS transistor 2304 is used to inhibit the bit line from being programmed, depending on the inhibition control circuit 2301 / 2302 / 2303. NMOS transistor 2306 provides a bias programming current to the bit line. NMOS transistor 2305 enables the bias programming current to the bit line, thereby enabling programming of the selected memory cell. Therefore, the programming control signal (such as Fig.19 Signal 1902 / 1903, Fig. 20 Signals in 2003 / 2004, Fig.21 Signals 2103 / 2104 in will enable programming of the selected cells.

[0113] Fig.24 An exemplary VMM system 2400 is shown, which includes reference array matrices 2401a, 2401b, 2401c, and 2401d and VMM matrices 2402a, 2402b, 2402c, and 2402d. Each VMM matrix has its own reference array matrix.

[0114] Fig.25 An exemplary VMM system 2500 is shown, which includes a single reference array matrix 2501 and VMM matrices 2502a, 2502b, 2502c, and 2502d. The single reference array matrix 2501 is shared across multiple VMM matrices.

[0115] Fig.26 An exemplary reference matrix 2600 is shown, which may be used as Fig.24 Reference matrices 2401a-d in Fig.25 Matrix 2501 in, or Fig.28 and 292801 or 2901 in the reference matrix 2600. The reference matrix 2600 includes reference memory cells 2602a, 2602b, 2602c, and 2602x coupled to a common control gate signal 2603 and a source line reference signal 2604, and a bit line reference decoder 2601 that provides a plurality of bit line reference signals for a read or verify operation. For example, the reference memory cells 2602a-x may provide an incremental current level of 100pA / 200pA / 300pA / ... / 30nA. Alternatively, the reference memory cells 2602a-x may provide a constant current level of 100pA for each reference memory cell. For this case, a combination of the reference memory cells 2602a-x is used to generate different reference current levels, such as generating 100pA / 200pA / 300pA / etc. in a thermometer code manner. Other combinations of reference current levels, constants, and / or increments may be used to generate desired reference current levels. Furthermore, the differential current between the two reference cells can be used to generate a reference memory cell current, such as 100 pA reference current = 250 pA reference current - 150 pA reference current. This is used, for example, to generate a reference current that is compensated with temperature.

[0116] Fig. 27 An exemplary reference matrix 2700 is shown, which may be used as Fig.24 Reference matrices 2401a-d in Fig.25 Matrix 2501 in, or Fig.28 and 29 2801 or 2901 in the reference matrix 2700. The reference matrix 2700 includes reference memory cells 2702a, 2702b, 2702c, and 2702x, which are coupled to a common control gate reference signal 2703, an erase gate reference signal 2704, and a source line reference signal 2705, and a bit line reference decoder 2701, which provides a plurality of bit line reference signals for read or verify operations. The combination of reference current levels, constants, and / or increments can be as shown in FIG. Fig.26 is used as in to generate the required reference current level.

[0117] Fig.28An Icell PMOS comparison circuit 2800 is shown, which includes PMOS transistors 2801 and 2804, NMOS cascode transistors 2802 and 2805, a memory cell 2803 selected from a VMM memory array 2820, and a reference matrix 2806 (such as reference matrix 2500 or 2600) arranged as shown. NMOS transistors 2802 and 2805 are used to bias the reference bit line to a desired voltage level. The output current Iout is a current value indicating the value stored in the selected memory cell 2803. The voltage level of the output node 2810 indicates the comparison result between the current in the selected memory cell 2803 and the reference current from the reference matrix 28006. If the current in the selected memory cell 2803 is greater than (or less than) the reference current from the reference matrix 2806, the voltage on the node 2801 rises to Vdd (or drops to ground).

[0118] Fig.29 An Icell PMOS comparison circuit 2900 is shown, which includes a PMOS transistor 2901, switches 2902, 2903 and 2904, NMOS cascode transistors 2905 and 2907, a selected memory cell 2908 from a VMM memory array 2920, a reference matrix 2906 (such as reference matrix 2600 or 2700), and a comparator 2909, arranged as shown. The output COMP_OUT is a voltage value indicating the relationship between the value stored in the selected memory cell 2908 and the reference current. The Icell comparison circuit operates by acting as a single PMOS current mirror with time multiplexing to eliminate the mismatch between the two mirror PMOS transistors. For the first time period, S0 and S1 are closed and S2 is open. The current from the reference memory matrix 2906 is stored (retained) in the PMOS transistor 2901. For the next time period, S0 and S1 are open and S2 is closed. The stored reference current is compared with the current from the memory cell 2908, and the comparison result is indicated on the output node 2910. Optionally, the comparator can compare the voltage on the node 2901 with the reference voltage VREF to indicate the result of the comparison. Here, the reference current is sampled and held, and the current from the selected memory cell 2902 is sampled and held in the PMOS transistor 2901 to be compared with the reference current.

[0119] In another embodiment, Fig.30The array leakage compensation circuit 3051 shown can be used with a single PMOS mirror circuit to sample (S3 closed) the leakage on the VMM array 3020 (all word lines are open, the leakage current on the bit line is sampled into the holding PMOS) and hold the leakage in the (S3 open) holding transistor. This leakage is then subtracted from the current in the selected memory cell during the comparison cycle to obtain the actual memory cell current from the VMM array 3020 for comparison. This can be used for all comparison circuits described herein. This can be used for reference array leakage compensation.

[0120] Fig.31 An Icell to digital data circuit 3100 is shown, which includes a current source 3101, a switch 3102, a capacitor 3103, a comparator 3104, and a counter 3105. At the beginning of the comparison cycle, a signal 3110 is pulled to ground. The signal 3110 then begins to rise in accordance with the cell current 3101 (extracted from a VMM memory array with array leakage compensation as described above). The ramp rate is proportional to the cell current 3101 and the capacitor 3103. The output 3112 of the comparator 3104 then enables the counter 3105 to begin counting digitally. Once the voltage on node 3110 reaches the voltage level VREF 3111, the comparator 3104 switches polarity and stops the counter 3105. The digital output Q <n:0>The value 3113 indicates the value of the cell current 3101.

[0121] Fig.32 Waveform 3200 is shown for the operation of Icell to digital data circuit 3100. Signal 3201 is a ramp voltage (corresponding to Fig.31 3110 in FIG. 3111). Different slope rates of signal 3201 are shown for different cell current levels. Signals 3205 and 3207 are the outputs of the comparators for two different cell currents (corresponding to Fig.31 Signals 3206 and 3208 are the digital outputs Q of the above two different unit currents. <n:0>.

[0122] Fig.33 An Icell to slope circuit 3300 is shown, which includes a memory cell current source 3301, a switch 3302, a capacitor 3303, and a comparator 3304. As described above, a memory cell current is extracted from a VMM memory array with array leakage compensation. At the beginning of a comparison cycle, a signal 3310 is pulled to ground. Then, the signal 3310 begins to rise according to the cell current 3301 (extracted from the VMM memory array). The slope rate is proportional to the cell current 3301 and the capacitor 3303. After a fixed comparison cycle, the comparator 3304 compares the voltage on the node 3310 with a reference voltage VREFx 3311. For 16 reference levels, VREFx 3311 is, for example, 0.1V, 0.2V, 0.3V, ..., 1.5V, 1.6V. Therefore, each level corresponds to a current level of 16 different current levels. The output of the comparator 3304 indicates the value of the cell current 3301. In order to compare the voltage on node 3310 (which can be maintained on capacitor 3303 by closing S1 after a fixed comparison period) with 16 reference levels, either 16 comparators with 16 reference levels are used, or one comparator with a reference level multiplexed 16 times for the 16 reference levels is used.

[0123] Fig.34 Waveform 3400 is shown for the operation of Icell to slope circuit 3300. Signal 3401 shows different ramp rates with different voltage levels (Vcellx) at the rising edge of enable signal 3402. Voltage Vcellx is compared to a reference voltage to indicate the value of the cell current ( Fig.33 The reference voltage VREFx in 3311).

[0124] Fig.35 Icell to slope conversion circuit 3500 is shown, which includes memory cell current source 3504, switch 3502, capacitor 3501, NMOS cascode transistor 3503 and comparator 3505. As described above, memory cell current 3501 is extracted from a VMM memory array with array leakage compensation. NMOS 3503 is used to bias the voltage on the bit line of the selected memory cell (shown as Icell3504). The operation is similar to Fig.31 The difference is that the slope direction is down instead of up.

[0125] Fig.36 Waveform 3600 for the operation of Icell to slope circuit 3500 is shown.

[0126] It should be noted that, as used herein, the terms "above" and "on" both inclusively include "directly on" (no intermediate material, element, or space is provided in between) and "indirectly on" (intermediate material, element, or space is provided in between). Similarly, the term "adjacent" includes "directly adjacent" (no intermediate material, element, or space is provided in between) and "indirectly adjacent" (intermediate material, element, or space is provided in between), "mounted to" includes "directly mounted to" (no intermediate material, element, or space is provided in between) and "indirectly mounted to" (intermediate material, element, or space is provided in between), and "electrically coupled to" includes "directly electrically coupled to" (no intermediate material or element electrically connecting the elements together) and "indirectly electrically coupled to" (intermediate material or element electrically connecting the elements together). For example, forming an element "above" a substrate may include directly forming the element on the substrate without intermediate materials / elements in between, and indirectly forming the element on the substrate with one or more intermediate materials / elements in between.

Claims

1. A circuit for comparing a current consumed by a selected memory cell of a vector matrix multiplier with a current consumed by a reference matrix, the circuit comprising: a first circuit comprising a first PMOS transistor coupled to a first NMOS transistor, the first NMOS transistor coupled to the selected memory cell; and a second circuit comprising a second PMOS transistor coupled to a second NMOS transistor, the second NMOS transistor coupled to the reference matrix; Wherein a node between the second PMOS transistor and the second NMOS transistor outputs a current indicative of a value stored in the selected memory cell.

2. The circuit of claim 1 further comprising an array leakage compensation circuit.

3. The circuit of claim 1 further comprising a reference array leakage compensation circuit.

4. The circuit of claim 1, wherein the selected memory cell is a split gate memory cell.

5. The circuit of claim 1, wherein the selected memory cell is a stacked gate memory cell.

6. A circuit for comparing a current consumed by a selected cell of a vector matrix multiplier with a current consumed by a reference matrix, the circuit comprising: a transistor for comparing the current received from a first node with a reference current, wherein the first node is selectively coupled to a reference matrix through a first switch or to a selected memory cell through a second switch; The voltage on the first node indicates a comparison result.

7. The circuit of claim 6, wherein the reference current is maintained across the transistor.

8. The circuit according to claim 6, further comprising: A comparator for comparing the current received from the first node with a reference current Compare; Wherein the output of the comparator indicates whether the value stored in the selected cell exceeds the value stored in the reference matrix.

9. The circuit of claim 6, further comprising an array leakage compensation circuit.

10. The circuit of claim 6, further comprising a reference array leakage compensation circuit.

11. The circuit of claim 6, wherein the selected cell is a split gate memory cell.

12. The circuit of claim 6, wherein the selected cell is a stacked gate memory cell.

13. A circuit for converting a memory cell current signal of a vector matrix multiplier into a set of digital bits, the circuit comprising: a memory cell current source; a switch coupled to the current source; a capacitor coupled to the current source in parallel with the switch; a comparator having one input coupled to the current source, the switch, and the capacitor and another input coupled to a voltage reference; and A counter is coupled to receive the output from the comparator and output a digital count signal. The circuit of claim 13 , wherein the memory current is held across a transistor.

15. The circuit of claim 13, further comprising an array leakage compensation circuit.

16. The circuit of claim 13, wherein the memory cell is a split gate memory cell.

17. The circuit of claim 13, wherein the memory cell is a stacked gate memory cell.

18. A circuit for converting a memory cell current signal of a vector matrix multiplier into a slope, the circuit comprising: a memory cell current source; a switch coupled to the current source; a capacitor coupled to the current source in parallel with the switch; A comparator having one input coupled to the current source, the switch, and the capacitor and another input coupled to a voltage reference, wherein an output comparison of the comparator is indicative of the slope of the current output by the current source.

19. The circuit of claim 18, wherein the memory current is held across a transistor.

20. The circuit of claim 18, further comprising an array leakage compensation circuit.

21. The circuit of claim 18, wherein the memory cell is a split gate memory cell.

22. The circuit of claim 18, wherein the memory cell is a stacked gate memory cell.

23. The circuit of claim 18, wherein the voltage reference is time multiplexed to indicate a value of the memory cell.

24. A circuit for converting a memory cell current signal of a vector matrix multiplier into a slope, the circuit comprising: a memory cell current source; a transistor coupled to the current source; a switch coupled to the transistor; a capacitor coupled to the transistor in parallel with the switch; A comparator having one input coupled to the transistor, the switch, and the capacitor and another input coupled to a voltage reference, wherein an output comparison of the comparator is indicative of a slope of the current output by the current source.

25. The circuit of claim 24, wherein the memory current is held across a transistor.

26. The circuit of claim 24, further comprising an array leakage compensation circuit.

27. The circuit of claim 24, wherein the memory cell is a split gate memory cell.

28. The circuit of claim 24, wherein the memory cell is a stacked gate memory cell.

Images