Input circuitry for analog neural memory in deep learning artificial neural network

Abstract

To provide an input block for an analog neural memory in a deep learning artificial neural network.SOLUTION: The invention provides an input block for providing an input to a vector-by-matrix multiplication array (VMM array 3506) in a neural memory system. The vector-by-matrix multiplication array comprises non-volatile memory cells arranged in rows and columns. The input block comprises: a global digital-to-analog converter 3501; a plurality of row registers 3502, each row register corresponding to a row in the VMM array; and a plurality of row sample-and-hold buffers 3504, each row sample-and-hold buffer corresponding to a row in the VMM array.SELECTED DRAWING: Figure 35A

Description

[Technical Field]

[0001] (Priority Claim) This application claims priority to U.S. Provisional Patent Application No. 63 / 228,570, entitled "Input Circuitry for Analog Neural Memory in a Deep Learning Artificial Neural Network," filed August 2, 2021, and U.S. Patent Application No. 17 / 520,396, entitled "Input Circuitry for Analog Neural Memory in a Deep Learning Artificial Neural Network," filed November 5, 2021.

[0002] FIELD OF THE INVENTION Numerous embodiments of input circuits for analog neural memories in deep learning artificial neural networks are disclosed. [Background technology]

[0003] Artificial neural networks mimic biological neural networks (the central nervous systems of animals, particularly the brain) and are used to estimate or approximate functions that may depend on multiple inputs and are generally unknown. Artificial neural networks generally contain layers of interconnected "neurons" that exchange messages between each other.

[0004] FIG. 1 illustrates an artificial neural network, where circles represent layers of inputs or neurons. Connections (called synapses) are represented by arrows and have numerical weights that can be tuned based on experience. This allows the neural network to adapt to the inputs and learn. Typically, a neural network contains multiple layers of inputs. There are typically one or more hidden layers of neurons, and an output layer of neurons that provide the neural network's output. At each level, neurons make decisions, individually or collectively, based on the data they receive from the synapses.

[0005] One of the major challenges in developing artificial neural networks for high-performance information processing is the lack of suitable hardware technology. In practice, practical neural networks rely on a very large number of synapses, which allows for high connectivity between neurons and therefore a very high degree of parallelization of computation. In principle, such complexity could be achieved using digital supercomputers or dedicated graphic processing unit clusters. However, in addition to high cost, these approaches also suffer from poor energy efficiency compared to biological networks, which primarily perform low-precision analog computations and therefore consume much less energy. While CMOS analog circuits have been used in artificial neural networks, most CMOS-implemented synapses are too bulky given the large number of neurons and synapses.

[0006] The applicant previously disclosed an artificial (analog) neural network utilizing one or more non-volatile memory arrays as synapses in U.S. Patent Application No. 15 / 594,439, which is incorporated by reference. The non-volatile memory array operates as an analog neural memory. The neural network device includes a first plurality of synapses configured to receive a first plurality of inputs and generate a first plurality of outputs therefrom, and a first plurality of neurons configured to receive the first plurality of outputs. The first plurality of synapses includes a plurality of memory cells, each including spaced apart source and drain regions formed in a semiconductor substrate with a channel region extending therebetween, a floating gate disposed insulated above a first portion of the channel region, and a non-floating gate disposed insulated above a second portion of the channel region. Each of the plurality of memory cells is configured to store a weight value corresponding to a number of electrons in the floating gate. The plurality of memory cells is configured to multiply the first plurality of inputs by the stored weight value to generate the first plurality of outputs. Nonvolatile Memory Cell

[0007] Nonvolatile memory is well known. For example, U.S. Pat. No. 5,029,130 ​​(the "'130 patent"), incorporated herein by reference, discloses an array of split-gate nonvolatile memory cells, which are a type of flash memory cell. Such a memory cell 210 is shown in FIG. 2. Each memory cell 210 includes a source region 14 and a drain region 16 formed in a semiconductor substrate 12, with a channel region 18 between the source region 14 and the drain region 16. A floating gate 20 is formed over and insulated from a first portion of the channel region 18 (and controls the conductivity of the first portion of the channel region 18) and over a portion of the source region 14. A word line terminal 22 (typically coupled to a word line) has a first portion disposed over and insulated from a second portion of the channel region 18 (and controls the conductivity of the second portion of the channel region 18), and a second portion extending upward above the floating gate 20. A floating gate 20 and a wordline terminal 22 are insulated from the substrate 12 by a gate oxide. A bitline 24 is coupled to the drain region 16.

[0008] The memory cell 210 is erased (electrons are removed from the floating gate) by applying a high positive voltage to the word line terminal 22, which causes electrons in the floating gate 20 to pass from the floating gate 20 to the word line terminal 22 through the insulator between them via Fowler-Nordheim (FN) tunneling.

[0009] The memory cell 210 is programmed by hot electron source side injection (SSI) (electrons are added to the floating gate) by applying a positive voltage to the word line terminal 22 and a positive voltage to the source region 14. Electrons flow from the drain region 16 toward the source region 14. The electrons accelerate and heat up when they reach the gap between the word line terminal 22 and the floating gate 20. Some of the heated electrons are injected into the floating gate 20 through the gate oxide due to electrostatic attraction from the floating gate 20.

[0010] The memory cell 210 is read by applying a positive read voltage to the drain region 16 and word line terminal 22 (turning on the portion of the channel region 18 below the word line terminal). When the floating gate 20 is positively charged (i.e., erased with electrons), the portion of the channel region 18 below the floating gate 20 is also turned on, and current flows through the channel region 18, which is sensed as an erased or "1" state. When the floating gate 20 is negatively charged (i.e., programmed with electrons), the portion of the channel region below the floating gate 20 is mostly or completely off, and no (or very little) current flows through the channel region 18, which is sensed as a programmed or "0" state.

[0011] Table 1 shows typical voltage / current ranges that may be applied to the terminals of memory cell 110 to perform read, erase, and program operations. Table 1: Operation of flash memory cell 210 of FIG. 3 [Table 1]

[0012] Other split-gate memory cell configurations, including other types of flash memory cells, are also known. For example, FIG. 3 shows a four-gate memory cell 310 including a source region 14, a drain region 16, a floating gate 20 above a first portion of a channel region 18, a select gate 22 (typically coupled to a word line, WL) above a second portion of the channel region 18, a control gate 28 above the floating gate 20, and an erase gate 30 above the source region 14. This configuration is described in U.S. Pat. No. 6,747,310, which is incorporated herein by reference for all purposes. Here, all gates, except for the floating gate 20, are non-floating gates, meaning they are electrically connected or connectable to a voltage source. Programming is performed by heated electrons injecting themselves from the channel region 18 into the floating gate 20. Erasing is performed by electrons tunneling from the floating gate 20 to the erase gate 30.

[0013] Table 2 shows typical voltage / current ranges that may be applied to the terminals of memory cell 310 to perform read, erase, and program operations. Table 2: Operation of the flash memory cell 310 of FIG. 3 [Table 2]

[0014] Figure 4 shows another type of flash memory cell, a three-gate memory cell 410. Memory cell 410 is identical to memory cell 310 of Figure 3, except that memory cell 410 does not have a separate control gate. Erase and read operations (erasure occurs through the use of an erase gate) are similar to those of Figure 3, except that no control gate bias is applied. Programming operations are also performed without a control gate bias, and as a result, a higher voltage must be applied to the source line during a program operation to compensate for the lack of control gate bias.

[0015] Table 3 shows typical voltage / current ranges that may be applied to the terminals of memory cell 410 to perform read, erase, and program operations. Table 3: Operation of flash memory cell 410 of FIG. 4 [Table 3]

[0016] 5 shows another type of flash memory cell, a stacked gate memory cell 510. Memory cell 510 is similar to memory cell 210 of FIG. 2, except that the floating gate 20 extends over the entire channel region 18, and a control gate 22 (where it is coupled to a word line) extends over the floating gate 20, separated by an insulating layer (not shown). Erasing is accomplished by FN tunneling of electrons from the FG to the substrate, and programming is accomplished by channel hot electron (CHE) injection in the region between the channel 18 and the drain region 16, by electrons flowing from the source region 14 toward the drain region 16, and by a read operation similar to that of memory cell 210, which has a higher control gate voltage.

[0017] Table 4 shows typical voltage ranges that may be applied to the terminals of memory cell 510 and substrate 12 to perform read, erase, and program operations. Table 4: Operation of flash memory cell 510 of FIG. 5 [Table 4]

[0018] The methods and means described herein may be applied to other non-volatile memory technologies such as, but not limited to, FINFET split-gate flash or stacked-gate flash memory, NAND flash, SONOS (silicon-oxide-nitride-oxide-silicon, charge traps in nitride), MONOS (metal-oxide-nitride-oxide-silicon, metal charge traps in nitride), ReRAM (resistive ram), PCM (phase change memory), MRAM (magnetic ram), FeRAM (ferroelectric ram), CT (charge trap) memory, CN (carbon-tube) memory, OTP (one time programmable), and CeRAM (correlated electron ram).

[0019] To utilize a memory array containing one of the non-volatile memory cell types in an artificial neural network, two modifications are made: First, as explained further below, the lines are configured so that each memory cell can be individually programmed, erased, and read without adversely affecting the memory state of other memory cells in the array; Second, continuous (analog) programming of the memory cells is provided.

[0020] Specifically, the memory state (i.e., the charge on the floating gate) of each memory cell in the array can be changed continuously from a fully erased state to a fully programmed state, independently and with minimal disturbance to other memory cells. In another embodiment, the memory state (i.e., the charge on the floating gate) of each memory cell in the array can be changed continuously from a fully programmed state to a fully erased state, and vice versa, independently and with minimal disturbance to other memory cells. This means that the cell storage is analog, or at a minimum, capable of storing one of a number of discrete values ​​(such as 16 or 64 different values), making every cell in the memory array very precisely and individually tunable and making the memory array ideal for storage and for fine tuning adjustments to the synaptic weights of neural networks. Neural networks using nonvolatile memory cell arrays

[0021] 6 conceptually illustrates a non-limiting example of a neural network utilizing the non-volatile memory array of the present embodiments. This example uses a non-volatile memory array neural network for a face recognition application, although other suitable applications can also be implemented using a non-volatile memory array-based neural network.

[0022] S0 is the input layer, which in this example is a 32x32 pixel RGB image with 5-bit precision (i.e., three 32x32 pixel arrays, one for each color R, G, and B, with each pixel having 5-bit precision). Synapse CB1 going from input layer S0 to layer C1 scans the input image with overlapping 3x3 pixel filters (kernels), applying different sets of weights to some instances and shared weights to other instances, and shifts the filters by one pixel (or two or more pixels, depending on the model). Specifically, the values ​​of nine pixels in the 3x3 portion of the image (i.e., referred to as filters or kernels) are provided to synapse CB1, which multiplies these nine input values ​​by the appropriate weights and, after summing the outputs of the multiplications, determines a single output value, which is applied by the first synapse of CB1 to generate one pixel of layer C1's feature map. The 3x3 filter is then shifted one pixel to the right in input layer S0 (i.e., adding a column of three pixels to the right and dropping a column of three pixels on the left), so that the nine pixel values ​​of this newly positioned filter are provided to synapse CB1, where they are multiplied by the same weights as above to determine a second single output value by the associated synapse. This process continues until the 3x3 filter has scanned the entire 32x32 pixel image of input layer S0 for all three colors and all bits (precision values). The process is then repeated using different sets of weights to generate different feature maps for layer C1 until all of layer C1's feature maps have been calculated.

[0023] In this example, there are 16 feature maps in layer C1, each having 30x30 pixels. Each pixel is a new feature pixel extracted from the multiplication of the input and the kernel, and therefore each feature map is a two-dimensional array. Thus, in this example, layer C1 comprises 16 layers of two-dimensional arrays. (Note that the layers and arrays referred to herein are logical, not necessarily physical, relationships; i.e., the arrays are not necessarily oriented in a physical two-dimensional array.) Each of the 16 feature maps in layer C1 is generated by one of 16 different sets of synaptic weights applied to the filter scans. The C1 feature maps can all target different aspects of the same image feature, such as boundary identification. For example, a first map (generated using a first set of weights shared by all scans used to generate this first map) can identify circular edges, while a second map (generated using a second set of weights different from the first set of weights) can identify rectangular edges or the aspect ratio of a particular feature, etc.

[0024] Before going from layer C1 to layer S1, an activation function P1 (pooling) is applied, which pools values ​​from non-overlapping, contiguous 2x2 regions in each feature map. The purpose of pooling function P1 is to average nearby locations (or a max function can be used), e.g., to reduce dependency on edge locations, and to reduce data size before going to the next stage. In layer S1, there are 16 15x15 feature maps (i.e., 16 different arrays of 15x15 pixels each). Synapse CB2 going from layer S1 to layer C2 scans the maps in layer S1 with a 4x4 filter with a filter shift of 1 pixel. In layer C2, there are 22 12x12 feature maps. Before going from layer C2 to layer S2, an activation function P2 (pooling) is applied, which pools values ​​from non-overlapping, contiguous 2x2 regions in each feature map. In layer S2, there are 22 6x6 feature maps. At synapse CB3 going from layer S2 to layer C3, an activation function (pooling) is applied, where every neuron in layer C3 connects to every map in layer S2 through a respective synapse in CB3. There are 64 neurons in layer C3. Synapse CB4 going from layer C3 to output layer S3 fully connects C3 to S3, i.e., every neuron in layer C3 connects to every neuron in layer S3. The output at S3 includes 10 neurons, where the neuron with the highest output determines the class. This output can indicate, for example, the identification or classification of the content of the original image.

[0025] Each layer of the synapse is implemented using an array or portion of an array of non-volatile memory cells.

[0026] Figure 7 is a block diagram of an array that can be used for this purpose. A vector-by-matrix multiplication (VMM) array 32 contains nonvolatile memory cells and is utilized as a synapse between one layer and the next (such as CB1, CB2, CB3, and CB4 in Figure 6). Specifically, the VMM array 32 includes an array of nonvolatile memory cells 33, an erase gate and word line gate decoder 34, a control gate decoder 35, a bit line decoder 36, and a source line decoder 37, which decode the respective inputs to the nonvolatile memory cell array 33. Inputs to the VMM array 32 can come from the erase gate and word line gate decoder 34 or from the control gate decoder 35. The source line decoder 37 in this example also decodes the output of the nonvolatile memory cell array 33. Alternatively, the bit line decoder 36 can decode the output of the nonvolatile memory cell array 33.

[0027] The non-volatile memory cell array 33 serves two purposes. First, it stores the weights used by the VMM array 32. Second, the non-volatile memory cell array 33 effectively multiplies the inputs by the weights stored in the non-volatile memory cell array 33 and adds them for each output line (source line or bit line) to produce an output that becomes the input to the next layer or the input to the last layer. Having the non-volatile memory cell array 33 perform the multiplication and addition functions eliminates the need for separate multiplication and addition logic and is also more power efficient due to in-memory computation.

[0028] The outputs of the non-volatile memory cell array 33 are fed to a differential summer (such as a summing op-amp or summing current mirror) 38, which sums the outputs of the non-volatile memory cell array 33 to create a single value for the convolution. The differential summer 38 is arranged to perform a summation of the positive and negative weights.

[0029] The summed output values ​​of the differential adder 38 are then provided to an activation function block 39, which rectifies the output. The activation function block 39 may provide a sigmoid, tanh, or ReLU function. The rectified output values ​​of the activation function block 39 become elements of a feature map as the next layer (e.g., C1 in FIG. 6) and are then applied to the next synapse to generate the next feature map layer or the final layer. Thus, in this example, the non-volatile memory cell array 33 constitutes multiple synapses (receiving input from a previous layer of neurons or from an input layer such as an image database), and the summing op-amps 38 and activation function block 39 constitute multiple neurons.

[0030] The inputs to the VMM array 32 of FIG. 7 (WLx, EGx, CGx, and optionally BLx and SLx) may be analog levels, binary levels, or digital bits (in which case a DAC is provided to convert the digital bits to the appropriate input analog levels), and the outputs may be analog levels, binary levels, or digital bits (in which case an output ADC is provided to convert the output analog levels to digital bits).

[0031] FIG. 8 is a block diagram illustrating the use of multiple layers of VMM array 32, labeled in the figure as VMM arrays 32a, 32b, 32c, 32d, and 32e. As shown in FIG. 8, input (denoted Inputx) is converted from digital to analog by digital-to-analog converter 31 and provided to input VMM array 32a. The converted analog input can be a voltage or current. The first layer's input D / A conversion can be performed by using a function or LUT (look up table) that maps input Inputx to the appropriate analog level of the matrix multiplier of input VMM array 32a. The input conversion can also be performed by an analog-to-analog (A / A) converter to convert an external analog input to the mapped analog input to input VMM array 32a.

[0032] The output generated by input VMM array 32a is then provided as input to the next VMM array (hidden level 1) 32b, which then generates an output that is provided as input to input VMM array (hidden level 2) 32c, and so on. The various layers of the VMM array 32 function as layers of synapses and neurons of a convolutional neural network (CNN). Each VMM array 32a, 32b, 32c, 32d, and 32e can be a standalone physical non-volatile memory array, or multiple VMM arrays can utilize different portions of the same physical non-volatile memory array, or multiple VMM arrays can utilize overlapping portions of the same physical non-volatile memory array. The example shown in FIG. 8 includes five layers (32a, 32b, 32c, 32d, and 32e): one input layer (32a), two hidden layers (32b and 32c), and two fully connected layers (32d and 32e). Those skilled in the art will appreciate that this is merely an example, and that the system may alternatively include more than two hidden layers and more than two fully connected layers. Vector Matrix Multiplication (VMM) Array

[0033] 9 shows a neuron VMM array 900 that is particularly suited for the memory cells 310 shown in FIG. 3 and that is utilized as part of the synapses and neurons between the input layer and the next layer. The VMM array 900 includes a memory array 901 of non-volatile memory cells and a reference array 902 of non-volatile reference memory cells (located at the top of the array). Alternatively, a separate reference array can be located at the bottom.

[0034] In VMM array 900, control gate lines, such as control gate line 903, run vertically (thus, row-oriented reference array 902 is orthogonal to control gate line 903), and erase gate lines, such as erase gate line 904, run horizontally. Here, inputs to VMM array 900 are provided to control gate lines (CG0, CG1, CG2, CG3), and outputs of VMM array 900 appear on source lines (SL0, SL1). In one embodiment, only even rows are used, and in another embodiment, only odd rows are used. The current in each source line (SL0, SL1, respectively) performs a function of the sum of all currents from memory cells connected to that particular source line.

[0035] As described herein for neural networks, the non-volatile memory cells of VMM array 900, ie, memory cells 310 of VMM array 900, are preferably configured to operate in the sub-threshold region.

[0036] The nonvolatile reference memory cells and nonvolatile memory cells described herein are biased in weak inversion (sub-threshold region) as follows: Ids=Io*e(Vg-Vth) / nVt=w*Io*e(Vg) / nVt where w=e(-Vth) / nVt; where Ids is the drain-source current, Vg is the gate voltage of the memory cell, Vth is the threshold voltage of the memory cell, Vt is the thermal voltage = k*T / q, k is Boltzmann's constant, T is temperature in Kelvin, q is the electron charge, n is the slope coefficient = 1 + (Cdep / Cox), Cdep = the capacitance of the depletion layer, and Cox is the capacitance of the gate oxide layer, Io is the memory cell current at a gate voltage equal to the threshold voltage, and Io is proportional to (Wt / L)*u*Cox*(n-1)*Vt2, where u is the carrier mobility, and Wt and L are the width and length of the memory cell, respectively.

[0037] When using an IV-log converter that converts input current to input voltage using a memory cell (such as a reference memory cell or peripheral memory cell) or transistor: Vg=n*Vt*log[Ids / wp*Io] where wp is the w of the reference or peripheral memory cell.

[0038] For a memory array used as a vector matrix multiplier VMM array with current inputs, the output current is: Iout=wa*Io*e(Vg) / nVt, i.e. Iout=(wa / wp)*Iin=W*Iin W=e(Vthp-Vtha) / nVt where wa=w of each memory cell in the memory array. Vthp is the effective threshold voltage of the peripheral memory cells, and Vtha is the effective threshold voltage of the main (data) memory cells. Note that the threshold voltage of a transistor is a function of the substrate body bias voltage, which is represented as Vsb, and can be modulated to compensate for various conditions at such temperature. The threshold voltage Vth can be expressed as:

[0039] The word line or control gate can be used as the input of the memory cell for the input voltage.

[0040] Alternatively, the flash memory cells of the VMM arrays described herein can be configured to operate in the linear region. Ids=beta*(Vgs-Vth)*Vds; beta=u*Cox*Wt / L W=α(Vgs-Vth) That is, the weight W in the linear region is proportional to (Vgs-Vth).

[0041] The word line or control gate or bit line or source line can be used as the input of a memory cell operating in the linear region, and the bit line or source line can be used as the output of the memory cell.

[0042] For the IV linear converter, memory cells (such as reference or peripheral memory cells) or transistors operating in the linear region can be used to linearly convert input and output currents to input and output voltages.

[0043] Alternatively, the memory cells of the VMM arrays described herein can be configured to operate in the saturation region. Ids=1 / 2*beta*(Vgs-Vth)2; beta=u*Cox*Wt / L Wα(Vgs-Vth)2, i.e., weight W, is proportional to (Vgs-Vth)2

[0044] The word line, control gate, or erase gate can be used as the input of a memory cell operating in the saturation region, and the bit line or source line can be used as the output of an output neuron.

[0045] Alternatively, the memory cells of the VMM arrays described herein may be used in all domains or combinations thereof (subthreshold, linear, or saturation) for each layer or layers of a neural network.

[0046] 7 is described in U.S. Patent No. 10,748,630, which is incorporated herein by reference. As described in that application, the source lines or bit lines can be used as neuron outputs (current sum outputs).

[0047] FIG. 10 shows a neuron VMM array 1000 that is particularly suited for the memory cells 210 shown in FIG. 2 and is utilized as a synapse between an input layer and the next layer. The VMM array 1000 includes a memory array 1003 of nonvolatile memory cells, a reference array 1001 of first nonvolatile reference memory cells, and a reference array 1002 of second nonvolatile reference memory cells. The reference arrays 1001 and 1002, arranged in columns of the array, function to convert current inputs flowing into terminals BLR0, BLR1, BLR2, and BLR3 into voltage inputs WL0, WL1, WL2, and WL3. In practice, the first and second nonvolatile reference memory cells are diode-connected through a multiplexer 1014 (only partially shown) with the current inputs flowing into them. The reference cells are tuned (e.g., programmed) to a target reference level, which is provided by a reference mini-array matrix (not shown).

[0048] Memory array 1003 serves two purposes. First, memory array 1003 stores the weights used by VMM array 1000 in each memory cell. Second, memory array 1003 effectively multiplies the inputs (i.e., the current inputs provided to terminals BLR0, BLR1, BLR2, and BLR3, which reference arrays 1001 and 1002 convert to input voltages and provide to word lines WL0, WL1, WL2, and WL3) by the weights stored in memory array 1003, and then adds all the results (memory cell currents) to generate outputs for each bit line (BL0-BLN), which serve as inputs to the next layer or the last layer. By performing the multiplication and addition functions, memory array 1003 eliminates the need for separate multiplication and addition logic and is also power efficient. Here, voltage inputs are provided to word lines WL0, WL1, WL2, and WL3, and outputs appear on respective bit lines BL0-BLN during a read (inference) operation. The current in each of the bit lines BL0-BLN performs a function of the sum of the currents from all the non-volatile memory cells connected to that particular bit line.

[0049] Table 5 shows the operating voltages and currents for the VMM array 1000. The columns in the table indicate the voltages applied to the word line of the selected cell, the word lines of the unselected cells, the bit lines of the selected cell, the bit lines of the unselected cells, the source lines of the selected cell, and the source lines of the unselected cells. The rows indicate the read, erase, and program operations. Table 5: Operation of VMM Array 1000 in Figure 10 [Table 5]

[0050] FIG. 11 shows a neuron VMM array 1100 that is particularly suited for the memory cells 210 shown in FIG. 2 and that is utilized as part of the synapses and neurons between the input layer and the next layer. The VMM array 1100 includes a memory array 1103 of nonvolatile memory cells, a reference array 1101 of first nonvolatile reference memory cells, and a reference array 1102 of second nonvolatile reference memory cells. The reference arrays 1101 and 1102 extend in the row direction of the VMM array 1100. The VMM array is similar to the VMM 1000, except that the word lines extend vertically in the VMM array 1100. Here, inputs are provided to the word lines (WLA0, WLB0, WLA1, WLB2, WLA2, WLB2, WLA3, WLB3), and outputs appear on the source lines (SL0, SL1) during a read operation. The current in each source line performs a function of the sum of all the currents from the memory cells connected to that particular source line.

[0051] Table 6 shows the operating voltages and currents for VMM array 1100. The columns in the table indicate the voltages applied to the word line of the selected cell, the word lines of the unselected cells, the bit lines of the selected cell, the bit lines of the unselected cells, the source lines of the selected cell, and the source lines of the unselected cells. The rows indicate the read, erase, and program operations. Table 6: Operation of VMM Array 1100 in Figure 11 [Table 6]

[0052] 12 shows a neuron VMM array 1200 that is particularly suited for the memory cells 310 shown in FIG. 3 and that is utilized as part of the synapses and neurons between the input layer and the next layer. VMM array 1200 includes a memory array 1203 of nonvolatile memory cells, a reference array 1201 of first nonvolatile reference memory cells, and a reference array 1202 of second nonvolatile reference memory cells. Reference arrays 1201 and 1202 function to convert current inputs flowing into terminals BLR0, BLR1, BLR2, and BLR3 into voltage inputs CG0, CG1, CG2, and CG3. In effect, the first and second nonvolatile reference memory cells are diode-connected through multiplexer 1212 (only a portion of which is shown), with the current inputs flowing through BLR0, BLR1, BLR2, and BLR3. Multiplexer 1212 includes a corresponding multiplexer 1205 and cascoding transistor 1204 to ensure a constant voltage on each bit line (e.g., BLR0) of the first and second non-volatile reference memory cells during each read operation, where the reference cells are tuned to a target reference level.

[0053] Memory array 1203 serves two purposes. First, memory array 1203 stores the weights used by VMM array 1200. Second, memory array 1203 effectively multiplies the weights stored in the memory array by the inputs (current inputs provided to terminals BLR0, BLR1, BLR2, and BLR3; reference arrays 1201 and 1202 convert these current inputs to input voltages provided to control gates (CG0, CG1, CG2, and CG3)), and then adds all the results (cell currents) to generate an output that appears on BL0-BLN and serves as the input to the next layer or the last layer. Having the memory array perform the multiplication and addition functions eliminates the need for separate multiplication and addition logic and is also power efficient. Here, the inputs are provided to the control gate lines (CG0, CG1, CG2, and CG3) and the outputs appear on the bit lines (BL0-BLN) during read operations. The current in each bit line is a function of the sum of all the currents from the memory cells connected to that particular bit line.

[0054] VMM array 1200 implements one-way tuning of the non-volatile memory cells in memory array 1203. That is, each non-volatile memory cell is erased and then partially programmed until the desired charge on the floating gate is reached. If too much charge is added to the floating gate (such as if an incorrect value is stored in the cell), the cell is erased and the series of partial programming operations starts over. As shown, two rows that share the same erase gate (such as EG0 or EG1) are erased together (known as a page erase), and then each cell is partially programmed until the desired charge on the floating gate is reached.

[0055] Table 7 shows the operating voltages and currents for VMM array 1200. The columns in the table indicate the voltages applied to the word line of the selected cell, the word lines of the unselected cells, the bit line of the selected cell, the bit lines of the unselected cells, the control gate of the selected cell, the control gates of the unselected cells in the same sector as the selected cell, the control gates of the unselected cells in a different sector from the selected cell, the erase gate of the selected cell, the erase gates of the unselected cells, the source line of the selected cell, and the source lines of the unselected cells. The rows indicate read, erase, and program operations. Table 7: Operation of VMM Array 1200 in Figure 12 [Table 7]

[0056] FIG. 13 shows a neuron VMM array 1300 that is particularly suited for the memory cells 310 shown in FIG. 3 and that is utilized as part of the synapses and neurons between the input layer and the next layer. The VMM array 1300 includes a memory array 1303 of nonvolatile memory cells, a reference array 1301 or first nonvolatile reference memory cells, and a reference array 1302 of second nonvolatile reference memory cells. EG lines EGR0, EG0, EG1, and EGR1 extend vertically, while CG lines CG0, CG1, CG2, and CG3 and SL lines WL0, WL1, WL2, and WL3 extend horizontally. The VMM array 1300 is similar to the VMM array 1400, except that the VMM array 1300 implements bidirectional tuning, meaning that each individual cell can be fully erased, partially programmed, and partially erased as needed to reach a desired amount of charge on the floating gate through the use of separate EG lines. As shown, reference arrays 1301 and 1302 convert input currents at terminals BLR0, BLR1, BLR2, and BLR3 into control gate voltages CG0, CG1, CG2, and CG3 (through the action of diode-connected reference cells via multiplexer 1314), which are applied to the memory cells in a row direction. The current outputs (neurons) are in bit lines BL0 through BLN, each bit line summing all the currents from the non-volatile memory cells connected to that particular bit line.

[0057] Table 8 shows the operating voltages and currents for VMM array 1300. The columns in the table indicate the voltages applied to the word line of the selected cell, the word lines of the unselected cells, the bit line of the selected cell, the bit lines of the unselected cells, the control gate of the selected cell, the control gates of the unselected cells in the same sector as the selected cell, the control gates of the unselected cells in a different sector from the selected cell, the erase gate of the selected cell, the erase gates of the unselected cells, the source line of the selected cell, and the source lines of the unselected cells. The rows indicate read, erase, and program operations. Table 8: Operation of VMM Array 1300 in Figure 13 [Table 8]

[0058] 22 shows a neuron VMM array 2200 that is particularly suited for memory cells 210 shown in FIG. 2 and that is utilized as part of the synapses and neurons between the input layer and the next layer. In VMM array 2200, inputs INPUT0...INPUTN are received on bit lines BL0...BLN, respectively, and outputs OUTPUT1, OUTPUT2, OUTPUT3, and OUTPUT4 are generated on source lines SL0, SL1, SL2, and SL3, respectively.

[0059] 23 shows a neuron VMM array 2300 that is particularly suited for memory cells 210 shown in FIG. 2 and that is utilized as part of the synapses and neurons between the input layer and the next layer. In this example, inputs INPUT0, INPUT1, INPUT2, and INPUT3 are received on source lines SL0, SL1, SL2, and SL3, respectively, and outputs OUTPUT0, ..., OUTPUTN are generated on bit lines BL0, ..., BLN.

[0060] 2 shows a neuron VMM array 2400 that is particularly suited for the memory cells 210 shown in FIG. 2 and that is utilized as part of the synapses and neurons between the input layer and the next layer. In this example, inputs INPUT0, ..., INPUTM are received on word lines WL0, ..., WLM, respectively, and outputs OUTPUT0, ..., OUTPUTN are generated on bit lines BL0, ..., BLN.

[0061] 25 shows a neuron VMM array 2500 that is particularly suited for the memory cells 310 shown in FIG. 3 and that is utilized as part of the synapses and neurons between the input layer and the next layer. In this example, inputs INPUT0, ..., INPUTM are received on word lines WL0, ..., WLM, respectively, and outputs OUTPUT0, ..., OUTPUTN are generated on bit lines BL0, ..., BLN.

[0062] 26 shows a neuron VMM array 2600 that is particularly suited for the memory cells 410 shown in FIG. 4 and that is utilized as part of the synapses and neurons between the input layer and the next layer. In this example, inputs INPUT0, ..., INPUTn are received on vertical control gate lines CG0, ..., CGN, respectively, and outputs OUTPUT1 and OUTPUT2 are generated on source lines SL0 and SL1.

[0063] 27 illustrates a neuron VMM array 2700 that is particularly suited for the memory cells 410 shown in FIG. 4 and that is utilized as part of the synapses and neurons between the input layer and the next layer. In this example, inputs INPUT0, ..., INPUTN are received at bit line control gates 2701-1, 2701-2, ..., 2701-(N-1), and 2701-N, which are coupled to bit lines BL0, ..., BLN, respectively. Exemplary outputs OUTPUT1 and OUTPUT2 are generated on source lines SL0 and SL1.

[0064] Figure 28 shows a neuron VMM array 2800 that is particularly suitable for memory cells 310 shown in Figure 3, memory cells 510 shown in Figure 5, and memory cells 710 shown in Figure 7, and that is utilized as part of the synapses and neurons between the input layer and the next layer. In this example, inputs INPUT0, ..., INPUTM are received on word lines WL0, ..., WLM, and outputs OUTPUT0, ..., OUTPUTN are generated on bit lines BL0, ..., BLN, respectively.

[0065] 29 shows a neuron VMM array 2900 that is particularly suitable for memory cells 310 shown in FIG. 3, memory cells 510 shown in FIG. 5, and memory cells 710 shown in FIG. 7, and that is utilized as part of the synapses and neurons between the input layer and the next layer. In this example, inputs INPUT0, ..., INPUTM are received on control gate lines CG0, ..., CGM. Outputs OUTPUT0, ..., OUTPUTN are generated on vertical source lines SL0, ..., SLN, respectively, with each source line SLi coupled to the source lines of all memory cells in column i.

[0066] 30 shows a neuron VMM array 3000 that is particularly suitable for memory cells 310 shown in FIG. 3, memory cells 510 shown in FIG. 5, and memory cells 710 shown in FIG. 7, and that is utilized as part of the synapses and neurons between the input layer and the next layer. In this example, inputs INPUT0, ..., INPUTM are received on control gate lines CG0, ..., CGM. Outputs OUTPUT0, ..., OUTPUTN are generated on vertical bit lines BL0, ..., BLN, respectively, with each bit line BLi coupled to the bit lines of all memory cells in column i. Long-term and short-term memory

[0067] Prior art includes a concept known as long short-term memory (LSTM). LSTM units are often used within neural networks. LSTM allows a neural network to store information for any predetermined period of time and use that information in subsequent operations. A traditional LSTM unit includes a cell, an input gate, an output gate, and a forget gate. The three gates regulate the flow of information into and out of the cell and the duration for which information is stored within the LSTM. VMMs are particularly useful in LSTM units.

[0068] Figure 14 shows an example LSTM 1400. LSTM 1400 in this example includes cells 1401, 1402, 1403, and 1404. Cell 1401 receives input vector x0 and generates output vector h0 and cell state vector c0. Cell 1402 receives input vector x1, output vector (hidden state) h0 from cell 1401, and cell state c0 from cell 1401, and generates output vector h1 and cell state vector c1. Cell 1403 receives input vector x2, output vector (hidden state) h1 from cell 1402, and cell state c1 from cell 1402, and generates output vector h2 and cell state vector c2. Cell 1404 receives input vector x3, output vector (hidden state) h2 from cell 1403, and cell state c2 from cell 1403, and generates output vector h3. Additional cells can be used; an LSTM with four cells is just an example.

[0069] Figure 15 shows an example implementation of an LSTM cell 1500 that can be used for cells 1401, 1402, 1403, and 1404 in Figure 14. LSTM cell 1500 receives an input vector x(t), a cell state vector c(t-1) from a previous cell, and an output vector h(t-1) from a previous cell, and produces a cell state vector c(t) and an output vector h(t).

[0070] LSTM cell 1500 includes sigmoid function devices 1501, 1502, and 1503, each of which applies a number between 0 and 1 to control the degree to which each component of the input vector contributes to the output vector. LSTM cell 1500 also includes tanh devices 1504 and 1505 for applying a hyperbolic tangent function to the input vector, multiplier devices 1506, 1507, and 1508 for multiplying two vectors, and adder device 1509 for adding the two vectors. The output vector h(t) can be provided to the next LSTM cell in the system or can be accessed for other purposes.

[0071] FIG. 16 shows LSTM cell 1600, an example of one implementation of LSTM cell 1500. For the convenience of the reader, the same numbering scheme from LSTM cell 1500 is used in LSTM cell 1600. Sigmoid function devices 1501, 1502, and 1503 and tanh device 1504 each include multiple VMM arrays 1601 and activation function blocks 1602. VMM arrays, therefore, prove particularly useful in LSTM cells used in certain neural network systems. Multiplier devices 1506, 1507, and 1508 and adder device 1509 are implemented in digital or analog fashion. Activation function block 1602 can be implemented in digital or analog fashion.

[0072] An alternative example of LSTM cell 1600 (and another example of one implementation of LSTM cell 1500) is shown in Figure 17. In Figure 17, sigmoid function devices 1501, 1502, and 1503 and tanh device 1504 share the same physical hardware (VMM array 1701 and activation function block 1702) in a time-multiplexed manner. LSTM cell 1700 also includes a multiplier device 1703 for multiplying two vectors, an adder device 1708 for adding two vectors, a tanh device 1505 (which includes an activation function block 1702), a register 1707 for storing the value i(t) when i(t) is output from sigmoid function block 1702, a register 1704 for storing the value f(t)*c(t−1) when that value is output from multiplier device 1703 via multiplexer 1710, a register 1705 for storing the value i(t)*u(t) when that value is output from multiplier device 1703 via multiplexer 1710, a register 1706 for storing the value o(t)*ĉ(t) when that value is output from multiplier device 1703 via multiplexer 1710, and a multiplexer 1709.

[0073] While LSTM cell 1600 includes multiple sets of VMM arrays 1601 and respective activation function blocks 1602, LSTM cell 1700 includes only one set of VMM arrays 1701 and activation function blocks 1702, which are used to represent multiple layers in embodiments of LSTM cell 1700. LSTM cell 1700 requires one-quarter the space for the VMMs and activation function blocks compared to LSTM cell 1600, and therefore LSTM cell 1700 requires less space than LSTM 1600.

[0074] It can be further appreciated that an LSTM unit typically includes multiple VMM arrays, each of which requires functionality provided by specific circuit blocks outside the VMM array, such as adder and activation function blocks and high-voltage generation blocks. Providing a separate circuit block for each VMM array would require a significant amount of space within a semiconductor device and would be somewhat inefficient. Accordingly, the embodiments described below reduce the circuitry required outside the VMM array itself. Gated Recurrent Unit

[0075] Analog VMM implementations can be used for gated recurrent unit (GRU) systems. GRUs are gating mechanisms within recurrent neural networks. GRUs are similar to LSTMs, except that GRU cells generally contain fewer components than LSTM cells.

[0076] 18 shows an exemplary GRU 1800. GRU 1800 in this example includes cells 1801, 1802, 1803, and 1804. Cell 1801 receives input vector x0 and generates output vector h0. Cell 1802 receives input vector x1 and output vector h0 from cell 1801 and generates output vector h1. Cell 1803 receives input vector x2 and output vector (hidden state) h1 from cell 1802 and generates output vector h2. Cell 1804 receives input vector x3 and output vector (hidden state) h2 from cell 1803 and generates output vector h3. Additional cells can be used; a GRU with four cells is merely an example.

[0077] FIG. 19 shows an example implementation of a GRU cell 1900 that may be used for cells 1801, 1802, 1803, and 1804 of FIG. 18. GRU cell 1900 receives an input vector x(t) and an output vector h(t-1) from a preceding GRU cell and generates an output vector h(t). GRU cell 1900 includes sigmoid function devices 1901 and 1902, each of which applies a number between 0 and 1 to components from the output vector h(t-1) and the input vector x(t). GRU cell 1900 also includes a tanh device 1903 for applying a hyperbolic tangent function to the input vector, multiple multiplier devices 1904, 1905, and 1906 for multiplying two vectors, an adder device 1907 for adding the two vectors, and a complement device 1908 for subtracting the input from 1 to generate the output.

[0078] FIG. 20 shows GRU cell 2000, an example of one implementation of GRU cell 1900. For convenience of the reader, the same numbering scheme as GRU cell 1900 is used in GRU cell 2000. As can be seen from FIG. 20, sigmoid function devices 1901 and 1902 and tanh device 1903 each include multiple VMM arrays 2001 and activation function blocks 2002. Therefore, it can be seen that VMM arrays are particularly used in GRU cells used in specific neural network systems. Multiplier devices 1904, 1905, and 1906, adder device 1907, and complementary device 1908 are implemented in a digital or analog manner. Activation function block 2002 can be implemented in a digital or analog manner.

[0079] An alternative example of GRU cell 2000 (and another example of one implementation of GRU cell 1900) is shown in Figure 21. In Figure 21, GRU cell 2100 utilizes a VMM array 2101 and an activation function block 2102, which, when configured as a sigmoid function, applies a number between 0 and 1 to control the degree to which each component of the input vector contributes to the output vector. In Figure 21, sigmoid function devices 1901 and 1902 and tanh device 1903 share the same physical hardware (VMM array 2101 and activation function block 2102) in a time-multiplexed manner. GRU cell 2100 also includes a multiplier device 2103 for multiplying two vectors, an adder device 2105 for adding the two vectors, a complementary device 2109 for subtracting an input from one to produce an output, a multiplexer 2104, a register 2106 for holding the value h(t-1)*r(t) as it is output from multiplier device 2103 via multiplexer 2104, a register 2107 for holding the value h(t-1)*z(t) as it is output from multiplier device 2103 via multiplexer 2104, and a register 2108 for holding the value h^(t)*(1-z((t)) as it is output from multiplier device 2103 via multiplexer 2104.

[0080] While GRU cell 2000 includes multiple sets of VMM array 2001 and activation function block 2002, GRU cell 2100 includes only one set of VMM array 2101 and activation function block 2102, which are used to represent multiple layers in embodiments of GRU cell 2100. GRU cell 2100 requires one-third the space for the VMM and activation function block compared to GRU cell 2000, so GRU cell 2100 requires less space than GRU cell 2000.

[0081] It can be further appreciated that a GRU system typically includes multiple VMM arrays, each of which requires functionality provided by specific circuit blocks outside the VMM array, such as adder and activation function blocks and high-voltage generation blocks. Providing separate circuit blocks for each VMM array would require a significant amount of space within a semiconductor device and would be somewhat inefficient. Accordingly, the embodiments described below reduce the circuitry required outside the VMM array itself.

[0082] The input to the VMM array can be an analog level, a binary level, a pulse, a time modulated pulse, or a digital bit (in which case a DAC is required to convert the digital bit to the appropriate input analog level), and the output can be an analog level, a binary level, a timing pulse, a pulse, or a digital bit (in which case an output ADC is required to convert the output analog level to a digital bit).

[0083] In general, for each memory cell in the VMM array, each weight W can be implemented by a single memory cell, a differential cell, or two blended memory cells (the average of two cells). In the case of a differential cell, two memory cells are required to implement the weight W as a differential weight (W=W+-W-). In the case of two blended memory cells, two memory cells are required to implement the weight W as the average of two cells.

[0084] FIG. 31 illustrates a VMM system 3100. In some embodiments, the weights W stored in the VMM array are stored as a differential pair, W+ (positive weight) and W− (negative weight), where W=(W+)−(W−). In VMM system 3100, half of the bit lines are designated as W+ lines, i.e., bit lines connecting to memory cells that store a positive weight W+, and the other half of the bit lines are designated as W− lines, i.e., bit lines connecting to memory cells that implement a negative weight W−. W− lines are interspersed alternately among the W+ lines. Subtraction operations are performed by summing circuits, such as summing circuits 3101 and 3102, that receive current from the W+ and W− lines. The outputs of the W+ and W− lines are combined together to effectively provide W=W+−W− for each pair of (W+, W−) cells on every pair of (W+, W−) lines. Although described above with respect to W- lines interspersed alternately among W+ lines, in other embodiments, the W+ and W- lines may be arbitrarily positioned anywhere within the array.

[0085] 32 shows another embodiment. In a VMM system 3210, the positive weights W+ are implemented in a first array 3211 and the negative weights W− are implemented in a second array 3212 that is separate from the first array, and the resulting weights are suitably combined together by a summing circuit 3213.

[0086] Figure 33 shows VMM system 3300. The weights W stored in the VMM array are stored as a differential pair, W+ (positive weight) and W- (negative weight), where W = (W+) - (W-). VMM system 3300 includes array 3301 and array 3302. Half of the bit lines in each of arrays 3301 and 3302 are designated as W+ lines, i.e., bit lines connecting to memory cells that store a positive weight W+, and the other half of the bit lines in each of arrays 3301 and 3302 are designated as W- lines, i.e., bit lines connecting to memory cells that implement a negative weight W-. W- lines are interspersed alternately among the W+ lines. Subtraction operations are performed by adder circuits, such as adder circuits 3303, 3304, 3305, and 3306, that receive current from the W+ and W- lines. The outputs on the W+ and W- lines from each array 3301, 3302 are combined together, respectively, to effectively give W = W+ - W- for each pair of (W+, W-) cells on every pair of (W+, W-) lines. Additionally, the W values ​​from each array 3301 and 3302 may be further combined via adder circuits 3307 and 3308, meaning that each W value is the result of subtracting the W value from array 3302 from the W value from array 3301, and the final result from adder circuits 3307 and 3308 is one of two difference values.

[0087] Each non-volatile memory cell used in an analog neural memory system should be erased and programmed to hold a very specific and precise amount of charge, i.e., number of electrons, in its floating gate. For example, each floating gate should hold one of N different values, where N is the number of different weights that can be represented by each cell. Examples of N include 16, 32, 64, 128, and 256.

[0088] Similarly, a read operation should be able to accurately distinguish between N different levels.

[0089] There is a need for an improved input block that can be used to quickly and accurately apply current or voltage to one or more rows of cells being programmed, read, or erased in a VMM system. Summary of the Invention

[0090] Numerous embodiments are disclosed for input circuits for analog neural memories in deep learning artificial neural networks.

[0091]

[0092]

[0093]

[0094]

[0095]

[0096]

[0097]

[0098]

[0099]

[0100]

[0101]

[0102]

[0103]

[0104]

[0105]

[0106]

[0107]

[0108]

[0109]

[0110]

[0111]

[0112]

[0113]

[0114]

[0115]

[0116]

[0117]

[0118]

[0119]

[0120]

[0121]

[0122]

[0123]

[0124]

[0125]

[0126]

[0127]

[0128]

[0129]

[0130]

[0131]

[0132]

[0133]

[0134]

[0135]

[0136]

[0137] [Brief explanation of the drawings]

[0138] [Figure 1] FIG. 1 illustrates an artificial neural network. [Figure 2] 1 shows a prior art split-gate flash memory cell. [Figure 3] 1 illustrates another prior art split-gate flash memory cell. [Figure 4] 1 illustrates another prior art split-gate flash memory cell. [Figure 5] 1 illustrates another prior art split-gate flash memory cell. [Figure 6] FIG. 1 illustrates various levels of an exemplary artificial neural network that utilizes one or more non-volatile memory arrays. [Figure 7] FIG. 1 is a block diagram illustrating a vector matrix multiplication system. [Figure 8] FIG. 1 is a block diagram illustrating an exemplary artificial neural network utilizing one or more vector-matrix multiplication systems. [Figure 9] 1 illustrates another embodiment of a vector matrix multiplication system. [Figure 10] 1 illustrates another embodiment of a vector matrix multiplication system. [Figure 11] 1 illustrates another embodiment of a vector matrix multiplication system. [Figure 12] 1 illustrates another embodiment of a vector matrix multiplication system. [Figure 13] 1 illustrates another embodiment of a vector matrix multiplication system. [Figure 14] 1 shows a prior art long-term memory system. [Figure 15] An exemplary cell for use in a long-term memory system is shown. [Figure 16]16 illustrates one embodiment of the exemplary cell of FIG. 15. [Figure 17] 16 illustrates another embodiment of the exemplary cell of FIG. 15. [Figure 18] 1 shows a prior art gated recurrent unit system. [Figure 19] 1 shows an exemplary cell for use in a gated recurrent unit system. [Figure 20] 20 illustrates one embodiment of the exemplary cell of FIG. 19. [Figure 21] 20 illustrates another embodiment of the exemplary cell of FIG. 19. [Figure 22] 1 illustrates another embodiment of a vector matrix multiplication system. [Figure 23] 1 illustrates another embodiment of a vector matrix multiplication system. [Figure 24] 1 illustrates another embodiment of a vector matrix multiplication system. [Figure 25] 1 illustrates another embodiment of a vector matrix multiplication system. [Figure 26] 1 illustrates another embodiment of a vector matrix multiplication system. [Figure 27] 1 illustrates another embodiment of a vector matrix multiplication system. [Figure 28] 1 illustrates another embodiment of a vector matrix multiplication system. [Figure 29] 1 illustrates another embodiment of a vector matrix multiplication system. [Figure 30] 1 illustrates another embodiment of a vector matrix multiplication system. [Figure 31] 1 illustrates another embodiment of a vector matrix multiplication system. [Figure 32] 1 illustrates another embodiment of a vector matrix multiplication system. [Figure 33] 1 illustrates another embodiment of a vector matrix multiplication system. [Figure 34] 1 illustrates another embodiment of a vector matrix multiplication system. [Figure 35A] 1 illustrates an embodiment of an input block. [Figure 35B] 1 illustrates an embodiment of an input block. [Figure 36] 10 illustrates another embodiment of an input block. [Figure 37A] 37 shows a waveform 3700 of a sampling of a global row DAC with local row sample and hold action. [Figure 37B] 37 shows a waveform 3700 of a sampling of a global row DAC with local row sample and hold action. [Figure 38] 1 shows a sample and hold buffer. [Figure 39] 1 shows a sample and hold buffer. [Figure 40A] Indicates the input block. [Figure 40B] Indicates the input block. [Figure 41A] Indicates the input block. [Figure 41B] Indicates the input block. [Figure 42] 1 shows an adjustable 2D thermometer code digital-to-analog converter. [Figure 43] 1 shows a reference subcircuit. [Figure 44] A configurable SAR (successive approximation register) analog-to-digital converter is shown. [Figure 45] 1 shows a voltage-to-current converter. [Figure 46] 1 shows a logarithmic current-to-voltage converter. [Figure 47] 1 shows a logarithmic current-to-voltage converter. [Figure 48] 1 shows a logarithmic current-to-voltage converter. DETAILED DESCRIPTION OF THE INVENTION

[0139] The artificial neural network of the present invention utilizes a combination of CMOS technology and non-volatile memory arrays. VMM System Overview

[0140] 34 shows a block diagram of a VMM system 3400. The VMM system 3400 includes a VMM array 3401, a row decoder 3402, a high-voltage decoder 3403, a column decoder 3404, a bit line driver 3405, input circuits 3406, output circuits 3407, control logic 3408, and a bias generator 3409. The VMM system 3400 further includes a high-voltage generation block 3410, which includes a charge pump 3411, a charge pump regulator 3412, and a high-voltage analog precision level generator 3413. The VMM system 3400 further includes a (program / erase or weight adjustment) algorithm controller 3414, analog circuitry 3415, a control engine 3416 (which may include specialized functions such as, but not limited to, arithmetic functions, startup functions, embedded microcontroller logic, etc.), and test control logic 3417. The systems and methods described below may be implemented in the VMM system 3400.

[0141] The input circuit 3406 may include circuits such as a DAC (digital to analog converter), a DPC (digital to pulses converter), an AAC (analog to analog converter, such as a current-to-voltage converter, a logarithmic converter), a PAC (pulse to analog level converter), or any other type of converter. The input circuit 3406 may implement normalization, linear or nonlinear up / downscaling functions, or arithmetic functions. The input circuit 3406 may implement a temperature compensation function for the input level. The input circuit 3406 may implement an activation function such as ReLU or sigmoid. The output circuit 3407 may include circuits such as an ADC (analog to digital converter, for converting neuron analog outputs to digital bits), an AAC (analog-to-analog converter, such as a current-to-voltage converter, a logarithmic converter), an APC (analog to pulse(s) converter, an analog-to-pulse converter, an analog-to-time-modulated pulse converter), or any other type of converter. The output circuit 3407 may perform activation functions such as a rectified linear activation function (ReLU) or a sigmoid. The output circuit 3407 may implement statistical normalization, regularization, up / downscaling / gain functions, statistical rounding, or arithmetic functions (e.g., addition, subtraction, division, multiplication, shift, log) of the neuron outputs. The output circuit 3407 may implement temperature compensation functions for the neuron outputs or array outputs (such as bit line outputs) to keep the power consumption of the array approximately constant or to improve the accuracy of the array (neuron) outputs, such as by keeping the IV slope approximately the same.

[0142] FIG. 35A shows an input block 3500 used to provide input to a VMM array 3506. Input block 3500 includes a global digital-to-analog converter (DAC) 3501, row registers 3502-0 through 3502-n, each corresponding to one of the rows numbered 0 through n in the array, digital comparator blocks 3503-0 through 3503-n, row sample-and-hold buffers 3504-0 through 3504-n, each corresponding to one of the rows numbered 0 through n, and output signals 3505-0 through 3505-n, each corresponding to one of the rows numbered 0 through n and labeled CGIN0, CGIN1, ..., CGINn-1, and CGINn, respectively. Signal GDACsup is a global DAC signal provided by global DAC 3501. Signals CGIN0-n couple to the row inputs of array 3506.

[0143] Digital comparator block 3503 compares the value stored in the associated row register 3502 with CLKCOUNTx, which is the result of counting clock signals during one interval. If there is a match, the corresponding row S / H 3504 is enabled to sample the value from the global DAC 3501 into its respective row S / H buffer. This technique is referred to as global row DAC sampling. Each row in VMM array 3506 has a corresponding row register 3502, digital comparator block 3503, and row S / H 3504.

[0144] In operation, row registers 3502-0 through 3502-n are loaded with the digital input bits DINx (where x is the number of bits, such as 8 or 16 bits) for that particular row and receive a clock signal CLK. The CLK signal is used to load data from DINx into row registers 3502-x. A global digital-to-analog converter 3501 is shared by all rows and performs digital-to-analog conversion on the digital bits DINx stored in a particular row register 3502 in a time-multiplexed manner. The conversion is performed by each of the digital comparator blocks 3503 comparing the digital input bits for a particular row with a digital count value, CLKCOUNTx. When the digital count value of global DAC 3501 matches the contents of its respective row register 3502, that row's corresponding row sample-and-hold buffer 3503 samples the analog output from global digital-to-analog converter 3501 and holds that value, which is applied as the output signal 3505 for that particular row. The output signal 3504 may be applied to a control gate line or word line during a programming operation in that particular row, for example, in the manner described above with respect to the other figures.

[0145] In another embodiment, the row sample and hold buffer 3504 may be shared among multiple rows by time multiplexing the row sample and hold buffer.

[0146] FIG. 35B shows an input block 3550 used to provide input to a VMM array 3556 . Input block 3550 includes a global digital-to-analog converter (DAC) 3551, row registers 3552-0 to 3552-n, each corresponding to a respective one of the rows numbered 0 to n in the VMM array, digital multiplexer (mux) blocks 3553-0 to 3553-n, each corresponding to a respective one of the rows numbered 0 to n, row sample-and-hold (S / H) buffers 3553-0 to 3553-n, each corresponding to a respective one of the rows numbered 0 to n, and output signals 3554-0 to 3554-n, designated CGIN0, CGIN1..., CGINn-1, and CGINn, respectively, each corresponding to a respective one of the rows numbered 0 to n. A digital mux block 3553 is used to multiplex the data in row register 3552 onto bus GDAC_DINx, which is applied as input to global DAC 3551. A corresponding row S / H buffer samples the value from the global DAC into a local S / H. Each row has its own row register 3552, S / H buffer 3553m, and output signal 3554.

[0147] In operation, row registers 3552-0 through 3552-n are loaded with the digital input bits DINx (where x is the number of bits, such as 8 or 16 bits) for that particular row and receive a clock signal CLK. The CLK signal is used to load data from DINx into row register 3552-x. A global digital-to-analog converter 3551 is shared by all rows and performs digital-to-analog conversion on the digital bits DINx stored in a particular row register 3552 in a time-multiplexed manner. The conversion is performed by multiplexing the row register data onto the data input (bus GDAC_DINx) of global DAC 3551. Multiplexing of row register data onto data input bus GDAC_DINx is enabled by signal EN-x 3557-x for each row. A corresponding row sample-and-hold buffer 3554 samples the analog output from global digital-to-analog converter 3551 and holds that value, which is applied as the output signal 3555 for that particular row. The output signal 3555 may be applied to a control gate line or word line during a programming operation in that particular row, for example, in the manner described above with respect to other figures. In another embodiment, the row sample and hold buffer 35554 may be shared among multiple rows by time multiplexing the row sample and hold buffer.

[0148] Figure 36 shows input block 3600 used to provide input to VMM array 3606. Input block 3600 is similar to input block 3500, but also provides a row decoder function for selecting one or more rows for operation. Input block 3600 includes global digital-to-analog converter and row decoder 3601, row registers 3602-0 through 3602-n, each corresponding to a respective one of the rows numbered 0 through n in the VMM array, digital comparator blocks 3603 through 3603-n, each corresponding to a respective one of the rows numbered 0 through n in the VMM array, row sample and hold buffers 3604-0 through 3604-n, and output signals 3605-0 through 3605-n, designated CGIN0, CGIN1..., CGINn-1, and CGINn, respectively, each corresponding to a respective one of the rows numbered 0 through n. A digital comparator block 3603 compares the value stored in each row register 3602 with a count value, CLKCOUNTx. When the digital count value of global DAC 3501 matches the contents of each row register 3602, each row S / H buffer 3604 samples the value from global DAC 3601 into its respective S / H buffer 3604. Each row has its own row register 3602, digital comparator block 3603, and row S / H buffer 3604.

[0149] In operation, row registers 3602-0 through 3602-n are loaded with the digital input bits DINx (where x is the number of bits, such as 8 or 16 bits) for the associated row and receive a clock signal CLK. The CLK signal is used to load data from DINx into row registers 3602-x. Global digital-to-analog converter 3601 (consisting of multiple global digital-to-analog converters, such as 3601-0 and 3601-1) is shared by all rows. In one embodiment, global DAC 3601-0 operates on even rows, and global DAC and row decoder 3601-1 operates on odd rows. Global digital-to-analog converter 3601 receives a row address via row data input bus GDAC_DINx to select the corresponding row. It then performs a digital-to-analog conversion on the digital bits DINx stored in the associated row register 3602 (via the GDAC_DINx bus). The row sample and hold buffer 3604 corresponding to that row samples the analog output from the global digital-to-analog converter 3601 and holds that value, which is applied as the output signal 3605 for that particular row. The output signal 3605 may be applied to control gate lines or word lines during programming operations in that particular row(s), for example, in the manner described above with respect to the other figures.

[0150] Figure 37A shows waveform 3700 illustrating exemplary linear voltage levels for an exemplary sample and hold operation by row sample and hold buffer 3504 of Figure 35A, row sample and hold buffer 3554 of Figure 35B, or row sample and hold buffer 3604 of Figure 36. This is suitable for memory cells operating in the linear region. Signal GDACsup 3701 is a voltage provided by a global linear DAC, such as circuit block 3501 of Figure 35A, circuit block 3551 of Figure 35B, or circuit block 3601x of Figure 36. A linear step is shown to illustrate that this is a linear DAC.

[0151] Figure 37B shows waveform 3720 illustrating exemplary logarithmic voltage levels for an exemplary sample and hold operation by row sample and hold buffer 3504 of Figure 35A, row sample and hold buffer 3554 of Figure 35B, or row sample and hold buffer 3604 of Figure 36. This is suitable for memory cells operating in the sub-threshold region. Alternatively, a global DAC voltage waveform can be provided for memory cells operating in the saturation region. Signal GDACsup 3721 is the voltage provided by a global (sub-threshold, linear, saturation) DAC, such as circuit block 3501 of Figure 35A, circuit block 3551 of Figure 35B, or circuit block 3601x of Figure 36. Logarithmic steps are shown to illustrate that this is a logarithmic DAC.

[0152] FIG. 38 shows a sample and hold buffer 3800 that may be used for row sample and hold buffer 3504 of FIG. 35A, row sample and hold buffer 3554 of FIG. 35B, or row sample and hold buffer 3604 of FIG. 36. Sample and hold buffer 3800 includes switch 3801, capacitor 3802, and buffer 3803, which may be implemented as a unity buffer using an operational amplifier. In operation, switch 3801 is closed (enabled, such as by a true result of a comparison between a digital comparator and a digital count value), thereby allowing an analog value (global DAC value) to be stored (held) in capacitor 3802. A value reflecting that stored value may then be output from buffer 3803, which drives each row input of the array. Capacitor 3802 may be an actual capacitor, such as the intrinsic capacitor found in a wire.

[0153] Figure 39 shows a sample and hold buffer 3900 that may be used for row sample and hold buffer 3504 of Figure 35A, row sample and hold buffer 3554 of Figure 35B, or row sample and hold buffer 3604 of Figure 36. Sample and hold buffer 3900 includes switch 3901 and capacitor 3902. In operation, switch 3901 is closed (enabled, such as by a true result of a comparison between a digital comparator and a digital count value), thereby storing (holding) an analog value in capacitor 3902. A value reflecting that value may then be output from capacitor 3902. Capacitor 3902 may be an actual capacitor, for example, the inherent capacitance found in a wire.

[0154] FIG. 40A shows an input block 4000. The input block 4000 includes a digital-to-analog converter 4001, a voltage-to-current converter 4002, and a current-to-voltage logarithmic converter 4003. The input block 4000 can be added to any of the preceding input blocks when it is desirable to have an input voltage that varies according to a logarithmic function, which is useful, for example, when memory cells in a VMM array are operating in a sub-threshold range. The digital-to-analog converter receives a digital input DINx and generates an analog voltage Vout. The voltage-to-current converter 4002 linearly converts the analog voltage Vout to a current Iout. The current-to-voltage logarithmic converter 4003 converts the current Iout to a voltage Vlog according to a logarithmic function, i.e., Vlog=A*log(Iout), where A is a constant. The input block 4000 can be used as a global DAC as in FIGS. 35A, 35B, and 36. It can generate the waveform 3720 shown in FIG. 37B. In the case of the global DAC of Figure 35A, CLK is used to generate the DAC output voltage in a stepwise manner, e.g., from 0 to 256 steps for an 8-bit DAC. In the case of the global DAC of Figures 35B and 36, DINx is used to generate the DAC output voltage.

[0155] FIG. 40B shows an input block 4020. The input block 4020 includes a digital-to-analog converter that can be used as a global DAC. This can be used, for example, for memory cells that operate in the linear region. The input block 4020 can be used as a global DAC as in FIGS. 35 and 36. It can generate the waveform 3700 shown in FIG. 37A.

[0156] FIG. 41A shows an input block 4100. The input block 4100 comprises a current digital-to-analog converter 4101 and a current-to-voltage logarithmic converter 4102. The input block 4100 can be added to any of the aforementioned input blocks when it is desirable to have an input voltage that varies according to a logarithmic function, which is useful, for example, when memory cells in a VMM array are operating in a sub-threshold range. The current digital-to-analog converter 4101 receives a digital input DINx as in the previous example and generates an analog current Iout. The current-to-voltage logarithmic converter 4102 converts the current Iout to a voltage Vlog according to a logarithmic function, for example, using a memory cell or a MOS transistor: e.g., Vlog=A*log(Iout)+Vlogoffset, where A is a constant and Vlogoffset is another constant to account for turning on or off, for example, the voltage of a memory cell or some other offset voltage from the array or the decoding circuitry or the input / output circuitry itself.

[0157] FIG. 41B shows an input block 4120. The input block 4120 comprises a current digital-to-analog converter 4121 and a current-to-voltage linear converter 4122. The input block 4120 can be added to any of the aforementioned input blocks when it is desirable to have an input voltage that varies according to a linear function, which is useful, for example, when the memory cells in the VMM array are operating in a linear range. The current digital-to-analog converter 4121 receives a digital input DINx as in the previous example and generates an analog current Iout. The current-to-voltage linear converter 4122 converts the current Iout to a voltage Vlin according to a linear function, for example, using a memory cell or MOS transistor: e.g., Vlin = A * Iout + Vlinoffset, where A is a constant and Vlogoffset is another constant to account for turning on or off, for example, the voltage of the memory cell or some other offset voltage from the array or the decoding circuitry or the input / output circuitry itself.

[0158] FIG. 42 illustrates an adjustable 2D thermometer code current digital-to-analog converter 4200. The adjustable 2D thermometer code current digital-to-analog converter 4200 comprises control logic 4202 and a 2D array 4205 including an array of i rows and j columns of devices 4206, where a particular device 4206 is indicated by the label 4206-(row)(column). A particular device 4206 may be a current mirror. As shown, there are 16 current mirrors (devices 4206) in the 2D array 4205. The adjustable 2D thermometer code current digital-to-analog converter 4200 essentially converts a four-digit input code into an output current 4201, Iout, having a value between 1 and 16 times the Ibias units provided by the bias source 4204.

[0159] For example, bias source 4204 may provide a 1 nA current Ibias unit that is mirrored to device 4206. Here, the first row consists of devices 4206-11 through 4206-1j, with devices 4206 sequentially enabled one at a time from left to right. The next row may then be enabled sequentially from left to right to add to the first column, meaning five, then six, then seven, then eight devices 4206 are enabled. Thus, by sequentially enabling devices 4206, any transistor mismatch associated with conventional binary decoding may be reduced. The sum of the enabled devices 4206 is then output as output current 4201. The illustrated tunable 4×4 2D thermometer-coded current digital-to-analog converter 4200 may be any other dimension, such as 32×32 or 8×32.

[0160] Figure 43 shows a reference subcircuit 4300 that may be used in device 4206 of Figure 42. Reference subcircuit 4300 includes NMOS transistors 4301 and 4302, configured as shown. Transistor 4302 is the current mirror bias transistor, transistor 4301 is the enable transistor (to allow bias transistor 4302 to be connected to output node OUTPUT), and the master bias current used to mirror into transistor 4302 is not shown.

[0161] Figure 44 shows a SAR (successive approximation register) analog-to-digital converter 4400. This circuit is a digital-to-analog converter that relies on charge redistribution using a binary capacitor. It includes a binary CDAC (capacitor-based DAC) 4401, an opamp / comparator 4402, and SAR logic 4403. GndV is a low voltage reference level, e.g., ground level. This is used, for example, in Figures 40A and 40B.

[0162] FIG. 45 shows a voltage-to-current converter 4500 comprising an operational amplifier 4501, an NMOS transistor 4501, and a resistor 4503 configured as shown. The voltage-to-current converter 4500 receives an input voltage V and produces an output current I. This may be used in combination with the DAC circuit 4400, for example, to convert a voltage to a current. This current may be used to convert to a voltage using a log IV converter such as those in FIGS. 46 / 47 / 48.

[0163] 46 shows a logarithmic current-to-voltage converter 4600. The logarithmic current-to-voltage converter 4600 comprises a switch 4601 and an NMOS transistor 4602 configured as shown. The NMOS 4602 operates in the sub-threshold region. The logarithmic current-to-voltage converter 4600 receives an input I-in and produces an output Vlog.

[0164] 47 shows a logarithmic current-to-voltage converter 4700. The logarithmic current-to-voltage converter 4700 includes switches 4701 and 4702 configured as shown, and a memory cell 4703. The memory cell 4703 operates in the sub-threshold region. The logarithmic current-to-voltage converter 4600 receives an input I-in and produces an output Vlog.

[0165] 48 shows a logarithmic current-to-voltage converter 4800. The logarithmic current-to-voltage converter 4800 includes switches 4801 and 4802, an NMOS transistor 4803, and a memory cell 4804 configured as shown. The memory cell 4804 operates in the sub-threshold region. The transistor 4803 provides a constant bias to the drain of the memory cell 4804. The logarithmic current-to-voltage converter 4600 receives an input I-in and produces an output Vlog. In Figures 47 and 48, a log I to V conversion is performed to recreate the log I vs. V slope of the memory cell current. In this case, the cell output current is = K*I-in, and the input current is from the output of the log DAC, which essentially represents the value of the input activation (row value). K is determined by the floating charge in the memory cell, which essentially represents the weight in the neural network.

[0166] It should be noted that, as used herein, both the terms "over" and "on" are inclusive of "directly" (with no intermediate material, element, or gap disposed therebetween) and "indirectly" (with an intermediate material, element, or gap disposed therebetween). Similarly, the term "adjacent" includes "directly adjacent" (with no intermediate material, element, or gap disposed therebetween) and "indirectly adjacent" (with an intermediate material, element, or gap disposed therebetween); "attached" includes "directly attached" (with no intermediate material, element, or gap disposed therebetween) and "indirectly attached" (with an intermediate material, element, or gap disposed therebetween); and "electrically coupled" includes "directly electrically coupled" (with no intermediate material or element disposed therebetween that electrically connects the elements together) and "indirectly electrically coupled" (with an intermediate material or element disposed therebetween that electrically connects the elements together). For example, forming an element "over a substrate" can include forming the element directly on the substrate with no intermediate materials / elements therebetween, and forming the element indirectly on the substrate with one or more intermediate materials / elements therebetween.

Claims

1. 1. An input block for providing input to a vector matrix multiplication array in a neural memory system, the vector matrix multiplication array comprising non-volatile memory cells arranged in rows and columns, the input block comprising: a global digital-to-analog converter; a plurality of row registers, each row register corresponding to a row in the array; a plurality of row sample and hold buffers, each row sample and hold buffer corresponding to a row in the array; an input block, wherein when a row is selected, the global digital-to-analog converter converts a digital value stored in a row register corresponding to the selected row into an analog value, and the analog value is sampled and held by the row sample-and-hold buffer corresponding to the selected row and applied to a line coupled to the selected row.

2. The input block of claim 1 , wherein the global digital-to-analog converter comprises a linear digital-to-analog converter.

3. The input block of claim 1 , wherein the global digital-to-analog converter comprises a logarithmic digital-to-analog converter.

4. 2. The input block of claim 1, wherein the global digital-to-analog converter comprises a digital-to-analog converter for odd rows and another digital-to-analog converter for even rows.

5. 2. The input block of claim 1, wherein the global digital-to-analog converter comprises a row decoder that selects the selected row in response to a row address.

6. 2. The input block of claim 1, wherein each row sample and hold buffer comprises a switch and a capacitor, the switch applying the analog input to the capacitor when closed.

7. 7. The input block of claim 6, wherein each row sample and hold buffer further comprises a buffer for outputting a value reflecting the value stored in the capacitor.

8. 1. An input block for providing input to a vector matrix multiplication array in a neural memory system, the vector matrix multiplication array comprising a plurality of non-volatile memory cells arranged in rows and columns, the input block comprising: a digital-to-analog converter; a voltage-to-current converter that generates a current responsive to the output of the digital-to-analog converter; a current-to-voltage logarithmic converter for generating a voltage that is a function of the logarithm of the current.

9. 9. The input block of claim 8, wherein the digital-to-analog converter comprises a 2D thermometer-coded current digital-to-analog converter.

10. 9. The input block of claim 8, wherein the digital-to-analog converter comprises a configurable successive approximation register.

11. The input block of claim 8 , wherein the voltage-to-current converter comprises an operational amplifier, an NMOS transistor, and a resistor.

12. The input block of claim 8 , wherein the current-to-voltage logarithmic converter comprises a switch and an NMOS transistor.

13. 9. The input block of claim 8, wherein the current-to-voltage logarithmic converter comprises a plurality of switches and memory cells.

14. The input block of claim 13 wherein the current-to-voltage logarithmic converter comprises an NMOS transistor.

15. 1. An input block for generating inputs to a vector matrix multiplication array in a neural network memory system, the vector matrix multiplication array comprising a plurality of non-volatile memory cells arranged in rows and columns, the input block comprising: a current digital-to-analog converter for generating a current; a current-to-logarithmic voltage converter that generates a voltage that is a function of the logarithm of the generated current.

16. 16. The input block of claim 15, wherein the digital-to-analog converter comprises a 2D thermometer-coded current digital-to-analog converter.

17. 16. The input block of claim 15, wherein the digital-to-analog converter comprises a successive approximation register.

18. The input block of claim 15, wherein the current-to-voltage logarithmic converter comprises a switch and an NMOS transistor.

19. The input block of claim 15, wherein the current-to-voltage logarithmic converter comprises a plurality of switches and memory cells.

20. 20. The input block of claim 19, wherein the current-to-voltage logarithmic converter comprises an NMOS transistor.

Images