Artificial neural network comprising reference array for i-v slope configuration

Abstract

This invention discloses numerous examples of an artificial neural network comprising a plurality of reference arrays for constructing a vector matrix multiplication array. In one example, a system comprises: a vector matrix multiplication array located in an artificial neural network; and a plurality of reference arrays characterized by different IV curves, wherein one or more of the plurality of reference arrays are used to generate input voltages to the vector matrix multiplication array during operation.

Description

Technical Field

[0001] [Priority Claim]

[0002] This application claims the priority of U.S. Provisional Patent Application No. 63 / 328,543, filed on April 7, 2022, with the title "Artificial Neural Network Comprising Monte Carlo Reference Array for I-V Slope Configuration" and U.S. Patent Application No. 17 / 848,381, filed on June 23, 2022, with the title "Artificial Neural Network Comprising Reference Array for I-V Slope Configuration".

[0003] Disclosed are numerous examples of artificial neural networks that include reference arrays for I-V slope configuration in a main array. Prior Art

[0004] Artificial neural networks simulate biological neural networks (the central nervous system of animals, specifically, the brain) and are used to estimate or approximate functions that may depend on a large number of inputs and are typically unknown. Artificial neural networks generally include layers of interconnected "neurons" that exchange information with each other.

[0005] FIG. 1 illustrates an artificial neural network, where the circles represent the inputs or layers of neurons. The connections (referred to as synapses) are represented by arrows and have numerical weights that can be tuned based on experience. This enables the neural network to adapt to the inputs and be able to learn. Typically, a neural network includes a layer of multiple inputs. Typically, there is one or more intermediate neuron layers and an output neuron layer that provides the output of the neural network. The neurons at each level make decisions individually or jointly based on the data received from the synapses.

[0006] One of the main challenges in the development of artificial neural networks for high-performance information processing lies in the lack of adequate hardware technology. In fact, practical neural networks rely on an extremely large number of synapses to achieve high connectivity between neurons, that is, extremely high computational parallelism. In principle, this complexity can be achieved using digital supercomputers or clusters of dedicated graphics processing units. However, in addition to high costs, these methods also suffer from medium energy efficiency compared to biological networks, mainly because biological networks perform low-precision analog calculations and thus consume much less energy. CMOS analog circuits have been used in artificial neural networks, but most of the implemented CMOS synapses have been too large given the large number of neurons and synapses.

[0007] The applicant previously disclosed in U.S. Patent Application Publication No. 2017 / 0337466A1 an artificial (analog) neural network that uses one or more non-volatile memory arrays as synapses, which is incorporated herein by reference. The non-volatile memory array operates as an analog neural memory and includes non-volatile memory cells arranged in columns and rows. The neural network includes: a first plurality of synapses configured to receive a first plurality of inputs and generate a first plurality of outputs from the first plurality of inputs; and a first plurality of neurons configured to receive the first plurality of outputs. The first plurality of synapses includes a plurality of memory cells, wherein each of the memory cells includes: spaced-apart source and drain regions formed in a semiconductor substrate, and a channel region extending between the source and drain regions; a floating gate disposed above a first portion of the channel region and insulated from the first portion; and a non-floating gate disposed above a second portion of the channel region and insulated from the second portion. Each of the plurality of memory cells stores a weight value corresponding to the number of electrons on the floating gate. The plurality of memory cells multiply the first plurality of inputs by the stored weight values to generate the first plurality of outputs. Non-volatile memory cell

[0008] Non-volatile memories are well known. For example, U.S. Patent 5,029,130 (the "'130 patent"), which is incorporated herein by reference, discloses a split-gate non-volatile memory cell array, which is a type of flash memory cell. This memory cell 210 is shown in FIG. 2. Each memory cell 210 includes a source region 14 and a drain region 16 formed in a semiconductor substrate 12 and a channel region 18 between the source region and the drain region. A floating gate 20 is formed over a first portion of the channel region 18 and is insulated from the first portion (and controls the conductivity of the first portion), and is formed over a portion of the source region 14. A word line terminal 22 (which is typically coupled to a word line) has: a first portion that is disposed over a second portion of the channel region 18 and is insulated from the second portion (and controls the conductivity of the second portion); and a second portion that extends over and above the floating gate 20. The floating gate 20 and the word line terminal 22 are insulated from the substrate 12 by a gate oxide. A bit line 24 is coupled to the drain region 16.

[0009] The memory cell 210 is erased by placing a high positive voltage on the word line terminal 22 (wherein electrons are removed from the floating gate), which causes electrons on the floating gate 20 to tunnel through an intervening insulator from the floating gate 20 to the word line terminal 22 via Fowler-Nordheim (FN) tunneling.

[0010] The memory cell 210 is programmed by applying a positive voltage to the word line terminal 22 and a positive voltage to the source region 14 by utilizing source-side injection (SSI) of hot electrons (wherein electrons are placed on the floating gate). An electron current will flow from the drain region 16 towards the source region 14. When the electrons reach the gap between the word line terminal 22 and the floating gate 20, the electrons will be accelerated and heated. Some of the heated electrons will be injected through the gate oxide onto the floating gate 20 due to the attractive electrostatic force from the floating gate 20.

[0011] The memory cell 210 is read by placing a positive read voltage on the drain region 16 and the word line terminal 22 (which turns on the portion of the channel region 18 under the word line terminal). If the floating gate 20 is positively charged (i.e., electrons are erased), then the portion of the channel region 18 under the floating gate 20 is also turned on, and current will flow across the channel region 18, which is sensed as the erased or "1" state. If the floating gate 20 is negatively charged (i.e., programmed with electrons), then the portion of the channel region under the floating gate 20 is mostly or completely turned off, and current will not flow across the channel region 18 (or only a very small amount of current will flow across the channel region), which is sensed as the programmed or "0" state.

[0012] Table 1 depicts the typical voltage and current ranges that can be applied to the terminals of memory cell 210 for performing read, erase, and program operations: Table 1: Operations of the flash memory cell 210 of FIG. 2 WL BL SL Read 2 - 3V 0.6 - 2V 0V Erase ~11 - 13V 0V 0V Program 1 - 2V 10.5 - 3μA 9 - 10V

[0013] Other split - gate memory cell configurations are known to us, which are other types of flash memory cells. For example, FIG. 3 depicts a four - gate memory cell 310, which includes a source region 14, a drain region 16, a floating gate 20 above a first portion of the channel region 18, a select gate 22 (commonly coupled to the word line WL) above a second portion of the channel region 18, a control gate 28 above the floating gate 20, and an erase gate 30 above the source region 14. This configuration is depicted in U.S. Patent 6,747,310, which is incorporated herein by reference for all purposes. Here, all gates except the floating gate 20 are non - floating gates, meaning that these gates are electrically connected or can be electrically connected to a voltage source. Programming is performed by hot electrons from the channel region 18 injecting themselves onto the floating gate 20. Erasing is performed by electron tunneling from the floating gate 20 to the erase gate 30.

[0014] Table 2 depicts the typical voltage and current ranges that can be applied to the terminals of memory cell 310 for performing read, erase, and program operations: Table 2: Operations of the flash memory cell 310 of FIG. 3 WL / SG BL CG EG SL Read 1.0 - 2V 0.6 - 2V 0 - 2.6V 0 - 2.6V 0V Erase -0.5V / 0V 0V 0V / -8V 8 - 12V 0V Program 1V 0.1 - 1μA 8 - 11V 4.5 - 9V 4.5 - 5V

[0015] FIG. 4 depicts a triple - gate memory cell 410, which is another type of flash memory cell. The memory cell 410 is the same as the memory cell 310 of FIG. 3, except that the memory cell 410 does not have a separate control gate. The erase operation (where erase is performed by using the erase gate) and the read operation are similar to the erase operation and the read operation of FIG. 3, except that no control - gate bias is applied. The program operation is also performed without a control - gate bias, and thus, a higher voltage is applied to the source line during the program operation to compensate for the lack of control - gate bias.

[0016] Table 3 depicts the typical voltage and current ranges that can be applied to the terminals of the memory cell 410 for performing read, erase, and program operations: Table 3: Operation of the Flash Memory Cell 410 of FIG. 4 WL / SG BL EG SL Read 0.7 - 2.2V 0.6 - 2V 0 - 2.6V 0V Erase -0.5V / 0V 0V 11.5V 0V Programmed 1V 0.2 - 3 μA 4.5V 7 - 9V

[0017] Figure 5 depicts a stacked - gate memory cell 510, which is another type of flash memory cell. Memory cell 510 is similar to memory cell 210 of Figure 2, except that the floating gate 20 extends over the entire channel region 18, and the control gate 22 (which will be coupled to the word line here) extends over the floating gate 20 and is separated by an insulating layer (not shown in the figure). The erasure is performed by FN tunneling of electrons from the FG to the substrate, the programming is performed by channel hot electron (CHE) injection at the region between the channel region 18 and the drain region 16, by electrons flowing from the source region 14 towards the drain region 16, and the read operation is similar to the read operation for the memory cell 210 with a higher control gate voltage.

[0018] Table 4 describes the typical voltage ranges that can be applied to the terminals and substrate 12 of the memory cell 510 for performing read, erase, and program operations: Table 4: Operation of the flash memory cell 510 of Figure 5 CG BL SL Substrate Read 2 - 5V 0.6 - 2V 0V 0V Erase - 8 to - 10V / 0V FLT FLT 8 - 10V / 15 - 20V Program 8 - 12V 3 - 5V 0V 0V

[0019] The methods and means depicted herein can be applied to other non-volatile memory technologies, such as but not limited to FINFET isolated gate flash or stacked gate flash memories, NAND flash, silicon oxide-nitride-oxide-silicon (SONOS, charge trapping in nitride), metal-oxide-nitride-oxide-silicon (MONOS, metal charge trapping in nitride), resistive ram (ReRAM), phase change memory (PCM), magnetic ram (MRAM), ferroelectric ram (FeRAM), charge trapping (CT) memory, carbon nanotube (CN) memory, bilayer or multi-level one-time programmable (OTP) and related electronic ram (CeRAM).

[0020] To utilize a memory array that includes one of the non-volatile memory cell types depicted above in an artificial neural network, two modifications are made. First, the lines are configured such that each memory cell can be individually programmed, erased, and read without adversely affecting the memory state of other memory cells in the array, as further explained below. Second, continuous (analog) programming of the memory cells is provided.

[0021] Specifically, the memory state of each memory cell in the array (i.e., the charge on the floating gate) can be continuously changed from a completely erased state to a completely programmed state independently and with minimal interference to other memory cells, and vice versa. This means that the cell storage effectively analogously or at least can store one of a number of discrete values (such as, 16 or 64 different values), which allows for very precise and individual tuning of all the memory cells in the memory array, and which makes the memory array ideal for storing and making fine-tuning adjustments to the synaptic weights of a neural network. Neural network employing a non-volatile memory cell array

[0022] FIG. 6 conceptually depicts a non-limiting example of a neural network utilizing a non-volatile memory array of an embodiment of the present invention. This example uses a non-volatile memory array neural network for a face recognition application, but any other suitable application can be implemented using a neural network based on a non-volatile memory array.

[0023] S0 is the input layer. For this example, the input layer is a 32×32 pixel RGB image with 5-bit precision (i.e., three 32×32 pixel arrays, one for each of the colors R, G, and B, and each pixel is 5-bit precision). In some cases, different sets of weights are applied to the synapses CB1 from the input layer S0 to the layer C1, and in other cases the weights are shared and the input image is scanned with a 3×3 pixel overlapping filter (kernel), shifting the filter by 1 pixel (or more than 1 pixel, as specified by the model). Specifically, the values of the 9 pixels in a 3×3 portion of the image (i.e., what is called the filter or kernel) are provided to the synapse CB1, where these 9 input values are multiplied by appropriate weights, and after summing the multiplied outputs, a single output value is determined and provided by the first synapse of CB1 for generating one of the pixels in the feature map of layer C1. The 3×3 filter is then shifted one pixel to the right within the input layer S0 (i.e., a row of three pixels is added on the right and a row of three pixels is discarded on the left), thereby providing the 9 pixel values in this newly positioned filter to the synapse CB1, where these pixel values are multiplied by the same weights, and a second single output value is determined by the associated synapse. This procedure continues for all three colors and for all bits (precision values) until the 3×3 filter has scanned across the entire 32×32 pixel image of the input layer S0. The procedure is then repeated with different sets of weights to generate different feature maps of layer C1 until all the feature maps of layer C1 have been computed.

[0024] In this example, there are 16 feature maps in layer C1, each feature map having 30×30 pixels. Each pixel is a new feature pixel extracted from the multiplication of the input with the kernel, and thus each feature map is a two-dimensional array, and thus in this example, layer C1 consists of 16 layers of two-dimensional arrays (it should be remembered that the layers and arrays mentioned herein are logical relationships, not necessarily physical relationships - i.e., the arrays do not necessarily tend to be physical two-dimensional arrays). Each of the 16 feature maps in layer C1 is generated by one of sixteen different sets of synapse weights applied to the filter scans. The C1 feature maps can all be for different aspects of the same image feature, such as edge identification. For example, the first map (generated using the first set of weights, shared for all scans used to generate this first map) may identify circular edges, the second map (generated using a second set of weights different from the first set of weights) may identify rectangular edges, or the aspect ratio of certain features, etc.

[0025] The activation function P1 (pooling) is applied before entering layer S1 from layer C1, and it pools the values of consecutive non-overlapping 2×2 regions in each feature map. The purpose of the pooling function P1 is to average the values at nearby positions (or the maximum function can also be used) to, for example, reduce the dependence on edge positions and reduce the data size before entering the next stage. At layer S1, there are 16 feature maps of 15×15 (i.e., sixteen different arrays each with 15×15 pixels). The synaptic CB2 from layer S1 to layer C2 scans the map in layer S1 using a 4×4 filter, with the filter shifted by 1 pixel. At layer C2, there are 22 feature maps of 12×12. The activation function P2 (pooling) is applied before entering layer S2 from layer C2, and it pools the values of consecutive non-overlapping 2×2 regions in each feature map. At layer S2, there are 22 feature maps of 6×6. The activation function (pooling) is applied at the synaptic CB3 from layer S2 to layer C3, where each neuron in layer C3 is connected to each map in layer S2 via the respective synapses of CB3. At layer C3, there are 64 neurons. The synaptic CB4 from layer C3 to the output layer S3 fully connects C3 to S3, that is, each neuron in layer C3 is connected to each neuron in layer S3. The output at layer S3 includes 10 neurons, and the highest output neuron determines the category. This output can, for example, indicate the identification or classification of the content of the original image.

[0026] Each synaptic layer is implemented using an array or a part of an array of non-volatile memory cells.

[0027] Figure 7 is a block diagram of an array that can be used for that purpose. The vector matrix multiplication (VMM) array 32 includes non-volatile memory cells and serves as the synapse between one layer and the next layer (such as CB1, CB2, CB3, and CB4 in Figure 6). Specifically, the VMM array 32 includes an array 33 of non-volatile memory cells, an erase gate and a word line gate decoder 34, a control gate decoder 35, a bit line decoder 36, and a source line decoder 37, which decode the respective inputs to the non-volatile memory cell array 33. The input to the VMM array 32 can come from the erase gate and the word line gate decoder 34 or from the control gate decoder 35. In this example, the source line decoder 37 also decodes the output of the non-volatile memory cell array 33. Alternatively, the bit line decoder 36 can decode the output of the non-volatile memory cell array 33.

[0028] The non-volatile memory cell array 33 serves two purposes. First, it stores the weights to be used by the VMM array 32. Second, the non-volatile memory cell array 33 effectively multiplies the input by the weights stored in the non-volatile memory cell array 33 and sums the results along the output lines (source lines or bit lines) to produce an output that will be the input to the next layer or the input to the final layer. By performing the multiplication and addition functions, the non-volatile memory cell array 33 eliminates the need for separate multiplication and addition logic circuits and is also power-efficient due to its in-situ memory computation.

[0029] The output of the non-volatile memory cell array 33 is supplied to a differential summer (such as a summing operational amplifier or a summing current mirror) 38 that sums the outputs of the non-volatile memory cell array 33 to produce a single value for the convolution. The differential summer 38 is configured to perform the summation of positive and negative weights.

[0030] The total output value of the differential summer 38 is then supplied to an activation function block 39 that corrects the output. The activation function block 39 can provide a sigmoid, tanh, or ReLU function. The corrected output value of the activation function block 39 becomes an element of the feature map for the next layer (e.g., C1 in FIG. 6) and is then applied to the next synapse to produce the next feature map layer or the final layer. Thus, in this example, the non-volatile memory cell array 33 constitutes a plurality of synapses that receive the inputs of the synapses from the previous neuron layer or from an input layer such as an image database, and the summing operational amplifier 38 and the activation function block 39 constitute a plurality of neurons.

[0031] The inputs (WLx, EGx, CGx, and optionally BLx and SLx) to the VMM array 32 in FIG. 7 can be at the analog level, binary level, or digital bit level (in which case, a DAC is provided to convert the digital bit to the appropriate input analog level), and the output can be at the analog level, binary level, or digital bit level (in which case, an output ADC is provided to convert the output analog level to a digital bit).

[0032] FIG. 8 is a block diagram depicting the use of multiple layers of the VMM arrays 32, labeled herein as VMM arrays 32a, 32b, 32c, 32d, and 32e. As shown in FIG. 8, an input, represented as Inputx, is converted from digital to analog by digital-to-analog converter 31 and provided to input VMM array 32a. The converted analog input can be a voltage or a current. The input D / A conversion of the first layer can be performed by using a function or a look-up table (LUT) that maps the input Inputx to an appropriate analog level for the matrix multiplier of the input VMM array 32a. The input conversion can also be performed by an analog-to-analog (A / A) converter to convert an external analog input to a mapped analog input to the input VMM array 32a.

[0033] The output generated by the input VMM array 32a is set as the input to the next VMM array (hidden layer 1) 32b, which in turn generates an output that is set as the input to the next VMM array (hidden layer 2) 32c, and so on. The various layers of the VMM array 32 act as different synaptic layers and neuron layers of a convolutional neural network (CNN). Each of the VMM arrays 32a, 32b, 32c, 32d, and 32e can be a separate physical non-volatile memory array, or multiple VMM arrays can utilize different portions of the same physical non-volatile memory array, or multiple VMM arrays can utilize overlapping portions of the same physical non-volatile memory array. The example shown in FIG. 8 contains five layers (32a, 32b, 32c, 32d, 32e): an input layer (32a), two hidden layers (32b, 32c), and two fully connected layers (32d, 32e). Those of ordinary skill in the art will understand that this is merely an example, and the system alternatively can include more than two hidden layers and more than two fully connected layers. Vector Matrix Multiplication (VMM) Array

[0034] FIG. 9 depicts a neuron VMM array 900, which is particularly suitable for the memory cells 310 as shown in FIG. 3 and serves as the synaptic and neuron portion between the input layer and the next layer. The VMM array 900 includes a memory array 901 of non-volatile memory cells and a reference array 902 of non-volatile reference memory cells (at the top of the array). Alternatively, another reference array can be placed at the bottom.

[0035] In the VMM array 900, control gate lines such as control gate line 903 extend in the vertical direction (thus, the reference array 902 in the column direction is orthogonal to the control gate line 903), and erase gate lines such as erase gate line 904 extend in the horizontal direction. Here, the input to the VMM array 900 is set on the control gate lines (CG0, CG1, CG2, CG3), and the output of the VMM array 900 appears on the source lines (SL0, SL1). In one example, only even columns are used, and in another example, only odd columns are used. The currents placed on each source line (SL0 and SL1 respectively) perform a summation function on all the currents from the memory cells connected to that particular source line.

[0036] As depicted herein for the neural network, the non-volatile memory cells of the VMM array 900, i.e., the memory cells 310 of the VMM array 900, can be configured to operate in the subthreshold region.

[0037] The non-volatile reference memory cells and non-volatile memory cells described herein are biased in weak inversion (subthreshold region): , where where Ids is the drain-to-source current; Vg is the gate voltage on the memory cell; Vth is the threshold voltage of the memory cell; Vt is the thermal voltage = k*T / q, where k is the Boltzmann constant, T is the temperature in Kelvin, and q is the electron charge; n is the slope factor = 1 + (Cdep / Cox), where Cdep = the capacitance of the depletion layer, and Cox is the capacitance of the gate oxide layer; Io is the memory cell current at the gate voltage equal to the threshold voltage, and Io is proportional to (Wt / L)*u*Cox*(n - 1)*Vt2, where u is the carrier mobility of the memory cell, and Wt and L are the width and length respectively.

[0038] For an I-to-V logarithmic converter that uses memory cells (such as reference memory cells or peripheral memory cells) or transistors to convert an input current into an input voltage: where wp is the w of the reference or peripheral memory cell.

[0039] For a memory array used as a vector matrix multiplier VMM array with a current input, the output current is: , that is Here, wa = w of each memory cell in the memory array. Vthp is the effective threshold voltage of the peripheral memory cell, and Vtha is the effective threshold voltage of the main (data) memory cell. It should be noted that the threshold voltage of the transistor is a function of the substrate bias voltage, and the substrate bias voltage denoted as Vsb can be modulated to compensate for various conditions at this temperature. The threshold voltage Vth can be expressed as: where Vth0 is the threshold voltage with zero substrate bias, φF is the surface potential, and γ is the body effect parameter.

[0040] The word line or control gate can be used as the input of the memory cell for the input voltage.

[0041] Alternatively, the flash memory cells of the VMM array depicted herein can be configured to operate in the linear region: ; This means that the weight W in the linear region is proportional to (Vgs - Vth).

[0042] The word line or control gate or bit line or source line can be used as the input of the memory cell operating in the linear region. The bit line or source line can be used as the output of the memory cell.

[0043] For the I to V linear converter, memory cells (such as reference memory cells or peripheral memory cells) or transistors operating in the linear region can be used to linearly convert the input / output current into the input / output voltage.

[0044] Alternatively, the memory cells of the VMM array depicted herein can be configured to operate in the saturation region: ; , this means that the weight W is proportional to (Vgs - Vth) 2.

[0045] A character line, a control gate, or an erase gate can be used as an input to a memory cell operating in the saturation region. A bit line or a source line can be used as an output of a neuron.

[0046] Alternatively, the memory cells of the VMM array depicted herein can be used in all regions or combinations thereof (sub-threshold region, linear region, or saturation region) of one or more layers of a neural network.

[0047] Other examples of the VMM array 32 of FIG. 7 are described in U.S. Patent No. 10,748,630, which is incorporated herein by reference. As described in that application, a source line or a bit line can be used as a neuron output (current summing output).

[0048] FIG. 10 depicts a neuron VMM array 1000, which is particularly suitable for the memory cell 210 as shown in FIG. 2 and serves as a synapse between an input layer and a next layer. The VMM array 1000 includes a memory array 1003 of non-volatile memory cells, a reference array 1001 of first non-volatile reference memory cells, and a reference array 1002 of second non-volatile reference memory cells. The reference arrays 1001 and 1002 disposed in the row direction of the array are used to convert current inputs flowing into terminals BLR0, BLR1, BLR2, and BLR3 into voltage inputs to WL0, WL1, WL2, and WL3. In fact, the first and second non-volatile reference memory cells are diode-connected through multiplexers 1014 (only partially described), into which the current inputs flow. The reference cells are adjusted (e.g., programmed) to a target reference level. The target reference level is provided by a reference small array matrix (not shown in the figure).

[0049] The memory array 1003 is used for two purposes. First, it stores the weights to be used by the VMM array 1000 on its respective memory cells. Second, the memory array 1003 effectively multiplies the inputs (i.e., the current inputs provided at terminals BLR0, BLR1, BLR2, and BLR3, which are converted by the reference arrays 1001 and 1002 into input voltages to be supplied to the word lines WL0, WL1, WL2, and WL3) by the weights stored in the memory array 1003, and then sums all the results (memory cell currents) to generate an output on the respective bit lines (BL0 to BLN), which will be the input to the next layer or the input to the final layer. By performing the multiplication and addition functions, the memory array 1003 eliminates the need for separate multiplication and addition logic circuits and is also power efficient. Here, the voltage inputs are set on the word lines WL0, WL1, WL2, and WL3, and the outputs appear on the respective bit lines BL0 to BLN during a read (inference) operation. The current placed on each of the bit lines BL0 to BLN performs a summing function on the currents from all the non-volatile memory cells connected to that particular bit line.

[0050] Table 5 depicts the operating voltages and currents for the VMM array 1000. The rows in the table indicate the voltages placed on: the word line for the selected cell, the word line for the unselected cell, the bit line for the selected cell, the bit line for the unselected cell, the source line for the selected cell, and the source line for the unselected cell. The columns indicate the operations of read, erase, and program. Table 5: Operations of the VMM array 1000 of FIG. 10: WL WL - unselected BL BL - unselected SL SL - unselected Read 1 - 3.5V -0.5V / 0V 0.6 - 2V (Ineuron) 0.6V - 2V / 0V 0V 0V Erase ~5 - 13V 0V 0V 0V 0V 0V Stylized 1 - 2V -0.5V / 0V 0.1 - 3 uA Vinh ~ 2.5V 4 - 10V 0 - 1V / FLT

[0051] Figure 11 depicts a neuron VMM array 1100, which is particularly suitable for the memory cell 210 as shown in Figure 2, and serves as the synaptic and neuron part between the input layer and the next layer. The VMM array 1100 includes a memory array 1103 of non-volatile memory cells, a reference array 1101 of first non-volatile reference memory cells, and a reference array 1102 of second non-volatile reference memory cells. The reference arrays 1101 and 1102 extend in the column direction of the VMM array 1100. The VMM array is similar to the VMM 1000, except that in the VMM array 1100 the word lines extend in the vertical direction. Here, the inputs are set on the word lines (WLA0, WLB0, WLA1, WLB1, WLA2, WLA2, WLA3, WLB3), and the outputs appear on the source lines (SL0, SL1) during a read operation. The current placed on each source line performs a summation function on all the currents from the memory cells connected to that particular source line.

[0052] Table 6 depicts the operating voltages and currents for the VMM array 1100. The rows in the table indicate the voltages placed on: the word line for the selected cell, the word line for the unselected cell, the bit line for the selected cell, the bit line for the unselected cell, the source line for the selected cell, and the source line for the unselected cell. The columns indicate the operations of read, erase, and program. Table 6: Operations of the VMM Array 1100 in Figure 11 WL WL - unselected BL BL - unselected SL SL - unselected Read 1 - 3.5V -0.5V / 0V 0.6 - 2V 0.6V - 2V / 0V ~0.3 - 1V (Ineuron) 0V Erase ~5 - 13V 0V 0V 0V 0V SL - Forbidden (~4 - 8V) Program 1 - 2V -0.5V / 0V 0.1 - 3 uA Vinh ~2.5V 4 - 10V 0 - 1V / FLT

[0053] Figure 12 depicts a neuron VMM array 1200, which is particularly suitable for the memory cell 310 as shown in Figure 3, and serves as the synapse and neuron part between the input layer and the next layer. The VMM array 1200 includes a memory array 1203 of non - volatile memory cells, a reference array 1201 of first non - volatile reference memory cells, and a reference array 1202 of second non - volatile reference memory cells. The reference arrays 1201 and 1202 are used to convert the current inputs flowing into the terminals BLR0, BLR1, BLR2, and BLR3 into voltage inputs CG0, CG1, CG2, and CG3. In fact, the first and second non - volatile reference memory cells are diode - connected through - multiplexers 1212 (only partially shown), where the current inputs flow into these multiplexers via BLR0, BLR1, BLR2, and BLR3. Each multiplexer 1212 includes a respective multiplexer 1205 and a stacked transistor 1204 to ensure a constant voltage on the bit line (such as BLR0) of each of the first and second non - volatile reference memory cells during a read operation. The reference cells are tuned to the target reference levels.

[0054] The memory array 1203 serves two purposes. First, it stores the weights to be used by the VMM array 1200. Second, the memory array 1203 effectively multiplies the inputs (the current inputs provided to terminals BLR0, BLR1, BLR2, and BLR3, where reference arrays 1201 and 1202 convert these current inputs into input voltages to supply to the control gates (CG0, CG1, CG2, and CG3)) by the weights stored in the memory array, and then sums all the results (cell currents) to produce an output that appears on BL0 to BLN and will be the input to the next layer or the input to the final layer. By performing the multiplication and addition functions, the memory array eliminates the need for separate multiplication and addition logic circuits and is also power efficient. Here, the inputs are set on the control gate lines (CG0, CG1, CG2, and CG3), and the outputs appear on the bit lines (BL0 to BLN) during a read operation. The current placed on each bit line performs a summation function on all the currents from the memory cells connected to that particular bit line.

[0055] The VMM array 1200 performs unidirectional tuning on the non-volatile memory cells in the memory array 1203. That is, each non-volatile memory cell is erased and then partially programmed until the desired charge on the floating gate is reached. If too much charge is placed on the floating gate (such that an incorrect value is stored in the cell), the cell is erased and the sequence of the partial programming operation is restarted. As shown, two columns sharing the same erase gate (such as EG0 or EG1) are erased together (this is known as page erase), and thereafter, each cell is partially programmed until the desired charge on the floating gate is reached.

[0056] Table 7 depicts the operating voltages and currents for the VMM array 1200. The rows in the table indicate the voltages placed on the following: the word line for the selected cell, the word line for the unselected cell, the bit line for the selected cell, the bit line for the unselected cell, the control gate for the selected cell, the control gate for the unselected cell in the same sector as the selected cell, the control gate for the unselected cell in a different sector from the selected cell, the erase gate for the selected cell, the erase gate for the unselected cell, the source line for the selected cell, and the source line for the unselected cell. The columns indicate the read, erase, and program operations. Table 7: Operation of the VMM array 1200 in FIG. 12 WL WL- Unselected BL BL- Not selected CG CG - not Same selection One sector CG - not Selected EG EG - not Selected SL SL - not Selected Read 1.0 - 2V - 0.5V / 0V 0.6 - 2V (Ineuron) 0V 0 - 2.6V 0 - 2.6V 0 - 2.6V 0 - 2.6V 0 - 2.6V 0V 0V Erase 0V 0V 0V 0V 0V 0 - 2.6V 0 - 2.6V 5 - 12V 0 - 2.6V 0V 0V Program 0.7 - 1V - 0.5V / 0V 0.1 - 1uA Vinh (1 - 2V) 4 - 11V 0 - 2.6V 0 - 2.6V 4.5 - 5V 0 - 2.6V 4.5 - 5V 0 - 1V

[0057] FIG. 13 depicts a neuron VMM array 1300, which is particularly suitable for the memory cell 310 as shown in FIG. 3, and serves as the synapse and neuron part between the input layer and the next layer. The VMM array 1300 includes a memory array 1303 of non-volatile memory cells, a reference array 1301 of first non-volatile reference memory cells, and a reference array 1302 of second non-volatile reference memory cells. The EG lines EGR0, EG0, EG1, and EGR1 extend vertically, while the CG lines CG0, CG1, CG2, and CG3 and the WL lines WL0, WL1, WL2, and WL3 extend horizontally. The VMM array 1300 is similar to the VMM array 1400, except that the VMM array 1300 implements bidirectional tuning, where due to the use of separate EG lines, each individual cell can be fully erased, partially programmed, and partially erased as needed to achieve the desired charge amount on the floating gate. As shown, the reference arrays 1301 and 1302 convert the input current in the terminals BLR0, BLR1, BLR2, and BLR3 into the control gate voltages CG0, CG1, CG2, and CG3 to be applied to the memory cells in the column direction (operation via the diode-connected reference cell multiplexer 1314). The current outputs (neurons) are in the bit lines BL0 to BLN, where each bit line sums all the currents from the non-volatile memory cells connected to that particular bit line.

[0058] Table 8 depicts the operating voltages and currents for the VMM array 1300. The rows in the table indicate the voltages placed on the following: the word line for the selected cell, the word line for the unselected cell, the bit line for the selected cell, the bit line for the unselected cell, the control gate for the selected cell, the control gate for the unselected cell in the same sector as the selected cell, the control gate for the unselected cell in a different sector from the selected cell, the erase gate for the selected cell, the erase gate for the unselected cell, the source line for the selected cell, and the source line for the unselected cell. The columns indicate the operations of read, erase, and program. Table 8: Operations of the VMM Array 1300 of FIG. 13 WL WL - unselected Selected BL BL - unselected Selected CG CG - unselected Selected same A sector CG- Unselected EG EG - Un Selected SL SL - Un Selected Read 1.0 - 2V - 0.5V / 0V 0.6 - 2V (Ineuron) 0V 0 - 2.6V 0 - 2.6V 0 - 2.6V 0 - 2.6V 0 - 2.6V 0V 0V Erase 0V 0V 0V 0V 0V 4 - 9V 0 - 2.6V 5 - 12V 0 - 2.6V 0V 0V Program 0.7 - 1V - 0.5V / 0V 0.1 - 1uA Vinh (1 - 2V) 4 - 11V 0 - 2.6V 0 - 2.6V 4.5 - 5V 0 - 2.6V 4.5 - 5V 0 - 1V

[0059] FIG. 22 depicts a neuron VMM array 2200, which is particularly suitable for the memory cell 210 as shown in FIG. 2, and serves as the synaptic and neuronal part between the input layer and the next layer. In the VMM array 2200, the inputs INPUT 0, …, INPUT N are respectively received on bit lines BL 0, …, BL N, and the outputs OUTPUT 1, OUTPUT 2, OUTPUT 3, and OUTPUT 4 are respectively generated on source lines SL 0, SL 1, SL 2, and SL 3.

[0060] FIG. 23 depicts a neuron VMM array 2300, which is particularly suitable for the memory cell 210 as shown in FIG. 2, and serves as the synaptic and neuronal part between the input layer and the next layer. In this example, the inputs INPUT 0, INPUT 1, INPUT 2, and INPUT 3 are respectively received on source lines SL 0, SL 1, SL 2, and SL 3, and the outputs OUTPUT 0, …, OUTPUT N are generated on bit lines BL 0, …, BL N.

[0061] FIG. 24 depicts a neuron VMM array 2400, which is particularly suitable for the memory cell 210 as shown in FIG. 2, and serves as the synaptic and neuronal part between the input layer and the next layer. In this example, the inputs INPUT 0, …, INPUT M are respectively received on word lines WL 0, …, WL M, and the outputs OUTPUT 0, …, OUTPUT N are generated on bit lines BL 0, …, BL N.

[0062] FIG. 25 depicts a neuron VMM array 2500, which is particularly suitable for the memory cell 310 as shown in FIG. 3, and serves as the synaptic and neuronal part between the input layer and the next layer. In this example, the inputs INPUT 0, ..., INPUT M are respectively received on word lines WL 0, ..., WL M, and the outputs OUTPUT 0, ..., OUTPUT N are generated on bit lines BL 0, ..., BL N.

[0063] FIG. 26 depicts a neuron VMM array 2600, which is particularly suitable for the memory cell 410 as shown in FIG. 4, and serves as the synaptic and neuronal part between the input layer and the next layer. In this example, the inputs INPUT 0, ..., INPUT N are respectively received on vertical control gate lines CG 0, ..., CG N, and the outputs OUTPUT 1 and OUTPUT 2 are generated on source lines SL 0 and SL 1.

[0064] Figure 27 depicts a neuron VMM array 2700, which is particularly suitable for the memory cell 410 as shown in FIG. 4, and serves as the synaptic and neuronal part between the input layer and the next layer. In this example, the inputs INPUT 0, …, INPUT N are respectively received on the gates of bit line control gates 2701-1, 2701-2, …, 2701-(N-1) and 2701-N, and these gates are respectively coupled to bit lines BL 0, …, BL N. The example outputs OUTPUT 1 and OUTPUT 2 are generated on source lines SL 0 and SL 1.

[0065] Figure 28 depicts a neuron VMM array 2800, which is particularly suitable for the memory cell 310 as shown in FIG. 3, the memory cell 510 as shown in FIG. 5, and the memory cell 710 as shown in FIG. 7, and serves as the synaptic and neuronal part between the input layer and the next layer. In this example, the inputs INPUT 0, …, INPUT M are received on word lines WL 0, …, WL M, and the outputs OUTPUT 0, …, OUTPUT N are respectively generated on bit lines BL 0, …, BL N.

[0066] Figure 29 depicts a neuron VMM array 2900, which is particularly suitable for the memory cell 310 as shown in FIG. 3, the memory cell 510 as shown in FIG. 5, and the memory cell 710 as shown in FIG. 7, and serves as the synaptic and neuronal part between the input layer and the next layer. In this example, the inputs INPUT 0, ..., INPUT M are received on control gate lines CG 0, ..., CG M. The outputs OUTPUT 0, ..., OUTPUT N are respectively generated on vertical source lines SL 0, ..., SL N, where each source line SL i is coupled to the source lines of all the memory cells in row i.

[0067] Figure 30 depicts a neuron VMM array 3000, which is particularly suitable for the memory cell 310 as shown in FIG. 3, the memory cell 510 as shown in FIG. 5, and the memory cell 710 as shown in FIG. 7, and serves as the synaptic and neuronal part between the input layer and the next layer. In this example, the inputs INPUT 0, ..., INPUT M are received on control gate lines CG 0, ..., CG M. The outputs OUTPUT 0, ..., OUTPUT N are respectively generated on vertical bit lines BL 0, ..., BL N, where each bit line BL i is coupled to the bit lines of all the memory cells in row i. Long short-term memory

[0068] The prior art includes the concept known as long short-term memory (LSTM). LSTM cells are often used in neural networks. LSTM allows a neural network to remember information over an arbitrary predetermined time interval and use that information in subsequent operations. Conventional LSTM cells include a cell, an input gate, an output gate, and a forget gate. The three gates regulate the flow of information into and out of the cell and the time interval for remembering information in the LSTM. The VMM is particularly applicable to LSTM cells.

[0069] Figure 14 depicts an example LSTM 1400. The LSTM 1400 in this example includes cells 1401, 1402, 1403, and 1404. Cell 1401 receives the input vector x0 and generates the output vector h0 and the cell state vector c0. Cell 1402 receives the input vector x1, the output vector (hidden state) h0 from cell 1401, and the cell state c0 from cell 1401, and generates the output vector h1 and the cell state vector c1. Cell 1403 receives the input vector x2, the output vector (hidden state) h1 from cell 1402, and the cell state c1 from cell 1402, and generates the output vector h2 and the cell state vector c2. Cell 1404 receives the input vector x3, the output vector (hidden state) h2 from cell 1403, and the cell state c2 from cell 1403, and generates the output vector h3. Additional cells may be used, and an LSTM with four cells is only an example.

[0070] Figure 15 depicts an exemplary implementation of an LSTM cell 1500, which can be used for cells 1401, 1402, 1403, and 1404 in Figure 14. The LSTM cell 1500 receives the input vector x(t), the cell state vector c(t-1) from the previous cell, and the output vector h(t-1) from the previous cell, and generates the cell state vector c(t) and the output vector h(t).

[0071] The LSTM cell 1500 includes sigmoid function components 1501, 1502, and 1503, each of which applies a number between 0 and 1 to control the amount of each component in the input vector that is allowed to pass through the output vector. The LSTM cell 1500 also includes hyperbolic tangent components 1504 and 1505 for applying the hyperbolic tangent function to the input vector, multiplier components 1506, 1507, and 1508 for multiplying two vectors together, and adder component 1509 for adding two vectors together. The output vector h(t) can be provided to the next LSTM cell in the system, or the output vector can be accessed for other purposes.

[0072] Figure 16 depicts the LSTM cell 1600, which is an example implementation of the LSTM cell 1500. For the convenience of the reader, the same numbers from the LSTM cell 1500 are used in the LSTM cell 1600. The sigmoid function components 1501, 1502, and 1503, and the hyperbolic tangent component 1504 each include a plurality of VMM arrays 1601 and activation function blocks 1602. Thus, it can be seen that the VMM arrays are particularly suitable for use in LSTM cells in certain neural network systems. The multiplier components 1506, 1507, and 1508, and the adder component 1509 are implemented digitally or analogously. The activation function blocks 1602 can be implemented digitally or analogously.

[0073] An alternative to the LSTM cell 1600 (and another example implementation of the LSTM cell 1500) is shown in Figure 17. In Figure 17, the sigmoid function components 1501, 1502, and 1503, and the hyperbolic tangent component 1504 share the same physical hardware (VMM array 1701 and activation function block 1702) in a time-division multiplexing manner. The LSTM cell 1700 also includes: a multiplier component 1703 for multiplying two vectors together; an adder component 1708 for adding two vectors together; a hyperbolic tangent component 1505 (which includes the activation function block 1702); a register 1707 for storing the value i(t) when it is output from the sigmoid function block 1702; a register 1704 for storing the value when the value f(t) * c(t-1) is output from the multiplier component 1703 via the multiplexer 1710; a register 1705 for storing the value when the value i(t) * u(t) is output from the multiplier component 1703 via the multiplexer 1710; and a register 1706 for storing the value when the value o(t) * c~(t) is output from the multiplier component 1703 via the multiplexer 1710 and the multiplexer 1709.

[0074] The LSTM cell 1600 contains multiple sets of VMM arrays 1601 and individual activation function blocks 1602, while the LSTM cell 1700 contains only one set of VMM array 1701 and activation function block 1702, which is used to represent multiple layers in an example of the LSTM cell 1700. The LSTM cell 1700 will require less space compared to the LSTM 1600 because the LSTM cell 1700 will require 1 / 4 the space for the VMM and activation function blocks compared to the LSTM cell 1600.

[0075] It can be further understood that an LSTM cell will typically include multiple VMM arrays, each of which requires functionality provided by some circuit blocks external to the VMM array, such as a summing unit, an activation function block, and a high voltage generation block. Providing individual circuit blocks to each VMM array will require a large amount of space within the semiconductor component and will be slightly inefficient. Therefore, the examples described below reduce the circuitry required outside of the VMM array itself. Gated Recurrent Unit

[0076] The analog VMM implementation can be used in a Gated Recurrent Unit (GRU) system. A GRU is a gating mechanism in a recurrent neural network. A GRU is similar to an LSTM, except that a GRU cell typically contains fewer components than an LSTM cell.

[0077] FIG. 18 depicts an exemplary GRU 1800. The GRU 1800 in this example includes cells 1801, 1802, 1803, and 1804. Cell 1801 receives an input vector x0 and generates an output vector h0. Cell 1802 receives an input vector x1, the output vector h0 from cell 1801, and generates an output vector h1. Cell 1803 receives an input vector x2 and the output vector (hidden state) h1 from cell 1802, and generates an output vector h2. Cell 1804 receives an input vector x3 and the output vector (hidden state) h2 from cell 1803 and generates an output vector h3. Additional cells can be used, and a GRU with four cells is only an example.

[0078] FIG. 19 depicts an exemplary implementation of a GRU cell 1900, which can be used in cells 1801, 1802, 1803, and 1804 of FIG. 18. The GRU cell 1900 receives an input vector x(t) and the output vector h(t−1) from the previous GRU cell, and generates an output vector h(t). The GRU cell 1900 includes sigmoid function components 1901 and 1902, each of which applies a number between 0 and 1 to components from the output vector h(t−1) and the input vector x(t). The GRU cell 1900 also includes a hyperbolic tangent component 1903 for applying a hyperbolic tangent function to the input vector, a plurality of multiplier components 1904, 1905, and 1906 for multiplying two vectors together, an adder component 1907 for adding two vectors together, and a complement component 1908 for subtracting the input from 1 to produce an output.

[0079] Figure 20 depicts GRU cell 2000, which is an example implementation of GRU cell 1900. For the convenience of the reader, the same numbers from GRU cell 1900 are used in GRU cell 2000. As can be seen in Figure 20, sigmoid function components 1901 and 1902, and hyperbolic tangent component 1903 each contain multiple VMM arrays 2001 and activation function blocks 2002. Thus, it can be seen that VMM arrays are particularly used in GRU cells for use in certain neural network systems. Multiplier components 1904, 1905, 1906, adder component 1907, and complement component 1908 are implemented digitally or analogously. Activation function block 2002 can be implemented digitally or analogously.

[0080] An alternative to GRU cell 2000 (and another example implementation of GRU cell 1900) is shown in Figure 21. In Figure 21, GRU cell 2100 utilizes VMM array 2101 and activation function block 2102, which applies a number between 0 and 1 when configured as a sigmoid function to control the amount by which each component in the input vector is allowed through to the output vector. In Figure 21, sigmoid function components 1901 and 1902, and hyperbolic tangent component 1903 share the same physical hardware (VMM array 2101 and activation function block 2102) in a time-division multiplexing manner. GRU cell 2100 also includes: multiplier component 2103, which is used to multiply two vectors together; adder component 2105, which is used to add two vectors together; complement component 2109, which is used to subtract the input from 1 to produce an output; multiplexer 2104; register 2106, which is used to save the value when the value h(t - 1) * r(t) is output from multiplier component 2103 via multiplexer 2104; register 2107, which is used to save the value when the value h(t - 1) * z(t) is output from multiplier component 2103 via multiplexer 2104; and register 2108, which is used to save the value when the value h^(t) * (1 - z(t)) is output from multiplier component 2103 via multiplexer 2104.

[0081] GRU cell 2000 contains multiple sets of VMM array 2001 and activation function block 2002, while GRU cell 2100 contains only one set of VMM array 2101 and activation function block 2102, which is used to represent multiple layers in an instance of GRU cell 2100. GRU cell 2100 will require less space compared to GRU cell 2000, because GRU cell 2100 will require one-third the space for VMM and activation function blocks compared to GRU cell 2000.

[0082] It can be further understood that the GRU system will typically include multiple VMM arrays, each of which requires functionality provided by certain circuit blocks external to the VMM array, such as summing circuits, activation function blocks, and high-voltage generation blocks. Providing separate circuit blocks to each VMM array would require a large amount of space within the semiconductor component and would be slightly inefficient. Therefore, the examples described below reduce the circuitry required outside of the VMM array itself.

[0083] The input to the VMM array can be at the analog level, binary level, pulse, time-modulated pulse, or digital bit (in which case a DAC is required to convert the digital bit to an appropriate input analog level), and the output can be at the analog level, binary level, timing pulse, pulse, or digital bit (in which case an output ADC is required to convert the output analog level to a digital bit).

[0084] Generally, for each memory cell in the VMM array, each weight W can be implemented by a single memory cell, or a differential cell, or two hybrid memory cells (the average of two cells). In the case of a differential cell, two memory cells are required to implement the weight W as a differential weight (W = W+ - W-). In two hybrid memory cells, two memory cells are required to implement the weight W as the average of two cells.

[0085] Figure 31 depicts a VMM system 3100. In some examples, the weights W stored in the VMM array are stored as a differential pair W+ (positive weight) and W- (negative weight), where W = (W+) - (W-). In the VMM system 3100, one half-bit line is designated as the W+ line, i.e., the bit line connected to the memory cell that will store the positive weight W+, and the other half-bit line is designated as the W- line, i.e., the bit line connected to the memory cell that implements the negative weight W-. The W- lines are interleaved among the W+ lines in an alternating fashion. The subtraction operation is performed by a summing circuit that receives currents from the W+ line and the W- line, such as summing circuits 3101 and 3102. The outputs of the W+ line and the W- line are combined together, effectively yielding W = W+ - W- for each pair (W+, W-) of cells for all pairs (W+, W-) of lines. Although the above has been described with the W- lines interleaved among the W+ lines in an alternating fashion, in other examples, the W+ lines and the W- lines can be located anywhere in the array arbitrarily.

[0086] FIG. 32 depicts another example. In the VMM system 3210, positive weights W+ are implemented in the first array 3211 and negative weights W- are implemented in the second array 3212, the second array 3212 being separated from the first array, and the resulting weights are appropriately combined together by a summing circuit 3213.

[0087] FIG. 33 depicts the VMM system 3300. The weights W stored in the VMM array are stored as a differential pair W+ (positive weight) and W- (negative weight), where W = (W+) - (W-). The VMM system 3300 includes an array 3301 and an array 3302. Half of the bit lines in each of the arrays 3301 and 3302 are designated as W+ lines, i.e., the bit lines connected to the memory cells that will store the positive weight W+, and the other half of the bit lines in each of the arrays 3301 and 3302 are designated as W- lines, i.e., the bit lines connected to the memory cells that implement the negative weight W-. The W- lines are interleaved among the W+ lines in an alternating manner. The subtraction operation is performed by a summing circuit that receives currents from the W+ lines and the W- lines, such as summing circuits 3303, 3304, 3305, and 3306. The outputs of the W+ lines and the outputs of the W- lines from each of the arrays 3301, 3302 are combined together respectively, so that for each (W+, W-) cell pair of all (W+, W-) line pairs, effectively W = W+ - W- is obtained. In addition, the W values from each of the arrays 3301 and 3302 can be further combined via summing circuits 3307 and 3308, such that each W value is the result of the W value from array 3301 minus the W value from array 3302, which means the final result from summing circuits 3307 and 3308 is the difference value of two difference values.

[0088] Each non-volatile memory cell in the analog neural memory system needs to be erased and programmed to maintain a very specific and accurate amount of charge, i.e., the number of electrons, in the floating gate. For example, each floating gate should store one of N different values, where N is the number of different weights that can be indicated by each cell. Examples of N include 16, 32, 64, 128, and 256.

[0089] To ensure the accuracy of the VMM system, it is crucial to compensate for the natural variations that occur. For example, the I-V slope can vary within the same memory array due to natural process variations, and it can change with the operating temperature. The I-V slope refers to the relationship between the currents drawn by the memory cell when a voltage is applied to a terminal such as the control gate terminal of the memory cell. It is desirable to be able to compensate for such changes in the I-V slope of part or all of the memory array. SUMMARY OF THE INVENTION

[0090] Describes a large number of examples for providing a neural network system, the neural network system including a plurality of reference arrays with a meaningful variation range, such that an appropriate part of the reference array closely approaching the operation array can be selected for operation. Brief description of the drawings

[0091] FIG. 1 is a diagram illustrating an artificial neural network.

[0092] FIG. 2 depicts a prior art isolated gate flash memory cell.

[0093] FIG. 3 depicts another prior art isolated gate flash memory cell.

[0094] FIG. 4 depicts another prior art isolated gate flash memory cell.

[0095] FIG. 5 depicts another prior art isolated gate flash memory cell.

[0096] FIG. 6 is a diagram illustrating different levels of an exemplary artificial neural network using one or more non-volatile memory arrays.

[0097] FIG. 7 is a block diagram illustrating a VMM system.

[0098] FIG. 8 is a block diagram illustrating an exemplary artificial neural network using one or more VMM systems.

[0099] FIG. 9 depicts another example of a VMM system.

[0100] FIG. 10 depicts another example of a VMM system.

[0101] FIG. 11 depicts another example of a VMM system.

[0102] FIG. 12 depicts another example of a VMM system.

[0103] FIG. 13 depicts another example of a VMM system.

[0104] FIG. 14 depicts a prior art long short-term memory system.

[0105] Figure 15 depicts an exemplary cell for use in a long short-term memory system.

[0106] Figure 16 depicts an exemplary implementation of the cell of Figure 15.

[0107] Figure 17 depicts another exemplary implementation of the cell of Figure 15.

[0108] Figure 18 depicts a prior art gated recurrent unit system.

[0109] Figure 19 depicts an exemplary cell for use in a gated recurrent unit system.

[0110] Figure 20 depicts an exemplary implementation of the cell of Figure 19.

[0111] Figure 21 depicts another exemplary implementation of the cell of Figure 19.

[0112] Figure 22 depicts another example of a VMM system.

[0113] Figure 23 depicts another example of a VMM system.

[0114] Figure 24 depicts another example of a VMM system.

[0115] Figure 25 depicts another example of a VMM system.

[0116] Figure 26 depicts another example of a VMM system.

[0117] Figure 27 depicts another example of a VMM system.

[0118] Figure 28 depicts another example of a VMM system.

[0119] Figure 29 depicts another example of a VMM system.

[0120] Figure 30 depicts another example of a VMM system.

[0121] Figure 31 depicts another example of a VMM system.

[0122] Figure 32 depicts another example of a VMM system.

[0123] Figure 33 depicts another example of a VMM system.

[0124] Figure 34 depicts an example of a VMM system.

[0125] Figure 35 depicts an example of a VMM array and a reference array.

[0126] Figure 36 depicts an example of a VMM array and a reference array.

[0127] Figure 37 depicts an example of a VMM array and a reference array.

[0128] Figure 38 depicts an example of a reference array.

[0129] Figure 39 depicts an example of a reference array.

[0130] Figures 40A and 40B depict the physical layout of an analog array and an embedded reference array.

[0131] Figure 41 depicts a method for selecting a reference array.

[0132] Figures 42A and 42B depict an example of data of cells with different I-V slopes.

[0133] Figure 43 depicts an example of the physical layout of memory cells.

[0134] Figure 44 depicts an example of a calibration circuit.

[0135] Figure 45 depicts a column DAC bias generator.

[0136] Figure 46 depicts a column DAC bias generator.

[0137] Figure 47 depicts a global column DAC decoder.

[0138] Figure 48 depicts a global column DAC decoder.

[0139] Figure 49 depicts an exemplary layout of a reference array.

[0140] Figure 50 depicts an exemplary layout of a reference array.

[0141] Figure 51 depicts an exemplary layout of a reference array. Embodiment

[0142] VMM System Architecture []

[0143] Figure 34 depicts a block diagram of a VMM system 3400. The VMM system 3400 includes a VMM array 3401, a column decoder 3402, a high voltage decoder 3403, a row decoder 3404, a bit line driver 3405, an input circuit 3406, an output circuit 3407, control logic 3408, and a bias generator 3409. The VMM system 3400 further includes a high voltage generation block 3410, which includes a charge pump 3411, a charge pump regulator 3412, and a high voltage level generator 3413. The VMM system 3400 further includes a (programming / erasing, or weight tuning) algorithm controller 3414, analog circuitry 3415, a control engine 3416 (which may include, but is not limited to, special functions such as arithmetic functions, activation functions, embedded microcontroller logic), and test control logic 3417.

[0144] The input circuit 3406 may include circuits such as a digital-to-analog converter (DAC), a digital-to-pulse converter (DPC, digital-to-time-modulated pulse converter), an analog-to-analog converter (AAC, such as a current-to-voltage converter, a logarithmic converter), a pulse-to-analog level converter (PAC), or any other type of converter. The input circuit 3406 may implement one or more of normalization, linear or non-linear scaling / downscaling functions, or arithmetic functions. The input circuit 3406 may implement a temperature compensation function for the input level. The input circuit 3406 may implement an activation function such as ReLU or sigmoid.

[0145] The output circuit 3407 may include circuits such as an analog-to-digital converter (ADC) which is used to convert the neuron analog output into digital bits, an analog-to-analog converter (AAC) such as a current-to-voltage converter, a logarithmic converter, an analog-to-pulse converter (APC), an analog-to-time-modulated pulse converter, or any other type of converter. The output circuit 3407 may implement an activation function such as a rectified linear activation function (ReLU) or a sigmoid. The output circuit 3407 may implement one or more of statistical normalization, regularization, scaling / downscaling / gain functions, statistical rounding, or arithmetic functions (e.g., addition, subtraction, division, multiplication, shift, logarithm) for the neuron output. The output circuit 3407 may implement a temperature compensation function for the neuron output or an array output (such as a bitline output) so as to keep the power consumption of the array approximately constant or improve the accuracy of the array (neuron) output such as by keeping the IV slope substantially the same.

[0146] FIGS. 35 and 36 depict embodiments of a reference array embedded in the same physical array as the VMM array.

[0147] FIG. 35 depicts a physical array 3500. The physical array 3500 selectively includes an array of non-volatile memory cells. The non-volatile memory cells may selectively include stacked-gate flash memory cells or split-gate flash memory cells.

[0148] The physical array 3500 is divided into two types of arrays, the VMM array 3401 (as in FIG. 34) and the reference array 3501. The reference array 3501 includes a plurality of reference arrays as shown in FIGS. 38 and 39. In one embodiment, the VMM array 3401 and the reference array 3501 share the same bitlines. In another embodiment, the VMM array 3401 and the reference array 3501 use separate sets of bitlines where the two sets of bitlines are disconnected. Exemplary physical layouts are shown in FIGS. 40A and 40B.

[0149] FIG. 36 depicts a physical array 3600 which is divided into two arrays, the VMM array 3401 and the reference array 3501. In one embodiment, the VMM array 3401 and the reference array 3501 share one or more sets of horizontal lines such as wordlines, control gate lines, and erase lines. In another embodiment, the VMM array 3401 and the reference array 3501 do not share any sets of horizontal lines.

[0150] FIG. 37 depicts an embodiment where the reference array 3501 and the VMM array 3401 are located in separate physical arrays. For example, there may be substrate separation or active diffusion separation between the two arrays. The physical array 3701 contains the VMM array 3401, and the physical array 3702 contains the reference array 3501. The VMM array 3401 and the reference array 3501 do not share any bit lines, word lines, control gate lines, or erase lines.

[0151] FIG. 38 depicts an embodiment of the reference array 3501. Here, the reference array 3501 includes a plurality of reference arrays, such as reference arrays 3801-0, 3801-1,..., 3801-(n-1), and 3801-n. Thus, the reference array 3501 includes n + 1 different reference arrays. The different reference arrays may have different characteristics that result in I-V curves that are distinct for each individual reference array from those of other reference arrays. For example, each individual reference array may vary in one or more of the following dimensions: (1) the width of the control gate line of the transistors in each individual reference array; (2) the width of the word line of the transistors in each individual reference array; (3) the width of the floating gate of the transistors in each individual reference array; (4) the total width of the non-volatile memory cells in each individual reference array; (5) the shallow trench isolation (STI) spacing within each individual reference array; or (6) other characteristics. Additionally, the reference arrays may vary respectively in one or more component implantation conditions or doping characteristics (such as, but not limited to, well implantation conditions, source implantation conditions, drain implantation conditions).

[0152] FIG. 39 depicts another embodiment of the reference array 3501. Here, the reference array 3501 includes a plurality of reference arrays, such as reference arrays 3901-0, 3901-1,..., 3901-(n-1), 3901-n, and 3902-0, 3902-1,..., 3902-(n-1), and 3902-n. Thus, the reference array 3501 includes 2*(n + 1) different reference arrays. As in FIG. 38, the different reference arrays may have different characteristics that result in I-V curves that are distinct for each individual reference array from those of other reference arrays. For example, each individual reference array may vary in one or more of the following dimensions: control gate width, word line width, floating gate width, total width of the non-volatile memory cells in the array, STI spacing, and component implantation conditions, but not limited to these.

[0153] FIG. 40A depicts a physical layout of a VMM array and a reference array. The VMM system 4001 includes a VMM array 4002 and a reference array 4004. An exemplary bit line BL0 is coupled to the VMM array 4002, and an exemplary bit line BLR0 is coupled to the reference array 4004. Since the embodiments use metal layers M1 to M4 in the VMM array 4002, the bit line BL0 coupled to the VMM array 4002 can be referred to as the VMM array bit line BL0, and the metal layers M2 to M4 are also connected to peripheral circuitry such as a row decoder, and the embodiments use only the metal layer M1 as an example, the bit line BLR0 coupled to the reference array 4004 can be referred to as the reference array bit line BLR0. The VMM array 4002 and the reference array 4004 are part of the same physical array 4003. The physical array 4003 includes a substrate 4005 and a diffusion layer 4006, both of which are shared by the VMM array 4002 and the reference array 4004. The physical array 4003 is electrically segmented for the VMM array 4002 and the reference array 4004 by using disconnected metal interconnects. For example, as can be seen, both the VMM array metal bit line BL0 and the reference array metal bit line BLR0 use the metal layer M1 but are disconnected from each other. Using the same physical array 4003 for both the VMM array 4002 and the reference array 4004 reduces any physical impacts from process uniformity and area overhead that would occur when using separate physical arrays.

[0154] FIG. 40B depicts another specific example of a physical layout for a VMM array and a reference array, the physical layout being similar to the physical layout in FIG. 40A, except that in this VMM system 4051, the diffusion layer 4056 of the VMM array 4052 and the diffusion layer 4057 of the reference array 4004 are separate.

[0155] FIG. 41 depicts a reference array selection method 4100 executed to calibrate a VMM system. The method 4100 can be executed during a manufacturing or testing process, or it can be executed during real-time operation of the VMM system. In operation 4101, a plurality of reference arrays are selected as candidates for a calibration process. For example, a plurality of reference arrays are selected as candidates for adjusting the I-V (current-voltage) slope of the VMM array. In operation 4102, for the plurality of reference arrays, individual metrics are determined. For example, the metric can be a target circuit performance metric or a target neural network performance metric. In operation 4103, the reference array among the plurality of reference arrays that produces a metric closest to the target value is identified. In operation 4104, the reference array identified in operation 4103 is used to calibrate the VMM array.

[0156] Figure 42A depicts data retrieved from a memory array having different physical characteristics. Figure 42B depicts data retrieved from a memory array having different electrical characteristics (such as by well implant). As can be seen, the I-V slope of the memory cell can vary based on physical or electrical variations within the memory cell.

[0157] Figure 43 depicts an embodiment of the physical layout of the reference memory cell 4300 in the reference array 3501, where we can understand that the dimensions of the control gate width, word line width, floating gate width, total width, and STI spacing can vary based on the manufacturing process and layout used. These are examples of the characteristics mentioned above with reference to FIGS. 38 and 39, which can vary from reference array to reference array.

[0158] Figures 44 to 48 provide examples of how the reference array 3501 can be used to calibrate the VMM array 3401.

[0159] Figure 44 depicts an exemplary column bias calibration circuit 4400, which includes a current digital-to-analog converter (DAC) 4401, an operational amplifier 4402, and a reference memory cell 4403. The reference memory cell 4403 can be a reference cell in the reference array of FIGS. 35 to 39. The column bias calibration circuit 4400 provides a reference CG bias. The CG bias can then be directly applied to the CG terminals in the VMM array to provide a bias voltage across those cells, or the CG bias can be used in a larger system, such as the column DAC bias generator 4500 in FIG. 45, the column DAC bias generator 4600 in FIG. 46, the input block 4700 in FIG. 47, or the input block 4800 in FIG. 48, which in turn generates a bias voltage that can be applied to the CG terminals or other terminals in the VMM array to provide a bias voltage across those cells.

[0160] The reference memory cell 4403 can specifically be from the reference array selected in operation 4103 of the method 4100 in FIG. 41. The input DIN[0:n] selects the current generated by the current DAC 4401. The current generated is the target current that the memory cells in the VMM array 3401 would ideally generate to store the weights corresponding to the value of DIN[0:n]. For example, if DIN is an 8-bit value, there will be 256 possible DIN values and 256 individual current levels that would ideally be generated across the memory cells in the VMM array 3401 storing the weight corresponding to that value during a read operation.

[0161] The voltage CGbias on the terminal CG of the output reference memory cell 4403 of the operational amplifier 4402. Based on the inherent properties of the operational amplifier, the operational amplifier 4402 adjusts this output voltage until the voltage on its inverting input terminal is equal to the voltage VREF on its non-inverting input terminal (which can be any reference value, such as 0.5V). The output voltage of the operational amplifier 4402 is sampled and can be applied as a bias voltage to the control gate terminal of the memory cells in the VMM array 3401, or alternatively as an analog input to the array. Alternatively, CGbias can be used in systems such as the column DAC bias generator 4500 in FIG. 45, the column DAC bias generator 4600 in FIG. 46, the input block 4700 in FIG. 47, or the input block 4800 in FIG. 48.

[0162] The reference cell 4403 can be a single cell or can include multiple cells. For example, multiple reference cells can be used to obtain an average effect or to reflect the weight distribution of a neural network layer. For example, multiple reference cells can be selected such that 50% of the reference cells reflect low weight values, 20% of the reference cells reflect intermediate weight values, and 30% of the reference cells reflect high weight values, in an attempt to approximate the typical weight distribution in the VMM array in an artificial neural network.

[0163] Alternatively, for example, an example of the column bias calibration circuit 4400 can be used to generate different CGbias levels, such as high, medium, and low CGbias levels. For example, a low CGbias level (VREFL) can be generated using multiple reference cells (e.g., 1 cell, 2 cells,..., 32 cells), where each cell has a target current of 3 nA (generated by the current digital-to-analog converter 4401), and this target current represents the first level among the 32 levels of a 5-bit cell memory cell; a high CGbias level (VREFH) can be generated using multiple reference cells (e.g., 1 cell, 2 cells,..., 32 cells), where each cell has a target current of 96 nA, and this target current represents the 32nd level among the 32 levels of a 5-bit cell memory cell; and an intermediate CGbias level (VREFM) can be generated using multiple reference cells (e.g., 1 cell, 2 cells,..., 32 cells), where each cell has a target current of 48 nA, and this target current represents the 16th level among the 32 levels of a 5-bit cell memory cell. In this way, three instances of the column bias calibration circuit 4400 can be used to generate a high voltage VREFH, an intermediate voltage VREFM, and a low voltage VREFL.

[0164] For the case of serial DAC inputs (such as inputs that include a sequence of digital input bits applied one at a time, where the corresponding digital outputs are shifted and added for each binary input bit position) or timing inputs (pulse width modulation inputs or pulse count inputs), the CG bias voltage applied to the VMM array can be provided by the column bias calibration circuit 4400 of FIG. 44.

[0165] FIG. 45 depicts a column DAC bias generator 4500, which includes a voltage ladder 4501, a mapping block 4502, and an output buffer 4503. The voltage ladder 4501 generates a plurality of voltage levels (L0, ..., Lk), which respectively supply voltages to the VMM array to operate the VMM array in a linear mode or a sub-threshold mode depending on the selected voltage mode (linear or sub-threshold). For example, for sub-threshold value operation, the voltage ladder 4501 can have a resolution of 8 to 14 bits. Control logic (not shown) controls the operation of the column DAC bias generator 4500 (by performing functions such as enabling various blocks using an enable signal such as EN, thereby providing control signals to multiplexers, and other control functions).

[0166] The mapping block 4502 converts the levels from the voltage ladder 4501 into respective analog outputs, such as voltages representing 8-bit outputs, and feeds the output of the mapping block 4502 to the output buffer 4503.

[0167] The output of the output buffer 4503 is depicted as DAC_OUT 4590, which is then applied as a bias voltage (such as on the CG terminal) to a column of memory cells in the VMM array to change the I-V slope of those cells. At least when the array operates in the sub-threshold region, local trimming for each level (represented by trimming blocks L0_trm, ..., Ln_trm) is set within the mapping block 4502. Applying this bias voltage to, for example, the control gate line of the memory cell column will achieve a non-linear I-V slope of the memory cells in the VMM array within the temperature range in the sub-threshold region. It can be understood that DAC_OUT 4590 is generated using multiple reference arrays (for example, one reference array to generate VREFH and another reference array to generate VREFL).

[0168] The voltage ladder 4501 receives a high reference voltage VREFH (such as VREFH generated by way of example by the column bias calibration circuit 4400 in FIG. 44 described above) and a low reference voltage VREFL (such as VREFL generated by way of example by the column bias calibration circuit 4400 in FIG. 44 described above). VREFH corresponds to the highest possible value in the cells that can be stored in the VMM array, and VREFL corresponds to the lowest possible value in the cells that can be stored in the VMM array. The voltage ladder 4501 includes a plurality of resistors for selectively generating a series of voltages between VREFL and VREFH according to a linear function or a logarithmic function. For example, the top node of the top resistor in the voltage ladder will have the voltage VREFH, and due to the voltage drop across the top resistor, the bottom of the resistor will have a lower voltage. The bottom node of the bottom resistor will have the voltage VREFL. The voltage ladder thereby generates a plurality of voltage levels (L0, ..., Lk) (for example, k can be 4095), which are required when it is desired to supply a voltage to the VMM array to operate the non-volatile memory cells of the VMM array in a linear mode or a sub-threshold mode.

[0169] The mapping block 4502 receives a digital input DIN[n:0], which is used to select one of m + 1 analog voltages using a sub-block 4563 inside the mapping block 4502, where (m + 1) = 2^(n + 1). For example, if (n + 1) = 8, then (m + 1) = 256. The mapping block 4502 includes (m + 1) trimming blocks 4562 and (m + 1) multiplexers 4563. The mapping block 4502 converts the k + 1 voltage levels from the DAC 4501 into respective analog outputs corresponding to DIN[n:0]. This is achieved by local trimming of each level (represented by trimming blocks L0_trm, ..., Lm_trm), which can be useful, for example, when the non-volatile memory cells in the array are operating in the sub-threshold region. This is desired to achieve an optimal matching I-V slope for the non-volatile memory cells in the VMM array within the temperature in the sub-threshold region or the linear region.

[0170] By adjusting the reference voltages VREFL and VREFH, the range of the k + 1 levels is also adjusted. This is, for example, to match the output range of this input block with the input range of the memory cells. This can also perform temperature compensation by adjusting the reference levels VREFL and VREFH (such as shifting lower at high temperatures and higher at low temperatures) to achieve the desired range of the gate bias of the memory cells that may be required due to temperature variations. Further individual level adjustment and temperature compensation are performed by the level trimming circuit of the mapping block 4502, for example, for optimal neural network accuracy.

[0171] Figure 46 depicts column DAC bias generator 4600. Column DAC bias generator 4600 is similar to column DAC bias generator 4500, except that voltage ladder 4601 is used instead of voltage ladder 4501. Voltage ladder 4601 is similar to voltage ladder 4501, with an additional intermediate reference level VREFM added (such as VREFM generated by way of example by column bias calibration circuit 4400 in FIG. 44 described above). The adjustment of the VREFM reference level and the control of the temperature behavior are similar to the adjustment of the VREFH and VREFL reference levels and their temperature behavior. It includes an additional buffer 4603 to apply the intermediate reference level to the ladder to further adjust the I-V slope matching between GDAC 5480 and the inputs of the memory array. Although voltage ladder 4601 uses three voltage reference levels (VREFH, VREFM, and VREFL), it is understood that other numbers of voltage reference levels may alternatively be used.

[0172] The reference voltages are generated from reference memory cells 4403 of n+1 reference arrays 3801 or 3901, from the CGbias output of the example of column bias calibration circuit 4400 in FIG. 44. Column DAC bias generator 4600 provides the bias voltage (DAC_OUT 4602) to be applied to the columns of the VMM array.

[0173] Figure 47 depicts exemplary input block 4700 for providing inputs to a VMM array, such as VMM array 3401 in FIG. 34. Input block 4700 includes: a global digital-to-analog converter (DAC) 4701; address column decoders 4702-0 to 4702-n, each corresponding to one of the columns numbered 0 to n in the VMM array; column registers 4703-0 to 4703-n, each corresponding to one of the columns numbered 0 to n in the VMM array; digital comparator blocks 4704-0 to 4704-n, each corresponding to one of the columns numbered 0 to n in the VMM array; column sample-and-hold (S / H) buffers 4705-0 to 4705-n, each corresponding to one of the columns numbered 0 to n in the VMM array; output signals 4706-0 to 4706-n, each corresponding to one of the columns numbered 0 to n in the VMM array; and a counter 4707.

[0174] The global DAC 4701 may optionally include the column DAC bias generator 4500 in FIG. 45 or the column DAC bias generator 4600 in FIG. 46, having the reference voltage levels VREFH, VREFL, and optionally VREFM provided by the example of the column bias calibration circuit 4400 in FIG. 44.

[0175] Each set of column registers 4703, digital comparators 4704, and column sample and hold buffers 4705 for a particular column can be regarded as a local digital-to-analog converter for that particular column, where the voltage supply for that local digital converter is provided by the global DAC 4701, as explained below.

[0176] The address column decoders 4702-0 to 4702-n receive the column address ADD[n:0] and the enable signal EN. When ADD[n:0] is the address of that particular column and when EN is established, the output of each address column decoder, designated as ENROW, is high. The column registers 4703-0 to 4703-n are loaded with the respective digital input bits DINx (where x is the number of bits, such as 8 or 16 bits) of that particular column, where the loading operation is triggered by the clock signal CLK, where DINx is the active input for that particular column to perform the vector multiplication matrix operation. When the output of a particular address column decoder 4702 is high, the associated column register 4703 is enabled and outputs its digital bit DINx. When enabled by the signal EN, the counter 4707 counts the pulses in another clock signal CLKB. The output of the counter 4707 is the count value. The digital comparator block 4704-0 to 4704-n compares the active value DINx stored in each of the respective column registers 4703 with the count value. If the count value matches the value stored in a particular column register 4702, then each respective digital comparator block 4704 enables the corresponding column S / H buffer 4705 to sample and hold the analog output value from the global DAC 4701. The global DAC 4701 performs a digital-to-analog conversion on the count value (which also matches the DINx of the column register 4703 for the column under discussion). Each column S / H buffer 4705 outputs the held analog value as the output signal 4706. If, for example, x = 8 and DINx is an 8-bit input (meaning there are 256 different values for DINx), then the counter 4707 will count up to 256 and then reset. At this time, it will have covered all possible values of DINx, and each column S / H buffer 4705 will have been loaded with its associated value DINx.

[0177] For example, the output signal 4706 can be applied to the control gate line or the word line during a programmed or read operation in that particular column or during a neural read operation that reads all columns. During neural read, all the S / H buffers 4705 are enabled to drive the array input terminals via respective output signals 4706, resulting in bit line currents being output from the VMM array, which are then processed by output circuits such as current-to-voltage converter (ITV) circuits and analog-to-digital converter (ADC) circuits.

[0178] Each respective output signal 4706 can be applied to a respective control gate line or word line, for example, during a programmed or read operation in that particular column.

[0179] In another embodiment, the column sample-and-hold buffer 4704 can be shared by multiple columns in a time-division multiplexed manner.

[0180] FIG. 48 depicts an exemplary input block 4800 for providing inputs to a VMM array, such as the VMM array 3401 in FIG. 34. The input block 4800 is similar to the input block 4400 in FIG. 44, but provides multiple outputs from a global digital-to-analog converter for neural read operations. The input block 4800 includes: a global digital-to-analog converter 4801; address column decoders 4802-0 to 4802-n, each corresponding to a respective column among the columns numbered 0 to n; column registers 4803-0 to 4803-n, each corresponding to a respective column among the columns numbered 0 to n; digital comparators 4804-0 to 4804-n, each corresponding to a respective column among the columns numbered 0 to n; column sample-and-hold buffers 4805-0 to 4805-n, each corresponding to a respective column among the columns numbered 0 to n; and output signals 4806-0 to 4806-n, each corresponding to a respective column among the columns numbered 0 to n.

[0181] The global DAC 4801 can optionally include the column DAC bias generator 4500 in FIG. 45 or the column DAC bias generator 4600 in FIG. 46, with reference voltage levels VREFH, VREFL, and optionally VREFM provided by the example of the column bias calibration circuit 4400 in FIG. 44.

[0182] Address decoders 4802-0 to 4802-n receive column address ADD[n:0] and enable signal EN. When ADD[n:0] is the address of that particular column and when EN is asserted, the output of each address column decoder 4802, designated as ENROW, is high. Column registers 4803-0 to 4803-n load the respective digital input bits DINx (where x is the number of bits, such as 8 or 16 bits) of that particular column, where the loading is triggered by clock signal CLK, and where DINx is the active input of that particular column. When the output ENROW of a particular address column decoder 4802 is high, the associated column register 4803 is enabled and outputs its digital bits DINx. When enabled by signal EN, counter 4807 counts the pulses in another clock signal CLKB. The output of counter 4807 is the count value. Digital comparator blocks 4804-0 to 4804-n compare the active value DINx stored in each of the respective column registers 4803 with the count value. If the count value matches the value stored in a particular column register 4802, the corresponding column S / H buffer 4805 is enabled by the respective digital comparator block 4804 to sample and hold the analog output value from GDAC 4801. As shown, there are two vertical analog output lines from GDAC 4801. For example, for an 8-bit GDAC 4801, one line can deliver outputs of 0 to 127 analog levels (corresponding to 00000000 to 01111111), and the other line can deliver outputs of 128 to 255 analog levels (corresponding to 10000000 to 11111111). The two lines can operate simultaneously to reduce the column DAC sampling time from 256 (DAC) clocks to 128 (DAC) clocks. Each column S / H buffer 4805 receives only one of the outputs from GDAC 4801. For example, output signal 4806 can be applied to a control gate line or a word line during a programmed operation in that particular column.

[0183] Optionally, global digital-to-analog converter 4801 can include a first global DAC for odd columns and a second global DAC for even columns.

[0184] FIG. 49 depicts an exemplary layout of the reference array 3501. The reference array 3501 includes reference arrays 4901-0, 4901-1, 4901-2, and 4901-3, which are coupled to respective common bit lines in the same metal layer (such as metal layer M1). For example, each reference array 4901 can implement the low level (VREFL), middle level (VREFM), or high level (VREFH) CG reference bias described above with respect to FIGS. 44 to 46, but is not limited thereto. When a specific bit line is used for one reference array, the cells in another reference array connected to the bit line are depth-programmed so that they do not conduct any current and are substantially deactivated. In this way, each bit line can be effectively dedicated to only one of the reference arrays.

[0185] FIG. 50 depicts an exemplary layout of the reference array 3501. The reference array 3501 includes reference arrays 5001-0, 5001-1, 5001-2, and 5001-3, which are respectively coupled to respective bit lines in different metal layers (such as metal layers M4, M3, M2, and M1). For example, each reference array 4901 implements the low level (VREFL), middle level (VREFM), or high level (VREFH) CG reference bias described above with respect to FIGS. 44 to 46, but is not limited thereto.

[0186] FIG. 51 depicts an exemplary layout of the reference array 3501. The reference array 3501 includes reference arrays 5101-0, 5101-1, 5101-2, and 5101-3, which are bit lines respectively coupled to the same metal layer (such as metal layer M1). For example, each reference array 4901 implements the low level (VREFL), middle level (VREFM), or high level (VREFH) CG reference bias described above with respect to FIGS. 44 to 46, but is not limited thereto. Here, each bit line is only electrically coupled to one of the reference arrays and is electrically disconnected from the other reference arrays.

[0187] It should be noted that, as used herein, the terms "above" and "on" both inclusively include "directly on" (with no intermediate material, element, or space therebetween) and "indirectly on" (with intermediate material, element, or space therebetween). Similarly, the term "adjacent" includes "directly adjacent" (with no intermediate material, element, or space therebetween) and "indirectly adjacent" (with intermediate material, element, or space therebetween), "mounted to" includes "directly mounted to" (with no intermediate material, element, or space therebetween) and "indirectly mounted to" (with intermediate material, element, or space therebetween), and "electrically coupled" includes "directly electrically coupled to" (with no intermediate material or element electrically connecting the components together) and "indirectly electrically coupled to" (with intermediate material or element electrically connecting the components together). For example, forming an element "above a substrate" may include forming the element directly on the substrate with no intermediate material / element therebetween, and forming the element indirectly on the substrate with one or more intermediate materials / elements therebetween.

[0188] 12: Semiconductor substrate 14: Source region 16: Drain region 18: Channel region 20: Floating gate 22: Word line terminal 24: Bit line 28: Control gate 30: Erase gate 31: Digital-to-analog converter 32, 32a, 32b, 32c, 32d, 32e: Vector matrix multiplication array 33: Non-volatile memory cell array 34: Erase gate and word line gate decoder 35: Control gate decoder 36: Bit line decoder 37: Source line decoder 38: Differential summer 39, 1602, 1702, 2002, 2102: Activation function block 210, 310, 410, 510: Memory cell 900, 1000, 1100, 1200, 1300, 1601, 1701, 2001, 2101, 2200, 2300, 2400, 2500, 2600, 2700, 2800, 2900, 3000, 3401, 4002: VMM array 901, 1003, 1103, 1203, 1303: Memory array 902, 1001, 1002, 1101, 1102, 1201, 1202, 1301, 1302, 3501, 3801-0, 3801-1,..., 3801-(n-1), 3801-n, 3901-0, 3901-1,..., 3901-(n-1), 3901-n, 3902-0, 3902-1,..., 3902-(n-1), 3902-n, 4004, 4901-0, 4901-1, 4901-2, 4901-3, 5001-0, 5001-1, 5001-2, 5001-3, 5101-0, 5101-1, 5101-2, 5101-3: Reference array 903: Control gate line 904: Erase gate line 1012, 1014: Diode-connected through multiplexer 1204: Stacked transistor 1205: Multiplexer 1314: Diode-connected reference cell through multiplexer 1400: LSTM 1401, 1402, 1403, 1404, 1801, 1802, 1803, 1804: Cells 1500, 1600, 1700: LSTM cells 1501, 1502, 1503, 1901, 1902: Sigmoid function components 1504, 1505, 1903: Hyperbolic tangent components 1506, 1507, 1508, 1703, 1904, 1905, 1906, 2103: Multiplier components 1509, 1708, 1907, 2105: Adder components 1704, 1705, 1706, 1707, 2106, 2107, 2108: Registers 1709, 1710, 2104: Multiplexers 1800: GRU 1900, 2000, 2100: GRU cells 1908, 2109: Complementary components 2701-1, 2701-2, …, 2701-(N-1), 2701-N: Bit line control gates 3100, 3210, 3300, 3400, 4001: VMM system 3101, 3102, 3213, 3303, 3304, 3305, 3306, 3307, 3308: Summing circuit 3211: First array 3212: Second array 3301, 3302: Arrays 3402: Column decoder 3403: High voltage decoder 3404: Row decoder 3405: Bit line driver 3406: Input circuit 3407: Output circuit 3408: Control logic 3409: Bias generator 3410: High voltage generation block 3411: Charge pump 3412: Charge pump regulator 3413: High voltage level generator 3414: Algorithm controller 3415: Analog circuit system 3416: Control engine 3417: Test control logic 3500, 3600, 3701, 3702, 4003: Physical arrays 4005: Substrate 4006, 4056, 4057: Diffusion layer 4100: Reference array selection method 4101, 4102, 4103, 4104: Operations 4300: Reference memory cell 4400: Column bias calibration circuit 4401: Current digital-to-analog converter 4402: Operational amplifier 4403: Reference memory cell 4500, 4600: Column DAC bias generator 4501, 4601: Voltage ladders 4502: Mapping block 4503: Output Buffer 4562_0, 4562_1, ..., 4562_m-1, 4562_m: Trimming Blocks 4563_0, 4563_1, ..., 4563_m-1, 4563_m: Sub-Blocks / Multiplexers 4590, 4602: DAC_OUT 4063: Additional Buffer 4700, 4800: Input Blocks 4701, 4801: Global Digital-to-Analog Converters 4702-0, 4702-1, ..., 4702-(n-1), 4702-n, 4802-0, 4802-1, ..., 4802-(n-1), 4802-n: Address Column Decoders 4703-0, 4703-1, ..., 4703-(n-1), 4703-n, 4803-0, 4803-1, ..., 4803-(n-1), 4803-n: Column Registers 4704-0, 4704-1, ..., 4704-(n-1), 4704-n, 4804-0, 4804-1, ..., 4804-(n-1), 4804-n: Digital Comparator Blocks 4705-0, 4705-1, ..., 4705-(n-1), 4705-n, 4805-0, 4805-1, ..., 4805-(n-1), 4805-n: Column Sample-and-Hold Buffers 4706-0, 4706-1, ..., 4706-(n-1), 4706-n, 4806-0, 4806-1, ..., 4806-(n-1), 4806-n: Output Signals 4707: Counter ADD[n:0]: Column Address BL0, BL1, BL2, BL3, …, BLN, BLBR0: Bit Lines BLR0, BLR1, BLR2, BLR3, CG: Terminals c 0, c 1, c 2, c 3, c(t-1), c(t): Cell State Vectors C1, C2, C3, S1, S2, S3: Layers CB1, CB2, CB3, CB4: Synapse CG0, CG1, CG2, CG3, CG M-1, CG M: Control gate line / Control gate voltage CGbias: Voltage CLK, CLKB: Clock signal DIN [n:0]: Digital input DINx: Digital input bit EG0, EG1: Erase gate / EG line EGR0, EGR1: EG line EN: Enable signal h 0, h 1, h 2, h 3, h(t-1), h(t): Output vector INPUT 0, INPUT 1, …, INPUT N-1, INPUT N, INPUT M-1, INPUT M, DIN[0:n]: Input L0, L1, …, Lk-1, Lk: Voltage level M1, M2, M3, M4: Metal layer OUTPUT 0, OUTPUT 1, OUTPUT 2, OUTPUT 3, OUTPUT 4, OUTPUT N-1, OUTPUT N, ENROW: Output P1, P2: Activation function S0: Input layer S1, S2, S3: Layer SL0, SL1, SL2, SL3: Source line VREFH: High voltage VREFL: Low voltage VREFM: Intermediate voltage WL0, WL1, WL2, WL3, WL4, WL5, WL6, WL7, WL M-1, WL M, WLA0, WLB0, WLA1, WLB1, WLA2, WLB2, WLA3, WLB3: Word line x 0, x 1, x 2, x 3, x(t): Input vector

Claims

1. A reference array selection method, comprising: determining individual metrics of a plurality of reference arrays; selecting one of the reference arrays whose determined metric is closest to a target value; and using the identified reference array to generate an input bias voltage for a vector matrix multiplication array in an artificial neural network.

2. As in request item 1, where, The plurality of reference arrays and the artificial neural network array are located in the same physical array.

3. As in request item 1, where, The plurality of reference arrays and the artificial neural network array are located in different physical arrays.

4. As in request item 1, where, Each of the plurality of reference arrays has different electrical characteristics.

5. As in request item 1, where, The individual reference arrays in the plurality of reference arrays differ in one doping characteristic.

6. As in request item 1, where, Calibration involves generating a bias voltage and applying that bias voltage to the memory cells in the vector matrix multiplication array.

7. As in request item 1, wherein, The reference array generates a bias voltage for the array of neural networks.

8. As in request item 1, where, The plurality of reference arrays and the artificial neural network array contain non-volatile memory cells.

9. As in request item 8, wherein, These non-volatile memory cells are stacked gate flash memory cells.

10. As in request item 8, wherein, These non-volatile memory cells are discrete gate flash memory cells.

11. An artificial neural network system comprising: a vector matrix multiplication array; and a plurality of reference arrays, wherein one or more of the plurality of reference arrays are selected and used to generate a bias input applied to one of the vector matrix multiplication arrays during operation.

12. The system as described in request item 11, wherein, This vector matrix multiplication array is a neural network array.

13. The system as described in request item 11, wherein, The complex array of reference arrays and the vector matrix multiplication array contain non-volatile memory cells.

14. The system as described in request item 13, wherein, These non-volatile memory cells are stacked gate flash memory cells.

15. The system as described in request item 13, wherein, These non-volatile memory cells are discrete gate flash memory cells.

16. An artificial neural network system comprising: a plurality of reference arrays, wherein one or more of the plurality of reference arrays are selected and used to generate a bias voltage of one of a vector matrix multiplication arrays during operation.

17. The system as described in request item 16, wherein, The plurality of reference arrays are coupled to bit lines in the same metal layer.

18. The system as described in request item 16, wherein, The cells in the multiple reference arrays are deeply programmed.

19. The system as described in request item 16, wherein, The plurality of reference arrays are coupled to bit lines in different metal layers.

20. The system as described in request item 19, wherein, The multiple reference arrays use the same metal layer to route bit lines to a peripheral circuit system.

Images