Determining the bias voltage to apply to one or more memory cells in a neural network.
Non-volatile memory arrays in artificial neural networks address the inefficiencies of CMOS synapses by enabling precise synaptic weight tuning and efficient analog calculations, enhancing energy efficiency and scalability.
Patent Information
- Application Number
- JP2025138394
- Authority / Receiving Office
- JP · JP
- Patent Type
- Patents
- Current Assignee / Owner
- Priority Date
- 2022-01-26
- Filing Date
- 2025-08-21
- Publication Date
- 2026-11-11
- Estimated Expiration
- 2042-01-27
AI Technical Summary
Existing artificial neural networks face challenges in achieving high-performance information processing due to the lack of suitable hardware technology, particularly in terms of energy efficiency and scalability, as CMOS-implemented synapses are too bulky for the large number of neurons and synapses required.
Utilizing non-volatile memory arrays as synapses in artificial neural networks, allowing for individual and continuous programming of memory cells with minimal disruption, enabling precise tuning of synaptic weights and eliminating the need for separate multiplication and addition logic circuits.
This approach enhances energy efficiency and scalability by performing high-precision analog calculations, reducing the need for separate multiplication and addition logic circuits, and enabling fine-tuning of synaptic weights in neural networks.
Smart Images

Figure 0007949123000010 
Figure 0007949123000011 
Figure 0007949123000012
Abstract
Description
[Technical Field]
[0001] (Claiming priority) This application claims priority to U.S. Provisional Patent Application No. 63 / 279,028, filed November 12, 2021, entitled "Optimization of Analog Neural Memory in a Deep Learning Artificial Neural Network as to Performance, Power, or Temperature," and U.S. Patent Application No. 17 / 585,452, filed January 26, 2022, entitled "Determination of a Bias Voltage to Apply to One or More Memory Cells in a Neural Network."
[0002] (Field of Invention) Numerous embodiments are disclosed for improving analog neural memory in deep learning artificial neural networks with respect to performance or power consumption in various temperature environments. [Background technology]
[0003] Artificial neural networks mimic biological neural networks (such as the central nervous system of animals, particularly the brain) and are used to estimate or approximate functions that may depend on numerous inputs and are generally unknown. Artificial neural networks typically consist of layers of interconnected "neurons" that exchange messages with one another.
[0004] Figure 1 illustrates an artificial neural network, where circles illustrate layers of inputs or neurons. Connections (called synapses) are represented by arrows and have numerical weights that can be tuned based on experience. This allows the neural network to adapt to inputs and learn. Typically, a neural network contains multiple layers of inputs. Typically, there are one or more hidden layers of neurons and output layers of neurons that provide the output of the neural network. Neurons at each level make decisions individually or collectively based on the data they receive from synapses.
[0005] One of the major challenges in developing artificial neural networks for high-performance information processing is the lack of suitable hardware technology. In fact, practical neural networks rely on a very large number of synapses, which enables high connectivity between neurons and thus very high levels of parallel processing. In principle, such complexity can be achieved with digital supercomputers or dedicated graphics processing unit clusters. However, in addition to their high cost, these approaches also suffer from poor energy efficiency compared to biological networks, which consume far less energy because they primarily perform low-precision analog calculations. While CMOS analog circuits have been used in artificial neural networks, most CMOS-implemented synapses are too bulky given the large number of neurons and synapses required.
[0006] The applicant previously disclosed, in U.S. Patent Application No. 15 / 594,439, incorporated by reference, an artificial (analog) neural network utilizing one or more non-volatile memory arrays as synapses. The non-volatile memory arrays operate as analog neural memories. The neural network device includes a first plurality of synapses configured to receive a first plurality of inputs and therefrom produce a first plurality of outputs, and a first plurality of neurons configured to receive the first plurality of outputs. The first plurality of synapses includes a plurality of memory cells, each of which includes spaced source and drain regions formed in a semiconductor substrate with a channel region extending between them, a floating gate isolated and disposed above a first portion of the channel region, and a non-floating gate isolated and disposed above a second portion of the channel region. Each of the plurality of memory cells is configured to store weight values corresponding to many electrons in the floating gate. The plurality of memory cells are configured to produce a first plurality of outputs by multiplying the first plurality of inputs by the stored weight values. <Non-volatile skin cell>
[0007] Non-volatile memory is well known. For example, U.S. Patent No. 5,029,130 ("'130"), incorporated herein by reference, discloses an array of split-gate non-volatile memory cells, a type of flash memory cell. Such a memory cell 210 is shown in Figure 2. Each memory cell 210 includes a source region 14 and a drain region 16 formed in a semiconductor substrate 12, with a channel region 18 between the source region 14 and the drain region 16. A floating gate 20 is formed insulated above a first portion of the channel region 18 (and controlling the conductivity of the first portion of the channel region 18) and extends above a portion of the source region 14. A word line terminal 22 (typically coupled to a word line) has a first portion disposed insulated above a second portion of the channel region 18 (and controlling the conductivity of the second portion of the channel region 18) and a second portion extending upward above the floating gate 20. The floating gate 20 and word line terminal 22 are insulated from the substrate 12 by the gate oxide. The bit line 24 is coupled to the drain region 16.
[0008] By applying a high-voltage positive voltage to the word line terminal 22, erasure is performed on the memory cell 210 (electrons are removed from the floating gate), causing the electrons in the floating gate 20 to pass through the insulator between them to the word line terminal 22 via a Fowler-Nordheim (FN) tunnel.
[0009] The memory cell 210 is programmed by source-side injection (SSI) using hot electrons by applying a positive voltage to the word line terminal 22 and a positive voltage to the source region 14 (electrons are added to the floating gate). The electron flow flows from the drain region 16 towards the source region 14. The electrons are accelerated and generate heat when they reach the gap between the word line terminal 22 and the floating gate 20. Some of the heated electrons are injected into the floating gate 20 via the gate oxide due to the electrostatic attraction from the floating gate 20.
[0010] The memory cell 210 is read by applying a positive read voltage to the drain area 16 and the word line terminal 22 (turning on the portion of the channel area 18 below the word line terminal). When the floating gate 20 is positively charged (i.e., electrons are erased), the portion of the channel area 18 below the floating gate 20 is also turned on, and current flows through the channel area 18, which is detected as the erased state, or the "1" state. When the floating gate 20 is negatively charged (i.e., programmed with electrons), the portion of the channel area below the floating gate 20 is almost or completely off, and no (or very little) current flows through the channel area 18, which is detected as the programmed state, or the "0" state.
[0011] Table 1 shows typical voltage and current ranges that may be applied to the terminals of the memory cell 110 to perform read, erase, and program operations. Table 1: Figure 2 Operation of flash memory cell 210 [Table 1]
[0012] Other types of flash memory cells, other split-gate memory cell configurations, are also known. For example, Figure 3 shows a four-gate memory cell 310 including a source region 14, a drain region 16, a floating gate 20 above a first portion of the channel region 18, a selection gate 22 (typically coupled to a word line, WL) above a second portion of the channel region 18, a control gate 28 above the floating gate 20, and an erase gate 30 above the source region 14. This configuration is described in U.S. Patent No. 6,747,310, which is incorporated herein by reference for all purposes. Here, all gates are non-floating gates, except for the floating gate 20; that is, they are electrically connected to or can be connected to a voltage source. Programming is performed by heated electrons injecting themselves from the channel region 18 into the floating gate 20. Erasing is performed by electrons tunneling from the floating gate 20 to the erase gate 30.
[0013] Table 2 shows typical voltage and current ranges that may be applied to the terminals of the memory cell 310 to perform read, erase, and program operations. Table 2: Operation of the flash memory cell 310 in Figure 3 [Table 2]
[0014] Figure 4 shows a different type of flash memory cell, a 3-gate memory cell 410. Memory cell 410 is identical to memory cell 310 in Figure 3, except that memory cell 410 does not have a separate control gate. The erase operation (erasure occurs through the use of an erase gate) and read operation are the same as those in Figure 3, except that no control gate bias is applied. The programming operation is also performed without a control gate bias; therefore, during the programming operation, a higher voltage must be applied to the source line to compensate for the lack of control gate bias.
[0015] Table 3 shows typical voltage / current ranges that can be applied to the terminals of memory cell 410 to perform a read operation, an erase operation, and a program operation. Table 3: Operation of the flash memory cell 410 of FIG. 4 [Table 3]
[0016] FIG. 5 shows a stacked-gate memory cell 510, which is another type of flash memory cell. The memory cell 510 is similar to the memory cell 210 of FIG. 2, except that the floating gate 20 extends over the entire channel region 18, and the control gate 22 (here coupled to the word line) extends over the floating gate 20, separated by an insulating layer (not shown). Erasing is performed by FN tunneling of electrons from the FG to the substrate, and programming is performed by channel hot electron (CHE) injection in the region between the channel 18 and the drain region 16, by electrons flowing from the source region 14 toward the drain region 16, and by a read operation that is similar to the read operation of the memory cell 210 having a higher control gate voltage.
[0017] Table 4 shows typical voltage ranges that can be applied to the terminals of the memory cell 510 and the substrate 12 to perform a read operation, an erase operation, and a program operation. Table 4: Operation of the flash memory cell 510 of FIG. 5 [Table 4]
[0018] The methods and means described in this specification can be applied to, but are not limited to, FINFET split gate flash or stacked gate flash memory, NAND flash, SONOS (silicon-oxide-nitride-oxide-silicon, charge trap in nitride), MONOS (metal-oxide-nitride-oxide-silicon, metal charge trap in nitride), ReRAM (resistive ram, resistive change memory), PCM (phase change memory), MRAM (ferroelectric ram, magnetoresistive memory), FeRAM (ferroelectric ram, ferroelectric memory), CT (charge trap) memory, CN (carbon-tube) memory, OTP (one time programmable, bilevel or multilevel one-time programmable only), and CeRAM (correlated electron ram, correlated electron memory) and other non-volatile memory technologies.
[0019] Two modifications are made to utilize a memory array including one of the types of non-volatile memory cells in the above artificial neural network. First, as further explained below, the lines are configured so that each memory cell can be individually programmed, erased, and read without adversely affecting the memory state of other memory cells in the array. Second, continuous (analog) programming of the memory cells is provided.
[0020] Specifically, the memory state (i.e., the charge on the floating gate) of each memory cell in the array can be changed independently and continuously, with minimal disruption to other memory cells, from a completely erased state to a completely programmed state. In another embodiment, the memory state (i.e., the charge on the floating gate) of each memory cell in the array can be changed independently and continuously, with minimal disruption to other memory cells, from a completely programmed state to a completely erased state, and vice versa. This means that the cell memory is analog, or can store at least one of a number of discontinuous values (such as 16 or 64 different values), which allows every cell in the memory array to be tuned very precisely and individually, and makes the memory array ideal for memory and fine-tuning of the synaptic weights of a neural network. <Neural network using non-volatile memory cell arrays>
[0021] Figure 6 conceptually illustrates a non-limiting example of a neural network utilizing a non-volatile memory array in this embodiment. While this example uses a non-volatile memory array neural network for a facial recognition application, it is also possible to implement other suitable applications using a non-volatile memory array-based neural network.
[0022] S0 is the input layer, which in this example is a 32x32 pixel RGB image with 5-bit precision (i.e., three 32x32 pixel arrays, one for each color R, G, and B, with each pixel having 5-bit precision). The synapse CB1, going from input layer S0 to layer C1, scans the input image with a 3x3 pixel overlapping filter (kernel), applying different weight sets to some instances and shared weights to others, and shifts the filter by one pixel (or more than two pixels depending on the model). Specifically, the values of nine pixels in the 3x3 portion of the image (i.e., referred to as the filter or kernel) are provided to synapse CB1, where these nine input values are multiplied by the appropriate weights, and after summing the outputs of the multiplications, a single output value is determined, which is then given by the first synapse of CB1 to generate one of the pixels in the feature map of layer C1. The 3x3 filter is then shifted one pixel to the right within the input layer S0 (i.e., a column of 3 pixels is added to the right and a column of 3 pixels is dropped to the left), thereby providing the 9 pixel values of this newly positioned filter to synapse CB1, where they are multiplied by the same weights as above, determining a second single output value by the associated synapse. This process continues until the 3x3 filter has scanned the entire 32x32 pixel image of the input layer S0 for all three colors and all bits (precision values). The process is then repeated with different weight sets to generate different feature maps of layer C1 until all feature maps of layer C1 have been computed.
[0023] In this example, layer C1 contains 16 feature maps, each with 30x30 pixels. Each pixel is a new feature pixel extracted from the multiplication of the input and the kernel; therefore, each feature map is a two-dimensional array, and thus in this example, layer C1 constitutes 16 layers of two-dimensional arrays (note that the layers and arrays referred to herein are logical relationships, not necessarily physical relationships; i.e., arrays are not necessarily oriented to physical two-dimensional arrays). Each of the 16 feature maps in layer C1 is generated by one of 16 different synaptic weight sets applied to the filtered scan. All C1 feature maps can target different aspects of the same image feature, such as boundary identification. For example, a first map (generated using a first weight set shared across all scans used to generate this first map) can identify circular edges, and a second map (generated using a second weight set different from the first) can identify rectangular edges or the aspect ratio of a particular feature, etc.
[0024] Before moving from layer C1 to layer S1, an activation function P1 (pooling) is applied that pools values from non-overlapping, consecutive 2x2 regions within each feature map. The purpose of the pooling function P1 is to average neighbor positions (or use the max function), reduce dependence on edge positions, and reduce data size before moving to the next stage. In layer S1, there are 16 15x15 feature maps (i.e., 16 different arrays of 15x15 pixels each). Synapse CB2, moving from layer S1 to layer C2, scans the maps in layer S1 with a 4x4 filter, shifting by 1 pixel. In layer C2, there are 22 12x12 feature maps. Before moving from layer C2 to layer S2, an activation function P2 (pooling) is applied that pools values from non-overlapping, consecutive 2x2 regions within each feature map. In layer S2, there are 22 6x6 feature maps. At synapse CB3, which goes from layer S2 to layer C3, an activation function (pooling) is applied, where all neurons in layer C3 are connected to all maps in layer S2 via each synapse of CB3. There are 64 neurons in layer C3. Synapse CB4, which goes from layer C3 to output layer S3, completely connects C3 to S3; that is, all neurons in layer C3 are connected to all neurons in layer S3. The output in S3 contains 10 neurons, where the neuron with the highest output determines the class. This output can, for example, indicate the identification or classification (classification) of the content of the original image.
[0025] Each layer of a synapse is implemented using an array or part of an array of non-volatile memory cells.
[0026] Figure 7 is a block diagram of an array that can be used for that purpose. The vector-by-matrix multiplication (VMM) array 32 contains non-volatile memory cells and is used as synapses between one layer and the next (e.g., CB1, CB2, CB3, and CB4 in Figure 6). Specifically, the VMM array 32 includes an array of non-volatile memory cells 33, erase gate and word line gate decoders 34, a control gate decoder 35, a bit line decoder 36, and a source line decoder 37, each of which decoders decodes its respective input to the non-volatile memory cell array 33. Input to the VMM array 32 can be from the erase gate and word line gate decoder 34 or from the control gate decoder 35. In this example, the source line decoder 37 also decodes the output of the non-volatile memory cell array 33. Alternatively, the bit line decoder 36 can decode the output of the non-volatile memory cell array 33.
[0027] The non-volatile memory cell array 33 serves two purposes. First, it stores the weights that will be used by the VMM array 32. Second, the non-volatile memory cell array 33 effectively multiplies the weights stored in the non-volatile memory cell array 33 by the inputs and adds them together for each output line (source line or bit line) to generate an output, which becomes the input to the next layer or the last layer. Because the non-volatile memory cell array 33 performs the multiplication and addition functions, the need for separate multiplication and addition logic circuits is eliminated, and the calculations are more power-efficient due to being performed in memory.
[0028] The outputs of the non-volatile memory cell array 33 are fed to a differential summer (such as a summing operational amplifier or summing current mirror) 38, which sums the outputs of the non-volatile memory cell array 33 to create a single value for its convolution. The differential summer 38 is configured to perform the summing of positive and negative weights.
[0029] The summed output values of the differential summer 38 are then fed to an activation function block 39 that rectifies the output. The activation function block 39 may provide a sigmoid, tanh, or ReLU function. The rectified output values of the activation function block 39 become elements of a feature map as the next layer (e.g., C1 in Figure 6), and are then applied to the next synapse to generate the next feature map layer or the final layer. Thus, in this example, the non-volatile memory cell array 33 constitutes multiple synapses (receiving input from the previous layer of the neuron or from an input layer such as an image database), and the summing operational amplifier 38 and activation function block 39 constitute multiple neurons.
[0030] The inputs to the VMM array 32 in Figure 7 (WLx, EGx, CGx, and optionally BLx and SLx) can be analog level, binary level, or digital bits (in which case a DAC is provided to convert the digital bits to the appropriate input analog level), and the outputs can be analog level, binary level, or digital bits (in which case an output ADC is provided to convert the output analog level to digital bits).
[0031] Figure 8 is a block diagram showing the use of multiple layers of the VMM array 32, labeled in the figure as VMM arrays 32a, 32b, 32c, 32d, and 32e. As shown in Figure 8, the input (indicated as Inputx) is converted from digital to analog by the digital-to-analog converter 31 and provided to the input VMM array 32a. The converted analog input can be voltage or current. Input D / A conversion of the first layer can be performed by using a function or LUT (look-up table) that maps the input Inputx to the appropriate analog level of the matrix multiplier of the input VMM array 32a. Input conversion can also be performed by an analog-to-analog (A / A) converter to convert an external analog input to the mapped analog input to the input VMM array 32a.
[0032] The output generated by input VMM array 32a is then provided as input to the next VMM array (hidden level 1) 32b, which generates an output that is then provided as input to input VMM array (hidden level 2) 32c, and so on. The various layers of VMM array 32 function as the synapses and neurons of a convolutional neural network (CNN). Each VMM array 32a, 32b, 32c, 32d, and 32e can be a standalone physical non-volatile memory array, or multiple VMM arrays can utilize different parts of the same physical non-volatile memory array, or multiple VMM arrays can utilize overlapping parts of the same physical non-volatile memory array. The example shown in Figure 8 includes five layers (32a, 32b, 32c, 32d, 32e), i.e., one input layer (32a), two hidden layers (32b, 32c), and two fully connected layers (32d, 32e). Those skilled in the art will understand that this is merely an example and that the system may instead include more than two hidden layers and more than two fully connected layers. <Vector Matrix Multiplication (VMM) Array>
[0033] Figure 9 shows a neuron VMM array 900, which is particularly suitable for the memory cell 310 shown in Figure 3 and is used as part of a synapse and neuron between the input layer and the next layer. The VMM array 900 includes a memory array 901 of non-volatile memory cells and a reference array 902 of non-volatile reference memory cells (located at the top of the array). Alternatively, another reference array may be located at the bottom.
[0034] In the VMM array 900, control gate lines such as control gate line 903 extend vertically (thus the row-direction reference array 902 is perpendicular to the control gate line 903), and erase gate lines such as erase gate line 904 extend horizontally. Here, inputs to the VMM array 900 are provided to the control gate lines (CG0, CG1, CG2, CG3), and outputs of the VMM array 900 appear on the source lines (SL0, SL1). In one embodiment, only even rows are used, and in another embodiment, only odd rows are used. The current on each source line (SL0, SL1, respectively) is a function of the sum of all currents from the memory cells connected to that particular source line.
[0035] As described herein with respect to neural networks, it is preferable that the non-volatile memory cells of the VMM array 900, i.e., the memory cells 310 of the VMM array 900, are configured to operate in the subthreshold region.
[0036] The non-volatile reference memory cells and non-volatile memory cells described herein are biased in a weak inversion (sub-threshold region) as follows: Ids=Io*e(Vg-Vth) / nVt=w*Io*e(Vg) / nVt In the equation, w = e(-Vth) / nVt, Ids is the drain-source current, Vg is the gate voltage of the memory cell, Vth is the threshold voltage of the memory cell, Vt is the thermal voltage = k*T / q, where k is Boltzmann's constant, T is the Kelvin temperature, and q is the electron charge, n is the gradient coefficient = 1 + (Cdep / Cox), where Cdep is the capacitance of the depletion layer, and Cox is the capacitance of the gate oxide layer, Io is the memory cell current at a gate voltage equal to the threshold voltage, and Io is proportional to (Wt / L)*u*Cox*(n-1)*Vt², where u is the carrier mobility, and Wt and L are the width and length of the memory cell, respectively.
[0037] When using an IV logarithmic converter that converts input current to input voltage using a memory cell (such as a reference memory cell or peripheral memory cell) or transistor: Vg = n * Vt * log[Ids / wp * Io] In the formula, wp is the w of the reference or peripheral memory cell.
[0038] For a memory array used as a vector matrix multiplier (VMM) array with current input, the output current is as follows: Iout = wa * Io * e(Vg) / nVt, that is, Iout = (wa / wp) * Iin = W * Iin W=e(Vthp-Vtha) / nVt Here, wa = w of each memory cell in the memory array. Vthp is the effective threshold voltage of the peripheral memory cell, and Vtha is the effective threshold voltage of the main (data) memory cell. Note that the threshold voltage of a transistor is a function of the substrate bias voltage, and the substrate bias voltage, denoted as Vsb, can be adjusted to compensate for various conditions at such temperatures. The threshold voltage Vth can be expressed as follows:
number
[0039] Word lines or control gates can be used as inputs to memory cells for input voltage.
[0040] Alternatively, the flash memory cells of the VMM array described herein can be configured to operate in a linear region. Ids=beta * (Vgs-Vth) * Vds; beta = u * Cox * Wt / L W ∝ (Vgs-Vth) In other words, the weight W in the linear region is proportional to (Vgs - Vth).
[0041] A word line or a control gate or a bit line or a source line can be used as an input of a memory cell operating in a linear region. A bit line or a source line can be used as an output of the memory cell.
[0042] For an I-V linear converter, a memory cell (such as a reference memory cell or a peripheral memory cell) or a transistor operating in a linear region can be used to linearly convert an input / output current into an input / output voltage.
[0043] Alternatively, the memory cells of the VMM array described herein can be configured to operate in a saturation region. Ids=1 / 2 * Beta * (Vgs-Vth) 2 ; Beta=u * Cox * Wt / L W ∝ (Vgs-Vth) 2 , that is, the weight W is (Vgs-Vth) 2 proportional to
[0044] A word line, a control gate, or an erase gate can be used as an input of a memory cell operating in a saturation region. A bit line or a source line can be used as an output of an output neuron.
[0045] Alternatively, the memory cells of the VMM array described herein may be used in all regions or combinations thereof (subthreshold, linear, or saturation) for each layer or multiple layers of a neural network.
[0046] Other embodiments for the VMM array 32 of FIG. 7 are described in U.S. Patent No. 10,748,630, which is incorporated herein by reference. As described in the above application, a source line or a bit line can be used as a neuron output (current sum output).
[0047] Figure 10 shows a neuron VMM array 1000, particularly suited to the memory cell 210 shown in Figure 2 and used as a synapse between the input layer and the next layer. The VMM array 1000 includes a memory array 1003 of non-volatile memory cells, a reference array 1001 of first non-volatile reference memory cells, and a reference array 1002 of second non-volatile reference memory cells. The reference arrays 1001 and 1002, arranged in the column direction of the array, function to convert current inputs flowing into terminals BLR0, BLR1, BLR2, and BLR3 into voltage inputs WL0, WL1, WL2, and WL3. In practice, the first and second non-volatile reference memory cells are diode-connected through a multiplexer 1014 (partially shown) with current inputs flowing in. The reference cells are tuned (e.g., programmed) to a target reference level. The target reference level is provided by a reference miniarray matrix (not shown).
[0048] The memory array 1003 serves two purposes. First, it stores the weights used by the VMM array 1000 in each memory cell. Second, the memory array 1003 effectively multiplies the weights stored in it by the inputs (i.e., the current inputs provided to terminals BLR0, BLR1, BLR2, and BLR3, which are converted into input voltages by the reference arrays 1001 and 1002 and supplied to the word lines WL0, WL1, WL2, and WL3), then adds all the results (memory cell currents) to produce outputs on each bit line (BL0~BLN), which become inputs to the next layer or the last layer. By performing multiplication and addition functions, the memory array 1003 eliminates the need for separate multiplication and addition logic circuits and is also power efficient. Here, voltage inputs are supplied to word lines WL0, WL1, WL2, and WL3, and outputs appear on the respective bit lines BL0 to BLN during the read (inference) operation. The current in each bit line BL0 to BLN is a function of the sum of the currents from all non-volatile memory cells connected to that particular bit line.
[0049] Table 5 shows the operating voltages and currents of the VMM array 1000. The columns in the table show the voltages applied to the word lines of selected cells, word lines of unselected cells, bit lines of selected cells, bit lines of unselected cells, source lines of selected cells, and source lines of unselected cells. The rows show the read, erase, and program operations. Table 5: Operation of VMM Array 1000 in Figure 10 [Table 5]
[0050] Figure 11 shows a neuron VMM array 1100, which is particularly suitable for the memory cell 210 shown in Figure 2 and is used as part of a synapse and neuron between the input layer and the next layer. The VMM array 1100 includes a memory array 1103 of non-volatile memory cells, a reference array 1101 of a first non-volatile reference memory cell, and a reference array 1102 of a second non-volatile reference memory cell. The reference arrays 1101 and 1102 extend in the row direction of the VMM array 1100. The VMM array is similar to the VMM 1000, except that the word lines in the VMM array 1100 extend vertically. Here, inputs are provided to the word lines (WLA0, WLB0, WLA1, WLB2, WLA2, WLB2, WLA3, WLB3), and outputs appear on the source lines (SL0, SL1) during read operations. The current on each source line performs a function of the sum of all currents from the memory cells connected to that particular source line.
[0051] Table 6 shows the operating voltages and currents of the VMM array 1100. The columns in the table show the voltages applied to the word lines of selected cells, word lines of unselected cells, bit lines of selected cells, bit lines of unselected cells, source lines of selected cells, and source lines of unselected cells. The rows show the read, erase, and program operations. Table 6: Operation of VMM Array 1100 in Figure 11 [Table 6]
[0052] Figure 12 shows a neuron VMM array 1200, which is particularly suitable for the memory cell 310 shown in Figure 3 and is used as part of a synapse and neuron between the input layer and the next layer. The VMM array 1200 includes a memory array 1203 of nonvolatile memory cells, a reference array 1201 of a first nonvolatile reference memory cell, and a reference array 1202 of a second nonvolatile reference memory cell. The reference arrays 1201 and 1202 function to convert the current inputs flowing into terminals BLR0, BLR1, BLR2, and BLR3 into voltage inputs CG0, CG1, CG2, and CG3. In practice, the first and second nonvolatile reference memory cells are diode-connected through a multiplexer 1212 (partially shown) with current inputs flowing through BLR0, BLR1, BLR2, and BLR3. The multiplexer 1212 includes a corresponding multiplexer 1205 and a cascoding transistor 1204 to ensure a constant voltage across the respective bit lines (such as BLR0) of the first and second non-volatile reference memory cells during read operations. The reference cells are tuned to a target reference level.
[0053] The memory array 1203 serves two purposes. First, it stores the weights used by the VMM array 1200. Second, the memory array 1203 effectively multiplies the weights stored in the memory array by the inputs (current inputs provided to terminals BLR0, BLR1, BLR2, and BLR3, which are converted into input voltages by the reference arrays 1201 and 1202 and supplied to the control gates (CG0, CG1, CG2, and CG3)), then adds all the results (cell currents) to produce an output, which appears in BL0~BLN and becomes the input to the next layer or the last layer. By having the memory array perform the multiplication and addition functions, the need for separate multiplication and addition logic circuits is eliminated, and power efficiency is also improved. Here, the inputs are provided to the control gate lines (CG0, CG1, CG2, and CG3), and the output appears in the bit lines (BL0~BLN) during read operations. The current in each bit line acts as a function of the sum of all currents from the memory cells connected to that particular bit line.
[0054] The VMM array 1200 implements one-way tuning of non-volatile memory cells within the memory array 1203. That is, each non-volatile memory cell is erased and then partially programmed until the desired charge on the floating gate is reached. If too much charge is applied to the floating gate (e.g., an incorrect value is stored in the cell), the cell is erased and the series of partial programming operations is restarted from the beginning. As shown, two rows sharing the same erase gate (e.g., EG0 or EG1) are erased together (known as page erase), and then each cell is partially programmed until the desired charge on the floating gate is reached.
[0055] Table 7 shows the operating voltages and currents of the VMM array 1200. The columns in the table show the voltages applied to the word lines of selected cells, word lines of unselected cells, bit lines of selected cells, bit lines of unselected cells, control gates of selected cells, control gates of unselected cells in the same sector as the selected cell, control gates of unselected cells in a different sector than the selected cell, erase gates of selected cells, erase gates of unselected cells, source lines of selected cells, and source lines of unselected cells. The rows show the read, erase, and program operations. Table 7: Operation of VMM Array 1200 in Figure 12 [Table 7]
[0056] Figure 13 shows a neuron VMM array 1300, which is particularly suitable for the memory cell 310 shown in Figure 3 and is used as part of a synapse and neuron between the input layer and the next layer. The VMM array 1300 is a memory array 1303 of nonvolatile memory cells, First non-volatile reference memory cell Reference array 130 1 andThe VMM array 1300 comprises a reference array 1302 of a second non-volatile reference memory cell. The EG lines EGR0, EG0, EG1, and EGR1 extend vertically, and the CG lines CG0, CG1, CG2, and CG3 and the SL lines WL0, WL1, WL2, and WL3 extend horizontally. The VMM array 1300 is a VMM array except that the VMM array 1300 implements bidirectional tuning. 1200 Similarly, each individual cell can be completely erased, partially programmed, and partially erased as needed to reach the desired amount of charge on the floating gate by using separate EG lines. As shown, reference arrays 1301 and 1302 convert the input currents at terminals BLR0, BLR1, BLR2, and BLR3 into control gate voltages CG0, CG1, CG2, and CG3 (through the action of diode-connected reference cells via multiplexer 1314), and these voltages are applied to memory cells in the row direction. Current outputs (neurons) are located in the bit lines BL0~BLN, and each bit line sums all the currents from the non-volatile memory cells connected to that particular bit line.
[0057] Table 8 shows the operating voltages and currents of the VMM array 1300. The columns in the table show the voltages applied to the word lines of selected cells, word lines of unselected cells, bit lines of selected cells, bit lines of unselected cells, control gates of selected cells, control gates of unselected cells in the same sector as the selected cell, control gates of unselected cells in a different sector than the selected cell, erase gates of selected cells, erase gates of unselected cells, source lines of selected cells, and source lines of unselected cells. The rows show the read, erase, and program operations. Table 8: Operation of VMM Array 1300 in Figure 13 [Table 8]
[0058] Figure 22 shows a neuron VMM array 2200, which is particularly suitable for the memory cell 210 shown in Figure 2 and is used as part of a synapse and neuron between the input layer and the next layer. In the VMM array 2200, inputs INPUT0..., INPUTN are received on bit lines BL0,...BLN, respectively, and outputs OUTPUT1, OUTPUT2, OUTPUT3, and OUTPUT4 are generated on source lines SL0, SL1, SL2, and SL3, respectively.
[0059] Figure 23 shows a neuron VMM array 2300 that is particularly suitable for the memory cell 210 shown in Figure 2 and is used as part of a synapse and neuron between the input layer and the next layer. In this example, inputs INPUT0, INPUT1, INPUT2, and INPUT3 are received on source lines SL0, SL1, SL2, and SL3, respectively, and outputs OUTPUT0, ... OUTPUTN are generated on bit lines BL0, ... BLN.
[0060] Figure 24 shows a neuron VMM array 2400, which is particularly suitable for the memory cell 210 shown in Figure 2 and is used as part of a synapse and neuron between the input layer and the next layer. In this example, inputs INPUT0, ..., INPUTM are received on word lines WL0, ..., WLM, respectively, and outputs OUTPUT0, ..., OUTPUTN are generated on bit lines BL0, ..., BLN.
[0061] Figure 25 shows a neuron VMM array 2500, which is particularly suitable for the memory cell 310 shown in Figure 3 and is used as part of a synapse and neuron between the input layer and the next layer. In this example, inputs INPUT0, ..., INPUTM are received on word lines WL0, ..., WLM, respectively, and outputs OUTPUT0, ..., OUTPUTN are generated on bit lines BL0, ..., BLN.
[0062] Figure 26 shows a neuron VMM array 2600 that is particularly suitable for the memory cell 410 shown in Figure 4 and is used as part of a synapse and neuron between the input layer and the next layer. In this example, inputs INPUT0, ..., INPUTn are received by vertical control gate lines CG0, ..., CGN, respectively, and outputs OUTPUT1 and OUTPUT2 are generated by source lines SL0 and SL1.
[0063] Figure 27 shows a neuron VMM array 2700, which is particularly suitable for the memory cell 410 shown in Figure 4 and is used as part of a synapse and neuron between the input layer and the next layer. In this example, inputs INPUT0, ..., INPUTN are received at the gates of bit line control gates 2701-1, 2701-2, ..., 2701-(N-1) and 2701-N, respectively, which are coupled to bit lines BL0, ..., BLN. Exemplary outputs OUTPUT1 and OUTPUT2 are generated at source lines SL0 and SL1.
[0064] Figure 28 shows a neuron VMM array 2800, which is particularly suitable for the memory cell 310 shown in Figure 3, the memory cell 510 shown in Figure 5, and the memory cell 710 shown in Figure 7, and is used as part of a synapse and neuron between the input layer and the next layer. In this example, inputs INPUT0, ..., INPUTM are received on word lines WL0, ..., WLM, respectively, and outputs OUTPUT0, ..., OUTPUTN are generated on bit lines BL0, ..., BLN.
[0065] Figure 29 shows a neuron VMM array 2900, particularly suitable for the memory cell 310 shown in Figure 3, the memory cell 510 shown in Figure 5, and the memory cell 710 shown in Figure 7, and used as part of synapses and neurons between the input layer and the next layer. In this example, inputs INPUT0, ..., INPUTM are received by control gate lines CG0, ..., CGM. Outputs OUTPUT0, ..., OUTPUTN are generated by vertical source lines SL0, ..., SLN, respectively, and each source line SLi is coupled to the source line of all memory cells in column i.
[0066] Figure 30 shows a neuron VMM array 3000, particularly suitable for the memory cell 310 shown in Figure 3, the memory cell 510 shown in Figure 5, and the memory cell 710 shown in Figure 7, and used as part of synapses and neurons between the input layer and the next layer. In this example, inputs INPUT0, ..., INPUTM are received by control gate lines CG0, ..., CGM. Outputs OUTPUT0, ..., OUTPUTN are generated by vertical bit lines BL0, ..., BLN, respectively, and each bit line BLi is coupled to the bit lines of all memory cells in column i. <Long-term and short-term memory>
[0067] Prior art includes the concept known as long short-term memory (LSTM). LSTM units are often used within neural networks. LSTMs allow neural networks to store information for a predetermined period and use that information in subsequent operations. A conventional LSTM unit includes a cell, an input gate, an output gate, and a forget gate. The three gates regulate the flow of information into and out of the cell, and the duration for which information is stored within the LSTM. VMMs are particularly useful in LSTM units.
[0068] Figure 14 shows an exemplary LSTM1400. In this example, the LSTM1400 includes cells 1401, 1402, 1403, and 1404. Cell 1401 receives the input vector x0 and generates the output vector h0 and the cell state vector c0. Cell 1402 receives the input vector x1, the output vector (hidden state) h0 from cell 1401, and the cell state c0 from cell 1401, and generates the output vector h1 and the cell state vector c1. Cell 1403 receives the input vector x2, the output vector (hidden state) h1 from cell 1402, and the cell state c1 from cell 1402, and generates the output vector h2 and the cell state vector c2. Cell 1404 receives the input vector x3, the output vector (hidden state) h2 from cell 1403, and the cell state c2 from cell 1403, and generates the output vector h3. Additional cells can also be used, and an LSTM with four cells is just one example.
[0069] Figure 15 shows an exemplary implementation of LSTM cell 1500 that can be used for cells 1401, 1402, 1403, and 1404 in Figure 14. LSTM cell 1500 receives an input vector x(t), a cell state vector c(t-1) from a preceding cell, and an output vector h(t-1) from a preceding cell, and generates the cell state vector c(t) and output vector h(t).
[0070] LSTM cell 1500 includes sigmoid function devices 1501, 1502, and 1503, each of which applies a number between 0 and 1 to control the extent to which each component of the input vector contributes to the output vector. LSTM cell 1500 also includes tanh devices 1504 and 1505 for applying a hyperbolic tangent function to the input vector, multiplier devices 1506, 1507, and 1508 for multiplying two vectors, and an adder device 1509 for adding two vectors. The output vector h(t) can be provided to the next LSTM cell in the system or accessed for other purposes.
[0071] Figure 16 shows an LSTM cell 1600, which is an example of an implementation of LSTM cell 1500. For the reader's convenience, the same numbering method used in LSTM cell 1500 is used in LSTM cell 1600. Sigmoid function devices 1501, 1502, and 1503, and tanh device 1504 each contain multiple VMM arrays 1601 and activation function blocks 1602. Thus, VMM arrays are found to be particularly useful in LSTM cells used in certain neural network systems. Multiplier devices 1506, 1507, and 1508, and adder device 1509 are implemented in a digital or analog manner. Activation function block 1602 can be implemented in a digital or analog manner.
[0072] Figure 17 shows an alternative example of LSTM cell 1600 (and another example of an implementation of LSTM cell 1500). In Figure 17, sigmoid function devices 1501, 1502, and 1503, and tanh device 1504 share the same physical hardware (VMM array 1701 and activation function block 1702) in a time-division multiplexed manner. The LSTM cell 1700 also includes a multiplier device 1703 for multiplying two vectors, an adder device 1708 for adding two vectors, a tanh device 1505 (including an activation function block 1702), a register 1707 for storing the value i(t) when i(t) is output from the sigmoid function block 1702, a register 1704 for storing the value f(t)*c(t-1) when its value is output from the multiplier device 1703 via the multiplexer 1710, a register 1705 for storing the value i(t)*u(t) when its value is output from the multiplier device 1703 via the multiplexer 1710, a register 1706 for storing the value o(t)*c~(t) when its value is output from the multiplier device 1703 via the multiplexer 1710, and a multiplexer 1709.
[0073] While an LSTM cell 1600 includes multiple sets of VMM arrays 1601 and their respective activation function blocks 1602, an LSTM cell 1700 includes only one set of VMM arrays 1701 and activation function blocks 1702, which are used to represent multiple layers in embodiments of the LSTM cell 1700. Compared to the LSTM cell 1600, the LSTM cell 1700 requires only one-quarter the space for the VMMs and activation function blocks, thus requiring less space than the LSTM cell 1600.
[0074] It can be further understood that an LSTM unit typically includes multiple VMM arrays, each of which requires functionality provided by specific circuit blocks outside the VMM array, such as adder and activation function blocks and high-voltage generation blocks. Providing separate circuit blocks for each VMM array would require a considerable amount of space within the semiconductor device and would be somewhat inefficient. Therefore, the embodiments described below reduce the circuitry required outside the VMM array itself. <Gated recurrent unit>
[0075] Analog VMM implementations can be used in GRU (gated recurrent unit) systems. A GRU is a gate mechanism within an iterative neural network. GRUs are similar to LSTMs, except that GRU cells generally contain fewer components than LSTM cells.
[0076] Figure 18 shows an exemplary GRU1800. In this example, the GRU1800 includes cells 1801, 1802, 1803, and 1804. Cell 1801 receives input vector x0 and produces output vector h0. Cell 1802 receives input vector x1 and output vector h0 from cell 1801 and produces output vector h1. Cell 1803 receives input vector x2 and output vector (hidden state) h1 from cell 1802 and produces output vector h2. Cell 1804 receives input vector x3 and output vector (hidden state) h2 from cell 1803 and produces output vector h3. Additional cells are also available, and a GRU with four cells is just an example.
[0077] Figure 19 shows an exemplary implementation of a GRU cell 1900 that may be used in cells 1801, 1802, 1803, and 1804 of Figure 18. The GRU cell 1900 takes an input vector x(t) and an output vector h(t-1) from a preceding GRU cell and produces an output vector h(t). The GRU cell 1900 includes sigmoid function devices 1901 and 1902, each of which applies a number between 0 and 1 to the components from the output vector h(t-1) and the input vector x(t). The GRU cell 1900 also includes a tanh device 1903 for applying a hyperbolic tangent function to the input vector, multiple multiplier devices 1904, 1905, and 1906 for multiplying two vectors, an adder device 1907 for adding two vectors, and a complementary device 1908 for subtracting the input from 1 to produce an output.
[0078] Figure 20 shows GRU cell 2000, an example of an implementation of GRU cell 1900. For the reader's convenience, the same numbering method used in GRU cell 1900 is used in GRU cell 2000. As can be seen from Figure 20, the sigmoid function devices 1901 and 1902, and the tanh device 1903, each contain multiple VMM arrays 2001 and activation function blocks 2002. Thus, it can be seen that VMM arrays are used in particular in GRU cells used in specific neural network systems. The multiplier devices 1904, 1905, 1906, the adder device 1907, and the complementary device 1908 are implemented in a digital or analog manner. The activation function block 2002 can be implemented in a digital or analog manner.
[0079] Figure 21 shows an alternative example of the GRU cell 2000 (and another example of one implementation of the GRU cell 1900). In Figure 21, the GRU cell 2100 utilizes the VMM array 2101 and the activation function block 2102, and when configured as a sigmoid function, it applies a number between 0 and 1 to control the extent to which each component of the input vector contributes to the output vector. In Figure 21, the sigmoid function devices 1901 and 1902, and the tanh device 1903, share the same physical hardware (VMM array 2101 and activation function block 2102) in a time-division multiplexed manner. The GRU cell 2100 also includes a multiplier device 2103 for multiplying two vectors, an adder device 2105 for adding two vectors, a complementary device 2109 for subtracting an input from 1 to produce an output, a multiplexer 2104, a register 2106 for holding the value h(t-1)*r(t) when that value is output from the multiplier device 2103 via the multiplexer 2104, a register 2107 for holding the value h(t-1)*z(t) when that value is output from the multiplier device 2103 via the multiplexer 2104, and a register 2108 for holding the value h^(t)*(1-z((t)) when that value is output from the multiplier device 2103 via the multiplexer 2104.
[0080] While GRU cell 2000 includes multiple sets of VMM arrays 2001 and activation function blocks 2002, GRU cell 2100 includes only one set of VMM arrays 2101 and activation function blocks 2102, which are used to represent multiple layers in embodiments of GRU cell 2100. GRU cell 2100 requires less space than GRU cell 2000 because it requires only one-third the space for the VMM and activation function blocks compared to GRU cell 2000.
[0081] It can be further understood that a GRU system typically includes multiple VMM arrays, each of which requires functionality provided by specific circuit blocks outside the VMM array, such as adder and activation function blocks and high-voltage generation blocks. Providing separate circuit blocks for each VMM array would require a considerable amount of space within the semiconductor device and would be somewhat inefficient. Therefore, the embodiments described below reduce the circuitry required outside the VMM array itself.
[0082] The input to the VMM array may be analog level, binary level, pulse, time-modulated pulse, or digital bit (in which case a DAC is required to convert the digital bit to an appropriate input analog level), and the output may be analog level, binary level, timing pulse, pulse, or digital bit (in which case an output ADC is required to convert the output analog level to a digital bit).
[0083] For each memory cell in a VMM array, each weight W can be implemented by a single memory cell, a differential cell, or two blended memory cells (the average of two cells). In the case of a differential cell, two memory cells are required to implement the weight W as a differential weight (W = W+-W-). In the case of two blended memory cells, two memory cells are required to implement the weight W as the average of two cells.
[0084] Figure 31 shows the VMM system 3100. In some embodiments, multiple weights W, differential pairs, W+ (positive weight) and W- (negative weight) are stored in the VMM array, with W = (W+) - (W-). In the VMM system 3100, half of the multiple bit lines are designated as W+ lines, i.e., bit lines that will connect to memory cells that will store the positive weight W+, and the other half of the multiple bit lines are designated as W- lines, i.e., bit lines that will connect to memory cells that will implement the negative weight W-. Multiple W- lines are interspersed alternately among multiple W+ lines. Subtraction operations are performed by adders, such as adders 3101 and 3102, which receive current from the W+ and W- lines. The outputs of the W+ lines and the W- lines are combined to effectively give W = W+ - W- for each pair of (W+, W-) cells in all pairs of (W+, W-) lines. Up to this point, we have described W- lines that are alternately scattered between W+ lines, but in other embodiments, W+ and W- lines can be arbitrarily placed anywhere within the array.
[0085] Figure 32 shows another embodiment. In the VMM system 3210, a plurality of positive weights W+ are implemented in the first array 3211, and a plurality of negative weights W- are implemented in the second array 3212, which is separate from the first array, and the resulting weights are appropriately combined by an adder circuit 3213.
[0086] Figure 33 shows the VMM system 3300. Multiple weights W stored in the VMM array are stored as differential pairs, W+ (positive weight) and W- (negative weight), where W = (W+) - (W-). The VMM system 3300 comprises arrays 3301 and 3302. Half of the multiple bit lines in each of arrays 3301 and 3302 are designated as W+ lines, i.e., bit lines connected to memory cells that store positive weights W+, and the other half of the multiple bit lines in each of arrays 3301 and 3302 are designated as W- lines, i.e., bit lines connected to memory cells that implement negative weights W-. Multiple W- lines are interspersed alternately among multiple W+ lines. Subtraction operations are performed by adders, such as adders 3303, 3304, 3305 and 3306, which receive current from the W+ and W- lines. The outputs of the W+ line and the W- line from each array 3301 and 3302 are combined together to effectively give W=W+-W- for each pair of (W+, W-) cells in all pairs of (W+, W-) lines. In addition, the W values from each array 3301 and 3302 can be further combined via adders 3307 and 3308, such that each W value is the result of subtracting the W value from array 3302 from the W value from array 3301, meaning that the final result from adders 3307 and 3308 is the difference of one of the two difference values.
[0087] Each non-volatile memory cell used in an analog neural memory system holds a charge, i.e., the number of electrons, in a very specific and precise quantity within a floating gate, corresponding to the erase / program. For example, each floating gate should hold one of N different values, where N is the number of different weights that can be represented by each cell. Examples of N include 16, 32, 64, 128, and 256.
[0088] Similarly, the read operation should be able to accurately distinguish between N different levels.
[0089] In some cases, accuracy is critical, and it is desirable to improve the system's accuracy (perhaps at the expense of power consumption). In other cases, power management is critical, and it is desirable to improve the system's power consumption (i.e., reduce power consumption) (perhaps at the expense of accuracy). In other cases, the ability to maintain accuracy when operating temperature changes is desirable. Instead of power consumption and accuracy, other characteristics such as latency or other performance criteria can be maximized.
[0090] It is desirable to be able to modify the characteristics of the neural network system to improve accuracy or power consumption in various temperature environments. [Overview of the project]
[0091] Numerous embodiments are disclosed for improving analog neural memory in deep learning artificial neural networks with respect to accuracy as a temperature change, power consumption, or other criteria. In some embodiments, a method is performed for determining in real time a bias value to be applied to one or more memory cells in the neural network. In other embodiments, the bias voltage is determined from a lookup table and applied to the terminals of the memory cell during a read operation.
[0092]
[0093]
[0094]
[0095]
[0096]
[0097]
[0098]
[0099]
[0100]
[0101]
[0102]
[0103]
[0104]
[0105]
[0106]
[0107]
[0108]
[0109]
[0110]
[0111]
[0112]
[0113]
[0114]
[0115]
[0116]
[0117]
[0118]
[0119]
[0120]
[0121]
[0122]
[0123]
[0124]
[0125]
[0126]
[0127]
[0128]
[0129]
[0130]
[0131]
[0132]
[0133]
[0134]
[0135]
[0136]
[0137]
[0138]
[0139]
[0140]
[0141]
[0142]
[0143] [Brief explanation of the drawing]
[0144] [Figure 1] This is a diagram illustrating an artificial neural network. [Figure 2] This shows a prior art split-gate flash memory cell. [Figure 3] This shows another prior art split-gate flash memory cell. [Figure 4] This shows another prior art split-gate flash memory cell. [Figure 5] This shows another prior art split-gate flash memory cell. [Figure 6]This diagram illustrates various levels of exemplary artificial neural networks that utilize one or more non-volatile memory arrays. [Figure 7] This is a block diagram illustrating a vector matrix multiplication system. [Figure 8] This is a block diagram illustrating an exemplary artificial neural network that utilizes one or more vector-matrix multiplication systems. [Figure 9] Another embodiment of the vector-matrix multiplication system is shown. [Figure 10] Another embodiment of the vector-matrix multiplication system is shown. [Figure 11] Another embodiment of the vector-matrix multiplication system is shown. [Figure 12] Another embodiment of the vector-matrix multiplication system is shown. [Figure 13] Another embodiment of the vector-matrix multiplication system is shown. [Figure 14] This demonstrates prior art long- and short-term memory systems. [Figure 15] This shows an example cell used in a long-term short-term memory system. [Figure 16] Figure 15 shows an exemplary cell embodiment. [Figure 17] Another embodiment of the exemplary cell shown in Figure 15 is presented. [Figure 18] This shows a prior art gated regression unit system. [Figure 19] An exemplary cell for use in a gated regressive unit system is shown. [Figure 20] Figure 19 shows one embodiment of an exemplary cell. [Figure 21] Another embodiment of the exemplary cell in Figure 19 is shown. [Figure 22] Another embodiment of the vector-matrix multiplication system is shown. [Figure 23] Another embodiment of the vector-matrix multiplication system is shown. [Figure 24]Another embodiment of the vector-matrix multiplication system is shown. [Figure 25] Another embodiment of the vector-matrix multiplication system is shown. [Figure 26] Another embodiment of the vector-matrix multiplication system is shown. [Figure 27] Another embodiment of the vector-matrix multiplication system is shown. [Figure 28] Another embodiment of the vector-matrix multiplication system is shown. [Figure 29] Another embodiment of the vector-matrix multiplication system is shown. [Figure 30] Another embodiment of the vector-matrix multiplication system is shown. [Figure 31] Another embodiment of the vector-matrix multiplication system is shown. [Figure 32] Another embodiment of the vector-matrix multiplication system is shown. [Figure 33] Another embodiment of the vector-matrix multiplication system is shown. [Figure 34] Another embodiment of the vector-matrix multiplication system is shown. [Figure 35] This shows performance data from a neural network. [Figure 36] This describes a neural network method. [Figure 37] This shows a neural network array. [Figure 38] Show the array. [Figure 39] This shows a neural network array. [Figure 40A] I will show you the method. [Figure 40B] A bias lookup table is shown. [Figure 41] I will show you the method. [Figure 42] I will show you the method. [Figure 43] This shows one implementation of a scaler and an analog-to-digital converter. [Figure 44A] The calibration circuit is shown. [Figure 44B] shows a calibration method. [Figure 45] shows a bias averaging circuit. [Figure 46A] shows a bias generation block. [Figure 46B] shows another bias generation block. [Figure 46C] shows still another bias generation block. [Figure 47] shows a neural network layer method. [Figure 48] shows a neural network method. [Figure 49] shows a neural network method. [Figure 50] shows a neural network method. [Mode for Carrying Out the Invention]
[0145] The artificial neural network of the present invention utilizes a combination of CMOS technology and a non-volatile memory array. <Overview of VMM System>
[0146] FIG. 34 shows a block diagram of a VMM system 3400. The VMM system 3400 includes a VMM array 3401, a row decoder 3402, a high voltage decoder 3403, a column decoder 3404, a bit line driver 3405, an input circuit 3406, an output circuit 3407, control logic 3408, and a bias generator 3409. The VMM system 3400 further includes a high voltage generation block 3410, which includes a charge pump 3411, a charge pump regulator 3412, and a high voltage analog accuracy level generator 3413. The VMM system 3400 further includes a (program / erase, or weight tuning) algorithm controller 3414, an analog circuit 3415, a control engine 3416 (which may include special functions such as, but not limited to, arithmetic functions, activation functions, embedded microcontroller logic, etc.), and test control logic 3417. The systems and methods described below may be implemented in the VMM system 3400.
[0147] The input circuit 3406 may include circuits such as a DAC (digital-to-analog converter), DPC (digital-to-pulses converter, digital-to-time-modulated pulse converter), AAC (analog-to-analog converter, such as a current-to-voltage converter or logarithmic converter), PAC (pulse-to-analog level converter), or any other type of converter. The input circuit 3406 may implement normalization, linear or nonlinear up / downscaling functions, or arithmetic functions. The input circuit 3406 may implement a temperature compensation function for the input level. The input circuit 3406 may implement an activation function such as ReLU or sigmoid. The output circuit 3407 may include circuits such as an ADC (analog to digital converter, an analog-to-digital converter for converting the neuron's analog output to digital bits), an AAC (analog to analog converter, such as a current-to-voltage converter or logarithmic converter), an APC (analog to pulse converter, analog to time-modulated pulse converter), or any other type of converter. The output circuit 3407 may implement an activation function such as a rectified linear activation function (ReLU) or a sigmoid. The output circuit 3407 may implement statistical normalization, regularization, up / down scaling / gain functions, statistical rounding, or arithmetic functions (e.g., addition, subtraction, division, multiplication, shift, logarithm) of the neuron output. The output circuit 3407 may implement a temperature compensation function for the neuron output or array output (such as a bit line output) in order to improve the accuracy of the array (neuron) output by keeping the power consumption of the array nearly constant or by keeping the slope of IV nearly the same.
[0148] As mentioned above, a neural network can contain many different layers, and within each layer, many computations will be performed that require weight values stored in one or more arrays within that layer. Some layers will be used more often than others, and it can be understood that such layers are more important to the overall accuracy of the neural network based on their high frequency of use.
[0149] Figure 35 shows Graph 3501, which reflects data collected by the inventors regarding the frequency of weight usage within a multi-layer perceptron (MLP) neural network for MNIST (Modified National Institute of Standards and Technology) digit classification. In the illustrated example, there are n levels, where each L (L0, ..., Ln) represents a range of weights. As can be seen, lower weights are used far more frequently than other weight ranges. In this graph, as an example, Ln does not contribute significantly to the overall network performance. Therefore, Ln can be set to 0, for example, by reducing the control gate voltage applied to the array at level Ln, which results in lower power consumption due to lower cell current drawn at lower control gate voltages without significantly affecting accuracy.
[0150] A neural network consists of multiple layers. Each layer can have a weight distribution unique to that layer. Therefore, different techniques may be required for each layer to improve the overall network performance. For example, Ln may contribute only a small amount in the first layer, but may contribute significantly in the second layer.
[0151] This embodiment provides a method for improving the operation of a neural network. While the term optimization may be used, this method does not necessarily guarantee absolute optimization, i.e., optimization that is as complete, functional, or effective as possible. Instead, as used herein, the term optimization simply means an improvement over prior art methods.
[0152] Figure 35 also shows Table 3502, which illustrates the accuracy of the read operation based on the change in the voltage VCG applied to the control gate of the memory cell during the read operation. As can be seen from the figure, lowering the VCG from 1.8V to 1.6V has no effect on accuracy, and lowering the VCG from 1.5V to 1.4V has a small effect on accuracy. As VCG (or VEG) decreases, the cell current decreases exponentially based on the subthreshold formula. This shows that, in some cases, power can be saved by lowering the voltage applied to the terminals of the memory cell, either without sacrificing accuracy or with accuracy sacrificed to an acceptable degree. Similarly, in the linear region, lower input row voltages result in lower current. Furthermore, it can be understood that changes in operating temperature can affect both accuracy and power consumption, and similarly, improved power and / or accuracy can be obtained as temperature changes by using VCG and / or EG adjustments (i.e., increasing or decreasing the magnitude).
[0153] Based on this consideration in Figure 35, it will be understood that by determining and applying different bias voltages to one or more terminals of a memory cell (CG, EG, WL, etc.), power consumption can be improved (perhaps at the expense of accuracy, for example by reducing the VCG used), accuracy can be improved under static temperature conditions (perhaps at the expense of power consumption, for example by increasing the VCG used), or accuracy can be improved or maintained under changing temperature conditions (perhaps at the expense of power consumption, for example by increasing the VCG as the temperature changes). Other performance characteristics can be maximized instead of accuracy and power consumption.
[0154] With these concepts in mind, various methods will be explained here.
[0155] Figure 36 shows a neural network layer method 3600 that is performed on a specific layer within a neural network. For example, this method may be performed on one (or more) layers that are considered more important due to their significant impact on the overall network accuracy.
[0156] In step 3601, a default voltage bias is applied to the terminals of the cells in the layer array (e.g., control gate terminals) during the read operation. This default voltage bias is typically the same bias value used during the verification operation when the programmed weights are verified.
[0157] In step 3602, performance inference is performed.
[0158] In step 3603, baseline data regarding the network's performance (e.g., accuracy) is collected when the default bias is applied to the array. This data, for example, shows the accuracy of the MNIST inference operation. This baseline data will serve as a reference point for checking the performance target in step 3605.
[0159] In step 3604, the bias is adjusted (for example, increased or decreased by a specific increment) and then applied to the terminals of the cells in the layer of the array (for example, the control gate terminals).
[0160] In step 3605, a performance target check is performed. If the performance data results are within the target range compared to the performance data collection performed in step 3603, the method proceeds to step 3604 until the performance target is no longer met. At this point, the method proceeds to the completion of step 3606, where the method remembers the previous bias conditions that were the last set of biases that resulted in performance data within the target range.
[0161] In step 3606, the previous set of biases is deemed good and is stored (in a lookup table, etc.) in relation to that layer for future use. Optionally, the current operating temperature may be stored along with the bias level.
[0162] Figure 37 shows a neural network array 3700. The neural network array 3700 comprises arrays 3701-0, ..., 3701-n, where n+1 is the number of arrays in the neural network 3700. The neural network 3700 also comprises temperature sensors 3703-i, where i is the number of sensors that detect the operating temperature at a particular location within the neural network 3700. Optionally, each array 3701-0, ..., 3701-n includes its own temperature sensor 3703 (i=n+1) such that each temperature sensor 3703 is associated with one of the arrays 3701-0, ..., 3701-n and the memory cells contained in such arrays. A temperature-voltage-bias lookup table (LUT) 3704-i (where i is the number of the voltage-bias lookup table) is referenced, and based on the detected temperature, a bias voltage is obtained for one or more terminals (e.g., control gate terminals or erase gate terminals, but not limited to these). These bias voltages, called temperature biases 3702, are then applied to each cell in the particular array in question. Thus, a temperature bias 3702-0 is applied to array 3701-0, and so on. Each array 3701-0, ..., 3701-n forms one or more neurons in the neural network.
[0163] Figure 38 shows array 3801. Array 3801 can be used, for example, as one of arrays 3701-0, 3701-n in Figure 37. In this embodiment, different bias voltages (e.g., VCG) can be used for different sub-arrays 3802-0, ..., 3802-k contained within the same array 3801, i.e., array 3801 is divided into multiple sub-arrays. For example, each sub-array 3802-0, ..., 3802-k can receive its own temperature bias 3803-0, ..., 3803-k, respectively. In addition to enabling compensation based on specific operating temperatures measured at different locations within array 3801, this embodiment is also suitable for situations where different types of weights are stored in each sub-array 3802. Since each current range may require a different temperature bias, for example, subarray 3802-0 may store weights in the range of 0 to 30 nA, array 3801-1 may store weights in the range of 30 to 60 nA, and so on.
[0164] This embodiment is also suitable for situations where memory cells in different arrays operate in different modes (regions). For example, because different modes (regions) may require different temperature biases, cells in subarray 3802-0 may operate in subthreshold mode, while cells in subarray 3802-n may operate in linear mode.
[0165] Figure 39 shows the neural network array 3900. In this embodiment, the teachings relating to Figure 38 are extended to m+1 arrays 3901-0, ..., 3901-m within the neural network array 3900. Each array 3901 is divided into k+1 arrays 3902-0a, ..., 3902-ka (where a is an array number ranging from 0 to m). Each array 3902 receives its own temperature bias 3903-0a, ..., 3903-ka, respectively. It should be further understood that each array 3901 can be divided into a different number of arrays and does not need to be divided into the same number of arrays as other arrays 3901.
[0166] Figure 40A shows the neural network array 4000. In a typical neural network readout (inference) operation within a single layer, a digital input value DIN[m:0] is applied to array 4001, resulting in a digital output DOUT[n:0] (or alternatively, an analog value). Array 4001 may be an array or part of an array.
[0167] In the neural network 4000, criteria are used to find one or more values in the lookup table 4003. These criteria may include, for example, desired input and output values, current operating temperature values, and minimum power consumption, target performance (e.g., accuracy or latency), or whether it is desired to target performance at a particular temperature. The lookup table 4003 then provides a bias based on these criteria. The bias is then applied to the array 4001 during the read operation, completing method 4000. The array 4001 may include non-volatile memory cells or volatile memory cells.
[0168] Figure 40B shows a bias look-up table (BLUT) 4020. Array 4021 is an array or part of an array of volatile or non-volatile memory cells. Array 4021 receives a digital input DIN[m:0] and outputs a digital output DOUT[n:0]. The digital output data pattern is programmable according to desired output, but is not limited to linear or subthreshold memory cell relationships or silicon characteristic data. The digital output data DOUT[n:0] is then applied to a digital-to-analog converter 4022, which outputs a desired bias analog voltage to be applied to the array or subarray in question. The BLUT 4020 is used, for example, to provide a bias value, i.e., a temperature bias, in conjunction with a temperature sensor to improve the performance of a neural network.
[0169] Figure 41 shows the bias generation circuit 4100. The temperature sensor 4101A detects the operating temperature and indicates it with a digital bit D[m:0]. Optionally, the timer 4104 can start temperature detection and subsequent bias generation, for example, every 10 to 100 ms (using 1 degree Celsius as an acceptable temperature change that does not significantly affect network performance, this is the time it takes for silicon to increase by 1 degree Celsius). These D[m:0] bits are used to perform a lookup in the lookup table 4102 to find the bias value to be applied based on its operating temperature, i.e., the appropriate temperature bias. The bias value is indicated by a digital bit D[k:0], which is provided to the digital-to-analog converter 4103, which converts the digital bit into a bias voltage, which can then be applied to the terminals of the memory cells in the array (e.g., control gate terminals) during readout (inference) operations.
[0170] Figure 42 shows the scaling circuit 4200. A temperature sensor 4201 detects the operating temperature and indicates it in digital bits D[n:0]. These digital bits are provided to a scaler 4202, which also receives the output neuron current Ineu from the array as a result of the neuron readout operation. The scaler 4202 performs a current-to-voltage conversion of Ineu and scales the signal based on D[n:0]. For example, in the subthreshold region, higher temperatures result in higher neuron currents (due to higher memory cell currents), and therefore it is desirable to scale down this current before it is applied to the ADC 4203. In the linear region, higher temperatures typically result in lower neuron currents (due to lower cell currents), and therefore it is desirable to scale up this current before it is applied to the ADC 4203. The result is a more balanced analog value across temperature, provided to the analog-to-digital converter 4203, yielding a digital output bit D[n:0] representing a scaled digital version of Ineu, this scaling at least partially compensates for sensing the operating temperature.
[0171] Figure 43 shows a scaling circuit 4300, which is an implementation of the scaler ITV (current-voltage converter) 4202 and the analog-to-digital converter 4203 from Figure 42. The scaler 4202 has a programmable gain, which can be programmed by programming the R value (for an ITV circuit that uses R to convert neuron current into a voltage that is digitized by the ADC) or the C value (for an ITV circuit that uses C to convert neuron current into a voltage that is digitized by the ADC). The scaler 4202 can also be implemented as a programmable current mirror (for neuron (bit line) current). The ADC 4203 is a programmable n-bit ADC, where n can be, for example, 4, 8, or 12 bits.
[0172] Figure 44A shows the calibration circuit 4400, and Figure 44B shows the calibration method 4450 which uses the calibration circuit 4400 to populate the lookup table 4470 with values. The current-to-digital-to-analog converter 4402 is coupled to the bit line of the memory cell 4401 and the non-inverting input of the comparator 4403, which also receives a reference voltage VREF at its inverting input. The memory cell 4401 may be a single cell or multiple cells (e.g., from a reference array or part of a main array).
[0173] As described above, each non-volatile or volatile memory cell used in the analog neural memory system can be erased and programmed to hold a very specific and precise amount of charge, i.e., the number of electrons, within its floating gate. For example, each floating gate should hold one of N different values, where N is the number of different weights that can be represented by each cell. Examples of N include 16, 32, 64, 128, and 256. Calibration method 4450 is performed for each of the N different values that can be stored in the memory cell 4401. Each time calibration method 4450 is performed, the memory cell 4401 is programmed (tuned) to one of the N different values, such as a read current of 10 nA (step 4451).
[0174] The voltage at the control gate of memory cell 4401 is measured according to calibration method 4450. The bit line current is changed by the current digital-to-analog converter 4402 from a low current (e.g., 1 nA) to a high current (e.g., 100 nA), resulting in a current of increasing size, and the output of comparator 4403 (called the comparison output) is monitored. At some point, the value of the comparison output changes (e.g., from "0" to "1") (step 4452). When a flip occurs, i.e., before any change in the bit line current by the current digital-to-analog converter 4402, the control gate voltage of memory cell 4401 is measured, and this control gate voltage can be stored in lookup table 4470. This method is repeated for the other N possible values that can be stored in the memory cell. If more than one cell is used, the current provided by the current DAC (IDAC) must be adjusted accordingly; for example, if four cells are used (e.g., for averaging) at 1 nA each, the IDAC current is 4 nA. The resulting CG voltage is stored in the lookup table 4470 (step 4454).
[0175] In another embodiment, the lookup table 4470 is further extended to include multiple temperature values within an expected operating range, such that the lookup table 4470 is a temperature bias lookup table (TBLUT).
[0176] For example, in the case of N=128 (corresponding to an 8-bit input value), the equivalent current range can be 1nA to 128nA, where each 1nA increment is associated with one of the N levels. Then, the calibration circuit 4400 and calibration method 4450 are used to calibrate at several different temperatures (e.g., -40 ℃ -39 ℃ ,...,0 ℃ ,...twenty five ℃ , 26 ℃ ,...,85 ℃For each of the 128 levels, the CG voltages are entered into the lookup table 4470. For example, if 10 different temperature points are used for N=128, the lookup table will have 1280 values (one value for each of the 128 levels for each of the 10 different temperatures).
[0177] In another calibration method, multiple cells are used to store the (weight) currents representing the samples in the array. A bias current from the IDAC4402 is then applied, and the CG is extracted as described above for each of the multiple cells and their corresponding stored values (weights). This can be determined over temperature and stored in a lookup table; therefore, CG bias changes over temperature can be retrieved from the lookup table for different stored values (weights) and applied to the array based on the stored values for the cells in question. Optionally, this can be done in real time, with biases applied to various cells in the array during operation.
[0178] In another embodiment, the calibration circuit 4400 and calibration method 4450 shown in Figures 44A and 44B can be used to calculate the average of the CG voltage applied for each of the N levels for each of several different temperatures. For example, for each of the N values and each temperature, M different readings can be obtained, and the average reading can be stored in the lookup table 4470. For example, if 10 different temperature points are used for N=128, 1280*M readings will be obtained, and 1280 different averages will be stored in the lookup table 4470.
[0179] In another embodiment, instead of taking measurements for all N possible values for each of the multiple temperatures, measurements can be taken for a smaller set of possible values (for example, for four of the N possible values instead of all N possible values), and the average of those smaller sets of possible values can be stored in the lookup table 4470 for the particular temperature used. Thus, if 10 different temperatures are used, the lookup table 4470 will contain 10 values (only one value for each of the 10 different temperatures).
[0180] In another embodiment, the EG bias voltage is also changed. The CG voltage is measured with different EG bias voltages, and the CG and EG biases are stored in the lookup table 4470.
[0181] Figure 45 shows a bias averaging circuit 4500 for determining the average bias based on measurements performed on n+1 different memory cells. Calibration method 4450 is performed on n+1 different cells, each yielding a voltage (e.g., VCG) that represents the "optimal" or average bias voltage for that cell.
[0182] Each cell is associated with a measurement block 4501, which is shown here as measurement blocks 4501-0 to 4501-n. Each measurement block 4501 is identical. Measurement block 4501-0 comprises an operational amplifier 4502-0, PMOS transistors 4503-0 and 4504 arranged as current mirrors, an NMOS transistor 4505-0, and a register 4506-0. Other measurement blocks 4501 contain the exact same components. During operation, each measurement block 4501 contributes to the mirror current through its PMOS transistor 4504, and its mirror current is summed at the upper terminal of register 4507, which may be a variable register. The output VOUT is the average of the various voltages provided as inputs to block 4501 (depending on the appropriate ratio of the values of register 4507 to 4506). The output voltage VOUT = (R-4507 / R-4506) * sum of VIN0 to VINn. For example, if n=3, R-4507 / R-4506 = 1 / 4, so VOUT = (1 / 4) * (VIN0 + VIN1 + VIN2 + VIN3) = the average voltage of the four input voltages VIN0 to 3.
[0183] The output voltage VOUT can be applied as a bias to the control gate terminal of one or more cells in the neural network memory array.
[0184] Figure 46A shows the bias generation block 4600. The bias generation block 4600 includes a current-to-digital-to-analog converter 4602 coupled to the bit line of the memory cell 4601 and the non-inverting input of comparator 4603, the comparator 4603 also receiving a reference voltage VREF at its inverting input (where VREF is the same VREF shown in Figure 44). The row register 4604 provides the digital value DRIN[0:7] to IDAX 4602, which converts the digital value into a current that flows through the bit line terminal of cell 4601. When switch 4605 is closed, an external voltage VIN is applied to the CG terminal. Switch 4606 is closed, and capacitor 4607 is charged to the same voltage as CG. When the output of comparator 4603 changes, switch 4606 is opened. The voltage across capacitor 4607 at that point represents the CG voltage obtained by changing the output of comparator 4603, which is the determined bias voltage. In other words, switch 4606 and capacitor 4607 form a sample-and-hold circuit. This voltage is held constant by buffer 4608 and applied to the control gate in the array. Memory cell 4601 can operate in the subthreshold region or the linear region.
[0185] Figure 46B shows bias generation block 4650, which is similar to bias generation block 4600 except that memory cell 4651 is diode-connected to generate the CG bias and does not use a comparator. Bias generation block 4650 can be used in Figure 44A to generate CG bias values for lookup table 4470. Bias generation block 4650 includes a current-to-digital-to-analog converter 4652 coupled to the bit line of memory cell 4651. The current-to-digital-to-analog converter 4652 is controlled by row register 4654. The voltage at the control gate of cell 4651 is sampled by switch 4656, which then charges capacitor 4657 to its voltage, and capacitor 4657 holds that voltage after switch 4656 is opened. That is, switch 4656 and capacitor 4657 form a sample-and-hold circuit. This voltage is held constant by buffer 4658 and applied to the control gate in the array. The memory cell 4651 can operate in the subthreshold region or the linear region. The bias generation block 4650 converts the input digital value DRIN[0:7] from the row register 4654 into an equivalent CG voltage applied to the array.
[0186] Figure 46C shows bias generation block 4680, which is similar to bias generation block 4650 except for the addition of level shifter 4685. Bias generation block 4680 can be used in Figure 44A to generate CG bias values for lookup table 4470. Bias generation block 4680 comprises a current-to-digital-to-analog converter 4652 coupled to the bit line of memory cell 4651. The current-to-digital-to-analog converter 4652 is controlled by row register 4654. Level shifter 4685 is placed between the output of current-to-digital-to-analog converter 4652 and the control gate terminal of memory cell 4651 to shift the voltage by, for example, the bias voltage (e.g., 0.2V to 0.5V). The voltage at the control gate of cell 4651 is sampled by switch 4656, which then charges capacitor 4657 to that voltage, and capacitor 4657 holds that voltage after switch 4656 is opened. In other words, the switch 4656 and capacitor 4657 form a sample-and-hold circuit. This voltage is held constant by the buffer 4658 and applied to the control gate in the array. The memory cell 4651 can operate in the subthreshold region or the linear region. The bias generation block 4650 converts the input digital value DRIN[0:7] from the row register 4654 into an equivalent CG voltage applied to the array.
[0187] Figure 47 shows a neural network neuron method 4700 performed on specific neurons in a neural network. In step 4701, a nominal bias is applied to the specific neurons to be included in the array. This method may be performed on neurons that are considered more important due to their frequency of use. Steps 4702-4706 are exactly the same as steps 3602-3606 in Figure 36.
[0188] Figure 48 shows a neural network method 4800. This method 4800 includes the steps of: sensing the operating temperature associated with a first set of memory cells (step 4801); determining a bias in a lookup table based on the sensed operating temperature (step 4802); applying the determined bias to the terminals of the first set of memory cells (step 4803); and performing a read operation on the first set of memory cells (step 4804). Optionally, the first set of memory cells may include all cells in the array. Optionally, the first set of memory cells may include all cells in the array. Optionally, method 4800 further includes the steps of: detecting an operating temperature associated with a second set of memory cells (step 4805); determining a bias in a lookup table based on the second detected operating temperature (step 4806); applying the determined bias to the terminals of the second set of memory cells (step 4807); and performing a read operation on the second set of memory cells (step 4808).
[0189] Figure 49 shows a neural network operation method 4900, which is similar to the neural network operation method 4800 except that bias calibration is performed in real time. The neural network operation method 4900 includes the steps of: sensing the operating temperature associated with a first set of memory cells (step 4901); determining a bias based on the sensed operating temperature (step 4902); applying the determined bias to the terminals of the first set of memory cells (step 4903); and performing a read operation on the first set of memory cells (step 4904). Optionally, the first set of memory cells may include all cells in the array. Optionally, the first set of memory cells may include all cells in the array. Optionally, method 4900 further includes the steps of: detecting an operating temperature associated with a second set of memory cells (step 4905); determining a bias based on the second detected operating temperature (step 4906); applying the determined bias to the terminals of the second set of memory cells (step 4907); and performing a read operation on the second set of memory cells (step 4908).
[0190] Figure 50 shows a neural network method 5000, which includes the steps of: programming one or more memory cells (step 5001); passing multiple currents through the programmed memory cells (step 5002); measuring the voltage at the control gate terminal of each programmed memory cell and storing the voltage as a decision bias for the cell that stores the value to be stored in the programmed memory cell (step 5003); applying a bias voltage to the terminals of the set of memory cells based on the use of the decided bias for the cell that stores the value to be stored in the set of memory cells (step 5004); and performing a read operation on the set of memory cells (step 5005).
[0191] It should be noted that, as used herein, the terms “over” and “on” both encompass “directly” (without intermediate material, element, or gap between them) and “indirectly to” (with intermediate material, element, or gap between them). Similarly, the term “adjacent” includes “directly adjacent” (without intermediate material, element, or gap between them) and “indirectly adjacent” (with intermediate material, element, or gap between them); “attached” includes “directly attached” (without intermediate material, element, or gap between them) and “indirectly attached to” (with intermediate material, element, or gap between them); and “electrically coupled” includes “directly electrically coupled” (without intermediate material or element between them electrically connecting the elements together) and “indirectly electrically coupled to” (with intermediate material or element between them electrically connecting the elements together). For example, forming an element "on top of a substrate" may include forming the element directly on the substrate without any intermediate materials / elements between them, and forming the element indirectly on the substrate with one or more intermediate materials / elements between them.
Claims
1. A method for loading into a bias lookup table, wherein the method is A memory cell capable of storing any of N values is programmed with one of the N values, The steps include passing a series of increasing currents through the bit lines of the programmed memory cell, To generate a comparison output, the steps include comparing the voltage of the bit line with a reference voltage, A method comprising the steps of measuring the voltage at the control gate terminal of the memory cell and storing the voltage in a lookup table when the comparison output changes its value.
2. The method according to claim 1, wherein the memory cell is a non-volatile memory cell.
3. The method according to claim 1, wherein the memory cell is a volatile memory cell.
4. The method according to claim 1, wherein the voltage at the control gate terminal is measured using a sample-and-hold circuit.
5. A voltage averaging circuit for generating a bias, A variable register coupled between the output node and ground, The system comprises multiple measurement blocks, each of which converts its respective input voltage into a current and mirrors the current to the output node. The output node is a voltage averaging circuit that provides the plurality of measurement blocks with a bias equal to the average of the input voltages.
6. The voltage averaging circuit according to claim 5, wherein the voltage at the output node is equal to the sum of the values provided by each measurement block, the value being obtained by multiplying the input voltage received by the measurement block by the ratio of the variable register, dividing by the sum of the variable registers of the measurement block, and multiplying by the sum of the respective input voltages.
7. The voltage averaging circuit according to claim 6, wherein the voltage at the output node is applied to the control gate terminal of one or more cells in a neural network memory array.
8. A method for generating a bias for a memory array, wherein the method is The steps include: programming the memory cell to store a value, The steps include passing a series of increasing currents through the bit lines of the programmed memory cell, A method comprising the step of measuring the voltage of the control gate terminal of the memory cell in order to determine the bias.
9. The method according to claim 8, further comprising the step of storing the determined bias.
10. The method according to claim 9, further comprising the step of applying the bias to one or more memory cells during operation on one or more memory cells in an array of memory cells.
11. The method according to claim 10, wherein the array is an analog neural memory array.
12. The method according to claim 9, comprising the steps of performing the programming step, the flow step, the measurement step, and the storage step for a plurality of different operating temperatures of the programmed memory cell.
13. A method for determining the bias for a memory array in a neural network in real time, wherein the method is The steps include: programming the memory cell to store a value, The steps include: passing a predetermined current through the bit line of the programmed memory cell; A method comprising the step of measuring the voltage of the control gate terminal of the memory cell in order to determine the bias.
14. The method according to claim 13, further comprising the step of storing the bias.
15. The method according to claim 14, further comprising the step of applying the bias to one or more memory cells during operation on one or more memory cells in an array of memory cells.
16. The method according to claim 15, wherein the array is an analog neural memory array.
17. The method according to claim 14, comprising the steps of performing the programming step, the application step, the measurement step, and the storage step for a plurality of different operating temperatures of the programmed memory cell.
18. A method for a neural network, wherein the method is The steps to program the memory cell, The steps include passing a series of increasing currents through the bit lines of the programmed memory cell, To generate a comparison output, the steps include comparing the voltage of the bit line with a reference voltage, The steps include measuring the voltage at the control gate terminal of the memory cell when the comparison output changes value, and storing the voltage as the determined bias, The steps include applying the determined bias to the terminals of the set of memory cells, A method comprising the step of performing a read operation on the set of memory cells.
Citation Information
Patent Citations
Compensation for reference transistors and memory cells in analog neuro memory in deep learning artificial neural network
WO2020013933A1
Temperature and leakage compensation for memory cells in an analog neural memory system used in a deep learning neural network
WO2020046495A1
Testing circuitry and methods for analog neural memory in artificial neural network
WO2021015812A1
