Masking of sparse inputs and outputs in neural network arrays

Abstract

Numerous examples of masking circuits for inputs and outputs in neural network arrays are disclosed. In one example, the system comprises a neural network array including a plurality of nonvolatile memory cells arranged in rows and columns, and a row circuit for each row in the neural network array, the row circuit including a masking circuit to prevent the application of sparse inputs to one or more rows in the array when a condition is met.

Description

[Technical Field]

[0001] (Claiming priority) This application claims priority from U.S. Patent Provisional Application No. 63 / 458,376, filed on April 10, 2023, entitled "Masking Circuit for Inputs and Outputs in Neural Network Array," and from U.S. Patent Application No. 18 / 212,066, filed on June 20, 2023, entitled "Masking Sparse Inputs and Outputs in Neural Network Array."

[0002] (Field of Invention) Numerous examples of masking circuits for inputs and outputs in neural network arrays are disclosed. [Background technology]

[0003] Artificial neural networks mimic biological neural networks (such as the central nervous system of animals, particularly the brain), can depend on a large number of inputs, and are generally used to estimate or approximate unknown functions. Artificial neural networks typically consist of layers of interconnected "neurons" that exchange messages with one another.

[0004] Figure 1 shows an artificial neural network, where circles represent the inputs or layers of neurons. Connections (called synapses) are represented by arrows and have numerical weights that can be tuned based on experience. This allows the neural network to adapt to inputs and learn. Typically, a neural network contains multiple input layers. Typically, there are one or more hidden layers of neurons and output layers of neurons that provide the output of the neural network. Neurons at each level make decisions individually or collectively based on the data they receive from synapses.

[0005] One of the major challenges in developing artificial neural networks for high-performance information processing is the lack of suitable hardware technology. In fact, practical neural networks rely on a very large number of synapses, which enables high connectivity between neurons and thus very high levels of parallel processing. In principle, such complexity can be achieved with digital supercomputers or dedicated graphics processing unit clusters. However, in addition to their high cost, these approaches also suffer from poor energy efficiency compared to biological networks, which consume far less energy because they primarily perform low-precision analog calculations. While CMOS analog circuits have been used in artificial neural networks, the synapses in most CMOS implementations are too large considering the large number of neurons and synapses.

[0006] The applicant previously disclosed, in U.S. Patent Application Publication No. 2017 / 0337466(A1), incorporated by reference, an artificial (analog) neural network utilizing one or more non-volatile memory arrays as synapses. The non-volatile memory arrays operate as analog neural memory and comprise non-volatile memory cells arranged in rows and columns. The neural network device includes a first plurality of synapses configured to receive a first plurality of inputs and therefrom produce a first plurality of outputs, and a first plurality of neurons configured to receive the first plurality of outputs. The first plurality of synapses comprises a plurality of memory cells, each of which includes spaced source and drain regions formed in a semiconductor substrate, with a channel region extending between them, a floating gate insulated and disposed above a first portion of the channel region, and a non-floating gate insulated and disposed above a second portion of the channel region. Each of the plurality of memory cells stores weight values ​​corresponding to the number of electrons on the floating gate. The plurality of memory cells generate a first plurality of outputs by multiplying the first plurality of inputs by the stored weight values. <Non-volatile membrane>

[0007] Non-volatile memory is well known. For example, U.S. Patent No. 5,029,130 ​​("Patent No. 130"), incorporated herein by reference, discloses an array of split-gate non-volatile memory cells, a type of flash memory cell. Such a memory cell 210 is shown in Figure 2. Each memory cell 210 includes a source region 14 and a drain region 16 formed in a semiconductor substrate 12, with a channel region 18 between the source region 14 and the drain region 16. A floating gate 20 is formed insulated above a first portion of the channel region 18 (and controlling the conductivity of the first portion of the channel region 18) and extends above a portion of the source region 14. A word line terminal 22 (typically coupled to a word line) is located above a second portion of the channel region 18 and has a first portion that is insulated from (and controls the conductivity of) the second portion, and a second portion that extends above the floating gate 20. The floating gate 20 and the word line terminal 22 are insulated from the substrate 12 by a gate oxide. Bit line 24 is connected to drain region 16.

[0008] By applying a positive high voltage to the word line terminal 22, the memory cell 210 is erased (electrons are removed from the floating gate), causing the electrons on the floating gate 20 to pass through the intermediate insulator from the floating gate 20 to the word line terminal 22 via a Fowler-Nordheim (FN) tunnel.

[0009] The memory cell 210 is programmed by source-side injection (SSI) with hot electrons by applying a positive voltage to the word line terminal 22 and a positive voltage to the source region 14 (electrons are added to the floating gate). The electron flow flows from the drain region 16 towards the source region 14. The electrons are accelerated and generate heat when they reach the gap between the word line terminal 22 and the floating gate 20. Some of the heated electrons are injected into the floating gate 20 via the gate oxide due to the electrostatic attraction from the floating gate 20.

[0010] The memory cell 210 is read by applying a positive read voltage to the drain region 16 and the word line terminal 22 (turning on the portion of the channel region 18 below the word line terminal). When the floating gate 20 is positively charged (i.e., electrons are erased), the portion of the channel region 18 below the floating gate 20 is also turned on, and current flows through the channel region 18, which is detected as the erased state, or the "1" state. When the floating gate 20 is negatively charged (i.e., electrons are programmed), the portion of the channel region below the floating gate 20 is almost or completely off, and no (or very little) current flows through the channel region 18, which is detected as the programmed state, or the "0" state.

[0011] Table 1 shows typical voltage / current ranges that may be applied to the terminals of the memory cell 210 to perform read, erase, and program operations. Table 1: Operation of flash memory cell 210 in Figure 2 [Table 1]

[0012] Other split-gate type memory cell configurations, which are other types of flash memory cells, are also known. For example, FIG. 3 shows a four-gate memory cell 310 including a source region 14, a drain region 16, a floating gate 20 above a first portion of a channel region 18, a select gate 22 (typically coupled to a word line WL) above a second portion of the channel region 18, a control gate 28 above the floating gate 20, and an erase gate 30 above the source region 14. This configuration is described in U.S. Patent No. 6,747,310, which is incorporated herein by reference for all purposes. Here, all gates except the floating gate 20 are non-floating gates, that is, they are electrically connected or connectable to a voltage source. Programming is performed by injecting the electrons themselves from the channel region 18 into the floating gate 20. Erasure is performed by tunneling of electrons from the floating gate 20 to the erase gate 30.

[0013] Table 2 shows typical voltage / current ranges that can be applied to the terminals of the memory cell 310 to perform read, erase, and program operations. Table 2: Operation of the Flash Memory Cell 310 of FIG. 3

Table 2

[0014] FIG. 4 shows a three-gate memory cell 410, which is another type of flash memory cell. The memory cell 410 is identical to the memory cell 310 of FIG. 3 except that the memory cell 410 does not have a separate control gate. (Erasure occurs through the use of the erase gate) The erase operation and the read operation are the same as those of FIG. 3 except that no control gate bias is applied. The programming operation is also performed without a control gate bias. As a result, a higher voltage is applied to the source line during the programming operation to compensate for the lack of control gate bias.

[0015] Table 3 shows typical voltage / current ranges that may be applied to the terminals of the memory cell 410 to perform read, erase, and program operations. Table 3: Operation of flash memory cell 410 in Figure 4 [Table 3]

[0016] Figure 5 shows another type of flash memory cell, a stacked gate memory cell 510. Memory cell 510 is similar to memory cell 210 in Figure 2, except that the floating gate 20 extends above the entire channel region 18, and the control gate 22 (coupled here to the word line) extends above the floating gate 20, separated by an insulating layer (not shown). Erase is performed by FN tunneling of electrons from the FG to the substrate, programming is performed by channel hot electron (CHE) injection in the region between the channel 18 and the drain region 16, and read operation is performed by electrons flowing from the source region 14 to the drain region 16, similar to the read operation of memory cell 210, which has a higher control gate voltage.

[0017] Table 4 shows typical voltage ranges that can be applied to the terminals of the memory cell 510 and the circuit board 12 for performing read, erase, and program operations. Table 4: Operation of flash memory cell 510 in Figure 5 [Table 4]

[0018] The methods and means described herein may be applied to other non-volatile memory technologies, including but not limited to FINFET split-gate flash or stack-gate flash memory, NAND flash, SONOS (silicon-oxide-nitride-oxide-silicon, charge trap in nitride), MONOS (metal-oxide-nitride-oxide-silicon, metal charge trap in nitride), ReRAM (resistive random-access memory), PCM (phase change memory), MRAM (magnetic random-access memory), FeRAM (ferroelectric random-access memory), CT (charge trap) memory, CN (carbon-tube) memory, OTP (one-time programmable, bi-level or multi-level), and CeRAM (correlated electron random-access memory).

[0019] Two modifications are made to utilize a memory array containing one of the non-volatile memory cell types in the artificial neural network described above. First, lines are configured to allow each memory cell to be programmed, erased, and read individually without adversely affecting the memory state of other memory cells in the array, as will be further described below. Second, sequential (analog) programming of the memory cells is provided.

[0020] Specifically, the memory state of each memory cell in the array (i.e., the charge on the floating gate) can be changed independently and continuously, with minimal disturbance to other memory cells, from a completely erased state to a fully programmed state, and vice versa. This means that cell memory is essentially analog, or can store at least one of a large number of discontinuous values ​​(such as 16 or 64 different values), making every memory cell in the memory array highly precise and individually tunable, and the memory array becomes ideal for memory and fine-tuning of synaptic weights in neural networks. <Neural networks using non-volatile memory cell arrays>

[0021] Figure 6 conceptually illustrates an unrestricted example of a neural network utilizing a non-volatile memory array in this example. While this example uses a non-volatile memory array neural network for a facial recognition application, it is also possible to implement other suitable applications using a non-volatile memory array-based neural network.

[0022] S0 is the input layer, which in this example is a 32x32 pixel RGB image with 5-bit precision (i.e., three 32x32 pixel arrays, one for each color R, G, and B, with each pixel having 5-bit precision). The synapse CB1, going from input layer S0 to layer C1, scans the input image with a 3x3 pixel overlapping filter (kernel), applying different weight sets to some instances and shared weights to others, and shifts the filter by one pixel (or more than one pixel depending on the model). Specifically, the values ​​of nine pixels in the 3x3 portion of the image (i.e., referred to as the filter or kernel) are provided to synapse CB1, where these nine input values ​​are multiplied by appropriate weights, and after adding the outputs of the multiplications, a single output value is determined, which is then given by the first synapse of CB1 to generate one of the pixels in the feature map of layer C1. The 3x3 filter is then shifted one pixel to the right within the input layer S0 (i.e., a column of 3 pixels is added to the right and a column of 3 pixels is dropped to the left), thereby providing the 9 pixel values ​​of this newly positioned filter to synapse CB1, where they are multiplied by the same weights as above, determining a second single output value by the associated synapse. This process continues until the 3x3 filter scans the entire 32x32 pixel image of the input layer S0 for all three colors and all bits (precision values). The process is then repeated with different weight sets to generate different feature maps of layer C1 until all feature maps of layer C1 have been computed.

[0023] In this example, layer C1 contains 16 feature maps, each having 30x30 pixels. Each pixel is a new feature pixel extracted from the multiplication of the input and the kernel; therefore, each feature map is a two-dimensional array, and thus in this example, layer C1 constitutes 16 layers of two-dimensional arrays (note that the layers and arrays referred to herein are logical relationships that may or may not correspond to physical relationships, i.e., arrays may not be oriented to physical two-dimensional arrays). Each of the 16 feature maps in layer C1 is generated by one of 16 different synaptic weight sets applied to the filtered scan. All C1 feature maps can target different aspects of the same image feature, such as boundary identification. For example, a first map (generated using a first weight set shared across all scans used to generate this first map) can identify circular edges, a second map (generated using a second weight set different from the first) can identify rectangular edges or the aspect ratio of a particular feature, and so on.

[0024] Before moving from layer C1 to layer S1, an activation function P1 (pooling) is applied that pools values ​​from non-overlapping, consecutive 2x2 regions within each feature map. The purpose of the pooling function P1 is to average neighbor positions (or use the max function), reduce dependence on edge positions, and reduce data size before proceeding to the next operation. In layer S1, there are 16 15x15 feature maps (i.e., 16 different arrays, each 15x15 pixels). Synapse CB2, moving from layer S1 to layer C2, scans the maps in layer S1 with a 4x4 filter, shifting by 1 pixel. In layer C2, there are 22 12x12 feature maps. Before moving from layer C2 to layer S2, an activation function P2 (pooling) is applied that pools values ​​from non-overlapping, consecutive 2x2 regions within each feature map. In layer S2, there are 22 6x6 feature maps. At synapse CB3, which goes from layer S2 to layer C3, an activation function (pooling) is applied, where all neurons in layer C3 are connected to all maps in layer S2 via each synapse of CB3. There are 64 neurons in layer C3. Synapse CB4, which goes from layer C3 to output layer S3, completely connects C3 to S3; that is, all neurons in layer C3 are connected to all neurons in layer S3. The output in S3 contains 10 neurons, where the neuron with the highest output determines the class. This output can, for example, indicate the identification or classification (classification) of the content of the original image.

[0025] Each layer of a synapse is implemented using an array or part of an array of non-volatile memory cells.

[0026] Figure 7 is a block diagram of an array that can be used for that purpose. The vector-by-matrix multiplication (VMM) array 32 contains non-volatile memory cells and is used as synapses between one layer and the next (e.g., CB1, CB2, CB3, and CB4 in Figure 6). Specifically, the VMM array 32 includes an array of non-volatile memory cells 33, erase gate and word line gate decoders 34, a control gate decoder 35, a bit line decoder 36, and a source line decoder 37, each of which decoders decodes its respective input to the non-volatile memory cell array 33. Input to the VMM array 32 can be from the erase gate and word line gate decoder 34 or from the control gate decoder 35. In this example, the source line decoder 37 also decodes the output of the non-volatile memory cell array 33. Alternatively, the bit line decoder 36 can decode the output of the non-volatile memory cell array 33.

[0027] The non-volatile memory cell array 33 serves two purposes. First, it stores the weights used by the VMM array 32. Second, the non-volatile memory cell array 33 effectively multiplies the weights stored in it by the inputs and adds them together for each output line (source line or bit line) to generate an output, which becomes the input to the next layer or the last layer. By having the non-volatile memory cell array 33 perform the multiplication and addition functions, the need for separate multiplication and addition logic circuits is eliminated, and the calculations are more power-efficient due to being performed in memory.

[0028] The output of the non-volatile memory cell array 33 is fed to a differential adder (such as an adding operational amplifier or adding current mirror) 38, which adds the outputs of the non-volatile memory cell array 33 to create a single value for its convolution. The differential adder 38 is configured to perform the summation of positive and negative weights.

[0029] The summed output values ​​of the differential adder 38 are then fed to an activation function block 39, which normalizes the output. The activation function block 39 may provide a sigmoid, tanh, or ReLU function. The normalized output values ​​of the activation function block 39 become elements of a feature map as the next layer (e.g., C1 in Figure 6), and are then applied to the next synapse to generate the next feature map layer or the final layer. Thus, in this example, the non-volatile memory cell array 33 constitutes multiple synapses (receiving input from the previous layer of the neuron or from an input layer such as an image database), and the summation amplifier 38 and activation function block 39 constitute multiple neurons.

[0030] The inputs to the VMM array 32 in Figure 7 (WLx, EGx, CGx, and optionally BLx and SLx) can be analog level, binary level, or digital bits (in which case a DAC is provided to convert the digital bits to the appropriate input analog level), and the outputs can be analog level, binary level, or digital bits (in which case an output ADC is provided to convert the output analog level to the digital bits).

[0031] Figure 8 is a block diagram showing the use of multiple layers of the VMM array 32, labeled in the figure as VMM arrays 32a, 32b, 32c, 32d, and 32e. As shown in Figure 8, the input (indicated as Inputx) is converted from digital to analog by the digital-to-analog converter 31 and provided to the input VMM array 32a. The converted analog input can be voltage or current. Input D / A conversion of the first layer can be performed by using a function or LUT (look-up table) that maps the input Inputx to the appropriate analog level of the matrix multiplier of the input VMM array 32a. Input conversion can also be performed by an analog-to-analog (A / A) converter to convert an external analog input to the mapped analog input to the input VMM array 32a.

[0032] The output generated by input VMM array 32a is provided as input to the next VMM array (hidden level 1) 32b, the next VMM array (hidden level 1) 32b generates an output that is provided as input to the next input VMM array (hidden level 2) 32c, and so on. The various layers of VMM array 32 function as the synapses and neurons of a convolutional neural network (CNN). Each VMM array 32a, 32b, 32c, 32d, and 32e can be a standalone physical non-volatile memory array, or multiple VMM arrays can utilize different parts of the same physical non-volatile memory array, or multiple VMM arrays can utilize overlapping parts of the same physical non-volatile memory array. The example shown in Figure 8 includes five layers (32a, 32b, 32c, 32d, 32e), namely one input layer (32a), two hidden layers (32b, 32c), and two fully connected layers (32d, 32e). Those skilled in the art will understand that this is merely an example, and that a system could instead include more than two hidden layers and more than two fully connected layers. <Vector × Matrix Multiplication (VMM) Array>

[0033] Figure 9 shows a neuron VMM array 900, which is particularly suitable for the memory cell 310 shown in Figure 3 and is used as part of a synapse and neuron between the input layer and the next layer. The VMM array 900 includes a memory array 901 of non-volatile memory cells and a reference array 902 of non-volatile reference memory cells (located at the top of the array). Alternatively, another reference array may be located at the bottom.

[0034] In the VMM array 900, control gate lines such as control gate line 903 extend vertically (thus the row-direction reference array 902 is perpendicular to the control gate line 903), and erase gate lines such as erase gate line 904 extend horizontally. Here, inputs to the VMM array 900 are provided to the control gate lines (CG0, CG1, CG2, CG3), and outputs of the VMM array 900 appear on the source lines (SL0, SL1). In one example, only even rows are used, and in another example, only odd rows are used. The current on each source line (SL0, SL1, respectively) performs the function of summing all currents from memory cells connected to that particular source line.

[0035] As described herein with respect to neural networks, the non-volatile memory cells of the VMM array 900, i.e., the memory cells 310 of the VMM array 900, may be configured to operate selectively in a region below a threshold.

[0036] The non-volatile reference memory cells and non-volatile memory cells described herein are biased with weak inversion (in the region below the threshold) as follows: Ids = Io × e (Vg-Vth) / nVt =w × Io × e (Vg) / nVt , In the formula, w=e (-Vth) / nVt And, Ids is the drain-source current, Vg is the gate voltage on the memory cell, Vth is the threshold voltage of the memory cell, Vt is the thermal voltage = k × T / q, where k is Boltzmann's constant, T is the Kelvin temperature, q is the electron charge, n is the gradient coefficient = 1 + (Cdep / Cox), where Cdep is the capacitance of the depletion layer, and Cox is the capacitance of the gate oxide layer, Io is the memory cell current at a gate voltage equal to the threshold voltage, and Io is (Wt / L) × u ​​× Cox × (n-1) × Vt 2 It is proportional to , where u is the carrier mobility, and Wt and L are the width and length of the memory cell, respectively.

[0037] When using an IV logarithmic converter that converts input current to input voltage using a memory cell (such as a reference memory cell or peripheral memory cell) or transistor: Vg = n × Vt × log[Ids / wp × Io] In the formula, wp is the w of the reference or peripheral memory cell.

[0038] For a memory array used as a vector × matrix multiplier VMM array with current input, the output current is as follows: Iout=wa×Io×e (Vg) / nVt That is to say Iout = (wa / wp) × Iin = W × Iin W=e (Vthp-Vtha) / nVt Here, wa = w of each memory cell in the memory array. Vthp is the effective threshold voltage of the peripheral memory cell, and Vtha is the effective threshold voltage of the main (data) memory cell. Note that the threshold voltage of a transistor is a function of the substrate bias voltage, and the substrate bias voltage, denoted as Vsb, can be modulated to compensate for various conditions at such temperatures. The threshold voltage Vth can be expressed as follows: Vth=Vth0+γ(SQRT|Vsb-2×φF)-SQRT|2×φF|) In the formula, Vth0 is the threshold voltage with zero substrate bias, φF is the surface potential, and γ is the body effect parameter.

[0039] Word lines or control gates can be used as inputs to memory cells for input voltage.

[0040] Alternatively, the flash memory cells of the VMM array described herein can be configured to operate in a linear region. Ids=β×(Vgs-Vth)×Vds, β=u×Cox×Wt / L W=α(Vgs-Vth) In other words, the weight W in the linear region is proportional to (Vgs - Vth).

[0041] A word line, a control gate, a bit line, or a source line can be used as an input to a memory cell operating in the linear region. A bit line or a source line can be used as an output of a memory cell.

[0042] For an I-V linear converter, a memory cell (such as a reference memory cell or a peripheral memory cell) or a transistor operating in the linear region can be used to linearly convert an input / output current into an input / output voltage.

[0043] Alternatively, the memory cells of the VMM array described herein can be configured to operate in the saturation region. Ids = 1 / 2×β×(Vgs - Vth) 2 , where β = u×Cox×Wt / L W ∝ (Vgs - Vth) 2 , that is, the weight W is proportional to (Vgs - Vth) 2 proportional.

[0044] A word line, a control gate, or an erase gate can be used as an input to a memory cell operating in the saturation region. A bit line or a source line can be used as an output of an output neuron.

[0045] Alternatively, the memory cells of the VMM array described herein can be used in all regions or combinations thereof (below threshold, linear, or saturation) for each layer or multiple layers of a neural network.

[0046] Another example for the VMM array 32 of FIG. 7 is described in U.S. Patent No. 10,748,630, which is incorporated herein by reference. As described in the above application, a source line or a bit line can be used as a neuron output (current sum output).

[0047] Figure 10 shows a neuron VMM array 1000, particularly suited to the memory cell 210 shown in Figure 2 and used as a synapse between the input layer and the next layer. The VMM array 1000 includes a memory array 1003 of non-volatile memory cells, a reference array 1001 of first non-volatile reference memory cells, and a reference array 1002 of second non-volatile reference memory cells. The reference arrays 1001 and 1002, arranged in the column direction of the array, function to convert current inputs flowing into terminals BLR0, BLR1, BLR2, and BLR3 into voltage inputs WL0, WL1, WL2, and WL3. In practice, the first and second non-volatile reference memory cells are diode-connected through a multiplexer 1014 (partially drawn) with current inputs flowing in. The reference cells are tuned (e.g., programmed) to a target reference level. The target reference level is provided by a reference miniarray matrix (not shown).

[0048] The memory array 1003 serves two purposes. First, it stores the weights used by the VMM array 1000 in each memory cell. Second, the memory array 1003 effectively multiplies the weights stored in it by the inputs (i.e., the current inputs supplied to terminals BLR0, BLR1, BLR2, and BLR3, which are converted into input voltages by the reference arrays 1001 and 1002 and supplied to word lines WL0, WL1, WL2, and WL3), and then adds all the results (memory cell currents) to generate the outputs of each bit line (BL0~BLN), which become inputs to the next layer or the last layer. By performing multiplication and addition functions, the memory array 1003 eliminates the need for separate multiplication and addition logic circuits and is also power efficient. Here, voltage inputs are supplied to word lines WL0, WL1, WL2, and WL3, and outputs appear on the respective bit lines BL0 to BLN during the read (inference) operation. Each current in bit lines BL0 to BLN performs the function of summing the currents from all non-volatile memory cells connected to that particular bit line.

[0049] Table 5 shows the operating voltages and currents of the VMM array 1000. The columns in the table show the voltages applied to the word lines of selected cells, word lines of unselected cells, bit lines of selected cells, bit lines of unselected cells, source lines of selected cells, and source lines of unselected cells. The rows show the read, erase, and program operations. Table 5: Operation of VMM Array 1000 in Figure 10 [Table 5]

[0050] Figure 11 shows a neuron VMM array 1100, which is particularly suitable for the memory cell 210 shown in Figure 2 and is used as part of a synapse and neuron between the input layer and the next layer. The VMM array 1100 includes a memory array 1103 of non-volatile memory cells, a reference array 1101 of a first non-volatile reference memory cell, and a reference array 1102 of a second non-volatile reference memory cell. The reference arrays 1101 and 1102 extend in the row direction of the VMM array 1100. The VMM array is similar to the VMM 1000, except that the word lines in the VMM array 1100 extend vertically. Here, inputs are provided to the word lines (WLA0, WLB0, WLA1, WLB2, WLA2, WLB2, WLA3, WLB3), and outputs appear on the source lines (SL0, SL1) during read operations. The current on each source line performs the function of summing all the currents from the memory cells connected to that particular source line.

[0051] Table 6 shows the operating voltages and currents of the VMM array 1100. The columns in the table show the voltages applied to the word lines of selected cells, word lines of unselected cells, bit lines of selected cells, bit lines of unselected cells, source lines of selected cells, and source lines of unselected cells. The rows show the read, erase, and program operations. Table 6: Operation of VMM Array 1100 in Figure 11 [Table 6]

[0052] Figure 12 shows a neuron VMM array 1200, which is particularly suitable for the memory cell 310 shown in Figure 3 and is used as part of a synapse and neuron between the input layer and the next layer. The VMM array 1200 includes a memory array 1203 of nonvolatile memory cells, a reference array 1201 of a first nonvolatile reference memory cell, and a reference array 1202 of a second nonvolatile reference memory cell. The reference arrays 1201 and 1202 function to convert the current inputs flowing into terminals BLR0, BLR1, BLR2, and BLR3 into voltage inputs CG0, CG1, CG2, and CG3. In practice, the first and second nonvolatile reference memory cells are diode-connected through a multiplexer 1212 (partially shown) with current inputs flowing through BLR0, BLR1, BLR2, and BLR3. Each multiplexer 1212 includes a separate multiplexer 1205 and a cascoding transistor 1204 to ensure that the respective bit lines (such as BLR0) of the first and second non-volatile reference memory cells maintain a constant voltage during read operations. The reference cells are tuned to a target reference level.

[0053] The memory array 1203 serves two purposes. First, it stores the weights used by the VMM array 1200. Second, the memory array 1203 effectively multiplies the weights stored in the memory array by the inputs (current inputs supplied to terminals BLR0, BLR1, BLR2, and BLR3, which the reference arrays 1201 and 1202 convert into input voltages and supply to the control gates (CG0, CG1, CG2, and CG3)), then adds all the results (cell currents) to produce an output, which appears in BL0~BLN and becomes the input to the next layer or the last layer. By having the memory array perform the multiplication and addition functions, the need for separate multiplication and addition logic circuits is eliminated, and power efficiency is also improved. Here, the inputs are provided to the control gate lines (CG0, CG1, CG2, and CG3), and the output appears in the bit lines (BL0~BLN) during read operations. The current in each bit line performs the function of adding up all the currents from the memory cells connected to that particular bit line.

[0054] The VMM array 1200 implements one-way tuning of non-volatile memory cells within the memory array 1203. That is, each non-volatile memory cell is erased and then partially programmed until the desired charge on the floating gate is reached. If too much charge is applied to the floating gate (resulting in an incorrect value being stored in the cell), the cell is erased and the series of partial programming operations is restarted from the beginning. As shown, two rows sharing the same erase gate (such as EG0 or EG1) are erased together (which may also be called a page erase), and then each cell is partially programmed until the desired charge on the floating gate is reached.

[0055] Table 7 shows the operating voltages and currents of the VMM array 1200. The columns in the table show the voltages applied to the word lines of selected cells, word lines of unselected cells, bit lines of selected cells, bit lines of unselected cells, control gates of selected cells, control gates of unselected cells in the same sector as the selected cell, control gates of unselected cells in a different sector than the selected cell, erase gates of selected cells, erase gates of unselected cells, source lines of selected cells, and source lines of unselected cells. The rows show the read, erase, and program operations. Table 7: Operation of VMM Array 1200 in Figure 12 [Table 7]

[0056] Figure 13 shows a neuron VMM array 1300, which is particularly suitable for the memory cell 310 shown in Figure 3 and is used as part of a synapse and neuron between the input layer and the next layer. The VMM array 1300 comprises a memory array 1303 of nonvolatile memory cells, a reference array 1301 or a first nonvolatile reference memory cell, and a reference array 1302 of a second nonvolatile reference memory cell. The EG lines EGR0, EG0, EG1, and EGR1 extend vertically, and the CG lines CG0, CG1, CG2, and CG3 and the SL lines WL0, WL1, WL2, and WL3 extend horizontally. The VMM array 1300 is similar to the VMM array 1400 except that the VMM array 1300 implements bidirectional tuning, and each individual cell can be completely erased, partially programmed, and partially erased as needed to reach a desired amount of charge on the floating gate by using individual EG lines. As shown, reference arrays 1301 and 1302 convert the input currents at terminals BLR0, BLR1, BLR2, and BLR3 into control gate voltages CG0, CG1, CG2, and CG3 (through the action of diode-connected reference cells via multiplexer 1314), and these voltages are applied to memory cells in the row direction. Current outputs (neurons) are located in the bit lines BL0~BLN, and each bit line sums all the currents from the non-volatile memory cells connected to that particular bit line.

[0057] Table 8 shows the operating voltages and currents of the VMM array 1300. The columns in the table show the voltages applied to the word lines of selected cells, word lines of unselected cells, bit lines of selected cells, bit lines of unselected cells, control gates of selected cells, control gates of unselected cells in the same sector as the selected cell, control gates of unselected cells in a different sector than the selected cell, erase gates of selected cells, erase gates of unselected cells, source lines of selected cells, and source lines of unselected cells. The rows show the read, erase, and program operations. Table 8: Operation of VMM Array 1300 in Figure 13 [Table 8]

[0058] Figure 22 shows a neuron VMM array 2200 that is particularly suitable for the memory cell 210 shown in Figure 2 and is used as part of the synapse and neuron between the input layer and the next layer. In the VMM array 2200, inputs INPUT0...., INPUT N These are bit lines BL0, ...BL, respectively. N The signal is received, and outputs OUTPUT1, OUTPUT2, OUTPUT3, and OUTPUT4 are generated on source lines SL0, SL1, SL2, and SL3, respectively.

[0059] Figure 23 shows a neuron VMM array 2300 that is particularly suitable for the memory cell 210 shown in Figure 2 and is used as part of the synapses and neurons between the input layer and the next layer. In this example, inputs INPUT0, INPUT1, INPUT2, and INPUT3 are received by source lines SL0, SL1, SL2, and SL3, respectively, and outputs OUTPUT0, ...OUTPUT N These are bit lines BL0, ..., BL N It is generated by [this method].

[0060] Figure 24 shows a neuron VMM array 2400 that is particularly suitable for the memory cell 210 shown in Figure 2 and is used as part of the synapses and neurons between the input layer and the next layer. In this example, inputs INPUT0, ..., INPUT M These are the word lines WL0, ..., WL, respectively. M Received by, output OUTPUT0, ...OUTPUT N These are bit lines BL0, ..., BL N It is generated by [this method].

[0061] Figure 25 shows a neuron VMM array 2500 that is particularly suitable for the memory cell 310 shown in Figure 3 and is used as part of the synapses and neurons between the input layer and the next layer. In this example, inputs INPUT0, ..., INPUT M These are the word lines WL0, ..., WL, respectively. MReceived by, output OUTPUT0, ...OUTPUT N These are bit lines BL0, ..., BL N It is generated by [this method].

[0062] Figure 26 shows a neuron VMM array 2600 that is particularly suitable for the memory cell 410 shown in Figure 4 and is used as part of the synapses and neurons between the input layer and the next layer. In this example, inputs INPUT0, ..., INPUT n However, each of them is a vertical control gate line CG0, ..., CG N The signal is received, and outputs OUTPUT1 and OUTPUT2 are generated on source lines SL0 and SL1.

[0063] Figure 27 shows a neuron VMM array 2700 that is particularly suitable for the memory cell 410 shown in Figure 4 and is used as part of the synapses and neurons between the input layer and the next layer. In this example, the inputs are INPUT0, ..., INPUT N These are bit lines BL0, ..., BL, respectively. N The bit line control gates 2701-1, 2701-2, ..., 2701-(N-1) and 2701-N, which are coupled to the bit line control gates, are received by the gates. Exemplary outputs OUTPUT1 and OUTPUT2 are generated on source lines SL0 and SL1.

[0064] Figure 28 shows a neuron VMM array 2800, which is particularly suitable for the memory cell 310 shown in Figure 3, the memory cell 510 shown in Figure 5, and the memory cell 710 shown in Figure 7, and is used as part of synapses and neurons between the input layer and the next layer. In this example, inputs INPUT0, ..., INPUT M These are the word lines WL0, ..., WL, respectively. M Received by, output OUTPUT0, ..., OUTPUT N These are bit lines BL0, ..., BL N It is generated in [location].

[0065] Figure 29 shows a neuron VMM array 2900 that is particularly suitable for the memory cell 310 shown in Figure 3, the memory cell 510 shown in Figure 5, and the memory cell 710 shown in Figure 7, and is used as part of synapses and neurons between the input layer and the next layer. In this example, inputs INPUT0, ..., INPUT M These are control gate lines CG0, ..., CG M It is received as follows. Output OUTPUT0, ..., OUTPUT N These are the vertical source lines SL0, ..., SL, respectively. N It is generated in each source line SL i It is coupled to the source lines of all memory cells in column i.

[0066] Figure 30 shows a neuron VMM array 3000, which is particularly suitable for the memory cell 310 shown in Figure 3, the memory cell 510 shown in Figure 5, and the memory cell 710 shown in Figure 7, and is used as part of synapses and neurons between the input layer and the next layer. In this example, inputs INPUT0, ..., INPUT M These are control gate lines CG0, ..., CG M It is received as follows. Output OUTPUT0, ..., OUTPUT N These are the vertical bit lines BL0, ..., BL N Generated in each bit line BL i It is coupled to the bit lines of all memory cells in column i. <Long-term and short-term memory>

[0067] Prior art includes a concept called long short-term memory (LSTM). LSTM units are often used within neural networks. LSTMs allow neural networks to store information for a predetermined period and use that information in subsequent operations. A conventional LSTM unit includes a cell, an input gate, an output gate, and a forget gate. The three gates regulate the flow of information into and out of the cell, and the duration for which information is stored within the LSTM. VMMs are particularly useful in LSTM units.

[0068] Figure 14 shows an exemplary LSTM1400. In this example, the LSTM1400 includes cells 1401, 1402, 1403, and 1404. Cell 1401 receives the input vector x0 and generates the output vector h0 and the cell state vector c0. Cell 1402 receives the input vector x1 and the output vector (hidden state) h0 from cell 1401. 、 Cell 1401 receives the cell state c0 from cell 1401 and generates the output vector h1 and the cell state vector c1. Cell 1403 receives the input vector x2, the output vector (hidden state) h1 from cell 1402, and the cell state c1 from cell 1402 and generates the output vector h2 and the cell state vector c2. Cell 1404 receives the input vector x3, the output vector (hidden state) h2 from cell 1403, and the cell state c2 from cell 1403 and generates the output vector h3. Additional cells are also available, and an LSTM with four cells is just an example.

[0069] Figure 15 shows an exemplary implementation of LSTM cell 1500 that can be used for cells 1401, 1402, 1403, and 1404 in Figure 14. LSTM cell 1500 receives an input vector x(t), a cell state vector c(t-1) from a preceding cell, and an output vector h(t-1) from a preceding cell, and generates the cell state vector c(t) and output vector h(t).

[0070] LSTM cell 1500 includes sigmoid function devices 1501, 1502, and 1503, each of which applies a number between 0 and 1 to control the degree to which each component of the input vector contributes to the output vector. LSTM cell 1500 also includes tanh devices 1504 and 1505 for applying a hyperbolic tangent function to the input vector, multiplier devices 1506, 1507, and 1508 for multiplying two vectors, and an adder device 1509 for adding two vectors. The output vector h(t) can be provided to the next LSTM cell in the system or accessed for other purposes.

[0071] Figure 16 shows an LSTM cell 1600, which is an example implementation of LSTM cell 1500. For the reader's convenience, the same numbering method used in LSTM cell 1500 is used in LSTM cell 1600. Sigmoid function devices 1501, 1502, and 1503, and tanh device 1504 each contain multiple VMM arrays 1601 and activation function blocks 1602. Thus, VMM arrays are found to be particularly useful in LSTM cells used in certain neural network systems. Multiplier devices 1506, 1507, and 1508, and adder device 1509 are implemented in a digital or analog manner. Activation function block 1602 can be implemented in a digital or analog manner.

[0072] Figure 17 shows an alternative example of LSTM cell 1600 (and another example of an implementation of LSTM cell 1500). In Figure 17, sigmoid function devices 1501, 1502, and 1503, and tanh device 1504 share the same physical hardware (VMM array 1701 and activation function block 1702) in a time-division multiplexed manner. The LSTM cell 1700 also includes a multiplier device 1703 for multiplying two vectors, an adder device 1708 for adding two vectors, a tanh device 1505 (including an activation function block 1702), a register 1707 for storing the value i(t) when i(t) is output from the sigmoid function block 1702, a register 1704 for storing the value f(t) × c(t-1) when its value is output from the multiplier device 1703 via the multiplexer 1710, a register 1705 for storing the value i(t) × u(t) when its value is output from the multiplier device 1703 via the multiplexer 1710, a register 1706 for storing the value o(t) × c(t) when its value is output from the multiplier device 1703 via the multiplexer 1710, and a multiplexer 1709.

[0073] While an LSTM cell 1600 contains multiple sets of VMM arrays 1601 and their respective activation function blocks 1602, an LSTM cell 1700 contains a single set of VMM arrays 1701 and activation function blocks 1702, which are used to represent multiple layers in the example of an LSTM cell 1700. Compared to an LSTM cell 1600, an LSTM cell 1700 requires only 1 / 4 the space for the VMMs and activation function blocks, thus requiring less space than an LSTM cell 1600.

[0074] An LSTM unit typically includes multiple VMM arrays, each of which utilizes functionality provided by specific circuit blocks outside the VMM array, such as adder and activation function blocks, as well as high-voltage generation blocks. Providing separate circuit blocks for each VMM array would require a considerable amount of space within the semiconductor device and would be somewhat inefficient. Therefore, the examples described below reduce the amount of circuitry provided outside the VMM array itself. <Gated recurrent unit>

[0075] Analog VMM implementations can be used in GRU (gated recurrent unit) systems. A GRU is a gate mechanism within an iterative neural network. GRUs are similar to LSTMs, except that GRU cells generally contain fewer components than LSTM cells.

[0076] Figure 18 shows an exemplary GRU1800. In this example, the GRU1800 includes cells 1801, 1802, 1803, and 1804. Cell 1801 receives input vector x0 and produces output vector h0. Cell 1802 receives input vector x1 and output vector h0 from cell 1801 and produces output vector h1. Cell 1803 receives input vector x2 and output vector (hidden state) h1 from cell 1802 and produces output vector h2. Cell 1804 receives input vector x3 and output vector (hidden state) h2 from cell 1803 and produces output vector h3. Additional cells are also available, and a GRU with four cells is just an example.

[0077] Figure 19 shows an exemplary implementation of a GRU cell 1900 that may be used in cells 1801, 1802, 1803, and 1804 of Figure 18. The GRU cell 1900 takes an input vector x(t) and an output vector h(t-1) from a preceding GRU cell and produces an output vector h(t). The GRU cell 1900 includes sigmoid function devices 1901 and 1902, each of which applies a number between 0 and 1 to the components from the output vector h(t-1) and the input vector x(t). The GRU cell 1900 also includes a tanh device 1903 for applying a hyperbolic tangent function to the input vector, multiple multiplier devices 1904, 1905, and 1906 for multiplying two vectors, an adder device 1907 for adding two vectors, and a complementary device 1908 for subtracting the input from 1 to produce an output.

[0078] Figure 20 shows GRU cell 2000, an example implementation of GRU cell 1900. For the reader's convenience, the same numbering method used in GRU cell 1900 is used in GRU cell 2000. As can be seen from Figure 20, the sigmoid function devices 1901 and 1902, and the tanh device 1903, each contain multiple VMM arrays 2001 and activation function blocks 2002. Thus, it can be seen that VMM arrays are used in particular in GRU cells used in specific neural network systems. The multiplier devices 1904, 1905, 1906, the adder device 1907, and the complementary device 1908 are implemented in a digital or analog manner. The activation function block 2002 can be implemented in a digital or analog manner.

[0079] Figure 21 shows an alternative example of the GRU cell 2000 (and another example of an implementation of the GRU cell 1900). In Figure 21, the GRU cell 2100 utilizes the VMM array 2101 and the activation function block 2102, which, when configured as a sigmoid function, applies a number between 0 and 1 to control the extent to which each component of the input vector contributes to the output vector. In Figure 21, the sigmoid function devices 1901 and 1902, and the tanh device 1903, share the same physical hardware (VMM array 2101 and activation function block 2102) in a time-division multiplexed manner. The GRU cell 2100 also includes a multiplier device 2103 for multiplying two vectors, an adder device 2105 for adding two vectors, a complementary device 2109 for subtracting an input from 1 to produce an output, a multiplexer 2104, a register 2106 for holding the value h(t-1)×r(t) when that value is output from the multiplier device 2103 via the multiplexer 2104, a register 2107 for holding the value h(t-1)×z(t) when that value is output from the multiplier device 2103 via the multiplexer 2104, and a register 2108 for holding the value h^(t)×(1-z(t)) when that value is output from the multiplier device 2103 via the multiplexer 2104.

[0080] While GRU cell 2000 contains multiple sets of VMM array 2001 and activation function block 2002, GRU cell 2100 contains one set of VMM array 2101 and activation function block 2102, which are used to represent multiple layers in the example of GRU cell 2100. GRU cell 2100 requires less space than GRU cell 2000 because it requires only one-third the space for the VMM and activation function block.

[0081] A GRU system typically includes multiple VMM arrays, each of which utilizes functions provided by specific circuit blocks outside the VMM array, such as adder and activation function blocks, and high-voltage generation blocks. Providing separate circuit blocks for each VMM array would require a considerable amount of space within the semiconductor device and would be somewhat inefficient. Therefore, the examples described below reduce the circuitry outside the VMM array itself.

[0082] The input to the VMM array may be analog level, binary level, pulse, time-modulated pulse, or digital bit (in which case a DAC is used to convert the digital bit to an appropriate input analog level), and the output may be analog level, binary level, timing pulse, pulse, or digital bit (in which case an output ADC is used to convert the output analog level to a digital bit).

[0083] Typically, for each memory cell in a VMM array, each weight W can be implemented by a single memory cell, a differential cell, or two blended memory cells (the average of two cells). In the case of a differential cell, two memory cells are used to implement the weight W as a differential weight (W = W+-W-). In the case of two blended memory cells, two memory cells are used to implement the weight W as the average of two cells.

[0084] Figure 31 shows the VMM system 3100. In some examples, the weights W stored in the VMM array are stored as differential pairs, W+ (positive weight) and W- (negative weight), where W = (W+) - (W-). In the VMM system 3100, half of the multiple bit lines are designated as W+ lines, i.e., bit lines that will connect to memory cells that will store the positive weight W+, and the other half of the multiple bit lines are designated as W- lines, i.e., bit lines that will connect to memory cells that will give the negative weight W-. The W- lines are interspersed alternately between the W+ lines. Subtraction operations are performed by adders, such as adders 3101 and 3102, which receive current from the W+ and W- lines. The outputs of the W+ lines and the W- lines are combined to effectively give W = W+ - W- for each pair of (W+, W-) cells in all pairs of (W+, W-) lines. Up to this point, we have described W- lines that are alternately scattered between W+ lines, but in other examples, W+ and W- lines can be arbitrarily placed anywhere within the array.

[0085] Figure 32 shows another example. In the VMM system 3210, positive weights W+ are implemented in the first array 3211, and negative weights W- are implemented in the second array 3212, which is separate from the first array, and the resulting weights are appropriately combined by the adder circuit 3213.

[0086] Figure 33 shows the VMM system 3300. The weights W stored in the VMM array are stored as differential pairs, W+ (positive weight) and W- (negative weight), where W = (W+) - (W-). The VMM system 3300 comprises arrays 3301 and 3302. Half of the multiple bit lines in each of arrays 3301 and 3302 are designated as W+ lines, i.e., bit lines connected to memory cells that store the positive weights W+, and the other half of the multiple bit lines in each of arrays 3301 and 3302 are designated as W- lines, i.e., bit lines connected to memory cells that give the negative weights W-. The W- lines are interspersed alternately between the W+ lines. Subtraction operations are performed by adders, such as adders 3303, 3304, 3305, and 3306, which receive current from the W+ and W- lines. The outputs of the W+ line and the W- line from each array 3301 and 3302 are combined together to effectively give W=W+-W- for each pair of (W+, W-) cells in all pairs of (W+, W-) lines. In addition, the W values ​​from each array 3301 and 3302 can be further combined through adders 3307 and 3308, so that each W value is the result of subtracting the W value from array 3302 from the W value from array 3301, and the final result from adders 3307 and 3308 is the difference between the two difference values.

[0087] Each non-volatile memory cell used in an analog neural memory system is erased and programmed to hold a very specific and precise amount of charge, i.e., the number of electrons, within a floating gate. For example, each floating gate holds one of N different values, where N is the number of different weights that each cell can represent. Examples of N include 16, 32, 64, 128, and 256.

[0088] During a neural readout operation, all rows in the VMM array are read out at once, which involves applying row inputs to all rows in the array and reading out the resulting output currents, which are typically received from the columns of the array. As described herein, row inputs can include activation inputs, feature inputs, inputs to neural networks, outputs received by one neural network layer from another, and other types of data that may be applied to the rows. Often, one or more rows receive sparse row inputs, but operations are still performed on those rows, ultimately resulting in unnecessary power consumption and sometimes latency.

[0089] What is needed is a mechanism to detect sparse row inputs and prevent their activation during neural readout operations. Similarly, what is needed is a mechanism to detect sparse output currents and prevent their activation during neural readout operations. [Overview of the project]

[0090] Numerous examples of circuits and methods for masking sparse inputs and outputs in neural network arrays are disclosed.

[0091]

[0092]

[0093]

[0094]

[0095]

[0096]

[0097]

[0098]

[0099]

[0100]

[0101]

[0102]

[0103]

[0104]

[0105]

[0106]

[0107]

[0108]

[0109]

[0110]

[0111]

[0112]

[0113]

[0114]

[0115]

[0116]

[0117]

[0118]

[0119]

[0120]

[0121]

[0122]

[0123]

[0124]

[0125]

[0126]

[0127]

[0128]

[0129]

[0130]

[0131]

[0132]

[0133]

[0134]

[0135]

[0136]

[0137]

[0138]

[0139]

[0140]

[0141]

[0142]

[0143]

[0144]

[0145]

[0146]

[0147]

[0148]

[0149] [Brief explanation of the drawing]

[0150] [Figure 1] This is a diagram of an artificial neural network. [Figure 2] This shows a prior art split-gate flash memory cell. [Figure 3] This shows another prior art split-gate flash memory cell. [Figure 4] This shows another prior art split-gate flash memory cell. [Figure 5] This shows another prior art split-gate flash memory cell. [Figure 6] This figure shows various levels of examples of artificial neural networks that utilize one or more non-volatile memory arrays. [Figure 7] This is a block of VMM systems. [Figure 8] This block diagram shows an exemplary artificial neural network utilizing one or more VMM systems. [Figure 9] Here is another example of a VMM system. [Figure 10] Here is another example of a VMM system. [Figure 11] Here is another example of a VMM system. [Figure 12] Here is another example of a VMM system. [Figure 13] Here is another example of a VMM system. [Figure 14] This demonstrates prior art long- and short-term memory systems. [Figure 15] This shows an example cell used in long- and short-term memory systems. [Figure 16] Figure 15 shows an exemplary embodiment of the cell. [Figure 17] Another exemplary embodiment of the cell shown in Figure 15 is presented. [Figure 18] This shows a prior art gated regression unit system. [Figure 19] An exemplary cell for use in a gated regressive unit system is shown. [Figure 20] Figure 19 shows an exemplary embodiment of the cell. [Figure 21]Another exemplary embodiment of the cell shown in Figure 19 is presented. [Figure 22] Here is another example of a VMM system. [Figure 23] Here is another example of a VMM system. [Figure 24] Here is another example of a VMM system. [Figure 25] Here is another example of a VMM system. [Figure 26] Here is another example of a VMM system. [Figure 27] Here is another example of a VMM system. [Figure 28] Here is another example of a VMM system. [Figure 29] Here is another example of a VMM system. [Figure 30] Here is another example of a VMM system. [Figure 31] Here is another example of a VMM system. [Figure 32] Here is another example of a VMM system. [Figure 33] Here is another example of a VMM system. [Figure 34] This shows the VMM system. [Figure 35] An example of a weight distribution in a VMM system is shown. [Figure 36] An example of row input in a VMM system is shown. [Figure 37] This document describes a method for analyzing row input in a VMM system. [Figure 38] This document presents another method for analyzing row input in a VMM system. [Figure 39] This shows a masking circuit for input in a VMM system. [Figure 40A] This shows another masking circuit for inputs in a VMM system. [Figure 40B] This shows a global deactivation circuit for the VMM system. [Figure 40C] This shows the circuitry for setting bits in the row register and tag bits. [Figure 40D]This shows a circuit to disable the sampling logic. [Figure 40E] This shows the circuit for disabling the sample-and-hold buffer. [Figure 41] This shows another masking circuit for inputs in a VMM system. [Figure 42] This shows another masking circuit for inputs in a VMM system. [Figure 43] This shows another masking circuit for inputs in a VMM system. [Figure 44] This shows another masking circuit for inputs in a VMM system. [Figure 45] This shows another masking circuit for inputs in a VMM system. [Figure 46] This shows another masking circuit for inputs in a VMM system. [Figure 47] This shows the threshold output detector in the VMM system. [Figure 48A] This shows another threshold output detector in the VMM system. [Figure 48B] This shows another threshold output detector in the VMM system. [Figure 49] This shows an output block containing masking circuits used in a VMM system. [Figure 50] This document describes a method for analyzing output in a VMM system. [Figure 51] Here's another method for analyzing output in a VMM system. [Figure 52] This document describes a method for masking input in a VMM system. [Figure 53] This demonstrates another method for masking input in a VMM system. [Figure 54] This shows another method for masking output in a VMM system. [Figure 55] This shows another method for masking output in a VMM system. [Modes for carrying out the invention]

[0151] <Structure of the VMM System> FIG. 34 shows a block diagram of a VMM system 3400. The VMM system 3400 includes a VMM array 3401 (which may also be referred to as a neural network array), a row decoder 3402, a high-voltage decoder 3403, a column decoder 3404, a bit-line driver 3405 (such as a bit-line control circuit for programming), an input circuit 3406, an output circuit 3407, a control logic 3408, and a bias generator 3409. The VMM system 3400 further includes a high-voltage generation block 3410 including a charge pump 3411, a charge pump regulator 3412, and a high-voltage level generator 3413. The VMM system 3400 further includes a (program / erase, or weight tuning) algorithm controller 3414, an analog circuit 3415, a control engine 3416 (which may include functions such as arithmetic functions, activation functions, embedded microcontroller logic, etc., but is not limited thereto), a test control logic 3417, and a static random access memory (SRAM) block 3418 for storing intermediate data such as for the input circuit (e.g., activation data) or the output circuit (neuron output data, partial sum output neuron data), or data for programming (such as data input for the whole row or multiple rows). The VMM array 3401 includes an array of non-volatile memory cells arranged in rows and columns, and the non-volatile memory cells are split-gate flash memory cells of the type shown in FIGS. 2, 3, or 4, or stacked-gate flash memory cells as shown as memory cells 210, 310, 410, or 510 in FIG. 5, or other types of non-volatile memory cells.

[0152] The input circuit 3406 may include circuits such as a DAC (digital-to-analog converter), DPC (digital-to-pulse converter, digital-to-time-modulated pulse converter), AAC (analog-to-analog converter such as a current-to-voltage converter or logarithmic converter), PAC (pulse-to-analog level converter), or any other type of converter. The input circuit 3406 may implement one or more of the following: normalization, linear or nonlinear up / downscaling functions, or arithmetic functions. The input circuit 3406 may implement a temperature compensation function for the input level. The input circuit 3406 may implement an activation function such as a normalized linear activation function (ReLU) or a sigmoid. The input circuit 3406 can store digital activation data that is applied as an input signal during program or read operation, or combined with an input signal. The digital activation data can be stored in a register. The input circuit 3406 may include circuits for driving array terminals such as CG lines, WL lines, EG lines, and SL lines, which may include sample-and-hold circuits and buffers. A DAC can be used to convert digital activation data into an analog input voltage applied to the array. The input circuit 3406 may also include the masking circuit described below.

[0153] The output circuit 3407 may include circuits such as an ITV (current-voltage circuit), an ADC (analog-to-digital converter for converting the analog output of a neuron to digital bits), an AAC (analog-to-analog converter, such as a current-to-voltage converter or logarithmic converter), an APC (analog-to-pulse converter, analog-to-time-modulated pulse converter), or any other type of converter. The output circuit 3407 may convert the array output to activation data. The output circuit 3407 may implement an activation function such as ReLU or sigmoid. The output circuit 3407 may implement one or more of the following for the neuron output: statistical normalization, regularization, up / downscaling / gain functions, statistical rounding, or arithmetic functions (e.g., addition, subtraction, division, multiplication, shift, logarithm). The output circuit 3407 may implement a temperature compensation function for the neuron output or array output (such as a bit line output) to keep the array's power consumption nearly constant or to improve the accuracy of the array (neuron) output by keeping the IV slope nearly the same with respect to temperature changes. The output circuit 3407 may include a register for storing output data. The output circuit 3407 may also include a masking circuit as described below.

[0154] Figure 35 shows an example of a weight distribution 3500 in a typical VMM operation in a neural network. Figures 3501 and 3511 show, as examples, the weight distributions of layers L0 and Ln, respectively. As shown, the majority of weights are "0" or close to "0". In L0, there are very few weights with values ​​>q or less than -q, and in Ln, there are very few weights with values ​​>p and <-p. Extreme values ​​are sometimes called "sparse". The applicant observed that the weight distributions shown are examples of typical distributions that occur in VMM operation.

[0155] Figure 36 shows an example of row inputs 3600 applied to rows of a VMM array during typical VMM operation. In Example 3601, rows 0 through 15 each receive row inputs containing all zeros, meaning that the multiplication of those rows with the stored weights in the array is intended to yield an output current corresponding to the value "0". It would be desirable to completely avoid that operation (i.e., multiplication and addition operations involving the input circuit (row DAC) and output circuit (column ITV+DAC)) and force the output to a predefined state (e.g., the value "0") to reduce power consumption and latency (power and latency are reduced because the operation of the DAC and ITV+ADC does not occur). In Example 3602, we can see that certain bits in the row input are "0" and certain bits are "1". When row inputs are applied sequentially to rows of an array (for example, one bit position in the row input is applied at a time, as shown by boxes 3602-0, 3602-1, 3602-2, 3602-3, 3602-4, 3602-5, 3602-6, and 3602-7), it is understandable that when the row input value of a row is "0", it is desirable to avoid operation and force the row output to a predefined state (e.g., a value of "0") to reduce power consumption and latency.

[0156] Figures 37 and 38 show the methods for performing these operations.

[0157] Figure 37 illustrates method 3700. The row input value (e.g., a value stored in the row register) is measured (3701). If the value is "0", no operation is performed (no OP) (3703), which means that a neural readout of the memory array is not performed (meaning the DAC, ITV, and ADC circuits in the output block are not activated), and the output value of the output block is left in a predefined state. If the value is "1", an operation such as a neural readout of the memory array is performed (3702).

[0158] Figure 38 shows method 3800. The row input value (e.g., a value stored in the row register) is measured (3801). If the value is "0", or if the value is below the low input threshold, or if the value is above the high input threshold, the operation is not performed (3803). Otherwise, the operation is performed (3802). The low and high input thresholds are selected so that no significant performance degradation is observed when the operation is not performed.

[0159] Figure 39 shows an input block 3900, which can be part of the input circuit 3406 in Figure 34. The input block 3900 includes row circuits 3910-0, ..., 3910-N for each of the n+1 rows in the VMM array 3401 of Figure 34. Each row circuit 3910 includes an address decoder 3901, a row register and tag bits 3902 (which store the input activation value for that row, such as an 8-bit value, and the corresponding tag bits), a row input detector 3903, deactivation logic 3904, sampling logic 3905, and a sample-and-hold buffer 3906. For example, row circuits 3910-0 and 3910-N each include address decoders 3901-0 and 3901-N, row registers and tag bits 3902-0 and 3902-N, row input detectors 3903-0 and 3903-N, invalidation logic 3904-0 and 3904-N, sampling logic 3905-0 and 3905-N, and sample-and-hold buffers 3906-0 and 3906-N, respectively. The sampling logic 3905 includes logic for sampling a global digital-to-analog converter (not shown) and storing the sampled value in the respective sample-and-hold buffer 3906, as described in U.S. Patent Application No. 18 / 077,686, filed December 8, 2022, entitled "Input Circuit for Artificial Neural Network Array," which is incorporated herein by reference. The invalidation logic 3904 for each row is coupled to the global invalidation logic 3930.

[0160] During the load operation, the row input data for that particular row is loaded into each row register and tag bit 3902. During the load operation, the address decoder 3901 receives an address, and the address decoder 3901 for the selected row described by the received address provides an enable signal to its respective row register and tag bit 3902. In response to this enable signal, each row register and tag bit 3902 loads the row input data into its respective row register and tag bit 3902. The tag bits in the row registers and tag bit 3902 are optionally set to enable or disable a row for neural readout operations. For example, the tag bit for a particular row can be set to a first value (e.g., "1") if the row is read during a neural readout operation, and to a second value (e.g., "0") if the row is not read during a neural readout operation.

[0161] During neural readout, rows are selected according to the tag bits stored in their respective row registers and tag bit 3902. For example, rows with a tag bit set to a first value (e.g., "1") are operated so that the signal is ultimately applied to the row in the VMM array (shown as CG0, ..., CGN in Figure 39), while rows with a tag bit setting to a second value (e.g., "0") are ignored so that the signal is ultimately not applied to the row.

[0162] As will be explained in more detail below, the row input detector 3903, the invalidation logic 3904, and the global invalidation logic 3930 together form a masking circuit 3920, which prevents the application of an input signal to any row where the row input data stored in the associated row register and tag bit 3902 is 0, below a low input threshold, or above a high input threshold.

[0163] The application of signals to a row (e.g., CG0, ..., CGN in Figure 39) can be prevented (meaning the row is disabled) through one of the following mechanisms: (1) correcting all row input data stored in the row register and tag bit 3902 of the row from their initial values ​​to "0"; (2) correcting the value of the tag bit in the row register and tag bit 3902 from its initial value to the row's second value (e.g., "0"); or (3) deasserting an enable signal to the sampling logic 3905 or the sample-hold buffer 3906 to prevent their use.

[0164] During neural readout operation, for rows that are not disabled, the sampling logic 3905 converts the received row input data into an analog signal, which is held in the sample-hold buffer 3906 and applied as signal CG to the control gate line of the corresponding row in the VMM array.

[0165] Optionally, if all rows are disabled, another mechanism asserts a control signal to output circuit 3407 indicating that output circuit 3407 should output a predefined signal rather than process the actual output of VMM array 3401 (for example, to prevent neural readout operations from being performed).

[0166] Figure 40A shows a masking circuit 4000, which is an example of a masking circuit 3920 that can be used in input block 3900, and which can be part of input circuit 3406 in Figure 34. Masking circuit 4000 includes row input detectors 4001-0, ..., 4001-N (each for each of the n+1 rows) corresponding to row input detectors 3903-0, ..., 3903-N in Figure 39, and invalidation logics 4002-0, ..., 4002-N corresponding to invalidation logics 3904-0, ..., 3904-N in Figure 39. Each row input detector 4001 includes logic 4003, which receives the input row input data for that row (in this example, RDIN[7:0], which is an 8-bit value) received from the row register and tag bit 3902 in Figure 39, and optionally receives one or more reference values ​​(in this example, RDIN_S[7:0]) which may include a low input threshold, a high input threshold, and a specific value of interest (optionally, each such value may be provided sequentially as RDIN_S[7:0]), and also receives the EN[0] enable signal. For example, row input detectors 4001-0 include logic 4003-0, and row input detectors 4001-N include logic 4003-N. Each disable logic 4002 includes buffer 4004. For example, disable logic 4002-0 includes buffer 4004-0, and disable logic 4002-N includes buffer 4004-N.

[0167] During the load operation, row input data RDIN[7:0] is loaded into the row register and tag bit 3902, and logic 4003 determines whether any of the following conditions exist: (1) RDIN[7:0] = 00000000, (2) a low input threshold identified by RDIN[7:0] ≤ RDIN_S[7:0], (3) a high input threshold identified by RDIN[7:0] ≥ RDIN_S[7:0], or (4) a specific data input pattern identified by RDIN[7:0] = RDIN_S[7:0]. To determine whether conditions (1) to (4) exist, logic 4003 uses a digital comparator to compare RDIN[7:0] with a fixed value 00000000 for condition (1), or with RDIN_S[7:0] for conditions (2), (3), and (4). If any of these four conditions is true, the output of logic 4003 is high, i.e., asserted. The disabling logic 4002 includes a buffer 4004 that receives the output of logic 4003 and generates a row disable signal 4005 for each row in response to the assertion of the output of logic 4003. For example, when the row disable signal 4005 is high (e.g., "1"), the row is disabled, but when the row disable signal 4005 is low (e.g., "0"), the row is not disabled.

[0168] Figure 40B shows the global invalidation logic 4050, which is an example of the global invalidation logic 3930 in the input block 3900 in Figure 39 and can be part of the input circuit 3406 in Figure 34. The global invalidation logic 4050 is used to measure each row one at a time and is an alternative to the design in Figure 40A where a separate instance of row input detectors and invalidation logic exists for each row. That is, Figure 40A uses (N+1) sets of circuits, while Figure 40B uses one set of circuits.

[0169] The global disable logic 4050 includes a row input detector 4051 and a disable logic 4052. The row input detector 4051 includes logic 4053, which receives input row input data (in this example, RDIN[7:0], which is an 8-bit value) received from the row register and tag bit 3902 of the selected row in Figure 39, and optionally receives one or more reference values ​​(in this example, RDIN_S[7:0]) which may include a low input threshold, a high input threshold, and a specific value of interest (optionally, each such value may be provided sequentially as RDIN_S[7:0]), and also receives the EN[0] enable signal. The disable logic 4052 includes a buffer 4054.

[0170] During the loading operation of a particular row, the row input data RDIN[7:0] is loaded into the row register and tag bit 3902 of that row, and logic 4053 determines whether any of the following conditions exist: (1) RDIN[7:0] = 00000000, (2) a low input threshold identified by RDIN[7:0] ≤ RDIN_S[7:0], (3) a high input threshold identified by RDIN[7:0] ≥ RDIN_S[7:0], or (4) a specific data input pattern identified by RDIN[7:0] = RDIN_S[7:0]. To determine whether conditions (1) to (4) exist, logic 4053 uses a digital comparator to compare RDIN[7:0] with a fixed value 00000000 for condition (1), or with RDIN_S[7:0] for conditions (2), (3), and (4). If any of these four conditions is true, the output of logic 4053 is high, i.e., asserted. The disabling logic 4052 includes a buffer 4054 that receives the output of logic 4053 and generates a row disable signal 4055 in response to the assertion of the output of logic 4053. For example, when the row disable signal 4055 is high (e.g., "1"), the row is disabled, but when the row disable signal 4055 is low (e.g., "0"), the row is not disabled.

[0171] As described above, the first mechanism for disabling a row is to correct all row input data stored in the row register and tag bit 3902 of that row from their initial values ​​to "0". In Figure 40C, the row disable signal 4005 or 4055 is received by the controller 4010. When the row disable signal 4005 or 4055 is high (indicating that the row is disabled), the controller loads all "0" into the row register and the row register in tag bit 3902 of that row.

[0172] Furthermore, as mentioned above, a second mechanism for disabling a row is to modify the row register and the value of the tag bit in tag bit 3902 to the row's second value (e.g., "0"). In Figure 40C, the row disable signal 4005 or 4055 is received by the controller 4010. When the row disable signal 4005 or 4055 is high (indicating that the row is disabled), the controller 4010 loads "0" into the row register and the tag bit in tag bit 3902 for that row.

[0173] Furthermore, as mentioned above, a third mechanism for disabling rows is to deassert an enable signal to the sampling logic 3905 or the sample-hold buffer 3906 to prevent their use. In Figures 40D and 40E, row disable signals 4005 or 4055 are applied to inverters 4011 and 4012, and the outputs of inverters 4011 and 4012 are applied to the enable ports of the sampling logic 3905 and the S / H buffer 3906, respectively, which disables the sampling logic 3905 and the S / H buffer 3906.

[0174] Therefore, during the neural readout operation, specific rows may be disabled using the mechanism described above. This can save power, for example, if many rows are disabled because their row input data is 0, or falls below a low input threshold, or exceeds a high input threshold. When all rows are disabled, the entire neural readout operation is not performed, and the output of the output circuit 3407 is kept at a predefined condition (such as "0" for a single-ended output, or 127 for an 8-bit output, or the midpoint for a differential output).

[0175] Figure 41 shows a masking circuit 4100, which is an example of a masking circuit 3920 that can be used in input block 3900, and which can be part of input circuit 3406 in Figure 34. Masking circuit 4100 includes row input detectors 4101-0, ..., 4101-N (each for each of the n+1 rows), corresponding to row input detectors 3903-0, ..., 3903-N in Figure 39, invalidation logics 4102-0, ..., 4102-N, corresponding to invalidation logics 3904-0, ..., 3904-N in Figure 39, and global invalidation logic, corresponding to global invalidation logic 3930 in Figure 34. Each row input detector 4001 includes OR logic 4104 (OR gate, or other logic that performs an OR function), which receives all bits of the input row input data (RDIN[7:0]) for that row received from the row register and tag bits 3902 in Figure 39. For example, row input detector 4101-0 includes OR logic 4104-0, and row input detector 4101-N includes OR logic 4104-N. When all bits are "0", the output of OR logic 4104 is "0". When any bit is "1", the output of OR logic 4104, indicated as ZDETR, is "1". Disabling logic 4102 includes inverter 4105 which inverts the output of OR logic 4104. For example, disabling logic 4102 includes inverter 4105-0, whose output signal is indicated as ZDETR[0], and disabling logic 4102-N includes inverter 4105-N, whose output signal is indicated as ZDETR[N]. The output of each inverter 4105 is provided to global disabling logic 4103, which includes AND logic 4106 (AND gate, or other logic that performs an AND function). The AND logic 4106 receives all outputs from the inverter 4105 and generates the output DISABLE, which is a neural readout disable signal. If all bits in all rows are "0", DISABLE goes high, in which case all neural readout operations for rows 0 through N can be completely disabled.For example, if DISABLE=1, all rows (all rows selected for neural readout) will have a "0" input for their row input data, meaning all rows will produce a 0 output, and neural readout can be completely eliminated (no OP), meaning that the input row input data is not applied to each row, resulting in reduced power consumption and latency. Thus, each signal ZDETR indicates whether a particular row should be disabled and whether no OP (readout neural operation) should be performed for that row (e.g., the CG terminal for that row = 0V), while the signal DISABLE indicates whether all rows should be disabled.

[0176] Figure 42 shows a masking circuit 4200, which is an example of a masking circuit 3920 that can be used in input block 3900, and which can be part of input circuit 3406 in Figure 34. Masking circuit 4200 integrates the functions of row input detector 3903, invalidation logic 3904, and global invalidation logic 3930. Masking circuit 4200 includes an OR logic 4201 for each row (an OR gate, or other logic that performs the OR function), which takes all bits in the row input data for that row from the row register and tag bits 3902 in Figure 37 as input. OR logic 4201-0 also takes "0" as an additional input (which can be ground, or another voltage representing "0", in the form of voltage V1), and all subsequent OR logic 4201 take the result of the preceding row's OR logic 4201 as an additional input (for example, OR logic 4201-1 takes the output of OR logic 4201-0 as input). The OR logic 4201 also receives enable signals EN_B[0], ..., EN_B[N] for OR logics 4201-0, ..., 4201-N, respectively. In this way, if any of the activation bits received by any row are non-zero, the output of the final OR logic 4201-N is 1, indicating that a neural read operation will be performed. If all activation bits for all rows are zero, the final OR logic 4201-N is 0, which indicates that the neural read operation can be skipped for the entire VMM array (i.e., all rows) as the output is 0.

[0177] Figure 43 shows a masking circuit 4300, which is an example of a masking circuit 3920 that can be used in input block 3900, and which can be part of input circuit 3406 in Figure 34. Masking circuit 4300 integrates the functions of row input detector 3903, invalidation logic 3904, and global invalidation logic 3930. For each row, masking circuit 4300 includes OR logic 4301 (OR gate, or other logic that performs the OR function) (e.g., OR logic 4301-0 for row 0 and OR logic 4301-N for row N), NMOS transistor 4302 (e.g., NMOS transistor 4302-0 for row 0 and NMOS transistor 4302-N for row N), and inverter 4303 (e.g., inverter 4303-0 for row 0 and inverter 4303-N for row N). OR logic 4301 receives all bits in the row input data for that row from the row register and tag bits 3902 in Figure 37 as input. The OR logic 4301 also receives enable signals EN_B[0], ..., EN_B[N] for OR logic 4301-0, ..., 4301-N, respectively. The output of the OR logic 4301 is supplied to the gate of the NMOS transistor 4302. The NMOS transistor 4302 acts as an open-drain circuit with a global load circuit 4304, such as a current source, resistor, transistor, or other device. The source of the NMOS transistor 4302 is grounded, and the drain of the NMOS transistor 4302 is coupled to the global load circuit 4304. If any bit in the input data of any row of any row is "1", the output of its OR logic is "1", the NMOS transistor 4302 is turned on, and the output DISABLE is "0", which means that a readout neural operation is performed. Otherwise, DISABLE is "1", which means that no operation is performed for this neural readout, preventing the input signal from being applied to all rows. If the output of any individual inverter 4303 in a given row is true, the corresponding ZDETR signal is asserted, resulting in no operation for that particular row (for example, CG = "0", and the circuitry in that row, especially the analog circuitry, is disabled to save power).

[0178] Figure 44 shows a masking circuit 4400, which is an example of a masking circuit 3920 that can be used in input block 3900, and which can be part of input circuit 3406 in Figure 34. Masking circuit 4400 integrates the functions of row input detector 3903, invalidation logic 3904, and global invalidation logic 3930. For each row, the masking circuit 4400 includes NAND logic 4401 (a NAND gate, or other logic that performs a NAND function) (e.g., NAND logic 4401-0 for row 0, NAND logic 4401-1 for row 1, NAND logic 4401-(N-1) for row (N-1), and NAND logic 4401-N for row N), which sequentially receives each bit in the row input data for that row from the row register and tag bits 3902 in Figure 39, as well as the enable bits EN[0], ..., EN[N] of the NAND logic 4401-0, ..., 4401-N, respectively. Thus, if the row input data for each row contains 8 bits, eight different sequential logical operations will be performed. For each cycle, the output of the NAND logic 4401 is provided to the AND logic 4402 (an AND gate, or other logic that performs an AND function). If each bit input to NAND logic 4401 is 0, the output of AND logic 4402 is 1, which means that the row can be disabled for that bit position, and the neural read operation for that particular row can be skipped with respect to that bit position (e.g., no OP for that particular row with respect to that bit position), which means that the input signal is prevented from being applied to each row. The same process is repeated bit by bit for the rest of the row input data.

[0179] Figure 45 shows a masking circuit 4500, which is an example of a masking circuit 3920 that can be used in input block 3900, and which can be part of input circuit 3406 in Figure 34. Masking circuit 4500 integrates the functions of row input detector 3903, invalidation logic 3904, and global invalidation logic 3930. For each row, masking circuit 4500 includes an NMOS transistor 4501 (open-drain NMOS similar to that in Figure 43) (e.g., NMOS transistor 4501-0 for row 0, NMOS transistor 4501-1 for row 1, NMOS transistor 4501-(N-1) for row (N-1), and NMOS transistor 4501-N for row N), and its gate sequentially receives 1 bit from the activation (n bits) data of that row. Thus, if the row input data for each row contains 8 bits, eight different sequential logical operations will be performed. For each row, the masking circuit 4500 also includes inverters 4502 (inverter 4502-0 for row 0, inverter 4502-1 for row 1, inverter 4502-(N-1) for row (N-1), and inverter 4502-N for row N, etc.). Each inverter 4502 receives the same bits as received by the gate of the NMOS transistor 4501 associated with the same row. The source of the NMOS transistor 4501 is grounded, and the drain of the NMOS transistor 4501 (as in Figure 43) is coupled to the load 4503. If any bit at a given bit position in the row input data of any row is "1", the associated NMOS transistor 4501 turns on, and the output DISABLE becomes "0", indicating that a neural readout operation will be performed for that bit position. Otherwise, DISABLE becomes "1", and no neural readout operation will be performed for that bit position, which means that the input signal is prevented from being applied to the respective row. If the output of any individual inverter 4502 for a given row is true, then there is no OP for that particular row with respect to that bit position (for example, CG = "0", and the circuitry for that row, especially the analog circuitry, is disabled to save power).

[0180] Figure 46 shows a masking circuit 4600, which is an example of a masking circuit 3920 that can be used in input block 3900, and which can be part of input circuit 3406 in Figure 34. Masking circuit 4600 contains many of the same components as masking circuit 4000 in Figure 40A, and these components will not be described again for the sake of efficiency. Masking circuit 4600 further includes AND logic 4601 (AND gate, or other logic that performs the AND function) that receives row disable signals 4005-0, 4005-1, ..., 4005-(N-1), and 4005-N and generates the output DISABLE. If the outputs from logic 4003 for all rows 0 to N are "1", then DISABLE is equal to "1", which means that the activation bit for each row satisfies one of the following conditions: (1) RDIN[7:0] = 00000000, (2) RDIN[7:0] ≤ low input threshold, (3) RDIN[7:0] ≥ high input threshold, or (4) RDIN[7:0] = specific data input pattern. In this case, the neural readout operation can be skipped.

[0181] Figures 52-53 show the methods that can be implemented using the circuits in Figures 39, 40A-40D, and 41-46.

[0182] Figure 52 illustrates method 5200. Operation 5201 includes receiving row input data for each row in a neural network array, which includes a plurality of non-volatile memory cells arranged in rows and columns. Operation 5202 includes preventing the application of a signal derived from the associated row input data to the array for rows in the array where the row input data is below a low threshold or above a high threshold. Optionally, operation 5202 includes deasserting an enable signal to sampling logic, a sample-hold buffer, or other circuitry used to generate a signal from the row input data when the row input data for each row is below a low threshold or above a high threshold.

[0183] FIG. 53 illustrates method 5300. Operation 5301 includes receiving row input data for each row within a neural network array that includes a plurality of non-volatile memory cells arranged in rows and columns. Operation 5302 includes de-asserting an enable signal to sampling logic, a sample and hold buffer, or other circuitry used to generate a signal from row input data that is below a low threshold or above a high threshold. Optionally, operation 5302 includes preventing one or more of a digital-to-analog converter and an analog-to-digital converter from being activated when the row input data for a row within the neural network array is below a low threshold or above a high threshold.

[0184] FIGS. 47-49 illustrate circuits that can be used in the output circuit 3407 of FIG. 34.

[0185] FIG. 47 illustrates threshold output detector 4700. Threshold output detector 4700 includes a reference current source 4701 that supplies a current ZN_IREF that is a threshold current. Currents below that threshold current are considered negligibly small and can be ignored, and the output current is treated as "0". Reference current source 4701 is coupled to the measured VMM array column, represented as current source 4702. Threshold output detector 4700 includes a comparator 4703 for indicating the result of the comparison of the threshold current ZN_IREF to the array current I-Array. When I-Array < ZN_IREF, the voltage to the non-inverting input of comparator 4703 increases and COMPPUT equals "1". When I-Array ≧ ZN_IREF, the voltage of the non-inverting input of comparator 4703 decreases to 0V and COMPPOUT equals "0". When COMPOUT = "0", the current I-Array from the array can be considered negligible and optionally, the sensing operation may not be performed, which will reduce power consumption and latency, which means that components within the output circuit 3407 of FIG. 34, such as a current-to-voltage converter or an analog-to-digital converter, are not activated.

[0186] Figure 48A shows the threshold output detector 4800. The threshold output detector 4800 includes a current-to-voltage converter 4801 and a comparator 4802. The current-to-voltage converter 4801 receives current from the VMM array column I-Array, converts it to a voltage, and then the comparator 4802 compares this voltage to a reference voltage ZN_VREF. If the equivalent voltage of I-Array exceeds the threshold voltage ZN_VREF, COMPOUT becomes high. When COMPOUT is high, the array current exceeds the threshold and is detected and measured. When COMPOUT is low, the array current does not exceed the threshold, and certain components in the output circuit 3407, such as the current-to-voltage converter or analog-to-digital converter, cannot be left unenabled to reduce power consumption. When ZN_VREF is set to 0V, the threshold output detector 4800 will then operate as a zero-voltage detector.

[0187] Figure 48B shows the threshold output detector 4850. The threshold output detector 4850 includes a current-to-voltage converter 4851 and a comparator 4852. The current-to-voltage converter 4851 receives the current I-Array from a column of the VMM array, converts it to a voltage, and then the comparator 4852 compares this voltage to a reference voltage TH_VREF. COMPOUT indicates whether the voltage of the I-Array exceeds the threshold voltage TH_VREF. If COMPOUT is low, the array current is not above the threshold and will be detected and measured. If COMPOUT is high, the array current is above the threshold and certain components in the output circuit 3407, such as the current-to-voltage converter or analog-to-digital converter, cannot be left unenabled to reduce power consumption. This is used to detect large output currents, such as those outside the target range, and after detection, no readout operation is performed. The output of the output circuit will be forced to a predefined level, for example, "1" in this case. When TH_VREF is set to 0V, the threshold output detector 4800 will then operate as a zero-voltage detector.

[0188] Figure 49 shows an output block 4900 that can be used in the output circuit 3407 of Figure 34. The output block 4900 is used during readout or neural readout operations. The current-to-voltage converter and analog-to-digital converter 4901 receive current from the VMM array and, if not disabled by the masking circuit 4903, convert the current to voltage and the voltage to digital output bits. The output register 4902 then stores and outputs these bits. The masking circuit 4903 uses threshold output detectors 4700 or 4800 to identify output currents that are zero or above a predetermined threshold and disables the readout of the current in such columns, in particular the respective analog-to-digital conversions, which reduces the power consumption and latency involved in the analog-to-digital conversions. Optionally, the masking circuit 4903 can also disable the current-to-voltage converter and analog-to-digital converter 4901 in response to a signal from the input block 3900 indicating that the neural readout operation can be skipped, such as a disable signal from the global disable logic 3930.

[0189] Figures 50 and 51 illustrate methods for performing these operations. Figure 50 shows method 5000. The column output is measured (5001). If the value is "0" (which can be determined by setting ZN_VREF to 0V in Figure 48A and TH_VREF to 0V in Figure 48B), no operation is performed (5002), meaning that the column is not read (which means that the current-to-voltage converter and the analog-to-digital converter in the analog-to-digital converter 4901 in output block 4900, as well as the output register 4902, are not activated), and the output value of output block 4900 is set to a predetermined value. If the value is "1", an operation such as neural readout of the memory array is performed (5003).

[0190] Figure 51 shows method 5100. The column output is measured (5101). If the value is "0", or if the value is below the low output threshold (in which case all output bits of the output circuit are forced to "0"), or if the value is above the high output threshold (in which case all output bits of the output circuit are forced to "1"), the operation is not performed (5102). Otherwise, the operation is performed (5103). The low and high output thresholds are selected so that no significant performance degradation is observed when the operation is not performed.

[0191] Figures 54-55 show the method that can be carried out by the circuits in Figures 47, 48A, 48B, and 49.

[0192] Figure 54 shows method 5400. Operation 5401 involves receiving output current from each column in the neural network array. Operation 5402 is

[0193] This includes disabling components in the output circuit for columns where the current is below a threshold. Optionally, operation 5402 includes preventing the activation of current-to-voltage converters or analog-to-digital converters for one or more columns when the output current of those columns is below a threshold.

[0194] Figure 55 illustrates method 5500. Operation 5501 includes receiving row input data for each row in a neural network array containing multiple nonvolatile memory cells arranged in rows and columns. Operation 5502 includes preventing the detection of cells to be received from the associated row input data for any row in the array where the row input data is below a low threshold or above a high threshold. Optionally, operation 5502 includes preventing the activation of one or more current-to-voltage converters or analog-to-digital converters when the row input data for all rows is below a low threshold or above a high threshold.

[0195] It should be noted that, as used herein, the terms “over” and “on” both encompass “directly on” (no intermediate material, element, or gap is located between them) and “indirectly on” (intermediate material, element, or gap is located between them). Similarly, the term “adjacent” includes “directly adjacent” (no intermediate material, element, or gap is located between them) and “indirectly adjacent” (intermediate material, element, or gap is located between them); “attached” includes “directly attached” (no intermediate material, element, or gap is located between them) and “indirectly attached to” (intermediate material, element, or gap is located between them); and “electrically coupled” includes “directly electrically coupled” (no intermediate material or element electrically connecting the elements together between them) and “indirectly electrically coupled to” (intermediate material or element electrically connecting the elements together between them). For example, forming an element "on top of a substrate" may include forming the element directly on the substrate without any intermediate materials / elements between them, and forming the element indirectly on the substrate with one or more intermediate materials / elements between them.

Claims

1. A neural network array containing multiple non-volatile memory cells arranged in multiple rows and multiple columns, A system comprising: row circuits for each row in the neural network array, each row circuit including a masking circuit for preventing the application of sparse input to one or more rows in the array when a condition is met.

2. The system according to claim 1, wherein the condition is one of the following: (1) the row input data of one or more rows is equal to "0", (2) the row input data of one or more rows is below a low threshold, and (3) the row input data of one or more rows is above a high threshold.

3. The system according to claim 2, wherein each row circuit includes logic and a buffer.

4. The system according to claim 3, wherein the masking circuit includes AND logic to receive all outputs from the buffer and generate a neural readout disable signal to prevent the application of input signals to the rows in the neural network array.

5. The system according to claim 1, wherein the condition is that the row input data for all rows is equal to "0".

6. The system according to claim 4, wherein the masking circuit includes OR logic and an inverter for each row.

7. The system according to claim 6, wherein the masking circuit includes AND logic for generating an output in response to the output of each inverter in the row.

8. The system according to claim 5, wherein the masking circuit includes OR logic for receiving row input data for the row.

9. The system according to claim 8, wherein the OR logic for the first row receives "0" as input, and the OR logic for all other rows receives the output from the OR logic for the preceding row.

10. The system according to claim 5, wherein the row circuit includes an OR logic for receiving row input data of the row and an NMOS transistor coupled to a load.

11. The system according to claim 5, wherein the masking circuit includes an inverter that receives a single bit in the row input data of the row.

12. The system according to claim 11, wherein the masking circuit includes AND logic for generating an output in response to the output of each inverter.

13. The system according to claim 5, wherein the masking circuit includes an NMOS transistor having a gate that receives a single bit in the row input data of the row and a drain coupled to a load.

14. The system according to claim 1, wherein the masking circuit prevents the application of an input signal to one or more rows in the array by preventing one or more of the digital-to-analog converters and analog-to-digital converters from being activated when the conditions are met.

15. The steps include receiving row input data for each row in a neural network array containing multiple non-volatile memory cells arranged in multiple rows and multiple columns, A method comprising the step of preventing the application of an input signal derived from the associated row input data for rows in the array where the row input data is below a low threshold or exceeds a high threshold.

16. The method according to claim 15, wherein the preventive step includes preventing one or more of the digital-to-analog converters and analog-to-digital converters from being activated when the row input data for each row is below a low threshold or above a high threshold.

17. A neural network array containing multiple non-volatile memory cells arranged in multiple rows and multiple columns, A system comprising: an output block including a threshold output detector for each column in the neural network array and a masking circuit, wherein the masking circuit excludes columns in the array from readout operations if the output current is less than or equal to a threshold.

18. The system according to claim 17, wherein the threshold output detector includes a reference current source.

19. The system according to claim 17, wherein the threshold output detector includes a reference voltage source.

20. The system according to claim 17, wherein the masking circuit excludes columns in the array from readout operations by preventing one or more digital-to-analog converters and analog-to-digital converters from being activated for one or more columns when the output current of one or more columns is below a threshold.

21. The steps include receiving output current from each column in the neural network array, A method comprising the step of preventing current detection for a column whose current is below a threshold.

22. The method according to claim 21, wherein the preventive step includes preventing one or more digital-to-analog converters and analog-to-digital converters from being activated for one or more columns when the output current of one or more columns is below a threshold.

23. The steps include receiving row input data for each row in a neural network array containing multiple non-volatile memory cells arranged in multiple rows and multiple columns, A method comprising the step of preventing a row analog circuit from being activated by the associated row input data for rows in the array where the associated row input data is below a low threshold or above a high threshold.

24. The method according to claim 23, wherein the preventive step includes preventing one or more digital-to-analog converters and analog-to-digital converters from being activated when the row input data of the plurality of rows in the neural network array is below a low threshold or above a high threshold.

Images