Power management for analog neural memory in deep learning artificial neural networks

Abstract

Multiple embodiments of power management techniques for various operations involving one or more vector-matrix multiplication (VMM) arrays within an artificial neural network are disclosed.

Description

[0001] CLAIM OF PRIORITY

[0002] This application claims priority to U.S. Provisional Application 62 / 814,813, filed March 6, 2019, entitled “System for Converting Neuron Current Into Neuron Current-Based Time Pulses in an Analog Neural Memory in a Deep Learning Artificial Neural Network,” U.S. Provisional Application 62 / 794,492, filed January 18, 2019, entitled “System for Converting Neuron Current Into Neuron Current-Based Time Pulses in an Analog Neural Memory in a Deep Learning Artificial Neural Network,” and U.S. Patent Application 16 / 354,040, filed March 14, 2019, entitled “Power Management For An Analog Neural Memory In A Deep Learning Artificial Neural Network.” TECHNICAL FIELD

[0003] Disclosed are various embodiments of power management techniques for various operations involving one or more vector-matrix multiplication (VMM) arrays within an artificial neural network. BACKGROUND

[0004] Artificial neural networks model biological neural networks (the central nervous system of animals, particularly the brain), and are used to estimate or approximate functions that can depend on a large number of inputs and are typically unknown. Artificial neural networks typically include layers of interconnected “neurons” that exchange messages with each other.

[0005] Figure 1 An artificial neural network is shown, in which the circles represent the inputs or layers of neurons. The connections, called synapses, are shown with arrows, and have numeric weights that can be tuned based on experience. This makes the neural network adaptable to inputs and enables it to learn. Typically, a neural network includes a layer of multiple inputs. There are usually one or more intermediate layers of neurons, and an output layer of neurons that provide the output of the neural network. The neurons at each level make decisions individually or collectively based on data received from the synapses.

[0006] One of the major challenges in developing artificial neural networks for high performance information processing is the lack of adequate hardware technology. Indeed, practical neural networks rely on a large number of synapses, enabling high connectivity between neurons, i.e., very high computational parallelism. In principle, such complexity can be achieved by digital supercomputers or clusters of specialized graphics processing units. However, compared to biological networks, these approaches are not only high cost, but also generally energy inefficient, biological networks consuming much less energy primarily due to their performing low-precision analog computations. CMOS analog circuits have been used for artificial neural networks, but most CMOS implemented synapses are too bulky due to the need for a large number of neurons and synapses.

[0007] Applicant previously disclosed in U.S. Patent Application 15 / 594,439 (published as U.S. Patent Publication 2017 / 0337466), which is incorporated by reference herein, an artificial (analog) neural network that utilizes one or more arrays of non-volatile memory as synapses. The non-volatile memory array operates as an analog neuromorphic memory. The neural network device includes a first plurality of synapses configured to receive a first plurality of inputs and generate a first plurality of outputs therefrom, and a first plurality of neurons configured to receive the first plurality of outputs. The first plurality of synapses includes a plurality of memory cells, where each of the memory cells includes: spaced apart source and drain regions formed in a semiconductor substrate, with a channel region extending between the source and drain regions; a floating gate disposed over and insulated from a first portion of the channel region; and a non-floating gate disposed over and insulated from a second portion of the channel region. Each of the plurality of memory cells is configured to store a weight value corresponding to a number of electrons on the floating gate. The plurality of memory cells is configured to multiply the first plurality of inputs by the stored weight values to generate the first plurality of outputs.

[0008] Each non-volatile memory cell used in an analog neuromorphic memory system must be erased and programmed to hold a very specific and precise amount of charge (i.e., number of electrons) in the floating gate. For example, each floating gate must hold one of N different values, where N is the number of different weights that can be indicated by each cell. Examples of N include 16, 32, 64, 128, and 256.

[0009] One challenge in systems utilizing VMM arrays is to minimize the amount of power consumed during the many programming, erasing, and reading operations that occur, which can be quite substantial.

[0010] What is needed are improved power management techniques for systems containing one or more VMM arrays. SUMMARY

[0011] Multiple embodiments of power management techniques for various operations involving one or more vector-matrix multiplication (VMM) arrays within an artificial neural network are disclosed. BRIEF DESCRIPTION OF DRAWINGS

[0012] Figure 1 A schematic diagram of an artificial neural network of the prior art is shown.

[0013] Figure 2 A split gate flash memory cell of the prior art is shown.

[0014] Figure 3 A split gate flash memory cell of another prior art is shown.

[0015] Figure 4 A split gate flash memory cell of another prior art is shown.

[0016] Figure 5 A split gate flash memory cell of another prior art is shown.

[0017] Figure 6 A split gate flash memory cell of another prior art is shown.

[0018] Figure 7 A stacked gate flash memory cell of the prior art is shown.

[0019] Figure 8 A schematic diagram of different levels of an exemplary artificial neural network using one or more non-volatile memory arrays is shown.

[0020] Figure 9 A block diagram of a vector-matrix multiplication system is shown.

[0021] Figure 10 A block diagram of an exemplary artificial neural network using one or more vector-matrix multiplication systems is shown.

[0022] Figure 11 Another embodiment of a vector-matrix multiplication system is shown.

[0023] Figure 12 Another embodiment of a vector-matrix multiplication system is shown.

[0024] Figure 13 Another embodiment of a vector-matrix multiplication system is shown.

[0025] Figure 14 Another embodiment of a vector-matrix multiplication system is shown.

[0026] Figure 15 Another embodiment of a vector-matrix multiplication system is shown.

[0027] Figure 16 This illustrates a prior art long short-term memory system.

[0028] Figure 17 An exemplary cell used in a long short-term memory system is shown.

[0029] Figure 18 Show Figure 17 An implementation of an exemplary unit.

[0030] Figure 19 Show Figure 17 Another implementation of the exemplary unit.

[0031] Figure 20 This illustrates a prior art gate-controlled recursive cell system.

[0032] Figure 21 An exemplary cell used in a gate-controlled recursive cell system is shown.

[0033] Figure 22 Show Figure 21 An implementation of an exemplary unit.

[0034] Figure 23 Show Figure 21 Another implementation of the exemplary unit.

[0035] Figure 24 Another implementation scheme of the vector-matrix multiplication system is shown.

[0036] Figure 25 Another implementation scheme of the vector-matrix multiplication system is shown.

[0037] Figure 26 Another implementation scheme of the vector-matrix multiplication system is shown.

[0038] Figure 27 Another implementation scheme of the vector-matrix multiplication system is shown.

[0039] Figure 28 Another implementation scheme of the vector-matrix multiplication system is shown.

[0040] Figure 29 Another implementation scheme of the vector-matrix multiplication system is shown.

[0041] Figure 30 Another implementation scheme of the vector-matrix multiplication system is shown.

[0042] Figure 31 Another implementation scheme of the vector-matrix multiplication system is shown.

[0043] Figure 32 The VMM system is shown.

[0044] Figure 33 An analog neural memory system is shown.

[0045] Figure 34A An integrating analog-to-digital converter is shown.

[0046] Figure 34B Voltage characteristics of the integrating analog-to-digital converter of Figure 34A

[0047] Figure 35A An integrating analog-to-digital converter is shown.

[0048] Figure 35B Voltage characteristics of the integrating analog-to-digital converter of Figure 35A

[0049] Figure 36A Analog-to-digital converter operation examples of 36B Figure 34A 35A Waveforms of the analog-to-digital converter operation examples of

[0050] Figure 36C A timing control circuit is shown.

[0051] Figure 37 A pulse-to-voltage converter is shown.

[0052] Figure 38 A current-to-voltage converter is shown.

[0053] Figure 39 A current-to-voltage converter is shown.

[0054] Figure 40 A current-to-log-voltage converter is shown.

[0055] Figure 41 A current-to-log-voltage converter is shown.

[0056] Figure 42 A digital data-to-voltage converter is shown.

[0057] Figure 43 A digital data-to-voltage converter is shown.

[0058] Figure 44 A reference array is shown.

[0059] Figure 45 A digital comparator is shown.

[0060] Figure 46 A converter and digital comparator are shown.

[0061] Figure 47 An analog comparator is shown. ​​​​

[0062] Figure 48 The converter and analog comparator are shown.

[0063] Figure 49 The output circuit is shown.

[0064] Figure 50 This shows one aspect of the output that is activated after digitization.

[0065] Figure 51 This shows one aspect of the output that is activated after digitization.

[0066] Figure 52 The circuit for a charge summer is shown.

[0067] Figure 53 The circuit for a current summer is shown.

[0068] Figure 54 The circuit for a digital summer is shown.

[0069] Figure 55A and 55B The digital bit-to-pulse line converter and waveforms are shown respectively.

[0070] Figure 56 The power management method is shown.

[0071] Figure 57 This demonstrates another power management method.

[0072] Figure 58 This demonstrates another power management method. Detailed Implementation

[0073] The artificial neural network of this invention utilizes a combination of CMOS technology and non-volatile memory arrays.

[0074] Non-volatile memory cell

[0075] Digital nonvolatile memory is well known. For example, U.S. Patent 5,029,130 ​​(“130 Patent”), which is incorporated herein by reference, discloses an array of split-gate nonvolatile memory cells, which is a type of flash memory cell. Such memory cells 210 in… Figure 2Each memory cell 210 includes a source region 14 and a drain region 16 formed in the semiconductor substrate 12 with a channel region 18 therebetween. A floating gate 20 is formed over and insulated from (and controls the conductivity of) a first portion of the channel region 18, and over a portion of the source region 14. A word line terminal 22 (which is typically coupled to a word line) has a first portion disposed over and insulated from (and controls the conductivity of) a second portion of the channel region 18, and a second portion that extends upward and is located over the floating gate 20. The floating gate 20 and the word line terminal 22 are insulated from the substrate 12 by a gate oxide. A bit line 24 is coupled to the drain region 16.

[0076] The memory cell 210 is erased (where electrons are removed from the floating gate) by placing a high positive voltage on the word line terminal 22, which causes the electrons on the floating gate 20 to tunnel through the intervening insulator from the floating gate 20 to the word line terminal 22 via Fowler-Nordheim tunneling.

[0077] The memory cell 210 is programmed (where electrons are placed on the floating gate) by placing a positive voltage on the word line terminal 22 and a positive voltage on the source region 14. Electron current will flow from the source region 14 to the drain region 16. As the electrons reach the gap between the word line terminal 22 and the floating gate 20, the electrons will accelerate and heat up. Some of the heated electrons will be injected through the gate oxide onto the floating gate 20 due to the electrostatic attraction from the floating gate 20.

[0078] The memory cell 210 is read by placing a positive read voltage on the drain region 16 and the word line terminal 22, which turns on the portion of the channel region 18 under the word line terminal. If the floating gate 20 is positively charged (i.e., erased of electrons), then the portion of the channel region 18 under the floating gate 20 is also turned on, and current will flow through the channel region 18, which is sensed as an erased state or "1" state. If the floating gate 20 is negatively charged (i.e., programmed with electrons), then the portion of the channel region under the floating gate 20 is mostly or completely turned off, and current will not (or very little current) flow through the channel region 18, which is sensed as a programmed state or "0" state.

[0079] Table 1 shows typical voltage ranges that can be applied to the terminals of the memory cell 110 for performing read, erase, and program operations:

[0080] Table 1: Figure 2 Operation of the flash memory cell 210 of Figure 2

[0081] WL BL SL Read 2-3V 0.6-2V 0V Erase Approx. 11-13V 0V 0V Program 1-2V 1-3pA 9-10V

[0082] Figure 3 A memory cell 310 is shown, which is similar to the memory cell 210 shown in FIG. 1, except that the memory cell 310 includes a charge trapping layer 312 in place of the floating gate 20. The charge trapping layer 312 is formed over and insulated from (and controls the conductivity of) a first portion of the channel region 18, and over a portion of the source region 14. A word line terminal 22 (which is typically coupled to a word line) has a first portion disposed over and insulated from (and controls the conductivity of) a second portion of the channel region 18, and a second portion that extends upward and is located over the charge trapping layer 312. The charge trapping layer 312 and the word line terminal 22 are insulated from the substrate 12 by a gate oxide. A bit line 24 is coupled to the drain region 16. Figure 2memory cell 210, but with the addition of a control gate (CG) 28. The control gate 28 is biased at a high voltage during programming (e.g., 10V), at a low or negative voltage during erase (e.g., 0v / -8V), and at a low or intermediate voltage during read (e.g., 0v / 2.5V). The other terminals are similarly biased as in Figure 2

[0083] Figure 4 A four-gate memory cell 410 is shown, which includes a source region 14, a drain region 16, a floating gate 20 over a first portion of a channel region 18, a select gate 22 (typically coupled to a word line WL) over a second portion of the channel region 18, a control gate 28 over the floating gate 20, and an erase gate 30 over the source region 14. This configuration is described in U.S. Patent 6,747,310, which is incorporated herein by reference for all purposes. Here, all of the gates are non-floating, meaning that they are electrically connected or can be electrically connected to a voltage source, except for the floating gate 20. Programming is performed by heated electrons from the channel region 18 that inject themselves into the floating gate 20. Erase is performed by electrons tunneling from the floating gate 20 to the erase gate 30.

[0084] Table 2 shows typical voltage ranges that can be applied to the terminals of the memory cell 310 for performing read, erase, and program operations:

[0085] Table 2: Figure 4 Operation of the flash memory cell 410 of FIG. 4

[0086] WL / SG BL CG EG SL Read 1.0-2V 0.6-2V 0-2.6V 0-2.6V 0V Erase -0.5V / 0V 0V 0V / -8V 8-12V 0V Program 1V 1pA 8-11V 4.5-9V 4.5-5V

[0087] Figure 5 A memory cell 510 is shown, which is similar to the memory cell 410 of Figure 4 except that it does not contain an erase gate EG. Erase is performed by biasing the substrate 18 to a high voltage and biasing the control gate CG 28 to a low or negative voltage. Alternatively, erase is performed by biasing the word line 22 to a positive voltage and biasing the control gate 28 to a negative voltage. Programming and read are similar to Figure 4 .

[0088] Figure 6 A three-gate memory cell 610 is shown, which is another type of flash memory cell. The memory cell 610 is the same as the memory cell 410 of Figure 4 except that the memory cell 610 does not have a separate control gate. Except for not applying a control gate bias, the erase operation (and thus erase by use of the erase gate) and the read operation are similar to Figure 4 ​The operation is similar. In the absence of control gate bias, the program operation is also accomplished, and as a result, a higher voltage must be applied on the source line during the program operation to compensate for the lack of control gate bias.

[0089] Table 3 shows typical voltage ranges that can be applied to the terminals of memory cell 610 for performing read, erase, and program operations:

[0090] Table 3: Figure 6 Operation of the flash memory cell 610 of FIG. 1

[0091] WL / SG BL EG SL Read 0.7-2.2V 0.6-2V 0-2.6V 0V Erase -0.5V / 0V 0V 11.5V 0V Program 1V 2-3pA 4.5V 7-9V

[0092] Figure 7 A stacked gate memory cell 710 is shown, which is another type of flash memory cell. Memory cell 710 is similar to memory cell 210 of Figure 2 The memory cell 210 of is similar, except that the floating gate 20 extends over the entire channel region 18, and the control gate 22, which here will be coupled to a word line, extends over the floating gate 20, separated by an insulating layer (not shown). Erase, program, and read operations operate in a similar manner as previously described for memory cell 210.

[0093] Table 4 shows typical voltage ranges that can be applied to the terminals of memory cell 710 and substrate 12 for performing read, erase, and program operations:

[0094] Table 4: Figure 7 Operation of the flash memory cell 710 of FIG. 1

[0095] CG BL SL Substrate Read 2-5V 0.6–2V 0V 0V Erase -8 to -10V / 0V FLT FLT 8-10V / 15-20V Program 8-12V 3-5V 0V 0V

[0096] To utilize a memory array comprising one of the above types of non-volatile memory cells in an artificial neural network, two modifications are made. First, the wiring is configured so that each memory cell can be individually programmed, erased, and read without adversely affecting the memory state of other memory cells in the array, as explained further below. Second, a continuous (analog) programming of the memory cells is provided.

[0097] In particular, the memory state (i.e., the charge on the floating gate) of each memory cell in the array can be continuously changed from a fully erased state to a fully programmed state independently and with minimal interference to other memory cells. In another embodiment, the memory state (i.e., the charge on the floating gate) of each memory cell in the array can be continuously changed from a fully programmed state to a fully erased state independently and with minimal interference to other memory cells, or vice versa. This means that the cell storage is analog, or at least can store one of many discrete values (such as 16 or 64 different values), which allows for very precise and individual tuning of all cells in the memory array, and this makes the memory array ideal for storing and fine-tuning adjustments to the synaptic weights of a neural network.

[0098] The methods and apparatus described herein can be applied to other non-volatile memory technologies such as SONOS (silicon-oxide-nitride-oxide-silicon, charge trapped in the nitride), MONOS (metal-oxide-nitride-oxide-silicon, metal charge trapped in the nitride), ReRAM (resistive ram), PCM (phase change memory), MRAM (magnetic ram), FeRAM (ferroelectric ram), OTP (two or more layers of one-time programmable), and CeRAM (correlated electron ram), among others. The methods and apparatus described herein can be applied to volatile memory technologies for neural networks such as, but not limited to, SRAM, DRAM, and / or volatile synaptic cells.

[0099] Neural network employing an array of non-volatile memory cells

[0100] Figure 8 A non-limiting example of a neural network using a non-volatile memory array of the present embodiment is conceptually illustrated. This example uses a non-volatile memory array neural network for a face recognition application, but any other suitable application can also be implemented using a non-volatile memory array based neural network.

[0101] For this example, S0 is an input layer that is a 32x32 pixel RGB image with 5-bit precision (i.e., three 32x32 pixel arrays, one for each color R, G, and B, each pixel being 5-bit precision). The synapses CB1 from input layer S0 to layer C1 apply different sets of weights in some cases, shared weights in other cases, and scan the input image with a 3x3 pixel overlap filter (kernel) that is shifted by 1 pixel (or more than 1 pixel as indicated by the model) at a time. Specifically, the values of the 9 pixels in a 3x3 portion of the image (i.e., referred to as the filter or kernel) are provided to synapses CB1, where the 9 input values are multiplied by appropriate weights, and after summing the outputs of this multiplication, a single output value is determined by the first synapse of CB1 and provided for generating a pixel of one of the layers C1 of feature maps. The 3x3 filter is then shifted one pixel to the right within the input layer S0 (i.e., adding a column of three pixels to the right and freeing a column of three pixels to the left), whereby the 9 pixel values in this newly positioned filter are provided to synapses CB1, where they are multiplied by the same weights and a second single output value is determined by the associated synapses. This process continues until the 3x3 filter scans all three colors and all bits (precision values) across the entire 32x32 pixel image of input layer S0. The process is then repeated using a different set of weights to generate a different feature map of C1 until all of the feature maps of layer C1 are computed.

[0102] At layer C1, in this example, there are 16 feature maps, each having 30x30 pixels. Each pixel is a new feature pixel extracted from the multiplication of the input and kernel, so each feature map is a two-dimensional array, and thus layer C1 is composed of a two-dimensional array of 16 layers in this example (remember that the layers and arrays referenced herein are logical relationships and not necessarily physical relationships, i.e., the arrays do not necessarily orient to a physical two-dimensional array). Each of the 16 feature maps in layer C1 is generated by one of sixteen different sets of synapse weights applied to the filter scan. The C1 feature maps can all relate to different aspects of the same image feature, such as edge identification. For example, a first map (generated using a first set of weights, shared for all scans used to generate this first map) can identify circular edges, a second map (generated using a second set of weights different from the first set of weights) can identify rectangular edges, or aspect ratios of certain features, and so on.

[0103] Before transitioning from layer C1 to layer S1, an activation function P1 (pooling) is applied that pools values from consecutive non-overlapping 2x2 regions in each feature map. The purpose of the pooling function is to average (or max function can also be used) over neighboring locations, for example, to reduce the dependence on edge locations, and to reduce the data size before entering the next stage. At layer S1, there are 16 feature maps of 15x15 (i.e., sixteen different arrays of 15x15 pixels per feature map). Synapses CB2 from layer S1 to layer C2 scan the maps in S1 with 4x4 filters, where the filters are shifted by 1 pixel. At layer C2, there are 22 feature maps of 12x12. Before transitioning from layer C2 to layer S2, an activation function P2 (pooling) is applied that pools values from consecutive non-overlapping 2x2 regions in each feature map. At layer S2, there are 22 feature maps of 6x6. An activation function (pooling) is applied to synapses CB3 from layer S2 to layer C3, where each neuron in layer C3 is connected to each map in layer S2 via a respective synapse of CB3. At layer C3, there are 64 neurons. Synapses CB4 from layer C3 to output layer S3 connect C3 to S3 completely, i.e., each neuron in layer C3 is connected to each neuron in layer S3. The output at S3 includes 10 neurons, where the highest output neuron determines the class. For example, the output can indicate a recognition or classification of the content of the original image.

[0104] An array or portion of an array of non-volatile memory cells is used to implement the synapses of each layer.

[0105] Figure 9 A block diagram of an array that can be used for this purpose. A vector-matrix multiplication (VMM) array 32 includes non-volatile memory cells and is used as the synapses between a layer and the next layer (such as CB1, CB2, CB3, and CB4 in Figure 6 In particular, VMM array 32 includes an array of non-volatile memory cells 33, erase gate and word line gate decoders 34, control gate decoder 35, bit line decoder 36, and source line decoder 37, which decode respective inputs to the array of non-volatile memory cells 33. Inputs to VMM array 32 can come from erase gate and word line gate decoders 34 or from control gate decoder 35. In this example, source line decoder 37 also decodes outputs from the array of non-volatile memory cells 33. Alternatively, bit line decoder 36 can decode outputs from the array of non-volatile memory cells 33.

[0106] The non-volatile memory cell array 33 serves two purposes. First, it stores the weights to be used by the VMM array 32. Second, the non-volatile memory cell array 33 effectively multiplies the inputs by the weights stored in the non-volatile memory cell array 33 and each output line (source line or bit line) adds them to produce an output that will be the input to the next layer or the input to the final layer. By performing the multiplication and addition functions, the non-volatile memory cell array 33 eliminates the need for separate multiplication and addition logic circuits and is also highly power efficient due to its in-place memory computation.

[0107] The output of the non-volatile memory cell array 33 is provided to a differential summer (such as a summing operational amplifier or summing current mirror) 38 that sums the outputs of the non-volatile memory cell array 33 to create a single value for the convolution. The differential summer 38 is arranged to perform the summing of positive and negative weights.

[0108] The output value of the differential summer 38 is then summed and provided to an activation function circuit 39 that modifies the output. The activation function circuit 39 can provide a sigmoid, tanh, or ReLU function. The modified output value of the activation function circuit 39 becomes an element of the feature map that is the next layer (e.g., layer Cl) in Figure 8

[0109] Figure 9 The inputs (WLx, EGx, CGx, and optionally BLx and SLx) to the VMM array 32 in FIG. 1 can be analog levels, binary levels, digital pulses (in which case a pulse-to-analog converter PAC can be needed to convert the pulses to the appropriate input analog levels), or digital bits (in which case a DAC is provided to convert the digital bits to the appropriate input analog levels); the outputs can be analog levels, binary levels, digital pulses, or digital bits (in which case an output ADC is provided to convert the output analog levels to digital bits).

[0110] Figure 10 A block diagram to show the use of multiple layers of VMM arrays 32 (here labeled VMM arrays 32a, 32b, 32c, 32d, and 32e). As Figure 10 ​As shown, input (denoted as Inputx) is converted from digital to analog by digital-to-analog converter 31 and provided to input VMM array 32a. The converted analog input can be voltage or current. The input D / A conversion of the first layer can be done by using a function or LUT (look-up table) that maps the input Inputx to the appropriate analog level of the input VMM array 32a using a matrix multiplier. The input conversion can also be done by an analog-to-analog (A / A) converter to convert an external analog input to a mapped analog input to the input VMM array 32a. The input conversion can also be done by a digital-to-digital pulse (D / P) converter to convert an external digital input to one or more mapped digital pulses to the input VMM array 32a.

[0111] The output generated by the input VMM array 32a is provided as input to the next VMM array (hidden layer 1) 32b, which in turn generates an output that is provided as input to the next VMM array (hidden layer 2) 32c, and so on. Each layer of VMM arrays 32 serves as different layers of synapses and neurons of a convolutional neural network (CNN). Each VMM array 32a, 32b, 32c, 32d, and 32e can be a separate physical non-volatile memory array, or multiple VMM arrays can utilize different portions of the same non-volatile memory array, or multiple VMM arrays can utilize overlapping portions of the same physical non-volatile memory array. Each VMM array 32a, 32b, 32c, 32d, and 32e can also be time-division multiplexed for different portions of its array or neurons. Figure 10 The example shown contains five layers (32a, 32b, 32c, 32d, 32e): one input layer (32a), two hidden layers (32b, 32c), and two fully connected layers (32d, 32e). Those of ordinary skill in the art will appreciate that this is merely exemplary, and that, conversely, the system can include more than two hidden layers and more than two fully connected layers.

[0112] Vector-matrix multiplication (VMM) array

[0113] Figure 11 A neuron VMM array 1100 is shown, which is particularly suitable for use as a synapse and component of a neuron between an input layer and a next layer. Figure 3 The memory cells 310 shown, and used as a synapse and component of a neuron between an input layer and a next layer. The VMM array 1100 includes a memory array 1101 of non-volatile memory cells and a reference array 1102 of non-volatile reference memory cells (at the top of the array). Alternatively, another reference array can be placed at the bottom.

[0114] In the VMM array 1100, the control gate lines, such as control gate line 1103, extend in the vertical direction (so reference array 1102 is orthogonal to control gate line 1103 in the row direction), and the erase gate lines, such as erase gate line 1104, extend in the horizontal direction. Here, the inputs to the VMM array 1100 are set on the control gate lines (CG0, CG1, CG2, CG3), and the outputs of the VMM array 1100 appear on the source lines (SL0, SL1). In one embodiment, only the even rows are used, and in another embodiment, only the odd rows are used. The current placed on each source line (SL0, SL1, respectively) performs a summing function of all the currents from the memory cells connected to that particular source line.

[0115] As described herein for neural networks, the non-volatile memory cells of the VMM array 1100 (i.e., the flash memory of the VMM array 1100) are preferably configured to operate in the sub-threshold region.

[0116] Biasing the non-volatile reference memory cells and non-volatile memory cells described herein in weak inversion:

[0117] Ids = Io * e (Vg-Vth) / kVt = w * Io * e (Vg) / kVt ,

[0118] where w = e (-Vth) / kVt

[0119] For an I-to-V logarithmic converter that uses a memory cell (such as a reference memory cell or a peripheral memory cell) or a transistor to convert an input current to an input voltage:

[0120] Vg = k * Vt * log[Ids / wp * Io]

[0121] Here, wp is the w of the reference memory cell or the peripheral memory cell.

[0122] For a memory array used as a vector matrix multiplier VMM array, the output current is:

[0123] Iout = wa * Io * e (Vg) / kVt , i.e.

[0124] Iout = (wa / wp) * Iin = W * Iin

[0125] W = e (Vthp-Vtha) / kVt

[0126] Here, wa = the w of each memory cell in the memory array.

[0127] A word line or control gate can be used as an input to a memory cell for an input voltage.

[0128] Alternatively, the flash memory cells of the VMM array described herein can be configured to operate in a linear region:

[0129] Ids = β * (Vgs - Vth) * Vds; β = u * Cox * W / L

[0130] W = a (Vgs - Vth)

[0131] A word line or control gate or bit line or source line can be used as an input to the memory cells operating in a linear region. A bit line or source line can be used as an output from the memory cells.

[0132] For an I-V linear converter, a memory cell (e.g., a reference memory cell or a peripheral memory cell) or a transistor or resistor running in a linear region can be used to linearly convert an input / output current to an input / output voltage.

[0133] U.S. Patent Application 15 / 826,345 describes Figure 9 Other embodiments of the VMM array 32 of As described herein, a source line or bit line can be used as a neuron output (current sum output). Alternatively, the flash memory cells of the VMM array described herein can be configured to operate in a saturation region:

[0134] Ids = a 1 / 2 * β * (Vgs - Vth) 2 ; β = u * Cox * W / L

[0135] W = a (Vgs - Vth) 2

[0136] A word line, control gate, or erase gate can be used as an input to the memory cells operating in a saturation region. A bit line or source line can be used as an output from the output neuron.

[0137] Alternatively, the flash memory cells of the VMM array described herein can be used in all regions or a combination thereof (subthreshold, linear, or saturation region).

[0138] Figure 12 A neuron VMM array 1200 is shown, which is particularly suitable for use in Figure 2The illustrated memory cell 210, and serves as a synapse between the input layer and the next layer. The VMM array 1200 includes a memory array 1203 of non-volatile memory cells, a reference array 1201 of first non-volatile reference memory cells, and a reference array 1202 of second non-volatile reference memory cells. The reference arrays 1201 and 1202, arranged along the column direction of the array, serve to convert current inputs into terminals BLR0, BLR1, BLR2, and BLR3 into voltage inputs WL0, WL1, WL2, and WL3. In effect, the first and second non-volatile reference memory cells are diode-connected through a multiplexer 1214 (shown in part only), where the current inputs flow into. The reference cells are tuned (e.g., programmed) to a target reference level. The target reference level is provided by a reference microarray matrix (not shown).

[0139] The memory array 1203 serves two purposes. First, it stores the weights that the VMM array 1200 will use on their respective memory cells. Second, the memory array 1203 effectively multiplies the inputs (i.e., the current inputs provided in terminals BLR0, BLR1, BLR2, and BLR3, which the reference arrays 1201 and 1202 convert into input voltages to provide to word lines WL0, WL1, WL2, and WL3) by the weights stored in the memory array 1203, and then adds all the results (memory cell currents) to produce an output on the respective bit lines (BL0-BLN), which will be the input to the next layer or the final layer. By performing the multiplication and addition functions, the memory array 1203 eliminates the need for separate multiplication and addition logic circuits, and is also highly power efficient. Here, the voltage inputs are provided on the word lines (WL0, WL1, WL2, and WL3), and the outputs appear on the respective bit lines (BL0-BLN) during a read (inference) operation. The current placed on each bit line BL0-BLN performs a summation function of the currents from all the non-volatile memory cells connected to that particular bit line.

[0140] Table 5 shows the operating voltages for the VMM array 1200. The columns in the table indicate the voltages placed on the word lines for selected cells, the word lines for unselected cells, the bit lines for selected cells, the bit lines for unselected cells, the source lines for selected cells, and the source lines for unselected cells. The rows indicate read, erase, and program operations.

[0141] Table 5: Figure 12 Operation of the VMM array 1200

[0142] WL WL - unselected BL BL - unselected SL SL - unselected Read 1-3.5V -0.5V / 0V 0.6-2V (Ineuron) 0.6V-2V / 0V 0V 0V Erase Approx. 5-13V 0V 0V 0V 0V 0V Program 1-2V -0.5V / 0V 0.1-3uA Vinh approx. 2.5V 4-10V 0-1V / FLT

[0143] Figure 13 A neuron VMM array 1300 is shown, which is particularly suitable for use in Figure 2The illustrated memory cell 210, and serves as the synapse and component of the neuron between the input layer and the next layer. The VMM array 1300 includes a memory array 1303 of non-volatile memory cells, a reference array 1301 of first non-volatile reference memory cells, and a reference array 1302 of second non-volatile reference memory cells. The reference arrays 1301 and 1302 extend in the row direction of the VMM array 1300. The VMM array is similar to the VMM 1000, except that in the VMM array 1300, the word lines extend in the vertical direction. Here, the input is placed on the word lines (WLA0, WLB0, WLA1, WLB2, WLA2, WLB2, WLA3, WLB3), and the output appears on the source lines (SL0, SL1) during a read operation. The current placed on each source line performs a summing function of all the currents from the memory cells connected to that particular source line.

[0144] Table 6 shows the operating voltages for the VMM array 1300. The columns in the table indicate the voltages placed on the word line for the selected cell, the word line for the unselected cell, the bit line for the selected cell, the bit line for the unselected cell, the source line for the selected cell, and the source line for the unselected cell. The rows indicate the read, erase, and program operations.

[0145] Table 6: Figure 13 Operation of the VMM array 1300

[0146]

[0147]

[0148] Figure 14 A neuron VMM array 1400 is shown, which is particularly suitable for use in a neural network. Figure 3The illustrated memory cells 310, and serve as the synapses and components of the neurons between the input layer and the next layer. The VMM array 1400 includes a memory array 1403 of non-volatile memory cells, a reference array 1401 of first non-volatile reference memory cells, and a reference array 1402 of second non-volatile reference memory cells. The reference arrays 1401 and 1402 are used to convert current inputs into the terminals BLR0, BLR1, BLR2, and BLR3 into voltage inputs CG0, CG1, CG2, and CG3. In effect, the first and second non-volatile reference memory cells are diode-connected through multiplexers 1412 (only a portion shown), with current inputs flowing into them through BLR0, BLR1, BLR2, and BLR3. The multiplexers 1412 each include a respective multiplexer 1405 and a cascaded transistor 1404 to ensure a constant voltage on the bit line of each of the first and second non-volatile reference memory cells, such as BLR0, during a read operation. The reference cells are tuned to a target reference level.

[0149] The memory array 1403 serves two purposes. First, it stores the weights to be used by the VMM array 1400. Second, the memory array 1403 effectively multiplies the inputs (current inputs provided to the terminals BLR0, BLR1, BLR2, and BLR3, which the reference arrays 1401 and 1402 convert into input voltages to provide to the control gates CG0, CG1, CG2, and CG3) by the weights stored in the memory array, and then adds all the results (cell currents) to produce an output, which appears on the BL0-BLN and will be the input to the next layer or the final layer. By performing the multiplication and addition functions, the memory array eliminates the need for separate multiplication and addition logic circuits, and is also highly power efficient. Here, the inputs are provided on the control gate lines (CG0, CG1, CG2, and CG3), and the outputs appear on the bit lines (BL0-BLN) during a read operation. The current placed on each bit line performs a summing function of all the currents from the memory cells connected to that particular bit line.

[0150] The VMM array 1400 implements one-way tuning for the non-volatile memory cells in the memory array 1403. That is, each non-volatile memory cell is erased, and then partially programmed until the desired charge on the floating gate is reached. This can be performed, for example, using the precise programming technique described below. If too much charge is placed on the floating gate (such that an incorrect value is stored in the cell), the cell must be erased, and the sequence of partial programming operations must start over. As shown, two rows that share the same erase gate (such as EG0 or EG1) need to be erased together (which is referred to as a page erase), and thereafter each cell is partially programmed until the desired charge on the floating gate is reached.

[0151] Table 7 shows operating voltages for VMM array 1400. The columns in the table indicate the voltages on the word line for selected cells, the word line for unselected cells, the bit line for selected cells, the bit line for unselected cells, the control gate for selected cells, the control gate for unselected cells in the same sector as the selected cells, the control gate for unselected cells in a different sector than the selected cells, the erase gate for selected cells, the erase gate for unselected cells, the source line for selected cells, and the source line for unselected cells. The rows indicate read, erase, and program operations.

[0152] Table 7: Figure 14 Operation of the VMM array 1400

[0153]

[0154] Figure 15 A neuron VMM array 1500 is shown, which is particularly suitable for use in a neural network as described above. Figure 3 The memory cells 310 shown, and serve as the synapses and components of the neurons between the input layer and the next layer. The VMM array 1500 includes a memory array 1503 of non-volatile memory cells, a reference array 1501 of first non-volatile reference memory cells, and a reference array 1502 of second non-volatile reference memory cells. The EG lines EGR0, EG0, EG1, and EGR1 run vertically, while the CG lines CG0, CG1, CG2, and CG3, and the SL lines WL0, WL1, WL2, and WL3 run horizontally. The VMM array 1500 is similar to the VMM array 1400, except that the VMM array 1500 implements bidirectional tuning, in which each individual cell can be fully erased, partially programmed, and partially erased as needed, to achieve the desired amount of charge on the floating gate due to the use of individual EG lines. As shown, the reference arrays 1501 and 1502 convert input current in the terminals BLR0, BLR1, BLR2, and BLR3 to control gate voltages CG0, CG1, CG2, and CG3 to be applied to the memory cells in the row direction (by action of the reference cells connected via diode connections through multiplexer 1514). The current output (neuron) is in the bit lines BL0-BLN, where each bit line sums all the current from the non-volatile memory cells connected to that particular bit line.

[0155] Table 8 shows the operating voltages used for the VMM array 1500. The columns in the table indicate the voltages applied to the word lines for selected cells, word lines for unselected cells, bit lines for selected cells, bit lines for unselected cells, control gates for selected cells, control gates for unselected cells in the same sector as the selected cell, control gates for unselected cells in different sectors from the selected cell, erase gates for selected cells, erase gates for unselected cells, source lines for selected cells, and source lines for unselected cells. Rows indicate read, erase, and program operations.

[0156] Table 8: Figure 15 Operation of the VMM array 1500

[0157]

[0158] Figure 24 The neuronal VMM array 2400 is shown, which is particularly suitable for Figure 2 The memory unit 210 shown serves as a synapse and component for neurons between the input layer and the next layer. In the VMM array 2400, inputs INPUT0 to INPUT... N In position line BL0 to BL N The signals are received separately; the outputs OUTPUT1, OUTPUT2, OUTPUT3 and OUTPUT4 are generated on the source lines SL0, SL1, SL2 and SL3 respectively.

[0159] Figure 25 The diagram shows a neuronal VMM array 2500, which is particularly suitable for... Figure 2 The memory unit 210 shown serves as a synapse and component for neurons between the input layer and the next layer. In this example, inputs INPUT0, INPUT1, INPUT2, and INPUT3 are received on source lines SL0, SL1, SL2, and SL3, respectively; outputs OUTPUT0 to OUTPUT3 are also shown. N In position line BL0 to BL N Generate above.

[0160] Figure 26 The neuronal VMM array 2600 is shown, which is particularly suitable for Figure 2 The memory unit 210 shown serves as a synapse and component for neurons between the input layer and the next layer. In this example, input INPUT0 is connected to INPUT... M On the word lines WL0 to WL respectively M The data is received; OUTPUT0 is output to OUTPUT. N In position line BL0 to BL N Generate above.

[0161] Figure 27A neuron VMM array 2700 is shown that is particularly suitable for use with Figure 3 The memory cells 310 shown, and serve as synapses and components of neurons between an input layer and a next layer. In this example, inputs INPUT0 through INPUT M are received on word lines WL0 through WL M respectively; outputs OUTPUT0 through OUTPUT N are generated on bit lines BL0 through BL N respectively.

[0162] Figure 28 A neuron VMM array 2800 is shown that is particularly suitable for use with Figure 4 The memory cells 410 shown, and serve as synapses and components of neurons between an input layer and a next layer. In this example, inputs INPUT0 through INPUT n are received on bit line control gates 2901-1, 2901-2 through 2901-(N-1), and 2901-N respectively, which are coupled to bit lines BL0 through BL N respectively. Exemplary outputs OUTPUT1 and OUTPUT2 are generated on erase gate lines SL0 and SL1.

[0163] Figure 29 A neuron VMM array 2900 is shown that is particularly suitable for use with Figure 4 The memory cells 410 shown, and serve as synapses and components of neurons between an input layer and a next layer. In this example, inputs INPUT0 through INPUT N are received on bit line control gates 2901-1, 2901-2 through 2901-(N-1), and 2901-N respectively, which are coupled to bit lines BL0 through BL N respectively. Exemplary outputs OUTPUT1 and OUTPUT2 are generated on erase gate lines SL0 and SL1.

[0164] Figure 30 A neuron VMM array 3000 is shown that is particularly suitable for use with Figure 3 The memory cells 310 shown, Figure 5 The memory cells 510 shown, and Figure 7 The memory cells 710 shown, and serve as synapses and components of neurons between an input layer and a next layer. In this example, inputs INPUT0 through INPUT M are received on word lines WL0 through WL M respectively; outputs OUPUT0 through OUTPUT N are generated on bit lines BL0 through BL N respectively.

[0165] Figure 31 A neuron VMM array 3100 is shown that is particularly suitable for use with Figure 3The illustrated memory cell 310, Figure 5 The illustrated memory cell 510 and Figure 7 The illustrated memory cell 710, and serves as a synapse and component of a neuron between an input layer and a next layer. In this example, inputs INPUT0 through INPUT M The control gate lines CG0 through CG M The outputs OUTPUT0 through OUTPUT N are generated on the source lines SL0 through SL N respectively, where each source line SL i is coupled to the source line terminals of all memory cells in column i.

[0166] Figure 32 A VMM system 3200 is shown. The VMM system 3200 includes a VMM array 3201 (which can be based on any of the previously discussed VMM designs, such as VMMs 900, 1000, 1100, 1200, and 1320, or other VMM designs), a low voltage row decoder 3202, a high voltage row decoder 3203, a reference cell low voltage column decoder 3204 (shown in the column direction, which means it provides an input to output conversion in the row direction), a bit line multiplexer 3205, control logic 3206, analog circuitry 3207, a neuron output block 3208, an input VMM circuit block 3209, a pre-decoder 3210, test circuitry 3211, erase- program control logic EPCTL 3212, analog and high voltage generation circuitry 3213, bit line PE drivers 3214, redundancy arrays 3215 and 3216, NVR sectors 3217, and reference sectors 3218. The input circuit block 3209 serves as an interface from external inputs to the input terminals of the memory array. The neuron output block 3208 serves as an interface from the memory array outputs to external interfaces.

[0167] The low voltage row decoder 3202 provides bias voltages for read operations and program operations, and provides decode signals for the high voltage row decoder 3203. The high voltage row decoder 3203 provides high voltage bias signals for program operations and erase operations. The reference cell low voltage column decoder 3204 provides a decode function for reference cells. The bit line PE drivers 3214 provide control functions for bit lines during program, verify, and erase operations. The analog and high voltage generation circuitry 3213 is a shared bias block that provides multiple voltages needed for various program, erase, program verify, and read operations. The redundancy arrays 3215 and 3216 provide array redundancy for replacing defective array portions. The NVR (non-volatile register, also called information sector) sectors 3217 are sectors that are array sectors for storing user information, device ID, password, security key, revision bit, configuration bit, and manufacturing information, not limited to.

[0168] Figure 33 An analog neural memory system 3300 is shown. The analog neural memory system 3300 includes macro blocks 3301a, 3301b, 3301c, 3301d, 3301e, 3301f, 3301g, and 3301h; neuron output blocks (such as summer circuits and sample-and-hold S / H circuits) 3302a, 3302b, 3302c, 3302d, 3302e, 3302f, 3302g, and 3302h; and input circuit blocks 3303a, 3303b, 3303c, 3303d, 3303e, 3303f, 3303g, and 3304h. Each of the macro blocks 3301a, 3301b, 3301c, 3301d, 3301e, and 3301f is a VMM subsystem that includes a VMM array that includes rows and columns of non-volatile memory cells (such as flash memory cells). The neural memory subsystem 3333 includes macro blocks 3301, input blocks 3303, and neuron output blocks 3302. The neural memory subsystem 3333 can have its own digital control block.

[0169] The analog neural memory system 3300 also includes a system control block 3304, an analog low voltage block 3305, a high voltage block 3306, and a timing control circuit 3670, which are discussed in further detail below with respect to FIG. 36.

[0170] The system control block 3304 can include one or more microcontroller cores such as ARM / MIPS / RISC_V cores to handle general control functions and arithmetic operations. The system control block 3304 can also include a SIMD (Single Instruction Multiple Data) unit to operate on multiple data with a single instruction. The system control block can include a DSP core. The system control block can include hardware or software to perform functions such as, but not limited to, pooling, averaging, min, max, softmax, add, subtract, multiply, divide, log, inverse log, ReLu, sigmoid, tanh, and data compression. The system control block can include hardware or software to perform functions such as activation approximator / quantizer / normalizer functions. The system control block can include the ability to perform functions such as input data approximator / quantizer / normalizer functions. The system control block can include hardware or software to perform activation approximator / quantizer / normalizer functions. The control block of the neural memory subsystem 3333 can include similar elements of the system control block 3304 such as microcontroller cores, SIMD cores, DSP cores, and other functional units.

[0171] In one embodiment, the neuron output blocks 3302a, 3302b, 3302c, 3302d, 3302e, 3302f, 3302g, and 3302h each include a buffered (e.g., operational amplifier) low impedance output type circuit that can drive a long and configurable interconnect. In one embodiment, the input circuit blocks 3303a, 3303b, 3303c, 3303d, 3303e, 3303f, 3303g, and 3303h each provide a summed high impedance current output. In another embodiment, the neuron output blocks 3302a, 3302b, 3302c, 3302d, 3302e, 3302f, 3302g, and 3302h each include an activation circuit, in which case an additional low impedance buffer is needed to drive the output.

[0172] In another embodiment, the neuron output blocks 3302a, 3302b, 3302c, 3302d, 3302e, 3302f, 3302g, and 3302h each include an analog-to-digital conversion block that outputs digital bits rather than an analog signal. In this embodiment, the input circuit blocks 3303a, 3303b, 3303c, 3303d, 3303e, 3303f, 3303g, and 3303h each include a digital-to-analog conversion block that receives the digital bits from the corresponding neuron output block and converts the digital bits to an analog signal.

[0173] Accordingly, the neuron output blocks 3302a, 3302b, 3302c, 3302d, 3302e, 3302f, 3302g, and 3302h receive output currents from the macro blocks 3301a, 3301b, 3301c, 3301d, 3301e, and 3301f and optionally convert the output currents to analog voltages, digital bits, or one or more digital pulses, where the width of each pulse or the number of pulses varies in response to the value of the output current. Similarly, the input circuit blocks 3303a, 3303b, 3303c, 3303d, 3303e, 3303f, 3303g, and 3303h optionally receive analog currents, analog voltages, digital bits, or digital pulses, where the width of each pulse or the number of pulses varies in response to the value of the output current, and provide analog currents to the macro blocks 3301a, 3301b, 3301c, 3301d, 3301e, and 3301f. The input circuit blocks 3303a, 3303b, 3303c, 3303d, 3303e, 3303f, 3303g, and 3303h optionally include voltage-to-current converters, analog or digital counters to count the number of digital pulses in the input signal or the length of the digital pulse width in the input signal, or digital-to-analog converters.

[0174] Long short-term memory

[0175] Existing technologies include a concept known as Long Short-Term Memory (LSTM). LSTM cells are commonly used in neural networks. LSTM allows neural networks to remember information for predetermined arbitrary time intervals and use that information in subsequent operations. A typical LSTM cell includes a cell, an input gate, an output gate, and a forget gate. The three gates regulate the flow of information into and out of the cell and the time interval at which information is remembered in the LSTM. Virtual Memory Models (VMMs) are particularly useful in LSTM cells.

[0176] Figure 16 An exemplary LSTM 1600 is shown. This example LSTM 1600 includes cells 1601, 1602, 1603, and 1604. Cell 1601 receives an input vector x0 and generates an output vector h0 and a cell state vector c0. Cell 1602 receives an input vector x1, an output vector (hidden state) h0 from cell 1601, and a cell state c0 from cell 1601, and generates an output vector h1 and a cell state vector c1. Cell 1603 receives an input vector x2, an output vector (hidden state) h1 from cell 1602, and a cell state c1 from cell 1602, and generates an output vector h2 and a cell state vector c2. Cell 1604 receives an input vector x3, an output vector (hidden state) h2 from cell 1603, and a cell state c2 from cell 1603, and generates an output vector h3. Additional cells may be used, and this four-cell LSTM is only an example.

[0177] Figure 17 Showing what can be used Figure 16 An exemplary specific implementation of LSTM cell 1700 in cells 1601, 1602, 1603 and 1604 is provided. LSTM cell 1700 receives input vector x(t), cell state vector c(t-1) from the previous cell and output vector h(t-1) from the previous cell, and generates cell state vector c(t) and output vector h(t).

[0178] LSTM unit 1700 includes sigmoid function devices 1701, 1702, and 1703, each applying a number between 0 and 1 to control the number of times each component of the input vector is allowed to pass through to the output vector. LSTM unit 1700 also includes tanh devices 1704 and 1705 for applying the hyperbolic tangent function to the input vector, multiplier devices 1706, 1707, and 1708 for multiplying two vectors together, and adder device 1709 for adding two vectors together. The output vector h(t) can be provided to the next LSTM unit in the system, or it can be accessed for other purposes.

[0179] Figure 18An LSTM unit 1800 is shown, which is an example of an implementation of the LSTM unit 1700. For the convenience of the reader, the same numbers are used in the LSTM unit 1800 as in the LSTM unit 1700. The sigmoid function devices 1701, 1702, and 1703 and the tanh device 1704 each include multiple VMM arrays 1801 and activation circuit blocks 1802. Thus, it can be seen that VMM arrays are particularly useful in LSTM units used in certain neural network systems. The multiplier devices 1706, 1707, and 1708 and the adder device 1709 are implemented digitally or analogically. The activation function blocks 1802 can be implemented digitally or analogically.

[0180] An alternative form of the LSTM unit 1800 (and another example of an implementation of the LSTM unit 1700) is shown in Figure 19 In Figure 19 the sigmoid function devices 1701, 1702, and 1703 and the tanh device 1704 share the same physical hardware (VMM arrays 1901 and activation function blocks 1902) in a time-multiplexed fashion. The LSTM unit 1900 also includes a multiplier device 1903 that multiplies two vectors together, an adder device 1908 that adds two vectors together, a tanh device 1705 that includes an activation circuit block 1902, a register 1907 that stores the value i(t) when it is output from the sigmoid function block 1902, a register 1904 that stores the value f(t)*c(t-1) when it is output from the multiplier device 1903 through a multiplexer 1910, a register 1905 that stores the value i(t)*u(t) when it is output from the multiplier device 1903 through the multiplexer 1910, a register 1906 that stores the value o(t)*c~(t) when it is output from the multiplier device 1903 through the multiplexer 1910, and a multiplexer 1909.

[0181] The LSTM unit 1800 contains multiple sets of VMM arrays 1801 and corresponding activation function blocks 1802, while the LSTM unit 1900 contains only one set of VMM arrays 1901 and activation function blocks 1902, which are used to represent multiple layers in an implementation of the LSTM unit 1900. The LSTM unit 1900 will require less space than the LSTM 1800 because the LSTM unit 1900 requires only ¼ of the space of the LSTM unit 1800 for VMMs and activation function blocks compared to the LSTM unit 1800.

[0182] It is also understood that an LSTM unit will typically include multiple VMM arrays, each of which requires functionality provided by certain circuit blocks outside of the VMM arrays, such as summation and activation circuit blocks and a high voltage generation block. Providing separate circuit blocks for each VMM array will require a large amount of space within the semiconductor device and will be somewhat inefficient.

[0183] Gate-controlled recurrent unit

[0184] Analog VMM implementations can be used in a gated recurrent unit (GRU) system. A GRU is a gating mechanism in recurrent neural networks. A GRU is similar to an LSTM, except that a GRU unit generally contains fewer components than an LSTM unit.

[0185] Figure 20 An example GRU 2000 is shown. The GRU 2000 in this example includes units 2001, 2002, 2003, and 2004. Unit 2001 receives an input vector xo and generates an output vector ho. Unit 2002 receives an input vector xi, the output vector ho from unit 2001, and generates an output vector hi. Unit 2003 receives an input vector x2 and the output vector (hidden state) hi from unit 2002, and generates an output vector h2. Unit 2004 receives an input vector x3 and the output vector (hidden state) h2 from unit 2003, and generates an output vector h3. Additional units can be used, and a GRU with four units is merely an example.

[0186] Figure 21 An example implementation of a GRU unit 2100 for units 2001, 2002, 2003, and 2004 of Figure 20 is shown. The GRU unit 2100 receives an input vector x(t) and an output vector h(t-1) from a previous GRU unit, and generates an output vector h(t). The GRU unit 2100 includes sigmoid function devices 2101 and 2102, each of which applies a number between 0 and 1 to a component from the output vector h(t-1) and the input vector x(t). The GRU unit 2100 also includes a tanh device 2103 for applying a hyperbolic tangent function to the input vector, multiple multiplier devices 2104, 2105, and 2106 for multiplying two vectors together, an addition device 2107 for adding two vectors together, and a complement device 2108 for subtracting an input from 1 to generate an output.

[0187] Figure 22GRU unit 2200 is shown as an example of a specific implementation of GRU unit 2100. For the reader's convenience, GRU unit 2200 uses the same numbering as GRU unit 2100. Figure 22 As shown, sigmoid function devices 2101 and 2102 and tanh device 2103 each include multiple VMM arrays 2201 and activation function blocks 2202. Therefore, it can be seen that VMM arrays are particularly useful in GRU units used in certain neural network systems. Multiplier devices 2104, 2105, and 2106, adder device 2107, and complement device 2108 are implemented digitally or analogically. Activation function blocks 2202 can be implemented digitally or analogously.

[0188] Alternative forms of GRU unit 2200 (and another example of a specific implementation of GRU unit 2300) in Figure 23 As shown in [the image]. Figure 23 In this configuration, the GRU unit 2300 utilizes a VMM array 2301 and an activation function block 2302. When this activation function block is configured as a sigmoid function, a number between 0 and 1 is applied to control the number of times each component of the input vector is allowed to reach the output vector. Figure 23 In this context, sigmoid function devices 2101 and 2102 and tanh device 2103 share the same physical hardware (VMM array 2301 and activation function block 2302) in a time-division multiplexing manner. GRU unit 2300 also includes a multiplier device 2303 for multiplying two vectors together, an adder device 2305 for adding two vectors together, a complement device 2309 for subtracting the input from 1 to generate the output, a multiplexer 2304, a register 2306 for holding the value h(t-1)*r(t) when it is output from the multiplier device 2303 via the multiplexer 2304, a register 2307 for holding the value h(t-1)*z(t) when it is output from the multiplier device 2303 via the multiplexer 2304, and a register 2308 for holding the value h^(t)*(1-z(t)) when it is output from the multiplier device 2303 via the multiplexer 2304.

[0189] GRU unit 2200 includes multiple sets of VMM arrays 2201 and activation function blocks 2202, while GRU unit 2300 includes only one set of VMM arrays 2301 and activation function blocks 2302, which are used to represent multiple layers in an implementation of GRU unit 2300. GRU unit 2300 will require less space than GRU unit 2200 because GRU unit 2300 only needs 1 / 3 of its space for VMMs and activation function blocks compared to GRU unit 2200.

[0190] It is also understood that a GRU system will typically include multiple VMM arrays, each of which requires functionality provided by certain circuit blocks outside of the VMM array, such as summing and activation circuit blocks and high voltage generation blocks. Providing separate circuit blocks for each VMM array would require a large amount of space within the semiconductor device and would be somewhat inefficient.

[0191] The input to the VMM array can be an analog level, a binary level, or a digital bit (in which case a DAC is needed to convert the digital bit to the appropriate input analog level), and the output can be an analog level, a binary level, or a digital bit (in which case an output ADC is needed to convert the output analog level to a digital bit).

[0192] For each memory cell in the VMM array, each weight w can be implemented by a single memory cell or by a differential cell or by two hybrid memory cells (average of 2 cells). In the case of a differential cell, two memory cells are needed to implement the weight w as a differential weight (w = w+ - w-). In the two hybrid memory cells, two memory cells are needed to implement the weight w as an average of two cells.

[0193] Output circuit

[0194] Figure 34A An integrating dual hybrid slope analog-to-digital converter (ADC) 3400 is shown applied to the output neuron to convert the output neuron current I NEU 3406 to a digital pulse or digital output bit.

[0195] In one embodiment, the ADC 3400 converts the analog output current in a neuron output block, such as the neuron output blocks 3302a, 3302b, 3302c, 3302d, 3302e, 3302f, 3302g, and 3302h in Figure 32 In one embodiment, the ADC 3400 converts the analog output current in a neuron output block, such as the neuron output blocks 3302a, 3302b, 3302c, 3302d, 3302e, 3302f, 3302g, and 3302h in NEU 3406 to a digital pulse or digital output bit.

[0196] Optionally, the IREF 3407 can include a bandgap filter with a temperature coefficient of 0 or a temperature coefficient that tracks the neuron current I NEU 3406. The latter temperature coefficient can optionally be obtained from a reference array containing values determined during a test phase.

[0197] Optionally, the calibration step can be performed when the circuit is at or above operating temperature to offset any leakage current present in the array or control circuit, and then the offset value can be subtracted from Figure 34B or Figure 35B Ineu in the Ineu.

[0198] During the initialization phase, switch 3408 is closed. Then, the input to Vout 3403 and the negative terminal of operational amplifier 3401 will become VREF. Thereafter, as shown in Figure 34B switch 33408 is opened and the neuron current I NEU 3406 is integrated upwards during a fixed time period tref. During the fixed time period tref, Vout rises and its slope changes as the neuron current changes. Thereafter, during a time period tmeas, the constant reference current IREF is integrated downwards during the time period tmeas (during which Vout falls), where tmeas is the time required to integrate Vout down to VREF.

[0199] When VOUT > VREFV, the output EC 3405 will be high and vice versa. EC 3405 thus generates a pulse whose width reflects the time period tmeas, which in turn is proportional to the current I NEU 3406. In Figure 34B EC 3405 is shown as waveform 3410 for the example where tmeas = Ineu1 and as waveform 3412 for the example where tmeas = Ineu2. Thus, the output neuron current I NEU 3406 is converted to a digital pulse EC 3405, where the width of the digital pulse EC 3405 changes in proportion to the magnitude of the output neuron current I NEU 3406.

[0200] The current I NEU 3406 = tmeas / tref * IREF. For example, for a required 10-bit output bit resolution, tref corresponds to a time period of 1024 clock periods. Depending on the value of I NEU 3406 and the value of Iref, the time period tmeas varies between equal to 0 and 1024 clock periods. Figure 34B Examples of two different values of I NEU 3406 are shown, where one I NEU 3406 = Ineu1 and the other I NEU 3406 = Ineu2. Thus, the neuron current I NEU 3406 affects the rate and slope of the charging.

[0201] Optionally, the output pulse EC 3405 can be converted into a series of pulses with a uniform period to be transmitted to the next stage of the circuit, such as the input block of another VMM array. At the beginning of the time period tmeas, the output EC 3405, along with the reference clock 3441, is input into AND gate 3440. During the time period VOUT > VREF, the output will be a pulse sequence 3442 (where the frequency of the pulses in pulse sequence 3442 is the same as the frequency of clock 3441). The number of pulses is proportional to the time period tmeas, which is related to the current I. NEU It is directly proportional to 3406.

[0202] Optionally, the pulse sequence 3443 can be input to a counter 3420, which counts the number of pulses in the pulse sequence 3442 and generates a count value 3421, which is a digital count of the number of pulses in the pulse sequence 3442, and this digital count is correlated with the neuronal current I. NEU 3406 is proportional. The count value 3421 includes a set of digital bits. In another implementation, the integrating dual-slope ADC 3400 can measure the neuronal current I. NEU 3407 is converted into a pulse, where the width of the pulse is related to the neuron current I. NEU The value of 3407 is inversely proportional. This inversion can be done digitally or analogically and converted into a series of pulses or digital bits for output to a follower circuit.

[0203] Figure 35A This illustrates the application of unit current I to the output neuron. NEU The 3504 is an integrating dual-mixed-slope ADC that converts digital pulses of varying widths or a series of digital output bits into digital pulses. For example, the ADC 3500 can be used to convert neuron output blocks (such as...) Figure 32 The analog output current in the neuron output blocks 3302a, 3302b, 3302c, 3302d, 3302e, 3302f, 3302g, and 3302h is converted into a set of digital output bits. An integrator, including an integrating operational amplifier 3501 and an integrating capacitor 3502, converts the neuron current I... NEU 3504 is the integral of the relative reference current IREF and 3503. Switch 3505 can be closed to reset VOUT.

[0204] During the initialization phase, switch 3505 is closed, and VOUT is charged to voltage V. BIAS .

[0205] After that, as Figure 35B As shown, switch 3505 is turned on, and during a fixed time tref, the cell current I... NEU3504integrates upward. Thereafter, the reference current IREF 3503 integrates downward for a period of time tmeas until Vout falls to zero. The current I NEU 3504= tmeas Ineu / tref * IREF. For example, for a desired 10-bit output bit resolution, tref corresponds to a period of 1024 clock cycles. Depending on the value of Ineu and Iref, the period tmeas varies between equal to 0 and 1024 clock cycles. NEU 3504and Iref, the period tmeas varies between equal to 0 and 1024 clock cycles. Figure 35B Examples are shown for two different Ineu values, one with current Ineu1 and the other with current Ineu2. Thus, the neuron current I NEU 3504will affect the rate and slope of the charging and discharging.

[0206] When VOUT > VREF, the output 3506 will be high, and vice versa. The output 3506 thus generates a pulse whose width reflects the period tmeas, which in turn is proportional to the current I NEU 3404. In Figure 35B the example with tmeas = Ineu1 is shown as waveform 3512 and in the example with tmeas = Ineu2 is shown as waveform 3515. Thus, the output neuron current I NEU 3504is converted into a pulse, namely the output 3506, whose width varies in proportion to the magnitude of the output neuron current I NEU 3504.

[0207] Optionally, the output 3506 can be converted into a series of pulses with uniform period for transmission to the next stage of the circuit, such as the input block of another VMM array. At the beginning of the period tmeas, the output 3506 is input into a AND gate 3508 along with a reference clock 3507. During the period when VOUT > VREF, the output will be a pulse train 3509 (where the frequency of the pulses in the pulse train 3509 is the same as the frequency of the reference clock 3507). The number of pulses is proportional to the period tmeas, which is proportional to the current I NEU 3504.

[0208] Optionally, the pulse train 3509 can be input into a counter 3510, which will count the number of pulses in the pulse train 3509 and will generate a count value 3511, which is a digital count of the number of pulses in the pulse train 3509, which as shown in waveforms 3514, 3517 is proportional to the neuron current I NEU 3504. The count value 3511 comprises a set of digital bits.

[0209] In another embodiment, the integral dual-slope ADC 3500 can convert the neuron current I NEU 3504 into a pulse, where the width of the pulse is inversely proportional to the magnitude of the neuron current I NEU 3504. This inversion can be done in digital or analog fashion and converted into one or more pulses or digital bits for output to follow-on circuitry.

[0210] Figure 35B Count values 3511 (digital bits) for two neuron current values Ineu1 and Ineu2 of I NEU 3504 are shown.

[0211] Figure 36A and 36B Waveforms associated with exemplary methods 3600 and 3650 performed in a VMM during operation are shown. In each method 3600 and 3650, word lines WL0, WL1, and WL2 receive a variety of different inputs, which can optionally be converted into analog voltage waveforms to apply to the word lines. In these examples, the voltage VC represents the voltage on the integrating capacitor 3402 or 3502 in Figure 34A and Figure 35A , respectively, in the ADC 3400 or 3500 in the output block of the first VMM; the OT pulse (=‘1’) represents the period in which the output of the neuron (which is proportional to the value of the neuron) is captured using the integral dual-slope ADC 3400 or 3500. As shown with reference to FIGS. 34 and 35, the output of the output block can be a pulse whose width varies in direct proportion to the output neuron current of the first VMM, or it can be a series of pulses with uniform width, where the number of pulses varies in direct proportion to the neuron current of the first VMM. Those pulses can then be applied as input to a second VMM.

[0212] During method 3600, the series of pulses (such as pulse sequence 3442 or pulse sequence 3509) or an analog voltage derived from the series of pulses is applied to the word lines of a second VMM array. Alternatively, the series of pulses or an analog voltage derived from the series of pulses can be applied to the control gates of cells within a second VMM array. The number of pulses (or clock cycles) directly corresponds to the magnitude of the input. In this particular example, the magnitude of the input on WL1 is 4 times that on WL0 (4 pulses vs. 1 pulse).

[0213] During the method 3650, a single pulse of varying width, such as EC 3405 or output 3506, or an analog voltage derived from the single pulse, is applied to the word lines of the second VMM array, but the pulse has a variable pulse width. Alternatively, the pulse or the analog voltage derived from the pulse can be applied to the control gates. The width of the single pulse directly corresponds to the magnitude of the input. For example, the magnitude of the input on WL1 is 4 times that on WL0 (the pulse width of WL1 is 4 times that of WL0).

[0214] Further, with reference to Figure 36C , the timing control circuit 3670 can be used to manage the power of the VMM system by managing the output interface and input interface of the VMM array and sequentially splitting the conversion of various outputs or various inputs. Figure 56 A power management method 5600 is shown. A first step: receiving a plurality of inputs for a vector-matrix multiplication array (step 5601); a second step: organizing the plurality of inputs into a plurality of groups of inputs (step 5602); a third step: sequentially providing each group of the plurality of groups of inputs to the array (step 5603).

[0215] Embodiments of the power management method 5600 are as follows. The inputs can be applied to the VMM system, such as the word lines or control gates of the VMM array, sequentially over time. For example, for a VMM array with 512 word line inputs, the word line inputs can be divided into 4 groups: WL0-127, WL128-255, WL256-383, and WL383-511. Each group can be enabled at different times, and an output read operation (converting neuron current to digital bits) can be performed on the corresponding one of the four groups of word lines, such as by the output integration type circuit in FIGS. 34-36. Then, after sequentially reading each of the four groups, the output digital bit results are combined together. This operation can be controlled by the timing control circuit 3670.

[0216] In another embodiment, the timing control circuit 3670 performs power management in a vector-matrix multiplication system, such as the analog neural memory system 3300 in FIG. 33. Figure 33 The timing control circuit 3670 can cause the inputs to be applied to the VMM subsystem 3333 sequentially over time, such as by enabling the input circuit blocks 3303a, 3303b, 3303c, 3303d, 3303e, 3303f, 3303g, and 3303h at different times. Similarly, the timing control circuit 3670 can cause the outputs from the VMM subsystem 333 to be read sequentially over time, such as by enabling the neuron output blocks 3302a, 3302b, 3302c, 3302d, 3302e, 3302f, 3302g, and 3302h at different times.

[0217] Figure 57 A power management method 5700 is shown. First step: receive a plurality of outputs from a vector-matrix multiplication array (step 5701). Second step: organize the plurality of outputs from the array into a plurality of groups of outputs (step 5702). Third step: provide each of the plurality of groups of outputs to a converter circuit in turn (step 5703).

[0218] Embodiments of the power management method 5700 are as follows. Power management can be achieved by the timing control circuit 3670 by reading the groups of neuron outputs in turn at different times, i.e., by multiplexing the output circuit (such as the output ADC circuit) across multiple groups of neuron outputs (bitlines). The bitlines can be placed into different groups, and the output circuit operates on one group at a time in a sequence under the control of the timing control circuit 3670.

[0219] Figure 58 A power management method 5800 is shown. First step: receive a plurality of inputs in a vector-matrix multiplication system comprising a plurality of arrays. Second step: enable one or more of the plurality of arrays in turn to receive some or all of the plurality of inputs (step 5802).

[0220] Embodiments of the power management method 5800 are as follows. The timing control circuit 3670 can operate on one neural network layer at a time. For example, if one neural network layer is represented in a first VMM array and a second neural network layer is represented in a second VMM array, the output read operation (such as the case where neuron outputs are converted to digital bits) can be performed in turn on one VMM array at a time, thereby managing the power of the VMM system.

[0221] In another embodiment, the timing control circuit 3670 can operate by enabling a plurality of neural memory subsystems 3333 or a plurality of macros 3301 as shown in FIG. 33B in turn. Figure 33

[0222] In another embodiment, the timing control circuit 3670 can operate by enabling a plurality of neural memory subsystems 3333 or a plurality of macros 3301 as shown in FIG. 33B in turn without discharging the array bias (e.g., bias on the wordline WL and / or bitline BL, bias on the control gate CG as input and bitline BL as output, or bias on the control gate CG and / or bitline BL, bias on the wordline WL as input and bitline BL as output) during the inactive period (i.e., the time period between turning on and turning off the sequential enablement). This is to save the power of unnecessary discharging and charging of the array bias that is to be used multiple times during one or more read operations (e.g., during an inference or classification operation). Figure 33

[0223] Figures 37-44 ​​Various circuits that can be used for the VMM input block are shown, such as Figure 33 The input circuit blocks 3303a, 3303b, 3303c, 3303d, 3303e, 3303f, 3303g, and 3303h or neuron output blocks shown, such as Figure 33 The neuron output blocks 3302a, 3302b, 3302c, 3302d, 3302e, 3302f, 3302g, and 3302h shown.

[0224] Figure 37 A pulse-to-voltage converter 3700 is shown that can optionally be used to convert the digital pulses generated by the integrating dual-slope ADC 3400 or 3500 into a voltage that can be applied as an input to a VMM memory array, for example, on a WL or CG line. The pulse-to-voltage converter 3700 includes a reference current generator 3701 that generates a reference current IREF, a capacitor 3702, and a switch 3703. An input is used to control the switch 3703. When a pulse is received on the input, the switch closes and charge accumulates on the capacitor 3702 such that the voltage of the capacitor 3702 after the input signal is complete will indicate the number of pulses received. The capacitor can optionally be a word line or control gate capacitor.

[0225] Figure 38 A current-to-voltage converter 3800 is shown that can optionally be used to convert the neuron output current into a voltage that can be applied as an input to a VMM memory array, for example, on a WL or CG line. The current-to-voltage converter 3800 includes a current generator 3801, which here represents the received neuron current Ineu (or Iin), and a variable resistor 3802. The output Vout will increase as the neuron current increases. The variable resistor 3802 can be adjusted as needed to increase or decrease the maximum range of Vout.

[0226] Figure 39 A current-to-voltage converter 3900 is shown that can optionally be used to convert the neuron output current into a voltage that can be applied as an input to a VMM memory array, for example, on a WL or CG line. The current-to-voltage converter 3900 includes an operational amplifier 3901, a capacitor 3902, a switch 3903, a switch 3904, and a current source 3905, which here represents the neuron current ICELL. During operation, the switch 3903 will be open and the switch 3904 will be closed. The amplitude of the output Vout will increase in direct proportion to the magnitude of the neuron current ICELL 3905.

[0227] Figure 40A current-to-logarithmic voltage converter 4000 is shown, which can optionally be used to convert neuron output current into a logarithmic voltage that can be applied as input to a VMM memory array, for example, on a WL or CG line. The current-to-logarithmic voltage converter 4000 includes a memory cell 4001, a switch 4002 that selectively connects the word line terminal of the memory cell 4001 to a node that generates Vout, and a current source 4003 that represents the neuron current Iinhere. During operation, the switch 4002 will be closed, and the amplitude of the output Vout will increase in direct proportion to the magnitude of the neuron current Iin.

[0228] Figure 41 A current-to-logarithmic voltage converter 4100 is shown, which can optionally be used to convert neuron output current into a logarithmic voltage that can be applied as input to a VMM memory array, for example, on a WL or CG line. The current-to-logarithmic voltage converter 4100 includes a memory cell 4101, a switch 4102 that selectively connects the control gate terminal of the memory cell 4101 to a node that generates Vout, and a current source 4103 that represents the neuron current Iinhere. During operation, the switch 4102 will be closed, and the amplitude of the output Vout will increase in direct proportion to the magnitude of the neuron current Iin.

[0229] Figure 42 A digital data-to-voltage converter 4200 is shown, which can optionally be used to convert digital data (i.e., data of Os and Is) into a voltage that can be applied as input to a VMM memory array, for example, on a WL or CG line. The digital data-to-voltage converter 4200 includes a capacitor 4201, an adjustable current source 4202 (here, a current from a reference array of memory cells), and a switch 4203. The digital data controls the switch 4203. For example, the switch 4203 can be closed when the digital data is a “1” and open when the digital data is a “0.” The voltage accumulated on the capacitor 4201 will be the output OUT and will correspond to the value of the digital data. Optionally, the capacitor can be a word line or control gate capacitor.

[0230] Figure 43 A digital data-to-voltage converter 4300 is shown, which can optionally be used to convert digital data (i.e., data of Os and Is) into a voltage that can be applied as input to a VMM memory array, for example, on a WL or CG line. The digital data-to-voltage converter 4300 includes a variable resistor 4301, an adjustable current source 4302 (here, a current from a reference array of memory cells), and a switch 4303. The digital data controls the switch 4303. For example, the switch 4303 can be closed when the digital data is a “1” and open when the digital data is a “0.” The output voltage will correspond to the value of the digital data.

[0231] Figure 44 Reference array 4400 is shown, which can be used to provide Figure 42 and 43 The reference currents of the adjustable current sources 4202 and 4302 in the system.

[0232] Figures 45-47 The diagram shows components for verifying, after programming operations, that a flash memory cell in a VMM contains the appropriate charge corresponding to the W value expected to be stored in that flash memory cell.

[0233] Figure 45 A digital comparator 4500 is shown, which receives a set of reference W values ​​and a sensed W digital value as digital inputs from a plurality of programmed flash memory cells. If a mismatch exists, the digital comparator 4500 generates a flag indicating that one or more flash memory cells have not been programmed with the correct values.

[0234] Figure 46 This demonstrates the collaboration with converter 4600. Figure 45 The digital comparator 4500. The sensed W value is provided by multiple instantiations of the converter 4600. The converter 4600 receives cell current ICELL from the flash memory cell and converts the cell current into digital data, which can be provided to the digital comparator 4500 using one or more of the aforementioned converters (such as ADC 3400 or 3500).

[0235] Figure 47 An analog comparator 4700 is shown, which receives a set of reference W values ​​and sensed analog W values ​​as analog inputs from a plurality of programmed flash memory cells. If a mismatch exists, the analog comparator 4700 generates a flag indicating that one or more flash memory cells have not been programmed with the correct values.

[0236] Figure 48 This demonstrates the collaboration with the converter 4800. Figure 47 The analog comparator 4700. The sensed W value is provided by converter 4800. Converter 4800 receives the digital values ​​of the sensed W value and converts them into an analog signal, which can be provided to analog comparator 4700 using one or more of the previously described converters (such as pulse-to-voltage converter 3700, digital-to-voltage converter 4200, or digital-to-voltage converter 4300).

[0237] Figure 49 Output circuit 4900 is shown. It should be understood that if the neuron's output is digitized (such as by using an integral dual-slope ADC 3400 or 3500 as described above), it may still be necessary to perform activation function operations on the neuron's output. Figure 49An implementation is shown in which activation occurs before the neuron output is converted to a variable width pulse or pulse train. The output circuit 4900 includes an activation circuit 4901 and a current-to-pulse converter 4902. The activation circuit receives Ineuron values from the various flash memory cells and generates Ineuron act, which is the sum of the received Ineuron values. The current-to-pulse converter 4902 then converts Ineuron act to a series of digital pulses and / or digital data representing the count of a series of digital pulses. Other converters such as the integrating dual-slope ADCs 3400 or 3500 described previously can be used in place of the 4902 converter.

[0238] In another implementation, activation can occur after the digital pulse is generated. In this implementation, the digital output bits are mapped to a new set of digital bits using an activation mapping table or function implemented by activation mapping unit 5010. Examples of such mapping are shown graphically in Figure 50 and Figure 51 The activation digital mapping can mimic a sigmoid, tanh, ReLu, or any activation function. In addition, the activation digital mapping can quantize the output neuron.

[0239] Figure 52 An example of a charge summer 5200 is shown, which can be used to sum the outputs of the VMM during a verify operation after a program operation to obtain a single analog value representing the output and which can then be optionally converted to a digital bit value. The charge summer 5200 includes a current source 5201 and a sample-and-hold circuit including switches 5202 and sample-and-hold (S / H) capacitors 5203. As shown for the example of a 4-bit digital value, there are 4 S / H circuits to hold the values from 4 evaluation pulses, which are added at the end of the process. The S / H capacitors 5203 are selected to have a proportionality associated with their 2^n*DINn-bit position; for example, C DIN3 = x8 Cu, C DIN2 = x4 Cu, C DIN1 = x2 Cu, DIN0 = x1 Cu. The current source 5201 is also proportionally assigned.

[0240] Figure 53A current summer 5300 is shown that can be used to sum the output of the VMM during the verify operation after the program operation. The current summer 5300 includes current sources 5301, switches 5302, 5303, and 5304, and 5305. As shown for the example of a 4-bit digital value, there are current source circuits to hold the values from the 4 evaluation pulses, where these values are added at the end of the process. The current sources are assigned a scale based on 2^n*DINn-bit position; for example, I_DIN3 = x8 Icell units, I_DIN2 = x4 Icell units, I_DIN1 = x2 Icell units, I_DIN0 = x1 Icell units.

[0241] Figure 54 A digital summer 5400 is shown that receives multiple digital values, sums them, and generates an output DOUT representing the sum of the inputs. The digital summer 5400 can be used during the verify operation after the program operation. As shown for the example of a 4-bit digital value, there are digital output bits to hold the values from the 4 evaluation pulses, where these values are added at the end of the process. The digital output is digitally scaled based on 2^n*DINn-bit position; for example, DOUT3 = x8 DOUT0, DOUT2 = x4 DOUT1, I_DOUT1 = x2 DOUT0, I_DOUT0 = DOUT0.

[0242] Figure 55A and 55B A digital bit-to-pulse width converter 5500 is shown to be used within an input block, row decoder, or output block. The pulse width output from the digital bit-to-pulse width converter 5500 is proportional to the value described above with respect to Figure 36B The digital bit-to-pulse width converter includes a binary counter 5501. The state Q[N:0] of the binary counter 5501 can be loaded by serial or parallel data in a load sequence. Row control logic 5510 outputs a voltage pulse with a pulse width proportional to the value of the digital data input provided from a block such as the integrating type ADC in FIGS. 34 and 35.

[0243] Figure 55B A waveform of the output pulse width is shown with a width proportional to its digital bit value. First, the data in the received digital bit is inverted and the inverted digital bit is loaded into the counter 5501 either serially or in parallel. A row pulse width is then generated by the row control logic 5510 as shown in waveform 5520 by counting in binary until it reaches the maximum counter value.

[0244] Optionally, a pulse sequence-to-pulse converter can be used to convert a pulse sequence (such as Figure 34Bthe signal 3411 or 3413 in the signal 3513 or 3516 in Figure 35B the signal 3411 or 3413 in the signal 3513 or 3516 in Figure 36B the signal 3411 or 3413 in the signal 3513 or 3516 in

[0245] An example of a 4-bit digital input is shown in Table 9:

[0246] Table 9: Digital input bits to output pulse width

[0247]

[0248]

[0249] Another embodiment uses an up binary counter and digital comparison logic. That is, the output pulse width is generated by counting up the binary counter until the digital output of the binary counter is the same as the digital input bit.

[0250] Another embodiment uses a down binary counter. First, the down binary counter is loaded with the digital data input pattern, either serially or in parallel. Then the output pulse width is generated by counting down the binary counter until the digital output of the binary counter reaches a minimum value (i.e., a “0” logic state).

[0251] It should be noted that as used herein, the terms “over” and “on” both encompass “directly on” (with no intervening material, elements, or space disposed between the two) and “indirectly on” (with intervening material, elements, or space disposed between the two). Similarly, the term “adjacent” includes “directly adjacent” (with no intervening material, elements, or space disposed between the two) and “indirectly adjacent” (with intervening material, elements, or space disposed between the two), “mounted to” includes “directly mounted to” (with no intervening material, elements, or space disposed between the two) and “indirectly mounted to” (with intervening material, elements, or space disposed between the two), and “electrically coupled to” includes “directly electrically coupled to” (with no intervening material or elements electrically connecting the elements together between the two) and “indirectly electrically coupled to” (with intervening material or elements electrically connecting the elements together between the two). For example, forming an element “over” a substrate can include forming the element directly on the substrate with no intervening material / elements between the two, as well as forming the element indirectly on the substrate with one or more intervening material / elements between the two.

Claims

1. A method of performing power management in a vector-matrix multiplication system, the vector-matrix multiplication system comprising an array of non-volatile memory cells arranged in rows and columns, the method comprising: receiving a plurality of inputs; dividing the array into a plurality of groups of rows; organizing the plurality of inputs into a plurality of groups of inputs, each group of inputs in the plurality of groups of inputs corresponding to a different group of rows in the array; providing each group of inputs in the plurality of groups of inputs into a corresponding group of rows in the array in sequence; receiving a plurality of groups of outputs from the array in sequence, each group of outputs generated in response to a group of inputs in the plurality of groups of inputs; converting the plurality of groups of outputs into a plurality of groups of digital bits; and combining the plurality of groups of digital bits into a digital output by a timing control circuit.

2. The method of claim 1, further comprising: receiving a plurality of outputs from the array; organizing the plurality of outputs from the array into a plurality of groups of outputs; and providing each group of outputs in the plurality of groups of outputs to a converter circuit in sequence.

3. The method of claim 1, wherein each group of inputs in the plurality of groups of inputs is provided to a different group of word lines of the array.

4. The method of claim 1, wherein each group of inputs in the plurality of groups of inputs is provided to a different group of control gates of the array.

5. The method of claim 2, wherein the converting step is performed by an analog-to-digital converter.

6. The method of claim 5, wherein the analog-to-digital converter is an integrating analog-to-digital converter.

7. The method of claim 1, wherein each input in the plurality of inputs comprises one or more pulses.

8. The method of claim 1, wherein the array comprises a group of neurons.

9. The method of claim 1, wherein the non-volatile memory cells are split gate flash memory cells.

10. The method of claim 1, wherein the non-volatile memory cells are stack gate flash memory cells.

11. The method of claim 1, wherein the vector-matrix multiplication system comprises a second array of non-volatile memory cells arranged in rows and columns, the method further comprising, after performing the receiving step, the organizing step, and the providing step in sequence: providing a plurality of inputs to the second array.

12. A method of performing power management in a vector-matrix multiplication system comprising a first array and a second array, the first array and the second array coupled to a timing control circuit and an input circuit, each of the first array and the second array comprising non-volatile memory cells arranged in rows and columns, the method comprising: receiving, by the input circuit, a plurality of inputs; causing, by the timing control circuit, the first array to receive a first group of inputs in the plurality of inputs from the input circuit during a first read operation and the second array to receive a second group of inputs in the plurality of inputs from the input circuit during a second read operation in sequence.

13. The method of claim 12, wherein the vector-matrix multiplication system further comprises an output circuit for processing a plurality of outputs from the first array and the second array. ​ 14. The method of claim 13, wherein the output circuits are enabled sequentially.

15. The method of claim 13, wherein the output circuits comprise analog-to-digital converters.

16. The method of claim 12, wherein the input circuits comprise digital bit-to-pulse converters.

17. The method of claim 12, wherein the input circuits comprise digital-to-analog converters.

18. The method of claim 12, wherein the first array and the second array do not discharge array bias voltages when not enabled.

Images