Memory device and method for changing program state spacing based on usage frequency

Abstract

A memory device includes a plurality of memory cells and a controller. The controller is configured to program each of the memory cells to one of a plurality of program states and to read the memory cells using a read operation that applies a voltage to the memory cells. During the read operation, a spacing between adjacent ones of the program states is varied based on a frequency of use of the program states in the plurality of memory cells.

Description

[0001] This application is a divisional application of the application filed on August 29, 2019, with application number 201980090606.X, entitled "Memory device and method for changing programming state spacing based on usage frequency".

[0002] Related patent applications

[0003] This application claims the benefit of U.S. Application No. 16 / 382,060, filed April 11, 2019, which claims priority to U.S. Provisional Application No. 62 / 798,417, filed January 29, 2019. Technical Field

[0004] This invention relates to neural networks. Background Technology

[0005] Artificial neural networks mimic biological neural networks (the central nervous system of animals, especially the brain). These artificial neural networks are used to estimate or approximate functions that may depend on a large number of inputs and are generally known. Artificial neural networks typically consist of interconnected layers of "neurons" that exchange messages with each other. Figure 1 An artificial neural network is illustrated, where circles represent layers of inputs or neurons. Connections (called synapses) are indicated by arrows and have numerical weights that can be adjusted empirically. This allows the neural network to adapt to its inputs and learn. Typically, a neural network consists of layers with multiple inputs. There are usually one or more intermediate layers of neurons, and an output layer of neurons that provide the output of the neural network. Neurons at each level make decisions individually or collectively based on the data received from the synapses.

[0006] One of the major challenges in developing artificial neural networks for high-performance information processing is the lack of sufficient hardware technology. Real-world neural networks rely on a large number of synapses to achieve high connectivity between neurons, i.e., very high computational parallelism. In principle, such complexity can be achieved using digital supercomputers or dedicated clusters of graphics processing units. However, compared to biological networks, these methods are generally energy inefficient, in addition to being costly, as biological networks consume less energy primarily due to their ability to perform low-precision analog computations. CMOS analog circuits have been used in artificial neural networks, but given the large number of neurons and synapses, most CMOS-implemented synapses are excessively large. Summary of the Invention

[0007] The previously mentioned problems and needs are addressed by a memory device comprising multiple memory cells and a controller. The controller is configured to program each memory cell in the memory cells into one of a plurality of programming states, and to read the memory cells using a read operation that applies a voltage to the memory cells, wherein during the read operation, the spacing between adjacent programming states varies based on the frequency of use of the programming states in the plurality of memory cells.

[0008] The neural network device includes a first plurality of synapses configured to receive a first plurality of inputs and generate a first plurality of outputs therefrom. The first plurality of synapses includes a plurality of memory cells and a controller. The controller is configured to program each memory cell in the memory cells to one of a plurality of programming states, and to read the memory cells using a read operation with an applied voltage to the memory cells, wherein during the read operation, the spacing between adjacent programming states varies based on the frequency of use of the programming states in the plurality of memory cells. The plurality of memory cells are configured to generate the first plurality of outputs based on the first plurality of inputs and the plurality of programming states. The first plurality of neurons are configured to receive the first plurality of outputs.

[0009] A method for programming a memory cell, the method comprising: programming each of a plurality of memory cells to a programming state among a plurality of programming states; reading the memory cell using a read operation applying a voltage to the memory cell; determining a usage frequency of each programming state among the programming states of the plurality of memory cells; and for each programming state, setting a spacing between one programming state and an adjacent programming state during a read operation based on the determined usage frequency of the programming state.

[0010] A neural network device includes a first plurality of synapses and a second plurality of synapses, as well as a first plurality of neurons and a second plurality of neurons. The first plurality of synapses are configured to receive a first plurality of inputs and generate a first plurality of outputs therefrom. Each first plurality of synapses includes a plurality of first memory cells and a controller configured to program each of the first memory cells to a first programming state among a plurality of first programming states, and to read the first memory cells using a read operation applying a voltage to the first memory cells. The plurality of first memory cells are configured to generate the first plurality of outputs based on the first plurality of inputs and the plurality of first programming states. The first plurality of neurons are configured to receive the first plurality of outputs. The second plurality of synapses are configured to receive a second plurality of inputs from the first plurality of neurons and generate a second plurality of outputs therefrom. The second plurality of synapses include a plurality of second memory cells, and the controller is configured to program each of the second memory cells to a second programming state among a plurality of second programming states, and to read the second memory cells using a second read operation applying a second voltage to the second memory cells. The plurality of second memory cells are configured to generate a second plurality of outputs based on a second plurality of inputs and a second plurality of second programming states. The second plurality of neurons are configured to receive the second plurality of outputs. The neural network device is further characterized by at least one of the following:

[0011] The total number of multiple first programming states is different from the total number of multiple second programming states.

[0012] The total number of multiple first programming states is only 2 and the total number of multiple second programming states is greater than 2, or the total number of multiple first programming states is greater than 2 and the total number of multiple second programming states is only 2.

[0013] The first memory cell is volatile and the second memory cell is non-volatile, or the first memory cell is non-volatile and the second memory cell is volatile.

[0014] The first and second memory cells are non-volatile.

[0015] The controller is configured to perform a first read operation above a threshold and a second read operation below the threshold, or to perform a first read operation below the threshold and a second read operation above the threshold.

[0016] The controller is configured to perform a first read operation and a second read operation above a threshold, or to perform a first read operation and a second read operation below a threshold.

[0017] Other objects and features of the invention will become apparent from a review of the specification, claims and drawings. Attached Figure Description

[0018] Figure 1 This is a schematic diagram of an artificial neural network.

[0019] Figure 2 This is a side cross-sectional view of a conventional 2-gate non-volatile memory cell.

[0020] Figure 3 To show Figure 2 A schematic diagram of a conventional array architecture for memory cells.

[0021] Figure 4 This is a side cross-sectional view of a conventional 2-gate non-volatile memory cell.

[0022] Figure 5 To show Figure 4 A schematic diagram of a conventional array architecture for memory cells.

[0023] Figure 6 This is a side cross-sectional view of a conventional 4-gate non-volatile memory cell.

[0024] Figure 7 To show Figure 6 A schematic diagram of a conventional array architecture for memory cells.

[0025] Figure 8A This is a schematic diagram illustrating the uniformly spaced weight levels of a neural network.

[0026] Figure 8B This is a schematic diagram illustrating the non-uniform spacing of neural network weight levels.

[0027] Figure 9 The flowchart illustrates the two-way optimization algorithm.

[0028] Figure 10 A block diagram illustrating the weight mapping using current comparison is provided.

[0029] Figure 11 A block diagram illustrating the weight mapping using voltage comparison is provided.

[0030] Figure 12 This is a schematic diagram illustrating different levels of an exemplary neural network utilizing an array of non-volatile memory.

[0031] Figure 13 A block diagram illustrating the vector multiplier matrix.

[0032] Figure 14 A block diagram illustrating the various levels of the vector multiplier matrix.

[0033] Figure 15 This is a schematic diagram illustrating a first architecture of a two-gate memory cell array.

[0034] Figure 16 This is a schematic diagram illustrating a second architecture of a two-gate memory cell array.

[0035] Figure 17 This is a schematic diagram illustrating a third architecture of a two-gate memory cell array.

[0036] Figure 18 This is a schematic diagram illustrating a current-to-voltage converter using a two-gate memory cell.

[0037] Figure 19 This is a schematic diagram illustrating a first architecture of a quad-gate memory cell array.

[0038] Figure 20 This is a schematic diagram illustrating a current-to-voltage converter using a four-gate memory cell.

[0039] Figure 21 This is a schematic diagram illustrating a second architecture of a quad-gate memory cell array.

[0040] Figure 22 This is a schematic diagram illustrating the third architecture of a quad-gate memory cell array.

[0041] Figure 23 This is a schematic diagram illustrating the fourth architecture of a four-gate memory cell array.

[0042] Figure 24 This is a schematic diagram illustrating the fifth architecture of a quad-gate memory cell array.

[0043] Figure 25 This is a side cross-sectional view of a 3-gate non-volatile memory cell.

[0044] Figure 26 This is a schematic diagram illustrating the architecture of a tri-gate memory cell array.

[0045] Figure 27 This is a side cross-sectional view of a 3-gate non-volatile memory cell.

[0046] Figures 28 to 31 To show Figure 27 A schematic diagram of an array architecture for a tri-gate memory cell.

[0047] Figure 32 This is a side cross-sectional view of a 2-gate non-volatile memory cell.

[0048] Figures 33 to 39 To show Figure 32 A schematic diagram of an array architecture for two-gate memory cells.

[0049] Figure 40 , Figure 42 , Figure 44and Figure 46 A graph showing the usage density as a function of the programming state level.

[0050] Figure 41 , Figure 43 , Figure 45 and Figure 47 This is a schematic diagram illustrating different programming state level spacing schemes.

[0051] Figure 48 This is a schematic diagram illustrating a controller on the same chip as the memory array used to implement the operation of the memory array. Detailed Implementation

[0052] The artificial neural network of this invention utilizes a combination of CMOS technology and non-volatile memory arrays. Digital non-volatile memory is well known. For example, U.S. Patent 5,029,130 ​​(“130 Patent”) discloses a split-gate non-volatile memory cell array. The memory cells disclosed in “130 Patent”... Figure 2 The image shows a memory cell 10. Each memory cell 10 includes a source region 14 and a drain region 16 formed in a semiconductor substrate 12, with a channel region 18 between them. A floating gate 20 is formed over and insulated from (and controls the conductivity of) a first portion of the channel region 18, and is formed over a portion of the drain region 16. A control gate 22 (i.e., a second channel control gate) has a first portion 22b and a second portion 22c, the first portion being disposed over and insulated from (and controlling the conductivity of) the second portion of the channel region 18, the second portion extending upward along the floating gate 20 and over the floating gate. The floating gate 20 and the control gate 22 are insulated from the substrate 12 by a gate oxide 26.

[0053] By applying a high positive voltage to the control gate 22, the memory cell 10 is erased (where electrons are removed from the floating gate 20), causing electrons on the floating gate 20 to tunnel from the floating gate 20 through the intermediate insulator 24 to the control gate 22 via the Fowler-Nordheim tunneling effect.

[0054] Memory cell 10 is programmed by applying a positive voltage to control gate 22 and a positive voltage to drain 16 (where electrons are placed on floating gate 20). Electron current flows from source 14 to drain 16. When electrons reach the gap between control gate 22 and floating gate 20, they accelerate and become hot. Due to electrostatic attraction from floating gate 20, some heated electrons are injected into floating gate 20 through gate oxide 26.

[0055] Memory cell 10 is read by applying a positive read voltage to drain 16 and control gate 22 (which turns on the portion of the channel region below the control gate). If floating gate 20 is positively charged (i.e., electrons are erased and a positive voltage is capacitively coupled to drain 16), the portion of channel region 18 below floating gate 20 is also turned on, and current flows through channel region 18, which is sensed as erased or "1". If floating gate 20 is negatively charged (i.e., programmed electronically), the portion of channel region 18 below floating gate 20 is mostly or completely turned off, and no current (or very little current) flows through channel region 18, which is sensed as programmed or "0".

[0056] Figure 3 The structure of a conventional array architecture for memory cells 10 is shown. Memory cells 10 are arranged in rows and columns. In each column, memory cells are arranged end-to-end in a mirror manner, such that they form pairs of memory cells, each pair sharing a common source region 14(S), and each adjacent pair sharing a common drain region 16(D). All source regions 14 of any given row of memory cells are electrically connected together via source lines 14a. All drain regions 16 of any given column of memory cells are electrically connected together via bit lines 16a. All control gates 22 of any given row of memory cells are electrically connected together via control gate lines 22a. Therefore, although memory cells can be programmed and read individually, memory cell erasure is performed row-by-row (each row of memory cells is erased together by applying a high voltage to the control gate lines 22a). If a particular memory cell is to be erased, all memory cells in the same row are also erased.

[0057] Those skilled in the art will understand that the source and drain can be interchangeable, wherein the floating gate 20 may extend partially over the source 14 instead of the drain 16, as shown below. Figure 4 As shown. Figure 5The corresponding memory cell architecture is best illustrated, including memory cell 10, source line 14a, bit line 16a, and control gate line 22a. As can be clearly seen from the figures, memory cells 10 in the same row share the same source line 14a and the same control gate line 22a, while the drain regions of all cells in the same column are electrically connected to the same bit line 16a. The array design is optimized for digital applications and allows for individual programming of selected cells, for example, by applying 1.6V and 7.6V to the selected control gate line 22a and source line 14a respectively, and grounding the selected bit line 16a. Interference with unselected memory cells in the same pair is avoided by applying a voltage greater than 2V to the unselected bit line 16a and grounding the remaining lines. Memory cell 10 cannot be erased individually because the process responsible for erasing (electron tunneling from the floating gate 20 to the control gate 22) is only slightly affected by the drain voltage (i.e., for two adjacent cells in the row direction sharing the same source line 14a, the only voltage may differ). Non-limiting examples of operating voltages may include:

[0058] Table 1

[0059]

[0060]

[0061] "Read 1" is a read mode in which the cell current flows out on the bit line. "Read 2" is a read mode in which the cell current flows out on the source line.

[0062] Split-gate memory cells with more than two gates are also known. For example, memory cells having a source region 14, a drain region 16, a floating gate 20 above a first portion of a channel region 18, a select gate 28 (i.e., a second channel control gate) above a second portion of the channel region 18, a control gate 22 above the floating gate 20, and an erase gate 30 above the source region 14 are known, such as... Figure 6 As shown in the diagram (see, for example, U.S. Patent 6,747,310). Here, all gates except for floating gate 20 are non-floating gates, meaning they are electrically connected to or can be electrically connected to a voltage or current source. Programming is illustrated by heated electrons from channel region 18, which inject themselves onto floating gate 20. Erasing is illustrated by electrons tunneling from floating gate 20 to erase gate 30.

[0063] The architecture of a quad-gate memory cell array can be as follows: Figure 7The configuration is shown. In this embodiment, each horizontal select gate line 28a electrically connects all select gates 28 of the row memory cell together. Each horizontal control gate line 22a electrically connects all control gates 22 of the row memory cell together. Each horizontal source line 14a electrically connects all source regions 14 of the two rows of memory cells sharing a source region 14 together. Each bit line 16a electrically connects all drain regions 16 of the column memory cell together. Each erase gate line 30a electrically connects all erase gates 30 of the two rows of memory cells sharing an erase gate 30 together. As with the previous architecture, individual memory cells can be programmed and read independently. However, memory cells cannot be erased individually. Erasure is performed by applying a high positive voltage to the erase gate line 30a, which results in the simultaneous erasure of two rows of memory cells sharing the same erase gate line 30a. Exemplary, non-limiting operating voltages may include those in Table 2 below (in this embodiment, the select gate line 28a may be referred to as the word line WL):

[0064] Table 2

[0065]

[0066]

[0067] "Read 1" is a read mode in which the cell current flows out on the bit line. "Read 2" is a read mode in which the cell current flows out on the source line.

[0068] To utilize the aforementioned non-volatile memory array in a neural network, two modifications can be made. First, the lines can be reconfigured so that each memory cell can be programmed, erased, and read individually without adversely affecting the memory state of other memory cells in the array, as explained further below. Second, continuous (analog) programming of the memory cells can be provided. Specifically, the memory state or programming state of each memory cell in the array (i.e., the charge on the floating gate, reflected by the number of electrons on the floating gate) can be continuously changed from a fully erased state to a fully programmed state and vice versa, independently and with minimal interference to other memory cells. This means that the cell storage device is analog, or at least can store one discrete value among many discrete values. This allows for very precise and individual tuning of all cells in the memory array, making the memory array ideal for both storage and fine-tuning of the synaptic weights of the neural network.

[0069] Memory cell programming and storage

[0070] The neural network weight levels stored in memory cells can be evenly spaced (e.g., Figure 8AAs shown), or unevenly spaced (as shown). Figure 8B (As shown). Examples such as Figure 9 The bidirectional tuning algorithm shown implements the programming of non-volatile memory cells. Icell is the read current of the target cell to be programmed, and Itarget is the desired read current when the cell is ideally programmed. The target cell read current Icell is read (step 1) and compared with the target read current Itarget (step 2). If the target cell read current Icell is greater than the target read current Itarget, a programming tuning process (step 3) is performed to increase the number of electrons on the floating gate 20 (where a lookup table or a silicon-based approximation function can be used to determine the desired initial and incremental programming voltages VCG on the control gate 22) (steps 3a-3b), which can be repeated as needed (step 3c). If the target cell read current Icell is less than the target read current Itarget, an erase tuning process (step 4) is performed to decrease the number of electrons on the floating gate 20 (where a lookup table or a silicon-based approximation function can be used to determine the desired initial and incremental erase voltages VEG on the erase gate 30) (steps 4a-4b), which can be repeated as needed (step 4c). If the programming tuning process exceeds the target read current, then the erase tuning process is executed (step 3d and starting from step 4a), and vice versa (step 4d and starting from step 3a), until the target read current is reached (within an acceptable delta value).

[0071] Conversely, a one-way tuning algorithm utilizing programming optimization can be used to program non-volatile memory cells. Using this algorithm, memory cell 10 is first completely erased, and then... Figure 9 The programming tuning steps 3a-3c are performed until the read current of the target memory cell 10 reaches the target threshold. Alternatively, a unidirectional tuning algorithm utilizing erase tuning can be used to tune the non-volatile memory cell. In this method, the memory cell is first fully programmed, and then... Figure 9 The erase optimization steps 4a-4c continue until the read current of the target memory cell reaches the target threshold.

[0072] Figure 10This diagram illustrates a weight mapping using current comparison. Weighted digital bits (e.g., 5-bit weights per synapse representing the target digital weight of the memory cell) are input to a digital-to-analog converter (DAC) 40, which converts the bits to a voltage Vout (e.g., 64 voltage levels - 5 bits). Vout is converted to a current Iout (e.g., 64 current levels - 5 bits) via a voltage-to-current converter V / I Conv 42. The current Iout is provided to a current comparator IComp 44. A programming or erasing algorithm enable input is provided to the memory cell 10 (e.g., erasure: increasing the EG voltage; or programming: increasing the CG voltage). The output memory cell current Icellout (i.e., from a read operation) is provided to the current comparator IComp 44. The current comparator IComp 44 compares the memory cell current Icellout with the current Iout derived from the weighted digital bits to generate a signal indicating the weights stored in the memory cell 10.

[0073] Figure 11 This diagram illustrates a weight mapping using voltage comparison. Weight digital bits (e.g., 5-bit weights per synapse) are input to a digital-to-analog converter (DAC) 40, which converts the bits to a voltage Vout (e.g., 64 voltage levels - 5 bits). Vout is provided to a voltage comparator VComp 46. A programming or erasing algorithm enable input is provided to memory cell 10 (e.g., erasure: increasing the EG voltage; or programming: increasing the CG voltage). The output memory cell current Icellout is provided to a current-to-voltage converter I / VConv 48 to be converted to a voltage V2out (e.g., 64 voltage levels - 5 bits). The voltage V2out is provided to the voltage comparator VComp 46. The voltage comparator VComp 46 compares voltages Vout and V2 to generate a signal indicating the weights stored in memory cell 10.

[0074] Another implementation for weight mapping comparison uses a variable pulse width (i.e., a pulse width that is directly or inversely proportional to the weight value) to input the weights and / or the output of the memory cell. In yet another implementation for weight mapping comparison, digital pulses (e.g., pulses generated from a clock, wherein the number of pulses is directly or inversely proportional to the weight value) are used to input the weights and / or the output of the memory cell.

[0075] Neural networks using non-volatile memory cell arrays

[0076] Figure 12A conceptual, non-limiting example of a neural network utilizing an array of nonvolatile memory is illustrated. This example uses a non-volatile memory array neural network for a face recognition application, but any other suitable application can also be implemented using a neural network based on a non-volatile memory array. For this example, S0 is the input layer, which is a 32×32 pixel RGB image with 5-bit precision (i.e., three 32×32 pixel arrays, one for each color R, G, and B, with 5-bit precision per pixel). The synapse CB1 from S0 to C1 has both different sets of weights and shared weights, and scans the input image with a 3×3 pixel overlapping filter (kernel), shifting the filter by 1 pixel (or more than 1 pixel as indicated by the model). Specifically, the values ​​of 9 pixels in a 3×3 portion of the image (i.e., referred to as the filter or kernel) are provided to synapse CB1, whereby these 9 input values ​​are multiplied by the appropriate weights, and after summing the output of this multiplication, a single output value is determined by the first synapse of CB1 to be used to generate the pixels of one of the layers C1 of the feature map. The 3×3 filter is then shifted one pixel to the right (i.e., adding a column of three pixels to the right and releasing a column of three pixels to the left), thereby providing the nine pixel values ​​from this newly positioned filter to synapse CB1, whereby they are multiplied by the same weights and a second single output value is determined by the associated synapse. This process continues until the 3×3 filter scans all three colors and all bits (precision values) across the entire 32×32 pixel image. This process is then repeated using different sets of weights to generate different feature maps for C1 until all feature maps for layer C1 are computed.

[0077] At layer C1, in this example, there are 16 feature maps, each with 30×30 pixels. Each pixel is a new feature pixel extracted from the product of the input and the kernel, so each feature map is a two-dimensional array. Therefore, in this example, synapse CB1 consists of a 16-layer two-dimensional array (remember that the neuron layers and arrays referred to in this article are logical relationships, not necessarily physical relationships; that is, the array does not have to be oriented to a physical two-dimensional array). Each of the 16 feature maps is generated by a set of sixteen different groups of synaptic weights applied to the filter scan. The C1 feature maps can all relate to different aspects of the same image features, such as boundary recognition. For example, the first map (generated using the first weight recombination, shared for all scans used to generate the first map) can recognize circular edges, the second map (generated using the second weight recombination, different from the first weight recombination) can recognize rectangular edges, or the aspect ratio of certain features, and so on.

[0078] Before transitioning from layer C1 to layer S1, activation function P1 (pooling) is applied, which pools values ​​from consecutive non-overlapping 2×2 regions in each feature map. The purpose of the pooling stage is to average the neighboring locations (or, alternatively, use a max function) to, for example, reduce dependence on edge locations and reduce the data size before moving to the next stage. At layer S1, there are 16 15×15 feature maps (i.e., sixteen different arrays, each 15×15 pixels). The synapses and associated neurons in CB2 from layer S1 to layer C2 scan the map in S1 using a 4×4 filter, where the filter is shifted by one pixel. At layer C2, there are 22 12×12 feature maps. Before transitioning from layer C2 to layer S2, activation function P2 (pooling) is applied, which pools values ​​from consecutive non-overlapping 2×2 regions in each feature map. At layer S2, there are 22 6×6 feature maps. An activation function is applied to synapse CB3 from layer S2 to layer C3, where each neuron in layer C3 is connected to each mapping in layer S2. There are 64 neurons at layer C3. Synapse CB4 from layer C3 to the output layer S3 completely connects S3 to C3. The output at layer S3 consists of 10 neurons, with the highest output neuron determining the class. For example, this output could indicate the recognition or classification of the content of the original image.

[0079] Each level of synapse is implemented using an array or a portion of an array of non-volatile memory cells. Figure 13 This is a block diagram of a vector matrix multiplication (VMM) array including non-volatile memory cells, and used as a synapse between the input layer and the next layer. Specifically, the VMM array 32 includes a non-volatile memory cell array 33, erase gate and word line gate decoders 34, control gate decoders 35, bit line decoders 36, and source line decoders 37, which decode the inputs of the memory cell array 33. In this example, the source line decoder 37 also decodes the outputs of the memory cell array 33. Alternatively, the bit line decoder 36 can decode the outputs of the non-volatile memory cell array 33. The memory array serves two purposes. First, it stores weights that will be used by the VMM array 32. Second, the memory cell array efficiently multiplies the inputs by the weights stored in the memory cell array and adds the results along each output line to produce an output that will be the input to the next layer or the last layer. By performing multiplication and addition functions, the memory array eliminates the need for separate multiplication and addition logic circuits and is also highly efficient due to its in-situ memory computation.

[0080] The output of the memory cell array is provided to a single or differential summing circuit 38, which sums the output of the memory cell array to create a single value for the convolution. The summed output value is then provided to an activation function circuit 39, which modifies the output. The activation function can be a sigmoid, tanh, or ReLU function. The modified output value from circuit 39 becomes an element of the feature map for the next layer (e.g., C1 as described above), and is then applied to the next synapse to produce the next feature map layer or the final layer. Thus, in this example, the memory cell array 33 constitutes multiple synapses (which receive their input from existing neuron layers or from an input layer such as an image database), and the summing circuit 38 and the activation function circuit 39 constitute multiple neurons.

[0081] Figure 14 A block diagram illustrating the use of a multilayer VMM array 32 (here labeled VMM arrays 32a, 32b, 32c, 32d, and 32e). For example... Figure 14 As shown, the input (denoted as Inputx) is converted from digital to analog by a digital-to-analog converter 31 and provided to the input VMM array 32a. The output generated by the input VMM array 32a is provided as the input to the next VMM array (hidden level 1) 32b, which in turn generates the output provided as the input to the next VMM array (hidden level 2) 32c, and so on. The layers of the VMM array 32 serve as different layers of synapses and neurons in a convolutional neural network (CNN). Each VMM array 32a, 32b, 32c, 32d, and 32e can be an independent physical non-volatile memory array, or multiple VMM arrays can utilize different portions of the same non-volatile memory array, or multiple VMM arrays can utilize overlapping portions of the same physical non-volatile memory array. Figure 14 The example shown contains five layers (32a, 32b, 32c, 32d, 32e): one input layer (32a), two hidden layers (32b, 32c), and two fully connected layers (32d, 32e). Those skilled in the art will recognize that this is merely exemplary, and conversely, a system may include more than two hidden layers and more than two fully connected layers.

[0082] Figure 15 This illustrates a two-gate memory cell (i.e., such as a source-sum matrix multiplier) arranged as a source-sum matrix multiplier. Figure 4 (As shown) array. Used for Figure 15 The array of various gate lines and region lines with Figure 5The array is identical to the one in the array (where the corresponding structures have the same element number). After each memory cell is programmed with the appropriate weight value, the array is used as a source-sum matrix multiplier. The matrix voltage input is Vin0-Vinn and is placed on bit line 16a. Figure 15 The matrix outputs Iout0…IoutN of the array are generated on source line 22a. For all cells in a row, each output Iout is the sum of the input current I multiplied by the weights W stored in the cell:

[0083] Iout = Σ(Ij * Wij)

[0084] Where "i" represents the row and "j" represents the column where the memory cell is located. When applying an input voltage instead of an input current (e.g., ... Figure 15 In the case of Vin0-Vinn), for all cells in that row, each output Iout is proportional to the sum of the input voltage multiplied by the weights W stored in the cell:

[0085] IoutαΣ(Vj*Wij)

[0086] Each row of memory cells serves as a single neuron, and the total weight value of this neuron is represented as the output current Iout, which is determined by the sum of the weight values ​​stored in the memory cells of that row. The output of any given neuron is in the form of a current, which, after being regulated by the activation function circuitry, can be used as the input to the next subsequent VMM array stage.

[0087] Figure 16 This illustrates a two-gate memory cell 10 arranged as a drain (e.g., bit line of a memory array) summation matrix multiplier (i.e., such as...). Figure 4 Another configuration of the array (shown). Figure 16 The lines of the array and Figure 15 The array lines are identical. However, the matrix voltage inputs Vin0...Vin3 are placed on source line 14a, and the matrix outputs Iout0...IoutN are generated on bit line 16a (i.e., for all cells in the column, each output Iout is the sum of the cell currents, which are proportional to the weights W stored in the cell). As in the previous implementation, the output of any given neuron is in the form of a current, which, after being regulated by the activation function circuitry, can be used as the input current for the next subsequent VMM array stage.

[0088] Figure 17 This illustrates a two-gate memory cell (i.e., such as a drain-sum matrix multiplier) arranged as a drain-sum matrix multiplier. Figure 4 Another configuration of the array (shown). Figure 17 The lines of the array and Figure 15The array lines are the same. However, the matrix voltage inputs Vin0...Vin7 are placed on the control gate line 22a, and the matrix outputs (currents) Iout0...IoutN are generated on the bit line 16a (i.e., for all cells in the column, each output Iout is the sum of the cell currents, which are proportional to the weights W stored in the cell).

[0089] exist Figure 17 Since the input is voltage and the output is current, each subsequent VMM stage after the first stage preferably includes circuitry for converting the input current from the previous VMM stage into a voltage that will be used as the input voltage Vin. Figure 18 An example of such a current-to-voltage conversion circuit is shown, which is a modified row of memory cells that logarithmically converts the input currents Iin0...IinN to input voltages Vin0...VinN for application in subsequent stages. The memory cells described herein are biased in weak inversion mode.

[0090] Ids = Io * e (Vg-Vth) / kVt =w*Io*e (Vg) / kVt

[0091] Where w = e (-Vth) / kVt

[0092] For an I-to-V logarithmic converter that uses a memory cell to convert input current to input voltage:

[0093] Vg = k * Vt * log[Ids / wp * Io]

[0094] Here, wp represents the w of the reference memory cell or peripheral memory cell. For a memory array used as a vector matrix multiplier (VMM), the output current is:

[0095] Iout = wa * Io * e (Vg) / kVt ,Right now

[0096] Iout = (wa / wp) * Iin = W * Iin

[0097] W = e (Vthp-Vtha) / kVt

[0098] Here, wa = w for each memory cell in the memory array. The control gate line 22a can be used as an input to the memory cell's input voltage, which is connected to the bit line 16a via a switch BLR closed during current-to-voltage conversion.

[0099] Alternatively, the non-volatile memory cells of the VMM array described herein can be configured to operate in a linear region:

[0100] Ids=β*(Vgs-Vth)*Vds; β=u*Cox*Wt / L,

[0101] Where Wt and L are the width and length of the transistor, respectively.

[0102] Wα(Vgs-Vth) means that the weight W is proportional to (Vgs-Vth).

[0103] Control gate lines, bit lines, or source lines can be used as inputs to memory cells operating in a linear region. Bit lines or source lines can be used as outputs of output neurons.

[0104] For an I-to-V linear converter, memory cells (e.g., reference memory cells or peripheral memory cells) or transistors or resistors operating in the linear region can be used to linearly convert input / output current into input / output voltage. Alternatively, the non-volatile memory cells of the VMM array described herein can be configured to operate in the saturation region:

[0105] Ids = 1 / 2 * β * (Vgs - Vth) 2 β=u*Cox*Wt / L

[0106] Wα(Vgs-Vth) 2 This refers to the weight W and (Vgs-Vth). 2 Proportional

[0107] Control gate lines can be used as inputs to memory cells operating in saturation regions. Bit lines or source lines can be used as outputs of output neurons. Alternatively, the non-volatile memory cells of the VMM array described herein can be used in all regions or combinations thereof (subthreshold, linear, or saturation regions). Any of the above-described current-to-voltage conversion circuits or techniques can be used with any embodiment of the embodiments described herein, such that the current output from any given neuron in the form of a current can be used as the input to the next subsequent VMM array stage after being regulated by the activation function circuitry.

[0108] Figure 19 This illustrates a four-gate memory cell (i.e., such as a drain (bit line) summation matrix multiplier) arranged as a drain (bit line) summation matrix multiplier. Figure 6 (As shown) Array configuration. Figure 19 The lines of the array and Figure 7 The array lines are identical. After each memory cell is programmed with an appropriate weight value, the array is used as a drain-sum matrix multiplier. The matrix voltage inputs are Vin0-Vin3 and are placed on the select gate line 28a. The matrix outputs Iout0...Ioutn are generated on bit line 16a. For all cells in the column, each output Iout is the sum of the cell currents, which are proportional to the weight W stored in the cell.

[0109] exist Figure 19 Since the input is voltage and the output is current, each subsequent VMM stage after the first stage preferably includes circuitry for converting the input current from the previous VMM stage into a voltage that will be used as the input voltage Vin. Figure 20 An example of such a current-to-voltage conversion circuit is shown, which is a modified row of memory cells that logarithmically converts input currents Iin0...IinN to input voltages Vin0...VinN. Similarly, a linear current-to-voltage converter can be used with modified memory cells operating in the linear region to linearly convert input currents Iin0...IinN to input voltages Vin0...VinN. Figure 20 As shown, the select gate line 28a is connected to the bit line 16a via the switch BLR for current-to-voltage conversion. Alternatively, the control gate line 22a can be connected to the bit line 16a for current-to-voltage conversion.

[0110] Figure 21 This illustrates a four-gate memory cell (i.e., such as a drain-sum matrix multiplier) arranged as a drain-sum matrix multiplier. Figure 6 Another configuration of the array (shown). Figure 21 The lines of the array and Figure 7 The array lines are identical. After each memory cell is programmed with an appropriate weight value, the array is used as a drain-sum matrix multiplier. The matrix voltage inputs are Vin0-Vin3 and are placed on control gate line 22a. The matrix outputs Iout0...Ioutn are generated on bit line 16a. For all cells in the column, each output Iout is the sum of the cell currents, which are proportional to the weight W stored in the cell.

[0111] Figure 22 A quad-gate memory cell 10 arranged as a source-sum matrix multiplier is shown (i.e., such as...). Figure 6 Another configuration of the array (shown). For Figure 22 The lines of the array and Figure 7 The array is the same, except that the control gate lines 22a extend vertically instead of horizontally, allowing each memory cell to be programmed, erased, and read independently. Specifically, each column of memory cells includes a control gate line 22a that connects all the control gates 22 of the memory cells in that column together. Matrix voltage inputs Vin0...VinN are provided on the control gate lines 22a, and matrix outputs Iout0...Iout1 are generated on the source lines 14a.

[0112] Figure 23 This illustrates a four-gate memory cell (i.e., such as a source-sum matrix multiplier) arranged as a source-sum matrix multiplier. Figure 6 Another configuration of the array (shown). Figure 23 The lines of the array and Figure 22 The arrays are identical. Matrix voltage inputs Vin0...VinN are provided on bit line 16a, and matrix outputs Iout0...Iout1 are generated on source line 14a.

[0113] Figure 24 This illustrates a four-gate memory cell (i.e., such as a source-sum matrix multiplier) arranged as a source-sum matrix multiplier. Figure 6 Another configuration of the array (shown). Figure 24 The lines of the array and Figure 22 The array is the same, except that each bit line includes a bit line buffer transistor 60, which acts as a switch to turn on the bit line (i.e., couple the bit line to its current or voltage source). Matrix inputs Vin0...VinN are provided to the gate of transistor 60, and matrix outputs Iout0...Iout1 are provided on source line 14a. The advantage of this configuration is that the matrix inputs can be provided as voltages (which operate transistor 60) rather than directly to the bit lines. This allows a constant voltage source to be applied to the bit lines (i.e., through transistor 60), which responds to the input voltage Vin provided to the transistor gate.

[0114] Figures 25 to 26 This illustrates the configuration of a three-gate memory cell array arranged as a drain-sum matrix multiplier. The memory cells are... Figure 25 As shown in the figure, and with Figure 6 The memory cells are the same, except that there is no erase gate. In one embodiment, cell erasure is performed by applying a positive voltage to the select gate 28, where electrons tunnel from the floating gate 20 to the select gate 28. Below is... Figure 25 A table of exemplary non-limiting operating voltages for a tri-gate memory cell.

[0115] Table 3

[0116] SG 28a BL 16a CG 22a SL 14a Read 1 0.50-2V 0.1-2V 0-2.6V 0.1-2V Read 2 0.5-2V 0-2V 0-2.6V 2-0.1V erase 3-12V 0V 0V to -13V 0V programming 1V 1uA 8-11V 4.5-8V

[0117] "Read 1" is a read mode in which the cell current flows out on the bit line. "Read 2" is a read mode in which the cell current flows out on the source line.

[0118] Lines used for memory cell arrays Figure 26 As shown in the figure, and with Figure 19The array has identical lines, except that gate line 30a is not erased (because the gate is not erased), and source line 14a extends vertically instead of horizontally, allowing each memory cell to be programmed, erased, and read independently. Specifically, each column of memory cells includes a source line 14a connecting all source regions 14 of the memory cells in that column together. After each memory cell is programmed with an appropriate weight value, the array is used as a source-sum matrix multiplier. The matrix voltage input is Vin0-Vin3 and is placed on the control gate line 22a. Alternatively, the matrix voltage input can be placed on the select gate line 28a. Matrix outputs Iout0...Ioutn are generated on source line 14a. For all cells in the column, each output Iout is the sum of the cell currents, which are proportional to the weight W stored in the cell. Alternatively, the matrix outputs are generated on bit line 16a. In another embodiment, source line 14a extends horizontally (i.e., each connecting all source regions of the memory cell row). Figures 25 to 26 An alternative erasure operation can place the p-type substrate 12 at a high voltage (e.g., 10V to 20V) and the control gate 22 at a low voltage or negative voltage (e.g., -10V to 0V), thereby allowing electrons to tunnel from the floating gate 20 to the substrate 12.

[0119] Figure 27 A memory cell 10 with three gates is shown, which is similar to Figure 6 The memory cells in the example lack a control gate 22. Reading, erasing, and programming are performed in a similar manner, but without any biasing of the control gate. Exemplary non-limiting operating voltages may include those in Table 4 below:

[0120] Table 4

[0121] SG 28a BL 16a EG 30a SL 14a Read 1 0.5-2V 0.1-2V 0-2.6V 0V Read 2 0.5-2V 0-2V 0-2.6V 2-0.1V erase -0.5V / 0V 0V 8-12V 0V programming 1V 1uA 4.5-7V 4.5-7V

[0122] "Read 1" is a read mode in which the cell current flows out on the bit line. "Read 2" is a read mode in which the cell current flows out on the source line.

[0123] Figure 28 It shows the use of Figure 27The memory cell array architecture of memory cell 10, wherein all lines except bit line 16a extend in the horizontal / row direction. After each memory cell is programmed with appropriate weight values, the array is used as a bit line summation matrix multiplier. The matrix voltage input is Vin0-Vin3 and is placed on the select gate line 28a. The matrix outputs Iout0...Ioutn are generated on bit line 16a. For all cells in the column, each output Iout is the sum of the cell currents, which are proportional to the weight W stored in the cell. It should be noted that for this array architecture, the matrix voltage input can be placed on the erase gate line 30a instead of the select gate line 28, as... Figure 29 As shown. Alternatively, the matrix input may be a voltage input located on the source line 14a, instead of a voltage input located on the select gate line 28a or the erase gate line 30a, as shown. Figure 30 As shown. In yet another alternative, the matrix input can be a voltage input placed on bit line 16a, with the current output on source line 14a, as shown. Figure 31 As shown.

[0124] Figure 32 A memory cell 10 with only two gates in a stacked gate configuration is shown. The memory cell includes a source region 14 and a drain region 16, with a channel region 18 between them, as in the memory cell described above. However, in the memory cell configuration of the present invention, a floating gate 20 extends over the entire channel region, and a control gate 22 is disposed above the floating gate. The memory cell is programmed on the source side using hot electron injection. Alternative programming can be performed on the drain side. The memory cell is erased by applying a high positive voltage to the substrate 12, the source region 14, or the drain region 16, where electrons tunnel from the floating gate 20 to the substrate 12. The memory cell is read by applying a read voltage to the control gate and the source region 14. Exemplary non-limiting operating voltages may include those in Table 5 below:

[0125] Table 5

[0126] CG 22a BL 16a SL 14a Substrate 12 Read 1 0-5V 0.1-2V 0-2V 0V Read 2 0.5-2V 0-2V 2-0.1V 0V erase 0V / -12V 0V 0V 20V / 8V programming 5-12V Approximately 10-50uA 4.5V 0V

[0127] "Read 1" is a read mode in which the cell current flows out on the bit line. "Read 2" is a read mode in which the cell current flows out on the source line.

[0128] Figure 33 It shows the use of Figure 32The memory cell array architecture of memory cell 10, wherein all lines except bit line 16a extend in the horizontal / row direction. After each memory cell is programmed with appropriate weight values, the array is used as a bit line summation matrix multiplier. The matrix voltage input is Vin0-Vin3 and is placed on the control gate line 22a. The matrix outputs Iout0...Ioutn are generated on bit line 16a. For all cells in the column, each output Iout is the sum of the cell currents, which are proportional to the weight W stored in the cell. It should be noted that for this array architecture, the matrix input can alternatively be a voltage input placed on the source line 14a instead of a voltage input placed on the control gate line 22a, as shown below. Figure 34 As shown. In yet another alternative, the matrix input can be a voltage input placed on bit line 16a, with the current output on source line 14a, as shown. Figure 35 As shown.

[0129] Figure 36 It shows the use of Figure 32 Another memory cell array architecture of memory cell 10. Figure 36 array architecture and Figure 33 The array architecture is the same, except that the control gate lines 22a extend vertically instead of horizontally (i.e., each control gate line 22a connects all the control gates 22 of the memory cells in a column together). After each memory cell is programmed with the appropriate weight values ​​for that cell, the array is used as a source-line summation matrix multiplier. The matrix voltage inputs are Vin0-Vin3 and are placed on the control gate lines 22a. The matrix outputs Iout0...Ioutn are generated on the source lines 14a. For all cells in a row, each output Iout is the sum of the cell currents, which are proportional to the weights W stored in the cell.

[0130] Figure 37 It shows the use of Figure 32 Another memory cell array architecture of memory cell 10. Figure 37 array architecture and Figure 33The array architecture is the same, except that the source lines 14a extend vertically instead of horizontally (i.e., each source line 14a connects all the source regions 14 of the memory cells in a column together). After each memory cell is programmed with the appropriate weight values ​​for that cell, the array is used as a source-line summation matrix multiplier. The matrix voltage inputs are Vin0-Vin3 and are placed on the control gate line 22a. The matrix outputs Iout0...Ioutn are generated on the source lines 14a. For all cells in the column, each output Iout is the sum of the cell currents, which are proportional to the weights W stored in the cell. Alternatively, the matrix outputs are generated on the bit line 16a instead of the source lines 14a, as shown below. Figure 38 As shown.

[0131] Figure 39 It shows the use of Figure 32 Another memory cell array architecture of memory cell 10. Figure 39 array architecture and Figure 36 The array architecture is the same, except that each column of memory cells has two control gate lines 22a (i.e., a first control gate line 22a1 and a second control gate line 22a2, the first control gate line connecting the control gates 22 of all memory cells in columns of even-numbered rows together, and the second control gate line connecting the control gates 22 of all memory cells in columns of odd-numbered rows together). After each memory cell is programmed with appropriate weight values, the array is used as a source-line summation matrix multiplier. The matrix voltage inputs are Vin0-Vinn and are placed on control gate lines 22a1 and 22a2. The matrix outputs Iout0...Iout2 are generated on source line 14a. For all cells in a row, each output Iout is the sum of the cell currents, which are proportional to the weights W stored in the cell. The two inputs for each column can be differential inputs.

[0132] All the foregoing embodiments can operate the memory cell either analog or digitally and above or below a threshold value. As used herein, digital operation means that the memory cell can be programmed or erased during operation to represent one of two programming states (also called programming levels), such as "0" and "1" represented by programming and erasing states. Analog operation means that the memory cell can be programmed or erased during operation to represent more than just two programming states, such as one of the 16 programming states of a 4-bit equivalent memory cell. Above-threshold operation means that the applied read voltage is sufficient to turn on the memory cell (depending on its programming state), meaning that it will conduct current through the channel region (i.e., read current or cell current) in a manner that has a linear or saturation relationship between the read voltage and the cell current. Below-threshold operation means that the applied read voltage is insufficient to strongly turn on the memory cell, meaning that any current through the memory cell is considered a subthreshold current. During subthreshold operation, the relationship between the read voltage and the cell current is not linear but predictable and can repeat exponentially. Therefore, subthreshold memory cell operations are better suited for extremely low-power analog operations, while the linear characteristics of above-threshold operations are better suited for high-speed digital operations. However, for certain applications or for extending the range of operations, it may be necessary to perform subthreshold digital operations and / or above-threshold analog operations.

[0133] If the memory cells operate in an analog manner, network accuracy can be improved by optimizing the programming state. Specifically, as mentioned above, relative to... Figure 8A and Figure 8B The weight levels stored in the memory cells correspond to different programming states (including an erase state, which is technically the lowest programming state) such that the stored weights are appropriately applied to the incoming input during a read operation, thus ideally realizing the mapping (tuned by programming or erasing) of weight values ​​into the memory cells. Different programming states reflect different numbers of electrons placed on the floating gate and correspond to different possible read currents through the channel region considering the same applied read operation voltage. The more electrons on the floating gate, the higher the programming state (and therefore the lower the channel read current during a read operation). Therefore, for Figure 8AThe 16 programming states are divided into L0 and L1 states. L0 reflects the programming state with the most erased states (where the floating gate contains the fewest electrons). L1 reflects the programming state with the first number of electrons programmed onto the floating gate. L2 reflects the programming state with the second number of electrons programmed onto the floating gate, which is greater than the first number of electrons, and so on. In this embodiment, the programming states are evenly spaced from each other, meaning that for a given read operation with an applied read voltage applied to the memory cell, any two adjacent programming states differ from each other by the same read current difference. In other words, programming the floating gate with additional electrons to move the memory cell from one programming state to the next will always result in the same amount of reduction in read current for a given read operation with an applied read voltage. Alternatively, the same applies if read current is converted to read voltage, but relative to read voltage rather than read current. Therefore, as used herein, the programming state interval or spacing between two programming states refers to the difference between the read current or read voltage derived from the read current under the same read operation condition with an applied read voltage applied to the memory cell. In contrast, in Figure 8B In this context, higher-level programming states are spaced closer together than lower-level programming states (i.e., the difference in read current or read voltage between adjacent programming states varies with the programming state level, where the difference in read current or read voltage between adjacent higher-level programming states is smaller than the difference in read current or read voltage between adjacent lower-level programming states).

[0134] It has been found that improved reliability and performance of neural network systems can be achieved by varying the spacing between programming states based on the usage density of each programming state, rather than solely on the number of programming state levels. Usage density reflects the relative number of memory cells programmed into that programming state (i.e., the frequency of use of that programming state). Compared to other programming states, for any given programming state, a higher usage density (i.e., a higher frequency of use) results in a larger relative number of memory cells programmed into that programming state. For example, Figure 40 An exemplary usage density of a system employing 16 programming states is shown, where the middle programming state is used most frequently, and the lower and higher programming states are used least frequently. In this example, more memory cells are programmed into the L7 programming state compared to any other programming state. To increase system reliability and performance, the most frequently used state is set to be spaced further apart from adjacent programming states than those used less frequently. This is in... Figure 41As shown, programming states L4-L11 are more widely spaced than programming states L0-L3 and L12-15 (i.e., the spacing S1 between adjacent states is greater than the spacing S2 between adjacent states). A larger programming state spacing results in better potential accuracy and reliability because a larger deviation can exist between the target state level and the actual tuned state level during programming, while still accurately reading the programming state of the memory cell. It can tolerate larger tolerances for read inaccuracies while still accurately reading memory cells programmed with larger spacing. Sources of inaccuracy that can be overcome by a larger programming spacing can include tuning inaccuracies, read inaccuracies, thermal or 1 / f noise, cell-to-cell coupling, etc.

[0135] Compared to Figure 41 and Figure 42 As shown, Figure 42 and Figure 43 Examples of different usage densities are shown, where more upper programming states are used less frequently, and more lower programming states are used more frequently. In this example, programming states L2-L9 are spaced further apart than programming states L0-L1 and L10-L15. Considering that lower programming states with larger spacing are used more frequently by multiple memory cells, the system favors more lower programming states with larger spacing.

[0136] Figure 44 and Figure 45 Examples of different usage densities are shown, where it is the least used intermediate programming state. In this example, programming states L0-L3 and L11-L15 are more spaced apart from each other than programming states L4-L10. Considering that the lower and upper programming states with larger spacing are used more frequently by multiple memory cells, the system favors the lower and upper programming states with larger spacing over the intermediate programming states.

[0137] Figure 46 and Figure 47 Examples of different usage densities are shown, where the programming states with higher usage densities (L3, L7, L10, and L11) are not all adjacent to each other. This implementation reflects that usage density can be considered as separate, per-programming-state. In another implementation, where certain programming states have the greatest impact on the accuracy of the neural network, those programming states are spaced further apart.

[0138] It should be noted that although the spacing between adjacent programming states is shown as S1 or S2, more than two spacing values ​​can be used, and / or the spacing can change only between less frequently used states and / or only between more frequently used states. Furthermore, one way to implement spacing variation is that the S1 spacing is logarithmic and the S2 spacing is linear, or vice versa. Alternatively, S1 and S2 can be different logarithmic differences.

[0139] Usage density can be determined, for example, during neural network training. Such training may involve providing the neural network with known inputs to achieve a desired output. The actual output is then compared to the desired output to determine an error, which is used to adjust the stored weights to improve the system output until the desired output is achieved or the system reaches a statistically desirable point. As part of this training, the usage density of individual programming states is determined and used to adjust the programming state spacing as described above. Usage density can be determined and implemented globally, either jointly or more individually, layer by layer down across multiple levels. Other techniques for determining usage density may include creating a predictive model that estimates which weights (and therefore which programming states) are used most frequently, based on application-specific information or historical data.

[0140] The operation type (digital vs. analog) and / or programming state spacing scheme can be the same or different depending on the neural network layer to maximize performance in terms of speed, accuracy, reliability, and power consumption. For example, two different adjacent or non-adjacent neural network layers can use two different operation modes (one digital and the other analog) or the same operation mode. Alternatively, some VMM arrays can operate below a threshold, while others operate above a threshold. Furthermore, two different adjacent or non-adjacent neural network layers operating analogly can utilize different programming state spacing schemes and / or different numbers of programming states (i.e., if a given neural network layer requires fewer programming states, fewer spaced programming states can be used), or the same programming state spacing scheme can be used. As a specific example, in Figure 12 In CB2, the synapses from layer S1 to layer C2 can be manipulated digitally, while the synapses in CB3 (from layer S2 to layer C3) and CB4 (from layer C3 to layer S3) can be manipulated analogically. Furthermore, some neural network layers within CB2 can be used... Figure 41 The programming state spacing scheme in Figure 8 uses 16 levels, while other neural network layers use only 8 levels, as shown in Figure 8. Alternatively, some neural network layers in CB2 can use... Figure 41The programming state spacing scheme (where intermediate states are spaced further apart than other programming states) is used to operate, while other neural network layers use... Figure 45 The programmable state spacing scheme (where intermediate states are spaced closer together than other programmable states) is used for operation.

[0141] Considering the purpose and needs of each neural network layer, performance can be enhanced by using VMM arrays that operate in different ways (digital vs. analog, above or below a threshold, more / fewer analog programming states, etc.). For example, operating memory cells below the threshold (subthreshold) reduces power consumption (e.g., current drops from hundreds of nanoamps to less than one picoamp), thus being better for extremely low-power analog operations. Operating memory cells above the threshold consumes more power (e.g., current drops from hundreds of nanoamps to tens of microamps) but provides characteristics better suited for digital operations, a wider operating range, higher state levels due to the higher current, and high-speed performance. Operating memory cells digitally provides greater flexibility for different machine learning (deep learning) operations such as convolution, pooling, and nonlinear functions, but consumes more power. Operating memory cells analogally is less flexible but consumes less power (e.g., it can be one-tenth or even one-hundredth of the power consumed by operating memory cells digitally). Therefore, the operation of each VMM array (digital / analog, above or below a threshold, programmed state spacing scheme, etc.) can be configured to maximize the performance of the neural network layer, which can be the same or different relative to other VMM arrays.

[0142] Another example of how performance can be further enhanced by using VMM arrays that operate or are configured differently, either alone or in combination with one or more of the different types of VMM array operations described above, is by using an analog volatile VMM array in conjunction with the analog nonvolatile VMMs described above. Such an analog volatile VMM array is formed from volatile memory cells (e.g., DRAM, SRAM, capacitor-based custom memory cells, etc., which lose their stored information when power is turned off, unlike nonvolatile memory cells such as those with floating gates, which retain their stored information even after power is turned off) and can be used as a convolutional layer. For example, filter weights are stored in an analog nonvolatile array as described above. These weights are then transferred to the analog volatile VMM array, where convolution is then performed. One advantage of this is the increased speed using volatile memory cells (which operate faster than nonvolatile memory cells) while retaining the stored weights in the nonvolatile memory cells after power loss. VMM arrays with volatile memory cells are more flexible for different machine learning (deep learning) operations, but also require more frequent refreshes (e.g., typically over millisecond periods), thus consuming more power. VMM arrays with non-volatile memory cells are less flexible for different machine learning (deep learning) operations, but require less frequent refreshes (e.g., typically over periods of months or years), thus consuming less power. Therefore, for any given application, multiple layers of volatile VMM arrays and / or multiple layers of non-volatile VMM arrays can be used, depending on the requirements of each VMM layer and the balance between the aforementioned advantages and disadvantages of volatile and non-volatile VMM arrays.

[0143] All of the above functions can be executed under the control of controller 100, which includes control circuitry connected to the memory array of the aforementioned memory unit 10 for neural network functions. For example... Figure 48 As shown, the controller 100 is preferably located on the same semiconductor chip or substrate 110 as the memory array 120. However, the controller 100 may also be located on a separate semiconductor chip or substrate, and may be a collection of multiple controllers or different control circuits disposed at different locations on or outside the semiconductor chip or substrate 110.

[0144] Some neural network applications use both positive and negative weights. In this case, when programming a memory cell using weights, the controller 100 maps negative weights to some programming states and positive weights to other programming states. Then, during operation, the controller 100 will perform appropriate actions on the output current generated from the memory cell programmed using negative weights.

[0145] It should be understood that the invention is not limited to the embodiments described above and shown herein, but covers any and all variations within the scope of any claim. For example, references to the invention herein are not intended to limit the scope of any claim or claim terminology, but only to one or more features that may be covered by one or more claims. The examples of materials, processes, and values ​​described above are merely exemplary and should not be considered as limiting the claims. A single layer of material may be formed as multiple layers of such or similar materials, and vice versa. While the outputs of each memory cell array are manipulated by filtering and shrinking before being sent to the next neuron layer, they need not be. Finally, for each of the matrix multiplier array embodiments described above, the nominal read voltages for this configuration of memory cells disclosed in the table herein may (but are not necessary) be applied to these lines during operation for any lines that are not being used for input voltage or output current.

[0146] It should be noted that, as used herein, the terms “above” and “on” inclusively include “directly on” (without intermediate material, elements, or spaces) and “indirectly on” (with intermediate material, elements, or spaces). Similarly, the term “adjacent” includes “directly adjacent” (without intermediate material, elements, or spaces) and “indirectly adjacent” (with intermediate material, elements, or spaces), “mounted to” includes “directly mounted to” (without intermediate material, elements, or spaces) and “indirectly mounted to” (with intermediate material, elements, or spaces), and “electrically coupled to” includes “directly electrically coupled to” (without intermediate material or elements electrically connecting the elements together) and “indirectly electrically coupled to” (with intermediate material or elements electrically connecting the elements together). For example, forming an element “above a substrate” can include forming the element directly on the substrate without intermediate material / elements between them, and forming the element indirectly on the substrate with one or more intermediate materials / elements between them.

Claims

1. A neural network device comprising: a first plurality of synapses configured to receive a first plurality of inputs and generate therefrom a first plurality of outputs, wherein the first plurality of synapses comprises: a plurality of first memory cells; a controller configured to: program each of the first memory cells to one of a plurality of first programmed states, and read the first memory cells using a first read operation of an applied voltage to the first memory cells, the plurality of first memory cells configured to generate the first plurality of outputs based on the first plurality of inputs and the plurality of first programmed states; and a first plurality of neurons configured to receive the first plurality of outputs; a second plurality of synapses configured to receive a second plurality of inputs from the first plurality of neurons and generate therefrom a second plurality of outputs, wherein the second plurality of synapses comprises: a plurality of second memory cells; the controller configured to: program each of the second memory cells to one of a plurality of second programmed states, read the second memory cells using a second read operation of a second applied voltage to the second memory cells, the plurality of second memory cells configured to generate the second plurality of outputs based on the second plurality of inputs and the plurality of second programmed states; and a second plurality of neurons configured to receive the second plurality of outputs; wherein at least one of: a total number of the plurality of first programmed states is different than a total number of the plurality of second programmed states, the first memory cells are volatile and the second memory cells are non-volatile, or the first memory cells are non-volatile and the second memory cells are volatile, or the first and second memory cells are non-volatile, the controller is configured to: perform the first read operation above a threshold of the first memory cells and the second read operation below a threshold of the second memory cells, or perform the first read operation below a threshold of the first memory cells and the second read operation above a threshold of the second memory cells, or perform the first and second read operations above a threshold of the first and second memory cells, or perform the first and second read operations below a threshold of the first and second memory cells.

2. The neural network device of claim 1, wherein a total number of the plurality of first programmed states is only 2 and a total number of the plurality of second programmed states is greater than 2, or a total number of the plurality of first programmed states is greater than 2 and a total number of the plurality of second programmed states is only 2.

Images