Method and apparatus for precision data tuning of analog neural memory in artificial neural networks

Abstract

Disclosed are multiple embodiments of an accurate programming algorithm and apparatus for accurately and quickly depositing the correct amount of charge onto the floating gates of non-volatile memory cells within a vector-matrix multiplication (VMM) array in an artificial neural network. Thus, selected cells can be programmed with extreme accuracy to hold one of N different values.

Description

[0001] This application is a divisional application of the patent application filed on May 19, 2021, with international application number PCT / US2021 / 033122 and Chinese national application date of May 19, 2021, with application number 202180093494.0, entitled "Precise Data Tuning Method and Apparatus for Analog Neural Memory in Artificial Neural Networks".

[0002] Priority Statement

[0003] This application claims priority to U.S. Patent Application No. 17 / 185,725, filed February 25, 2021, entitled “Precise Data Tuning Method and Apparatus for Analog Neural Memory in an Artificial Neural Network,” which is a continuation-in-part of U.S. Patent Application No. 16 / 829,757, filed March 25, 2020, entitled “Precise Data Tuning Method and Apparatus for Analog Neuromorphic Memory in an Artificial Neural Network,” which claims priority to U.S. Provisional Patent Application No. 62 / 957,013, filed January 3, 2020, entitled “Precise Data Tuning Method and Apparatus for Analog Neuromorphic Memory in an Artificial Neural Network.” Technical Field

[0004] This invention discloses several embodiments of a method and apparatus for precisely tuning on the floating gate of a non-volatile memory cell within a vector-matrix multiplication (VMM) array in an artificial neural network, accurately and rapidly depositing the correct amount of charge. Background Technology

[0005] Artificial neural networks mimic biological neural networks (the central nervous system of animals, especially the brain) and are used to estimate or approximate functions that can depend on a large number of inputs and are often unknown. Artificial neural networks typically consist of interconnected layers of "neurons" that exchange messages with each other.

[0006] Figure 1An artificial neural network is illustrated, where circles represent the inputs or layers of neurons. Connections (called synapses) are indicated by arrows and have numerical weights that can be adjusted empirically. This allows the artificial neural network to adapt to its inputs and learn. Typically, an artificial neural network consists of layers with multiple inputs. There are usually one or more intermediate layers of neurons, and an output layer of neurons that provide the output of the neural network. Neurons at each level make decisions individually or collectively based on the data received from the synapses.

[0007] One of the major challenges in developing artificial neural networks for high-performance information processing is the lack of sufficient hardware technology. In reality, practical artificial neural networks rely on a large number of synapses to achieve high connectivity between neurons, i.e., very high computational parallelism. In principle, such complexity can be achieved using digital supercomputers or dedicated clusters of graphics processing units. However, compared to biological networks, these methods are generally energy inefficient, in addition to being costly, as biological networks consume far less energy primarily due to their ability to perform low-precision analog computations. CMOS analog circuits have been used in artificial neural networks, but given the large number of neurons and synapses, most CMOS-implemented synapses are excessively large.

[0008] The applicant previously disclosed an artificial (simulated) neural network utilizing one or more non-volatile memory arrays as synapses in U.S. Patent Application No. 15 / 594,439 (published as U.S. Patent Publication 2017 / 0337466), which is incorporated herein by reference. The non-volatile memory array operates as a simulated neuromorphic memory. As used herein, the term "neuromorphic" refers to a circuit that implements a model of a nervous system. The simulated neuromorphic memory includes a first plurality of synapses configured to receive a first plurality of inputs and generate a first plurality of outputs therefrom, and a first plurality of neurons configured to receive the first plurality of outputs. The first plurality of synapses includes a plurality of memory cells, wherein each memory cell includes: spaced-apart source and drain regions formed in a semiconductor substrate, wherein a channel region extends between the source and drain regions; a floating gate disposed over and insulated from a first portion of the channel region; and a non-floating gate disposed over and insulated from a second portion of the channel region. Each memory cell is configured to store weight values ​​corresponding to a plurality of electrons on the floating gate. Multiple memory cells are configured to multiply a first plurality of inputs by stored weight values ​​to generate a first plurality of outputs. An array of memory cells arranged in this manner may be called a vector matrix multiplication (VMM) array.

[0009] Each non-volatile memory cell in the VMM must be erased and programmed to maintain a very specific and precise amount of charge (i.e., the number of electrons) in the floating gate. For example, each floating gate must hold one of N distinct values, where N is the number of different weights that can be indicated by each cell. Examples of N include 16, 32, 64, 128, and 256. One challenge lies in being able to program selected cells with the required precision and granularity for different values ​​of N. For example, if selected cells could include one of 64 distinct values, extremely high precision is required in the programming operation.

[0010] There is a need for improved programming systems and methods suitable for use with VMM arrays in analog neuromorphic memories. Summary of the Invention

[0011] This invention discloses several embodiments of a precise tuning algorithm and apparatus for accurately and rapidly depositing the correct amount of charge onto the floating gates of non-volatile memory cells within a VMM array in an analog neuromorphic memory system. Therefore, selected cells can be programmed with extreme precision to maintain one of N distinct values.

[0012] In one embodiment, the neural network includes a vector-matrix multiplication array of non-volatile memory cells, wherein weight values ​​w are stored as difference pairs w+ and w- in a first and second non-volatile memory cell in the array according to the formula w=(w+)-(w-), wherein w+ and w- include non-zero offset values.

[0013] In another embodiment, the neural network includes a vector-matrix multiplication array of non-volatile memory cells, the array being organized into rows and columns of non-volatile memory cells, wherein weight values ​​w are stored as difference pairs w+ and w- in a first non-volatile memory cell and a second non-volatile memory cell according to the formula w=(w+)-(w-), wherein the storage of w+ and w- values ​​is approximately uniformly distributed in all columns of the array.

[0014] In another embodiment, a method for programming, verifying, and reading zero values ​​in differential pairs of non-volatile memory cells in a vector-matrix multiplication array includes: programming a first cell w+ in the differential pair to a first current value; verifying the first cell by applying a voltage equal to a first voltage plus a bias voltage to a control gate terminal of the first cell; programming a second cell w- in the differential pair to the first current value; reading and verifying the second cell by applying a voltage equal to the first voltage plus a bias voltage to a control gate terminal of the second cell; reading the first cell by applying a voltage equal to the first voltage to a control gate terminal of the first cell; reading the second cell by applying a voltage equal to the first voltage to a control gate terminal of the second cell; and calculating a value w according to the formula w=(w+)-(w-).

[0015] In another embodiment, a method for programming, verifying, and reading zero values ​​in differential pairs of non-volatile memory cells in a vector-matrix multiplication array includes: programming a first cell w+ in the differential pair to a first current value; verifying the first cell by applying a voltage equal to a first voltage plus a bias voltage to a control gate terminal of the first cell; programming a second cell w- in the differential pair to the first current value; reading and verifying the second cell by applying a voltage equal to the first voltage plus a bias voltage to a control gate terminal of the second cell; reading the first cell by applying a voltage equal to the first voltage to a control gate terminal of the first cell; reading the second cell by applying a voltage equal to the first voltage to a control gate terminal of the second cell; and calculating a value w according to the formula w=(w+)-(w-).

[0016] In another embodiment, a neural network includes a vector-matrix multiplication array of non-volatile memory cells, the array being organized into rows and columns of non-volatile memory cells, wherein weight values ​​w are stored as difference pairs w+ and w- according to the formula w=(w+)-(w-), wherein w+ is stored as a difference pair in a first and a second non-volatile memory cell in the array, and w- is stored as a difference pair in a third and a fourth non-volatile memory cell in the array, wherein the storage of w+ and w- values ​​is offset by a bias value.

[0017] In another embodiment, a neural network includes a vector-matrix multiplication array of nonvolatile memory cells, the array being organized into rows and columns of nonvolatile memory cells, wherein weight values ​​w are stored as difference pairs w+ and w- in a first nonvolatile memory cell and a second nonvolatile memory cell according to the formula w=(w+)-(w-), wherein the value of w+ is selected from a first range of non-zero values ​​and the value of w- is selected from a second range of non-zero values, wherein the first range and the second range do not overlap.

[0018] In another embodiment, a method for reading non-volatile memory cells in a vector-matrix multiplication array includes reading weights stored in selected cells in the array, including: applying a zero-voltage bias to a control gate terminal of the selected cell, and sensing a neuron output current including a current output from the selected cell.

[0019] In another embodiment, a method of operating a non-volatile memory cell in a vector multiplication matrix multiplication array includes: reading the non-volatile memory cell by applying a first bias voltage to the control gate of the non-volatile memory cell; and applying a second bias voltage to the control gate of the non-volatile memory cell during one or more of a standby operation, a deep power-down operation, or a test operation. Attached Figure Description

[0020] Figure 1 A schematic diagram illustrating an existing artificial neural network.

[0021] Figure 2 This illustrates a split-gate flash memory cell from the prior art.

[0022] Figure 3 This illustrates another prior art split-gate flash memory cell.

[0023] Figure 4 This illustrates another prior art split-gate flash memory cell.

[0024] Figure 5 This illustrates another prior art split-gate flash memory cell.

[0025] Figure 6 This illustrates another prior art split-gate flash memory cell.

[0026] Figure 7 This illustrates a stacked gate flash memory cell from the prior art.

[0027] Figure 8 This is a schematic diagram illustrating different layers of an exemplary artificial neural network using one or more VMM arrays.

[0028] Figure 9 A block diagram of a VMM system, including a VMM array and other circuitry, is shown.

[0029] Figure 10 A block diagram illustrating an exemplary artificial neural network using one or more VMM systems is provided.

[0030] Figure 11 Another implementation of the VMM array is shown.

[0031] Figure 12 Another implementation of the VMM array is shown.

[0032] Figure 13 Another implementation of the VMM array is shown.

[0033] Figure 14 Another implementation of the VMM array is shown.

[0034] Figure 15 Another implementation of the VMM array is shown.

[0035] Figure 16 Another implementation of the VMM array is shown.

[0036] Figure 17 Another implementation of the VMM array is shown.

[0037] Figure 18 Another implementation of the VMM array is shown.

[0038] Figure 19 Another implementation of the VMM array is shown.

[0039] Figure 20 Another implementation of the VMM array is shown.

[0040] Figure 21 Another implementation of the VMM array is shown.

[0041] Figure 22 Another implementation of the VMM array is shown.

[0042] Figure 23 Another implementation of the VMM array is shown.

[0043] Figure 24 Another implementation of the VMM array is shown.

[0044] Figure 25 This illustrates a prior art long short-term memory system.

[0045] Figure 26 An exemplary cell used in a long short-term memory system is shown.

[0046] Figure 27 Show Figure 26 An implementation of an exemplary unit.

[0047] Figure 28 Show Figure 26 Another implementation of the exemplary unit.

[0048] Figure 29 This illustrates a prior art gate-controlled recursive cell system.

[0049] Figure 30 An exemplary cell used in a gate-controlled recursive cell system is shown.

[0050] Figure 31 Show Figure 30 An implementation of an exemplary unit.

[0051] Figure 32 Show Figure 30 Another implementation of the exemplary unit.

[0052] Figure 33 The VMM system is shown.

[0053] Figure 34 The tuning and correction method is shown.

[0054] Figure 35A The tuning and correction method is shown.

[0055] Figure 35B The sector tuning correction method is shown.

[0056] Figure 36A This shows the effect of temperature on the values ​​stored in the cells.

[0057] Figure 36B This illustrates the problems caused by data drift during the operation of a VMM system.

[0058] Figure 36C The block used to compensate for data drift is shown.

[0059] Figure 36D The data drift monitor is shown.

[0060] Figure 37 The bit line compensation circuit is shown.

[0061] Figure 38 Another line compensation circuit is shown.

[0062] Figure 39 Another line compensation circuit is shown.

[0063] Figure 40 Another line compensation circuit is shown.

[0064] Figure 41 Another line compensation circuit is shown.

[0065] Figure 42 Another line compensation circuit is shown.

[0066] Figure 43 The neuron circuit is shown.

[0067] Figure 44Another neuron circuit is shown.

[0068] Figure 45 Another neuron circuit is shown.

[0069] Figure 46 Another neuron circuit is shown.

[0070] Figure 47 Another neuron circuit is shown.

[0071] Figure 48 Another neuron circuit is shown.

[0072] Figure 49A The block diagram of the output circuit is shown.

[0073] Figure 49B A block diagram of another output circuit is shown.

[0074] Figure 49C A block diagram of another output circuit is shown. Detailed Implementation

[0075] The artificial neural network of this invention utilizes a combination of CMOS technology and non-volatile memory arrays.

[0076] Non-volatile memory cells

[0077] Digital nonvolatile memory is well known. For example, U.S. Patent 5,029,130 ​​(“130 Patent”), which is incorporated herein by reference, discloses an array of split-gate nonvolatile memory cells, which is a type of flash memory cell. Such memory cells 210 in Figure 2 As shown in the figure. Each memory cell 210 includes a source region 14 and a drain region 16 formed in a semiconductor substrate 12, with a channel region 18 therebetween. A floating gate 20 is formed over and insulated from (and controls the conductivity of) a first portion of the channel region 18, and is formed over a portion of the source region 14. A word line terminal 22 (which is typically coupled to a word line) has a first portion disposed over and insulated from (and controlling the conductivity of) a second portion of the channel region 18, and a second portion extending upward and located over the floating gate 20. The floating gate 20 and the word line terminal 22 are insulated from the substrate 12 by a gate oxide. A bit line terminal 24 is coupled to the drain region 16.

[0078] The memory cell 210 is erased by applying a high positive voltage to the word line terminal 22 (where electrons are removed from the floating gate), which causes electrons on the floating gate 20 to tunnel from the floating gate 20 to the word line terminal 22 through the intermediate insulator via the Fowler-Nordheim tunnel.

[0079] Memory cell 210 is programmed by applying a positive voltage to word line terminal 22 and a positive voltage to source region 14 (where electrons are placed on the floating gate). Electron current flows from source region 14 (source line terminal) to drain region 16. When electrons reach the gap between word line terminal 22 and floating gate 20, they accelerate and become hot. Due to electrostatic attraction from floating gate 20, some heated electrons are injected into floating gate 20 through gate oxide.

[0080] Memory cell 210 is read by applying a positive read voltage to the drain region 16 and word line terminal 22 (which connects the portion of channel region 18 below the word line terminal). If the floating gate 20 is positively charged (i.e., electrons are erased), the portion of channel region 18 below the floating gate 20 is also turned on, and current flows through channel region 18, which is sensed as an erased state or a "1" state. If the floating gate 20 is negatively charged (i.e., programmed electronically), the portion of channel region below the floating gate 20 is mostly or completely turned off, and current does not flow (or very little current) through channel region 18, which is sensed as a programmed state or a "0" state.

[0081] Table 1 shows the typical voltage ranges that can be applied to the terminals of memory cell 110 for performing read, erase, and program operations: Table 1: Figure 2 Operation of flash memory cell 210

[0082] "Read 1" is the read mode where the cell current is output on the bit line. "Read 2" is the read mode where the cell current is output on the source line terminal.

[0083] Figure 3 The memory cell 310 is shown, which is related to Figure 2 The memory cell 210 is similar, but with the addition of a control gate (CG) terminal 28. The control gate terminal 28 is biased at a high voltage (e.g., 10V) during programming, at a low or negative voltage (e.g., 0V / -8V) during erasure, and at a low or medium voltage (e.g., 0V / 2.5V) during reading. Other terminals are similar. Figure 2 That kind of bias.

[0084] Figure 4A quad-gate memory cell 410 is shown, comprising a source region 14, a drain region 16, a floating gate 20 over a first portion of a channel region 18, a select gate 22 over a second portion of the channel region 18 (typically coupled to the word line WL), a control gate 28 over the floating gate 20, and an erase gate 30 over the source region 14. This configuration is described in U.S. Patent 6,747,310, which is incorporated herein by reference for all purposes. Here, all gates except the floating gate 20 are non-floating gates, meaning they are electrically connected to or can be electrically connected to a voltage source. Programming is performed by heated electrons from the channel region 18 that inject themselves into the floating gate 20. Erasing is performed by electrons tunneling from the floating gate 20 to the erase gate 30.

[0085] Table 2 shows the typical voltage ranges that can be applied to the terminals of memory cell 410 for performing read, erase, and program operations: Table 2: Figure 4 Operation of flash memory cell 410

[0086] "Read 1" is the read mode where the cell current is output on the bit line. "Read 2" is the read mode where the cell current is output on the source line terminal.

[0087] Figure 5 Memory cell 510 is shown, except that it does not have the erase gate EG terminal. Memory cell 510 is similar to... Figure 4 The memory cell 410 is similar. Erasure is performed by biasing the substrate 18 to a high voltage and the control gate CG terminal 28 to a low voltage or a negative voltage. Alternatively, erasure is performed by biasing the word line terminal 22 to a positive voltage and the control gate terminal 28 to a negative voltage. Programming and reading are similar. Figure 4 As it is.

[0088] Figure 6 A tri-gate memory cell 610 is shown, which is another type of flash memory cell. Memory cell 610 and... Figure 4 The memory cell 410 is identical to the memory cell 610, except that the memory cell 610 does not have a separate control gate terminal. Except that no control gate bias is applied, the erase operation (erasing via the erase gate terminal) and read operation are similar. Figure 4 The programming operation is performed without a control gate bias, and as a result, a higher voltage must be applied to the source line terminals during the programming operation to compensate for the lack of a control gate bias.

[0089] Table 3 shows the typical voltage ranges that can be applied to the terminals of memory cell 610 for performing read, erase, and program operations: Table 3: Figure 6 Operation of flash memory cell 610

[0090] "Read 1" is the read mode where the cell current is output on the bit line. "Read 2" is the read mode where the cell current is output on the source line terminal.

[0091] Figure 7 The stacked gate memory cell 710 is shown, which is another type of flash memory cell. Memory cell 710 and... Figure 2 The memory cell 210 is similar, except that the floating gate 20 extends over the entire channel region 18, and the control gate terminal 22 (which will be coupled to the word line here) extends over the floating gate 20, separated by an insulating layer (not shown). Erase, program, and read operations operate in a manner similar to those previously described for memory cell 210.

[0092] Table 4 shows the typical voltage ranges that can be applied to the terminals of memory cell 710 and substrate 12 to perform read, erase, and program operations: Table 4: Figure 7 Operation of flash memory cell 710

[0093] "Read 1" is a read mode in which the cell current is output on the bit line. "Read 2" is a read mode in which the cell current is output at the source line terminal. Optionally, in an array comprising rows and columns of memory cells 210, 310, 410, 510, 610, or 710, the source line can be coupled to a row of memory cells or two adjacent rows of memory cells. That is, the source line terminal can be shared by memory cells in adjacent rows.

[0094] To utilize memory arrays comprising one of the aforementioned types of non-volatile memory cells in artificial neural networks, two modifications were made. First, the circuitry was configured such that each memory cell could be individually programmed, erased, and read without adversely affecting the memory state of other memory cells in the array, as explained further below. Second, continuous (simulated) programming of the memory cells was provided.

[0095] Specifically, the memory state (i.e., the charge on the floating gate) of each memory cell in the array can be continuously changed from a fully erased state to a fully programmed state independently with minimal interference to other memory cells. In another embodiment, the memory state (i.e., the charge on the floating gate) of each memory cell in the array can be continuously changed from a fully programmed state to a fully erased state and vice versa, independently with minimal interference to other memory cells. This means that the cell storage device is analog, or at least can store one discrete value from many discrete values ​​(such as 16 or 64 different values), which allows for very precise and individual tuning of all cells in the memory array, and makes the memory array ideal for storage and fine tuning of synaptic weights in neural networks.

[0096] The methods and apparatus described herein can be applied to other non-volatile memory technologies, such as, but not limited to, SONOS (silicon-oxide-nitride-oxide-silicon, with charge trapped in the nitride), MONOS (metal-oxide-nitride-oxide-silicon, with metal charge trapped in the nitride), ReRAM (resistive RAM), PCM (phase-change memory), MRAM (magnetic RAM), FeRAM (ferroelectric RAM), OTP (two- or multi-layer programmable-once), and CeRAM (associated electron RAM). The methods and apparatus described herein can also be applied to volatile memory technologies for neural networks, such as, but not limited to, SRAM, DRAM, and / or volatile synaptic cells.

[0097] Neural networks using non-volatile memory cell arrays

[0098] Figure 8 This conceptually illustrates a non-limiting example of a neural network using a non-volatile memory array in this embodiment. This example uses a non-volatile memory array neural network for a facial recognition application, but any other suitable application can also be implemented using a neural network based on a non-volatile memory array.

[0099] In this example, S0 is the input layer, which is a 32x32 pixel RGB image with 5-bit precision (i.e., three 32x32 pixel arrays, one for each color R, G, and B, with 5-bit precision per pixel). The synapse CB1 from the input layer S0 to layer C1 applies different sets of weights in some cases and shared weights in others, and scans the input image with a 3x3 pixel overlapping filter (kernel), shifting the filter by one pixel (or more than one pixel depending on the model). Specifically, the values ​​of nine pixels in a 3x3 portion of the image (i.e., called the filter or kernel) are provided to synapse CB1, where these nine input values ​​are multiplied by appropriate weights, and after summing the output of this multiplication, a single output value is determined by the first synapse of CB1 and provided for generating the pixels of one of the feature maps in layer C1. The 3x3 filter is then shifted one pixel to the right within the input layer S0 (i.e., adding a column of three pixels to the right and releasing a column of three pixels to the left), thereby providing the nine pixel values ​​from this newly positioned filter to synapse CB1, where they are multiplied by the same weights and a second single output value is determined by the associated synapse. This process continues until the 3x3 filter scans all three colors and all bits (precision values) across the entire 32x32 pixel image of the input layer S0. This process is then repeated using different sets of weights to generate different feature maps for C1 until all feature maps for layer C1 are computed.

[0100] At layer C1, in this example, there are 16 feature maps, each with 30x30 pixels. Each pixel is a new feature pixel extracted from the product of the input and the kernel, so each feature map is a two-dimensional array. Therefore, in this example, layer C1 consists of a 16-layer two-dimensional array (remember that the layers and arrays referred to in this article are logical relationships, not necessarily physical relationships; that is, the array does not have to be oriented as a physical two-dimensional array). Each of the 16 feature maps in layer C1 is generated by one set of sixteen different groups of synaptic weights applied to the filter scan. The C1 feature maps may all relate to different aspects of the same image features, such as boundary recognition. For example, the first map (generated using a first weight recombination, shared for all scans used to generate the first map) may recognize circular edges, the second map (generated using a second weight recombination different from the first weight recombination) may recognize rectangular edges, or the aspect ratio of certain features, and so on.

[0101] Before transitioning from layer C1 to layer S1, activation function P1 (pooling) is applied, which pools the values ​​from consecutive non-overlapping 2x2 regions in each feature map. The purpose of the pooling function is to average the neighboring locations (or, alternatively, use a max function) to, for example, reduce the dependence on edge locations and reduce the data size before moving to the next stage. At layer S1, there are 16 15x15 feature maps (i.e., sixteen different arrays, each 15x15 pixels). The synapse CB2 from layer S1 to layer C2 scans the map in S1 using a 4x4 filter, where the filter is shifted by 1 pixel. At layer C2, there are 22 12x12 feature maps. Before transitioning from layer C2 to layer S2, activation function P2 (pooling) is applied, which pools the values ​​from consecutive non-overlapping 2x2 regions in each feature map. At layer S2, there are 22 6x6 feature maps. An activation function (pooling) is applied to the synapse CB3 from layer S2 to layer C3, where each neuron in layer C3 is connected to each mapping in layer S2 via a corresponding synapse in CB3. There are 64 neurons in layer C3. The synapse CB4 from layer C3 to the output layer S3 completely connects C3 to S3, meaning each neuron in layer C3 is connected to every neuron in layer S3. The output at S3 comprises 10 neurons, with the highest-output neuron determining the class. For example, this output could indicate the recognition or classification of the content of the original image.

[0102] Synapses for each layer are implemented using an array or a portion of an array of non-volatile memory cells.

[0103] Figure 9 This is a block diagram of a system that could be used for this purpose. The VMM system 32 includes non-volatile memory cells and serves as synapses between layers (such as...). Figure 6 (CB1, CB2, CB3, and CB4 in the original text). Specifically, the VMM system 32 includes a VMM array 33 (comprising non-volatile memory cells arranged in rows and columns), an erase gate and word line gate decoder 34, a control gate decoder 35, a bit line decoder 36, and a source line decoder 37, which decode the corresponding inputs to the non-volatile memory cell array 33. The inputs to the VMM array 33 may come from the erase gate and word line gate decoder 34 or from the control gate decoder 35. In this example, the source line decoder 37 also decodes the outputs of the VMM array 33. Alternatively, the bit line decoder 36 may decode the outputs of the VMM array 33.

[0104] The VMM array 33 serves two purposes. First, it stores weights that will be used by the VMM system 32. Second, the VMM array 33 efficiently multiplies the inputs with the weights stored in the VMM array 33 and adds them together at each output line (source line or bit line) to produce an output that will serve as the input to the next layer or the final layer. By performing multiplication and addition functions, the VMM array 33 eliminates the need for separate multiplication and addition logic circuits and is also highly efficient due to its in-situ memory computation.

[0105] The output of VMM array 33 is provided to a differential summer (such as a summing operational amplifier or a summing current mirror) 38, which sums the output of VMM array 33 to create a single value for the convolution. Differential summer 38 is arranged to perform the summation of both the positive and negative weight inputs to output a single value.

[0106] The output values ​​of the difference summer 38 are then summed and provided to the activation function circuit 39, which corrects the output. The activation function circuit 39 can provide a sigmoid, tanh, ReLU function, or any other nonlinear function. The corrected output value of the activation function circuit 39 becomes the next layer's (e.g., ...) Figure 8 The elements of the feature map of layer C1 are then applied to the next synapse to produce the next feature map layer or the final layer. Thus, in this example, the VMM array 33 constitutes multiple synapses (which receive their input from existing neuron layers or from input layers such as an image database), and the summer 38 and activation function circuit 39 constitute multiple neurons.

[0107] Figure 9 The inputs to the VMM system 32 (WLx, EGx, CGx, and optional BLx and SLx) can be analog levels, binary levels, digital pulses (in which case a pulse-to-analog converter (PAC) may be required to convert the pulses to the appropriate input analog level), or digital bits (in which case a DAC is provided to convert the digital bits to the appropriate input analog level); the outputs can be analog levels, binary levels, digital pulses, or digital bits (in which case an output ADC is provided to convert the output analog level to digital bits).

[0108] Figure 10 A block diagram illustrating the use of a multi-layer VMM system 32 (here labeled VMM systems 32a, 32b, 32c, 32d, and 32e) is provided. Figure 10As shown, the input (denoted as Inputx) is converted from digital to analog by a digital-to-analog converter 31 and provided to the input VMM system 32a. The converted analog input can be voltage or current. The first-level input D / A conversion can be accomplished by using a function or LUT (lookup table) of appropriate analog level to map Inputx to the input VMM system 32a. Input conversion can also be accomplished by an analog-to-analog (A / A) converter to convert an external analog input to a mapped analog input to the input VMM system 32a. Input conversion can also be accomplished by a digital-to-digital pulse (D / P) converter to convert an external digital input to one or more digital pulses mapped to the input VMM system 32a.

[0109] The output generated by the input VMM system 32a is provided as input to the next VMM system (hidden level 1) 32b, which in turn generates the output provided as input to the next VMM system (hidden level 2) 32c, and so on. The layers of the VMM system 32 serve as different layers of synapses and neurons in a convolutional neural network (CNN). Each VMM system 32a, 32b, 32c, 32d, and 32e can be an independent physical system comprising a corresponding non-volatile memory array, or multiple VMM systems can utilize different portions of the same physical non-volatile memory array, or multiple VMM systems can utilize overlapping portions of the same physical non-volatile memory array. Each VMM system 32a, 32b, 32c, 32d, and 32e can also be time-division multiplexed for different portions of its array or neurons. Figure 10 The example shown contains five layers (32a, 32b, 32c, 32d, 32e): one input layer (32a), two hidden layers (32b, 32c), and two fully connected layers (32d, 32e). Those skilled in the art will recognize that this is merely exemplary, and conversely, a system may include more than two hidden layers and more than two fully connected layers.

[0110] VMM array

[0111] Figure 11 The neuronal VMM array 1100 is shown, which is particularly suitable for Figure 3 The memory cell 310 shown serves as a synapse and component for neurons between the input layer and the next layer. The VMM array 1100 includes a memory array 1101 of non-volatile memory cells and a reference array 1102 of non-volatile reference memory cells (at the top of the array). Alternatively, another reference array may be placed at the bottom.

[0112] In VMM array 1100, control gate lines (such as control gate line 1103) extend vertically (therefore reference array 1102 is orthogonal to control gate line 1103 in the row direction), and erase gate lines (such as erase gate line 1104) extend horizontally. Here, the inputs of VMM array 1100 are located on control gate lines (CG0, CG1, CG2, CG3), and the outputs of VMM array 1100 appear on source lines (SL0, SL1). In one embodiment, only even rows are used, and in another embodiment, only odd rows are used. The currents placed on each source line (SL0, SL1, respectively) perform a summation function of all currents from the memory cells connected to that particular source line.

[0113] As described herein with respect to neural networks, the non-volatile memory cells of the VMM array 1100 (i.e., the flash memory of the VMM array 1100) are preferably configured to operate in the subthreshold region.

[0114] Biasing the non-volatile reference memory cell and non-volatile memory cell described herein during weak inversion: Ids = Io e (Vg- Vth) / nVt = w Io e (Vg) / nVt , Where w = e (- Vth) / nVt Where Ids is the drain-to-source current; Vg is the gate voltage on the memory cell; Vth is the threshold voltage of the memory cell; and Vt is the thermal voltage = k T / q, where k is Boltzmann's constant, T is the temperature in Kelvin, and q is the electron charge; n is the slope factor = 1 + (Cdep / Cox), where Cdep = the capacitance of the depletion layer, and Cox is the capacitance of the gate oxide layer; Io is the memory cell current at the gate voltage equal to the threshold voltage, and Io is related to (Wt / L). u Cox (n-1) Vt 2 Proportional, where u is the carrier mobility, and Wt and L are the width and length of the memory cell, respectively.

[0115] For I-to-V logarithmic converters that use memory cells (such as reference memory cells or peripheral memory cells) or transistors to convert input current Ids to input voltage Vg: Vg= n Vt log [Ids / wp Io] Here, wp refers to the w in the reference memory cell or the peripheral memory cell.

[0116] For I-to-V logarithmic converters that use memory cells (such as reference memory cells or peripheral memory cells) or transistors to convert input current Ids to input voltage Vg: Vg= n Vt log [Ids / wp Io] Here, wp refers to the w in the reference memory cell or the peripheral memory cell.

[0117] For a memory array used as a VMM array for vector matrix multipliers, the output current is: Iout = wa Io e (Vg) / nVt ,Right now Iout = (wa / wp) Iin = W Iin W = e (Vthp - Vtha) / nVt Iin = wp Io e (Vg) / nVt

[0118] Here, wa = w for each memory cell in the memory array.

[0119] Word lines or control gates can be used as inputs to memory cells that accept input voltages.

[0120] Alternatively, the non-volatile memory cells of the VMM array described herein can be configured to operate in a linear region: Ids = β (Vgs-Vth) Vds;β = u Cox Wt / L, Wα(Vgs-Vth), This means that the weight W in the linear region is proportional to (Vgs-Vth). Word lines, control gates, bit lines, or source lines can be used as inputs to memory cells operating in a linear region. Bit lines or source lines can be used as outputs to memory cells.

[0121] For an I-to-V linear converter, a memory cell (such as a reference memory cell or a peripheral memory cell) or a transistor or resistor operating in the linear region can be used to linearly convert the input / output current into the input / output voltage.

[0122] Alternatively, the memory cells of the VMM array described herein can be configured to operate in a saturation region: Ids = ½ β (Vgs-Vth) 2 ;β=u Cox Wt / L Wα(Vgs-Vth) 2 This means that the weight W is related to (Vgs-Vth). 2 proportional Word lines, control gates, or erase gates can be used as inputs to memory cells operating in saturation regions. Bit lines or source lines can be used as outputs of output neurons.

[0123] Alternatively, the memory cells of the VMM array described herein can be used in all regions or combinations thereof (subthreshold, linear, or saturated regions).

[0124] U.S. Patent Application No. 15 / 826,345 describes Figure 9 Other embodiments of the VMM array 33 are described herein, and this application is incorporated herein by reference. As described herein, source lines or bit lines can be used as neuron outputs (current summation outputs).

[0125] Figure 12 The neuronal VMM array 1200 is shown, which is particularly suitable for Figure 2 The memory cell 210 shown serves as a synapse between the input layer and the next layer. The VMM array 1200 includes a memory array 1203 of non-volatile memory cells, a reference array 1201 of first non-volatile reference memory cells, and a reference array 1202 of second non-volatile reference memory cells. The reference arrays 1201 and 1202, arranged along the column direction of the array, are used to convert current inputs flowing into terminals BLR0, BLR1, BLR2, and BLR3 into voltage inputs WL0, WL1, WL2, and WL3. In practice, the first and second non-volatile reference memory cells are diode-connected via a multiplexer 1214 (partially shown) through which current inputs flow. The reference cells are tuned (e.g., programmed) to a target reference level. The target reference level is provided by a reference microarray matrix (not shown).

[0126] Memory array 1203 serves two purposes. First, it stores the weights used by VMM array 1200 on their respective memory cells. Second, memory array 1203 efficiently multiplies the inputs (i.e., the current inputs provided in terminals BLR0, BLR1, BLR2, and BLR3, which reference arrays 1201 and 1202 convert into input voltages to provide to word lines WL0, WL1, WL2, and WL3) by the weights stored in memory array 1203, and then sums all the results (memory cell currents) to produce an output on the corresponding bit lines (BL0-BLN), which will be the input to the next layer or the final layer. By performing multiplication and addition functions, memory array 1203 eliminates the need for separate multiplication and addition logic circuits and is also highly efficient. Here, voltage inputs are provided on word lines (WL0, WL1, WL2, and WL3), and the output appears on the corresponding bit lines (BL0-BLN) during read (inference) operations. The current placed on each bit line in the bit lines BL0-BLN performs a summation function of the currents from all non-volatile memory cells connected to that particular bit line.

[0127] Table 5 shows the operating voltages used for the VMM array 1200. The columns in the table indicate the voltage applied to the word line for the selected cell, the word line for the unselected cell, the bit line for the selected cell, the bit line for the unselected cell, the source line for the selected cell, and the source line for the unselected cell, where FLT indicates floating, i.e., no voltage applied. The rows indicate read, erase, and program operations.

[0128] Table 5: Figure 12 Operation of VMM array 1200

[0129] Figure 13 The neuronal VMM array 1300 is shown, which is particularly suitable for Figure 2 The memory cell 210 shown serves as a synapse and component for neurons between the input layer and the next layer. The VMM array 1300 includes a memory array 1303 of non-volatile memory cells, a reference array 1301 of first non-volatile reference memory cells, and a reference array 1302 of second non-volatile reference memory cells. Reference arrays 1301 and 1302 extend in the row direction of the VMM array 1300. The VMM array is similar to the VMM 1000, except that in the VMM array 1300, word lines extend in the vertical direction. Here, inputs are set on word lines (WLA0, WLB0, WLA1, WLB2, WLA2, WLB2, WLA3, WLB3), and outputs appear on source lines (SL0, SL1) during read operations. The current placed on each source line performs a summation function of all currents from the memory cells connected to that particular source line.

[0130] Table 6 shows the operating voltages used for the VMM array 1300. The columns in the table indicate the voltage applied to the word line for the selected cell, the word line for the unselected cell, the bit line for the selected cell, the bit line for the unselected cell, the source line for the selected cell, and the source line for the unselected cell. The rows indicate read, erase, and program operations.

[0131] Table 6: Figure 13 Operation of VMM array 1300

[0132] Figure 14 The neuronal VMM array 1400 is shown, which is particularly suitable for Figure 3 The memory cell 310 shown serves as a synapse and component for neurons between the input layer and the next layer. The VMM array 1400 includes a memory array 1403 of non-volatile memory cells, a reference array 1401 of first non-volatile reference memory cells, and a reference array 1402 of second non-volatile reference memory cells. Reference arrays 1401 and 1402 are used to convert current inputs flowing into terminals BLR0, BLR1, BLR2, and BLR3 into voltage inputs CG0, CG1, CG2, and CG3. In practice, the first and second non-volatile reference memory cells are diode-connected via a multiplexer 1412 (partially shown), through which current inputs flow via BLR0, BLR1, BLR2, and BLR3. Each multiplexer 1412 includes a corresponding multiplexer 1405 and a cascode transistor 1404 to ensure that the voltage on the bit lines (such as BLR0) of each of the first and second nonvolatile reference memory cells remains constant during read operations. The reference cells are tuned to a target reference level.

[0133] Memory array 1403 serves two purposes. First, it stores weights that will be used by VMM array 1400. Second, memory array 1403 efficiently multiplies the inputs (current inputs provided to terminals BLR0, BLR1, BLR2, and BLR3, which reference arrays 1401 and 1402 convert into input voltages to be provided to the control gates (CG0, CG1, CG2, and CG3)) by the weights stored in the memory array, and then sums all the results (cell currents) to produce an output that appears on BL0-BLN and will be the input to the next layer or the final layer. By performing multiplication and addition functions, the memory array eliminates the need for separate multiplication and addition logic circuits and is also highly efficient. Here, the inputs are provided on the control gate lines (CG0, CG1, CG2, and CG3), and the outputs appear on the bit lines (BL0–BLN) during read operations. The currents placed on each bit line perform a summation function of all the currents from the memory cells connected to that particular bit line.

[0134] The VMM array 1400 implements unidirectional tuning for the non-volatile memory cells in the memory array 1403. That is, each non-volatile memory cell is erased and then partially programmed until the desired charge is reached on the floating gate. This can be performed, for example, using the precise programming techniques described below. If too much charge is placed on the floating gate (causing an incorrect value to be stored in the cell), the cell must be erased, and the sequence of partial programming operations must restart. As shown, two rows sharing the same erase gate (such as EG0 or EG1) need to be erased together (this is called page erasure), and thereafter, each cell is partially programmed until the desired charge is reached on the floating gate.

[0135] Table 7 shows the operating voltages used for the VMM array 1400. The columns in the table indicate the voltage applied to the word lines for the selected cell, the word lines for the unselected cell, the bit lines for the selected cell, the bit lines for the unselected cell, the control gate for the selected cell, the control gate for the unselected cell in the same sector as the selected cell, the control gate for the unselected cell in a different sector from the selected cell, the erase gate for the selected cell, the erase gate for the unselected cell, the source line for the selected cell, and the source line for the unselected cell. The rows indicate read, erase, and program operations.

[0136] Table 7: Figure 14 Operation of VMM array 1400

[0137] Figure 15 The diagram shows a neuronal VMM array 1500, which is particularly suitable for... Figure 3The memory cell 310 shown serves as a synapse and component for neurons between the input layer and the next layer. The VMM array 1500 includes a memory array 1503 of non-volatile memory cells, a reference array 1501 of first non-volatile reference memory cells, and a reference array 1502 of second non-volatile reference memory cells. EG lines EGR0, EG0, EG1, and EGR1 extend vertically, while CG lines CG0, CG1, CG2, and CG3 and SL lines WL0, WL1, WL2, and WL3 extend horizontally. The VMM array 1500 is similar to the VMM array 1400, except that the VMM array 1500 implements bidirectional tuning, where each individual cell can be completely erased, partially programmed, and partially erased as needed to achieve the desired amount of charge on the floating gate due to the use of individual EG lines. As shown in the figure, reference arrays 1501 and 1502 convert the input currents in terminals BLR0, BLR1, BLR2, and BLR3 into control gate voltages CG0, CG1, CG2, and CG3 to be applied to memory cells in the row direction (through the operation of reference cells connected via diodes of multiplexer 1514). The current outputs (neurons) are in bit lines BL0-BLN, where each bit line sums all currents from non-volatile memory cells connected to that particular bit line.

[0138] Table 8 shows the operating voltages used for the VMM array 1500. The columns in the table indicate the voltage applied to the word lines for the selected cell, the word lines for the unselected cell, the bit lines for the selected cell, the bit lines for the unselected cell, the control gate for the selected cell, the control gate for the unselected cell in the same sector as the selected cell, the control gate for the unselected cell in a different sector from the selected cell, the erase gate for the selected cell, the erase gate for the unselected cell, the source line for the selected cell, and the source line for the unselected cell. The rows indicate read, erase, and program operations.

[0139] Table 8: Figure 15 Operation of VMM array 1500

[0140] Figure 16 The neuronal VMM array 1600 is shown, which is particularly suitable for Figure 2 The memory unit 210 shown serves as a synapse and component for neurons between the input layer and the next layer. In the VMM array 1600, inputs INPUT0…, INPUT… N On bit lines BL0, …BL respectively N The data is received and outputs OUTPUT1, OUTPUT2, OUTPUT3 and OUTPUT4 are generated on source lines SL0, SL1, SL2 and SL3 respectively.

[0141] Figure 17 The neuronal VMM array 1700 is shown, which is particularly suitable for Figure 2 The memory unit 210 shown serves as a synapse and component for neurons between the input layer and the next layer. In this example, the inputs are INPUT0 and INPUT... 1、 INPUT2 and INPUT3 are received on source lines SL0, SL1, SL2, and SL3 respectively, and OUTPUT0, ..., OUTPUT are output. N In the bit lines BL0, ..., BL N Generate above.

[0142] Figure 18 The neuronal VMM array 1800 is shown, which is particularly suitable for Figure 2 The memory unit 210 shown serves as a synapse and component for neurons between the input layer and the next layer. In this example, the inputs are INPUT0, ..., INPUT0. M On the word lines WL0, ..., WL respectively M The data is received and outputs OUTPUT0, ..., OUTPUT. N In the bit lines BL0, ..., BL N Generate above.

[0143] Figure 19 The neuronal VMM array 1900 is shown, which is particularly suitable for Figure 3 The memory unit 310 shown serves as a synapse and component for neurons between the input layer and the next layer. In this example, the inputs are INPUT0, ..., INPUT0. M On the word lines WL0, ..., WL respectively M The data is received and outputs OUTPUT0, ..., OUTPUT. N In the bit lines BL0, ..., BL N Generate above.

[0144] Figure 20 The neuronal VMM array 2000 is shown, which is particularly suitable for... Figure 4 The memory unit 410 shown serves as a synapse and component for neurons between the input layer and the next layer. In this example, the inputs are INPUT0, ..., INPUT0. n On the vertical control grid lines CG0, ..., CG respectively N The data is received, and outputs OUTPUT1 and OUTPUT2 are generated on source lines SL0 and SL1.

[0145] Figure 21 The neuronal VMM array 2100 is shown, which is particularly suitable for Figure 4The memory unit 410 shown serves as a synapse and component for neurons between the input layer and the next layer. In this example, input INPUT0 is connected to INPUT... N They are received on the gates of bit line control gates 2901-1, 2901-2 to 2901-(N-1) and 2901-N, respectively, and these gates are coupled to bit lines BL0 to BL0, respectively. N Exemplary outputs OUTPUT1 and OUTPUT2 are generated on source lines SL0 and SL1.

[0146] Figure 22 This illustrates a neuronal VMM array 2200, which is particularly suitable for... Figure 3 The memory unit 310 shown Figure 5 The memory cell 510 shown and Figure 7 The memory unit 710 shown serves as a synapse and component for neurons between the input layer and the next layer. In this example, the inputs are INPUT0, …, INPUT0. M In the word lines WL0, …, WL M The data is received and output as OUTPUT0, …, OUTPUT. N On bit lines BL0, ..., BL respectively N Generate above.

[0147] Figure 23 The diagram shows a neuronal VMM array 2300, which is particularly suitable for... Figure 3 The memory unit 310 shown Figure 5 The memory cell 510 shown and Figure 7 The memory unit 710 shown serves as a synapse and component for neurons between the input layer and the next layer. In this example, input INPUT0 is connected to INPUT... M In control grid lines CG0 to CG M The data is received. Outputs OUTPUT0, ..., OUTPUT N On the vertical source lines SL0, ..., SL respectively N The above is generated, where each source line SL i The source line is coupled to all memory cells in column i.

[0148] Figure 24 The diagram shows a neuronal VMM array 2400, which is particularly suitable for... Figure 3 The memory unit 310 shown Figure 5 The memory cell 510 shown and Figure 7The memory unit 710 shown serves as a synapse and component for neurons between the input layer and the next layer. In this example, input INPUT0 is connected to INPUT... M In control grid lines CG0 to CG M The data is received. Outputs OUTPUT0, ..., OUTPUT N On the vertical position lines BL0, ..., BL respectively N Generate on, where each bit line BL i Bit lines coupled to all memory cells in column i.

[0149] Long Short-Term Memory

[0150] Existing technologies include a concept known as Long Short-Term Memory (LSTM). LSTMs are commonly used in artificial neural networks. LSTMs allow artificial neural networks to remember information for predetermined arbitrary time intervals and use that information in subsequent operations. A typical LSTM includes cells, input gates, output gates, and forget gates. These three gates regulate the flow of information into and out of the cells and the time interval at which information is remembered in the LSTM. Virtual Memory Models (VMMs) are particularly useful in LSTMs.

[0151] Figure 25 An exemplary LSTM 2500 is illustrated. This example LSTM 2500 includes cells 2501, 2502, 2503, and 2504. Cell 2501 receives an input vector x0 and generates an output vector h0 and a cell state vector c0. Cell 2502 receives an input vector x1, an output vector (hidden state) h0 from cell 2501, and a cell state c0 from cell 2501, and generates an output vector h1 and a cell state vector c1. Cell 2503 receives an input vector x2, an output vector (hidden state) h1 from cell 2502, and a cell state c1 from cell 2502, and generates an output vector h2 and a cell state vector c2. Cell 2504 receives an input vector x3, an output vector (hidden state) h2 from cell 2503, and a cell state c2 from cell 2503, and generates an output vector h3. Additional cells can be used, and this four-cell LSTM is merely an example.

[0152] Figure 26 Showing what can be used Figure 25 An exemplary specific implementation of LSTM unit 2600, comprising units 2501, 2502, 2503, and 2504, is provided. LSTM unit 2600 receives an input vector x(t), a unit state vector c(t-1) from the previous unit, and an output vector h(t-1) from the previous unit, and generates the unit state vector c(t) and the output vector h(t).

[0153] LSTM unit 2600 includes sigmoid function devices 2601, 2602, and 2603, each applying a number between 0 and 1 to control the amount of each component in the input vector allowed to pass through to the output vector. LSTM unit 2600 also includes tanh devices 2604 and 2605 for applying a hyperbolic tangent function to the input vector, multiplier devices 2606, 2607, and 2608 for multiplying two vectors together, and adder device 2609 for adding two vectors together. The output vector h(t) can be provided to the next LSTM unit in the system, or it can be accessed for other purposes.

[0154] Figure 27 The diagram shows LSTM unit 2700, which is an example of a specific implementation of LSTM unit 2600. For the reader's convenience, LSTM unit 2700 uses the same numbering as LSTM unit 2600. The sigmoid function devices 2601, 2602, and 2603, and the tanh device 2604 each include multiple VMM arrays 2701 and activation circuit blocks 2702. Therefore, it can be seen that VMM arrays are particularly useful in LSTM units used in certain neural network systems.

[0155] Alternative forms of the LSTM unit 2700 (and another example of a specific implementation of the LSTM unit 2600) are in Figure 28 As shown in [the image]. Figure 28 In this LSTM unit, sigmoid function devices 2601, 2602, and 2603, and tanh device 2604 share the same physical hardware (VMM array 2801 and activation function block 2802) in a time-division multiplexing manner. The LSTM unit 2800 also includes a multiplier device 2803 for multiplying two vectors together, an adder device 2808 for adding two vectors together, a tanh device 2605 (which includes activation circuit block 2802), and a register 2807 for storing the value i(t) when the value f(t) is output from sigmoid function block 2802. When c(t-1) is output from multiplier device 2803 via multiplexer 2810, register 2804 stores the value of i(t). When u(t) is output from multiplier device 2803 via multiplexer 2810, register 2805 stores the value of o(t). The register 2806 and the multiplexer 2809 store the value of c~(t) when it is output from the multiplier device 2803 via the multiplexer 2810.

[0156] LSTM unit 2700 includes multiple VMM arrays 2701 and corresponding activation function blocks 2702, while LSTM unit 2800 includes only one set of VMM arrays 2801 and activation function blocks 2802, which are used to represent multiple layers in an implementation of LSTM unit 2800. LSTM unit 2800 will require less space than LSTM 2700 because LSTM unit 2800 only needs 1 / 4 of its space for VMM and activation function blocks compared to LSTM unit 2700.

[0157] It is also understood that an LSTM cell will typically comprise multiple VMM arrays, each requiring functionality provided by certain circuit blocks outside the VMM array itself (such as summer and activation circuit blocks, and high-voltage generation blocks). Providing a separate circuit block for each VMM array would require a significant amount of space within the semiconductor device and would be inefficient to some extent. Therefore, the implementation described below attempts to minimize the circuitry required outside the VMM arrays themselves.

[0158] Gate control recursive unit

[0159] A simulated VMM implementation can be used with GRUs (Gate-Controlled Recurrent Units). GRUs are gate-controlled mechanisms in recurrent artificial neural networks. GRUs are similar to LSTMs, but GRU units generally contain fewer components than LSTM units.

[0160] Figure 29 An exemplary GRU 2900 is shown. The GRU 2900 in this example includes units 2901, 2902, 2903, and 2904. Unit 2901 receives an input vector x0 and generates an output vector h0. Unit 2902 receives an input vector x1, the output vector h0 from unit 2901, and generates an output vector h1. Unit 2903 receives an input vector x2 and the output vector (hidden state) h1 from unit 2902 and generates an output vector h2. Unit 2904 receives an input vector x3 and the output vector (hidden state) h2 from unit 2903 and generates an output vector h3. Additional units may be used, and the GRU with four units is merely an example.

[0161] Figure 30 Showing what can be used Figure 29An exemplary specific implementation of GRU unit 3000, comprising units 2901, 2902, 2903, and 2904, is described. GRU unit 3000 receives an input vector x(t) and an output vector h(t-1) from a previous GRU unit, and generates an output vector h(t). GRU unit 3000 includes sigmoid function devices 3001 and 3002, each of which applies a number between 0 and 1 to the components from the output vector h(t-1) and the input vector x(t). GRU unit 3000 also includes a tanh device 3003 for applying a hyperbolic tangent function to the input vector, multiple multiplier devices 3004, 3005, and 3006 for multiplying two vectors together, an adder device 3007 for adding two vectors together, and a complementary device 3008 for subtracting the input from 1 to generate the output.

[0162] Figure 31 GRU unit 3100 is shown as an example of a specific implementation of GRU unit 3000. For the reader's convenience, GRU unit 3100 uses the same numbering as GRU unit 3000. Figure 31 As shown, sigmoid function devices 3001 and 3002 and tanh device 3003 each include multiple VMM arrays 3101 and activation function blocks 3102. Therefore, it can be seen that VMM arrays are particularly useful in GRU units used in certain neural network systems.

[0163] Alternative forms of the GRU unit 3100 (and another example of a specific implementation of the GRU unit 3000) are in Figure 32 As shown in [the image]. Figure 32 In this configuration, the GRU unit 3200 utilizes a VMM array 3201 and an activation function block 3202, which, when configured as a sigmoid function, applies numbers between 0 and 1 to control how much of each component in the input vector is allowed to pass through to the output vector. Figure 32 In this configuration, sigmoid function devices 3001 and 3002, as well as tanh device 3003, share the same physical hardware (VMM array 3201 and activation function block 3202) in a time-division multiplexing manner. The GRU unit 3200 also includes a multiplier device 3203 that multiplies two vectors together, an adder device 3205 that adds two vectors together, a complementary device 3209 that subtracts the input from 1 to generate the output, and a multiplexer 3204, which operates when the value h(t-1) is... When r(t) is output from multiplier device 3203 via multiplexer 3204, register 3206 holds the value of h(t-1). The register 3207 holds the value of z(t) when it is output from the multiplier device 3203 via the multiplexer 3204, and the value h^(t) is also held. (1-z(t)) is output from the multiplier device 3203 via the multiplexer 3204 and is held in register 3208.

[0164] GRU unit 3100 includes multiple sets of VMM arrays 3101 and activation function blocks 3102, while GRU unit 3200 includes only one set of VMM arrays 3201 and activation function blocks 3202, which are used to represent multiple layers in an embodiment of GRU unit 3200. GRU unit 3200 will require less space than GRU unit 3100 because GRU unit 3200 only needs 1 / 3 of its space for VMMs and activation function blocks compared to GRU unit 3100.

[0165] It is also understandable that systems utilizing GRUs typically include multiple VMM arrays, each requiring functionality provided by certain circuit blocks outside the VMM array itself (such as summer and activation circuit blocks, and high-voltage generation blocks). Providing a separate circuit block for each VMM array would require a significant amount of space within the semiconductor device and would be inefficient to some extent. Therefore, the implementation described below attempts to minimize the circuitry required outside the VMM arrays themselves.

[0166] The inputs of a VMM array can be analog, binary, timing pulse, or digital bits, and the outputs can be analog, binary, timing pulse, or digital bits (in which case an output ADC is needed to convert the output analog current or voltage into digital bits).

[0167] For each memory cell in the VMM array, each weight w can be implemented by a single memory cell, a differential cell, or a hybrid memory cell (the average of two or more cells). In the case of differential cells, two memory cells are needed to implement the weight w as a differential weight (w = w+ – w-). In the case of two hybrid memory cells, two memory cells are needed to implement the weight w as the average of the two cells.

[0168] Implementation scheme for precise tuning of cells in VMM

[0169] Figure 33A block diagram of a VMM system 3300 is shown. The VMM system 3300 includes a VMM array 3301, a row decoder 3302, a high-voltage decoder 3303, a column decoder 3304, a bitline driver 3305, input circuitry 3306, output circuitry 3307, control logic unit 3308, and a bias generator 3309. The VMM system 3300 further includes a high-voltage generation block 3310, which includes a charge pump 3311, a charge pump regulator 3312, and a high-voltage level generator 3313. The VMM system 3300 also includes an algorithm controller 3314, analog circuitry 3315, control logic unit 3316, and test control logic unit 3317. The systems and methods described below can be implemented in the VMM system 3300.

[0170] Input circuitry 3306 may include circuitry such as a DAC (digital-to-analog converter), DPC (digital-to-pulse converter), AAC (analog-to-analog converter, such as a current-to-voltage converter), PAC (pulse-to-analog level converter), or any other type of converter. Input circuitry 3306 may implement normalization, scaling, or arithmetic functions. Input circuitry 3306 may implement a temperature compensation function for the input. Input circuitry 3306 may implement activation functions such as ReLU or sigmoid functions.

[0171] Output circuit 3307 may include circuitry such as an ADC (analog-to-digital converter, used to convert the analog output of a neuron into digital bits), an AAC (analog-to-analog converter, such as a current-to-voltage converter), an APC (analog-to-pulse converter), or any other type of converter. Output circuit 3307 may implement activation functions such as ReLU or the sigmoid function. Output circuit 3307 may implement normalization, scaling, or arithmetic functions for the neuron output. Output circuit 3307 may implement temperature compensation functions for the neuron output or array outputs (such as bitline outputs), as described below.

[0172] Figure 34A tuning (programming or erasing a memory cell to a target) correction method 3400 is shown, which can be executed by an algorithm controller 3314 in a VMM system 3300. The tuning correction method 3400 generates an adaptive target based on a final error generated from a cell output and a cell initial target. The method generally starts (step 3401) in response to receiving a tuning command. An initial current target (for a programming / verification algorithm) for a selected cell or a selected group of cells Itargetv(i) is determined using a prediction target model (such as by using a function or a look-up table), and a variable DeltaError is set to 0 (step 3402). The target function (if used) will be based on the I-V programming curve of the selected memory cell or group of cells. The target function also depends on various variations caused by array characteristics, such as the degree of programming interference exhibited by the cells (which depends on the cell address and cell level within a sector, where if a cell exhibits relatively large interference, the cell undergoes more programming time under a suppression condition, and cells with higher currents generally have more interference), coupling between cells, and various types of array noise. These variations for silicon can be characterized in terms of PVT (process, voltage, temperature). The look-up table (if used) can be characterized in the same way to simulate the I-V curve and various variations.

[0173] Then, a soft erase is performed on all cells in the VMM, which erases all cells to an intermediate weak erase level such that each cell will consume, for example, approximately 3 µA - 5 µA of current during a read operation (step 3403). For example, the soft erase is performed by applying an incremental erase pulse voltage to the cells until an intermediate cell current is reached. Next, a programming operation (such as by deep programming or by coarse / fine programming to the target) is performed on all unused cells (step 3404) to reach a current level of <pA or to reach an equivalent zero weight. Then, a target adjustment (correction) based on the error result is performed. If DeltaError > 0, meaning the cell has experienced an overshoot during programming, then Itargetv (i+1) is set to Itarget + θ DeltaError, where θ is, for example, 1 or a number close to 1 (step 3405A).

[0174] Itarget (i+1) can also be adjusted based on a proper error target adjustment / correction with respect to the previous Itarget(i). If DeltaError < 0, meaning the cell has experienced an undershoot during programming, which means the cell current has not reached the target, then Itargetv (i+1) is set to the previous target Itargetv (i) (step 3405B).

[0175] Next, coarse and / or fine programming and verification operations are performed (step 3406). Several adaptive coarse programming methods can be used to accelerate programming, such as by targeting multiple progressively smaller coarse targets before performing precise (fine) programming steps. Adaptive precise programming is accomplished, for example, with fine (precise) incremental programming voltage pulses or constant programming timing pulses. Embodiments of systems and methods for performing coarse and fine programming are described in U.S. Provisional Patent Application No. 62 / 933,809, filed November 11, 2019, by the same assignee as this application, entitled “Precise Programming Method and Apparatus for AnalogNeural Memory in a Deep Learning Artificial Neural Network,” which is incorporated herein by reference.

[0176] Measure the Icell in the selected cell (step 3407). For example, the cell current can be measured by a galvanometer circuit. For example, the cell current can be measured by an ADC (analog-to-digital converter) circuit, where the output is represented by digital bits in this case. For example, the cell current can be measured by an IV (current-to-voltage converter) circuit, where the output is represented by an analog voltage in this case. Calculate DeltaError, i.e., Icell - Itarget, which represents the difference between the actual current (Icell) and the target current (Itarget) in the measured cell. If |DeltaError| < DeltaMargin, the cell has reached the target current within a certain tolerance (DeltaMargin), and the method ends (step 3410). |DeltaError| = abs(DeltaError) = the absolute value of DeltaError. If not, the method returns to step 3403 and executes the steps in sequence again (step 3410).

[0177] Figure 35A and Figure 35B A tuning correction method 3500 is shown, which can be executed by the algorithm controller 3314 in the VMM system 3300. (See reference...) Figure 35A, the start of the method (step 3501) is typically carried out in response to receiving a tuning command. Erase the entire VMM array (step 3502), such as by a soft erasure method. Perform a programming operation on all unused cells (such as by deep programming or by coarse / fine programming of the target) (step 3503) to achieve a cell current at the <pA level or to achieve an equivalent zero weight. Use coarse and / or fine programming loops to program all cells in the VMM array to an intermediate value, such as 0.5 µA - 1.0 µA (step 3504). Embodiments of systems and methods for performing coarse programming and fine programming are described in U.S. Provisional Patent Application No. 62 / 933,809, filed on November 11, 2019, and titled "Precise Programming Method and Apparatus for Analog Neural Memory in a Deep Learning Artificial Neural Network" by the same assignee as this application, which is incorporated herein by reference. Set the prediction target for the used cells using the function or look-up table as described above (step 3505). Then, perform the sector tuning method 3507 on each sector in the VMM (step 3506). A sector typically consists of two or more adjacent rows in the array.

[0178] Figure 35B Shows the adaptive target sector tuning method 3507. Program all cells in the sector to the final expected value (e.g., 1 nA - 50 nA) using individual or combined programming / verification (P / V) methods, such as: (1) coarse / fine / constant P / V loops; (2) CG+ (only CG increment) or EG+ (only EG increment) or complementary CG+ / EG- (CG increment and EG decrement); and (3) first program the deepest programmed cells (such as progressive grouping, meaning dividing the cells into different groups, and the group of cells with the lowest current is programmed first) (step 3508A). Next, determine whether Icell < Itarget. If so, the method proceeds to step 3509. If not, the method repeats step 3508A. In step 3509, measure DeltaError, which is equal to the measured Icell - Itarget(i+1) (step 3509). Determine whether |DeltaError| < DeltaMargin (step 3510). If so, the method is completed (step 3511). If not, perform target adjustment. If DeltaError > 0, meaning the cell has experienced overshoot during programming, then set the new target to Itarget + θ DeltaError is used to adjust the target, where θ is typically = 1 (step 3512A). Itarget(i+1) can also be adjusted / corrected based on the previous Itarget(i) with an appropriate error target. If DeltaError < 0, it means the cell has experienced an undershoot in programming, implying the cell has not yet reached the target; in this case, the target is adjusted by maintaining the previous target, i.e., Itargetv(i+1) = Itargetv(i) (step 3512B). A soft erase is performed on the sector (step 3513). All cells in the sector are programmed to intermediate values ​​(step 3514), and the process returns to step 3509.

[0179] A typical neural network may have positive weights w+ and negative weights w-, as well as a combined weight = w+ - w-. w+ and w- are implemented by memory cells (Iw+ and Iw-, respectively), and the combined weight (Iw = Iw+ - Iw-, current subtraction) can be performed at peripheral circuit levels (e.g., by using an array bitline output circuit). Therefore, weight tuning implementations for combining weights may include, for example, simultaneously tuning both w+ and w- cells, tuning only the w+ cell, or tuning only the w- cell, as shown in the examples in Table 8. (Using previous references...) Figure 34 / Figure 35A / Figure 35B The described programming / verification and error target adjustment methods are used to perform tuning operations. Verification operations can be performed only for the combined weights (e.g., measuring / reading the combined weighted currents instead of individual positive w+ or w- cell currents), only for the w+ cell current, or only for the w- cell current.

[0180] For example, for a combination Iw of 3nA, Iw+ can be 3nA and Iw- can be 0nA; or, Iw+ can be 13nA and Iw- can be 10nA, meaning that both the positive weight Iw+ and the negative weight Iw- are not zero (e.g., where zero would represent a deep programming unit). Under certain operating conditions, this may be preferred because it will make both Iw+ and Iw- less susceptible to noise.

[0181] Table 8: Weighted Tuning Method (Values ​​are current values ​​in nA)

[0182] Therefore, the differential weight mapping according to the formula w=(w+)-(w-) can be used to store the tuned value w in the neural network for use. The mapping of w+ and w can be optimized to address specific problems that arise in VMM arrays in neural networks, for example, by including offset values ​​in w+ and w- when each value is stored.

[0183] In one implementation, the weight mapping is optimized to reduce RTN noise. For example, with an expected value of 0nA for w (zero w or unused memory cells), one possible mapping is (w+) = 0nA and (w-) = 0nA, and another possible mapping is (w+) = 30nA and (w-) = 30nA, where 30nA is an example of a non-zero offset value added to each w+ and w- value before storage. Including such a non-zero offset value will ultimately consume more power and introduce greater inaccuracy during the tuning process, but it will minimize the impact of any noise. Similarly, with an expected value of 1nA for w, one possible mapping is (w+) = 1nA and (w-) = 0nA, and another possible mapping is (w+) = 30nA and (w-) = 29nA. The latter will consume more power and may introduce greater inaccuracy during the tuning process, but it will minimize the impact of any noise.

[0184] In another implementation, for zero-weight w, both w+ and w- can be tuned to approximately 0, such as by using a bias offset voltage for verification operation, which will now be described. In this method, for example, w+ and w- are tuned to 5nA at a control gate voltage (VCG) higher than the normal CG voltage used for inference (reading). For example, for dVCG / Ir = 2mV / 1nA, if a value of VCG = 1.5V is used in inference, then in the tuning algorithm (the programmed verification algorithm for weight tuning), a value of VCG = 1.510V will be used to verify that the zero-weight cell reaches the target of 5nA. In the inference operation, because VCG = 1.5V is used, the cell current drops by 2mV for every 1nA, so the 5nA in the verification operation becomes approximately 0nA in the inference operation.

[0185] In another implementation, for zero-weight w, both w+ and w- can be tuned to have negative currents, such as by using a negative current tuning method with a bias offset voltage for verification operation, which will now be described. In this case, for example, at a control gate voltage (VCG) higher than the normal CG voltage used for inference (reading), both w+ and w- are tuned to -10nA. For example, for dVCG / Ir = 2mV / 1nA, if a value of VCG = 1.5V is used in inference, then in the tuning algorithm (the programmed verification algorithm for weight tuning), VCG = 1.530V is used to verify that the zero-weight cell reaches the target of 5nA. In the inference operation, because VCG = 1.5V is used, the cell current decreases by 2mV for every 1nA, thus the 5nA in the verification operation becomes approximately -10nA in the inference operation.

[0186] Similar methods using offset bias conditions or bias (voltage and / or current and / or timing and / or temperature) sequences can be used for test screening purposes to detect anomalous bits, such as cells susceptible to significant noise (such as RTN noise, thermal noise, or any other noise source). Essentially, there exist bias conditions or bias sequences that can be used to better detect noise by attenuating the noise from the memory cell to a greater extent than other bias conditions or bias sequences (noise attenuation testing). For example, for a 20nA memory cell, it might be advantageous to detect undesirable behavior by modulating the bias conditions of this cell (e.g., by changing the control gate bias voltage during test screening) to place this cell under a different condition. For example, it might be advantageous to detect bits / cells susceptible to noise (such as RTN noise) at higher current levels by modulating bias conditions due to tester or circuit limitations.

[0187] Another method for screening or verifying noise levels in memory cells (such as RTN noise screening) is to sample the memory cell output (e.g., by measuring the output multiple times, such as 4 / 8 / ... / 1024 times). The screening criterion is that the value of any sample instance is greater than the average of the samples by a certain amount. Another screening criterion is that the value of one sample is greater than the value of the next sample by a certain amount. These techniques are described by the applicant in U.S. Provisional Patent Application No. 62 / 933,809, filed November 11, 2019, entitled “Precise Programming Method and Apparatus for Analog Neural Memory in a Deep Learning Artificial Neural Network,” which is incorporated herein by reference.

[0188] A method for weighting (programming or erasing) a memory cell (which incorporates some of the weighting assignments described above) may include: soft erasing the cell, then programming a zero-weight cell (as described above), and then using noise filtering to perform coarse and fine and / or ultra-fine tuning algorithms, as described above. Techniques for coarse, fine, and ultra-fine tuning algorithms were previously disclosed in U.S. Patent Application No. 16 / 829757, filed March 25, 2020, entitled “Precise Data Tuning Method and Apparatus for Analog Neural Memory in an Artificial Neural Network,” which is incorporated herein by reference.

[0189] In another implementation, the impact of noise can be reduced by using a method that includes bias condition ordering for reads (inferences) or verifications. For example, cumulative conditions are applied to the memory cells before performing a read or verification operation.

[0190] In another embodiment, the noise effect can be reduced by applying a negative voltage range on the control gate. In yet another embodiment, the background data for the array of zero-weighted and unused cells can be a specific pattern to reduce variation. For example, a high current level background might be desired for noise such as RTN noise reduction. Conversely, a low current level background might be desired for noise such as data drift. During standby or deep power-down, the array is placed in the correct conditions by modulating the control gate voltage (e.g., by using a specific voltage level compared to the control gate voltage used during test operations), meaning the control gate level can be set to decrease or increase the current level during standby or deep power-down operations.

[0191] In another embodiment, the read or verify method is performed by applying a 0V, approximately 0V, or low bias voltage to the control gate during the read or verify operation. Word lines are used instead of control gate lines to receive row data inputs (activation values), such as pulse-width modulated inputs or analog voltages applied to the word lines.

[0192] In another implementation, a "0" value for w (zero w or an unused cell) can be defined as <10nA or another predetermined threshold. That is, if (w+) - (w-) < 10nA, then w is assigned the value "0". This provides greater tolerance each time w = 0 and is more robust to inaccuracies caused by noise, temperature changes, or other forces.

[0193] In another implementation, the weight mapping is optimized to reduce temperature variations. For example, with an expected value of w of 5 nA, one possible mapping is (w+) = 5 nA and (w-) = 0 nA, and another possible mapping is (w+) = 30 nA and (w-) = 25 nA. The latter will consume more power and may cause greater inaccuracies during the tuning process, but will minimize temperature variations.

[0194] In another implementation, the weight mapping is optimized to reduce the total noise or temperature variation for the neurons. For example, the number of w+ and w- values ​​stored in each bit line (e.g., 30nA-25nA, 50nA-45nA, or 80nA-75nA for 5nA) can be mapped to balance across all bit lines such that the number of values ​​stored in each bit line is approximately the same for all bit lines.

[0195] In another implementation, the weight mapping is optimized to reduce the total noise for the neurons (bit lines). For example, the number of stored w+ and w- values ​​(units) for each bit line can be balanced across all bit lines so that the total noise impact of all weights (units) in the neurons (bit lines) is optimal (with minimal noise).

[0196] Table 9A: Weight Tuning Methods

[0197] Table 9B: Weight Tuning Methods

[0198] Tables 9A and 9B illustrate exemplary implementations for 16 levels (states) in nA. This means that a memory cell can have 16 levels, as shown. Table 9A describes the case where w can be one of 16 different positive values, and Table 9B describes the case where w can be one of 16 different negative values, according to the formula Iw = Iw+ - Iw-. The current range shown is 0-80 nA.

[0199] Table 10A: Weight Tuning Methods

[0200] Table 10B: Weight Tuning Methods

[0201] Tables 10A and 10B show implementation schemes that compress the dynamic total current range of the level from 0-80nA to 40nA-85nA.

[0202] For Tables 10A / 11A / 12A, Iw- can be adjusted using coarse, fine, or ultra-fine tuning steps (e.g., programming and verifying the Iw- unit), and Iw+ can be adjusted using fine or ultra-fine steps (e.g., programming and verifying Iw = (Iw+ - Iw-), or simply verifying the Iw+ unit). For Tables 10B / 11B / 12B, w+ can be adjusted using coarse, fine, or ultra-fine tuning steps (e.g., programming and verifying the w+ unit), and Iw- can be adjusted using fine or ultra-fine steps (e.g., programming and verifying Iw = (Iw+ - Iw-), or simply verifying the IIw- unit).

[0203] This is advantageous in reducing variations and mismatches caused by processes, temperatures, noise, operating stress, or operating conditions, similar to... Figures 11-12 The concept shown.

[0204] As shown in Tables 10A and 10B, the values ​​in the lower half of the table are increased by a positive amount (offset bias) so that the total range of those values ​​is approximately the same as that in the upper half of the table. This offset is approximately equal to half of the maximum current (level).

[0205] Table 11A: Weight Tuning Methods

[0206] Table 11B: Weight Tuning Methods

[0207] Tables 11A and 11B show implementation schemes similar to those in Tables 10A and 10B, where zero weight (w=0) is equal to the offset value. Examples show a range of weight values ​​for one offset value and two sub-ranges of weights for two offset values.

[0208] Table 12A: Weight Tuning Methods

[0209] Table 12B: Weight Tuning Methods

[0210] Table 13: Weight Tuning Methods

[0211] Table 13 shows the implementation scheme for using varying offsets for positive and negative weights.

[0212] Table 14: Weight Tuning Methods

[0213] Table 14 shows an implementation scheme using a common offset for positive and negative weights.

[0214] Table 15: Weight Tuning Methods

[0215] Table 15 shows an implementation scheme that uses an incremental offset for the weights.

[0216] Table 16: Weight Tuning Methods

[0217] Table 16 illustrates an implementation using a decreasing offset for the weights. It also shows a constant offset for Iw+. Furthermore, it shows the maximum constant value for Iw+, which essentially moves all weights towards the maximum value.

[0218] Tables 12A and 12B show implementation schemes similar to those in Tables 10A and 10B, wherein each w+ or w+ value is implemented by two memory units to further reduce the total dynamic range by approximately another half.

[0219] It should be understood that the values ​​of w+ and w- provided in the above implementation scheme are merely examples, and other values ​​may be used based on the disclosed concepts.

[0220] For example, the offset value can be any value used for shifting for each level, or it can be a fixed value for all levels. In practice, each w is implemented as a differential cell, which can effectively minimize variations or mismatches from process, temperature, noise (such as RTN or power supply noise), stress, or operating conditions.

[0221] Figure 36A The data behavior (IV curve) is shown as a function of temperature (e.g., in the subthreshold region). Figure 36B This illustrates the problems caused by data drift during the operation of a VMM system, and Figure 36C and Figure 36D Showing blocks used to compensate for data drift and about Figure 36C This shows a block used to compensate for temperature changes.

[0222] Figure 36A The known characteristics of a VMM system as operating temperature increases are shown. The sensed current in any given selected non-volatile memory cell in the VMM array increases in the subthreshold region, decreases in the saturation region, or generally decreases in the linear region.

[0223] Figure 36B The diagram illustrates the array current distribution over time (data drift), and shows that the total output from the VMM array (which is the sum of the currents from all bit lines in the VMM array) shifts to the right (or left, depending on the technology used) with operating time. This means that the total output will drift over the lifetime of the VMM system. This phenomenon is called data drift because the data drifts due to usage conditions and degrades due to environmental factors.

[0224] Figure 36C The bit line compensation circuit 3600 is shown, which may include compensating current i COMP The output of the bit line output circuit 3610 is injected to compensate for data drift. The bit line compensation circuit 3600 may include amplification or reduction of the output by a scaling circuit based on a resistor or capacitor network. The bit line compensation circuit 3600 may include shifting or offset of the output by a shifter circuit based on its resistor or capacitor network.

[0225] Figure 36DA data drift monitor 3620 is shown that detects the amount of data drift. This information is then used as an input to a bit line compensation circuit 3600 so that an appropriate level of i can be selected. COMP .

[0226] Figure 37 The diagram shows a bitline compensation circuit 3700, which is one embodiment of the bitline compensation circuit 3600 in FIG. 36. The bitline compensation circuit 3700 includes an adjustable current source 3701 and an adjustable current source 3702, which together generate i COMP , where i COMP It equals the current generated by the adjustable current source 3701 minus the current generated by the adjustable current source 3701.

[0227] Figure 38 The bit line compensation circuit 3700 is shown, which is an embodiment of the bit line compensation circuit 3600 in FIG. 36. The bit line compensation circuit 3800 includes an operational amplifier 3801, an adjustable resistor 3802, and an adjustable resistor 3803. The operational amplifier 3801 receives a reference voltage VREF at its non-inverting terminal and receives a reference voltage V at its inverting terminal. INPUT V INPUT From Figure 36C The bit line output circuit 3610 in the middle receives the voltage and generates an output V. OUTPUT V OUTPUT It is V INPUT A scaled version of V, used to compensate for data drift based on the ratio of resistors 3803 and 3802. V can be amplified or reduced by configuring the values ​​of resistors 3803 and / or 3802. OUTPUT .

[0228] Figure 39 The diagram shows a bitline compensation circuit 3900, which is one embodiment of the bitline compensation circuit 3600 in Figure 36. The bitline compensation circuit 3900 includes an operational amplifier 3901, a current source 3902, a switch 3904, and an adjustable integrating output capacitor 3903. Here, the current source 3902 is actually the output current on a single bitline or a collection of multiple bitlines in the VMM array (such as one for summing positive weights w+ and one for summing negative weights w-). The operational amplifier 3901 receives a reference voltage VREF at its non-inverting terminal and a reference voltage Vo at its inverting terminal. INPUT V INPUT From Figure 36C The bit line output circuit 3610 receives the voltage. The bit line compensation circuit 3900 acts as an integrator, integrating the current Ineu passing through capacitor 3903 within an adjustable integration time to generate the output voltage V. OUTPUT V OUTPUT = Ineu Integral Time / C 3903 C 3903 This is the value of capacitor 3903. Therefore, the output voltage V OUTPUT The output current Ineu is directly proportional to the (bit line) output current, directly proportional to the integration time, and inversely proportional to the capacitance of capacitor 3903. The bit line compensation circuit 3900 generates the output V. OUTPUT V OUTPUT The value is scaled based on the configuration value of capacitor 3903 and / or the integration time to compensate for data drift.

[0229] Figure 40 The diagram shows a bit line compensation circuit 4000, which is one embodiment of the bit line compensation circuit 3600 in Figure 36. The bit line compensation circuit 4000 includes a current mirror 4010 with an M:N ratio, meaning I... COMP = (M / N) i input The current mirror 4010 receives current i INPUT And mirror the current, and optionally scale the current to generate i COMP Therefore, by configuring the M and / or N parameters, i can be amplified or reduced. COMP .

[0230] Figure 41 A bit line compensation circuit 4100 is shown, which is one embodiment of the bit line compensation circuit 3600 in FIG. 36. The bit line compensation circuit 4100 includes an operational amplifier 4101, an adjustable scaling resistor 4102, an adjustable shift resistor 4103, and an adjustable resistor 4104. The operational amplifier 4101 receives a reference voltage V at its non-inverting terminal. REF and receives V at its inverting terminal IN V IN Response to V INPUT Generates with Vshft, where V INPUT From Figure 36C The bit line output circuit 3610 in the middle receives the voltage, and Vshft is designed to achieve V INPUT With V OUTPUT The voltage shifted between them.

[0231] Therefore, V OUTPUT It is V INPUT A scaled and shifted version to compensate for data drift.

[0232] Figure 42The diagram shows a bit line compensation circuit 4200, which is one embodiment of the bit line compensation circuit 3600 in Figure 36. The bit line compensation circuit 4200 includes an operational amplifier 4201, an input current source Ineu 4202, a current shifter 4203, switches 4205 and 4206, and an adjustable integrating output capacitor 4204. Here, the current source 4202 is actually the output current Ineu on one or more bit lines in the VMM array. The operational amplifier 4201 receives a reference voltage VREF at its non-inverting terminal and receives Ineu at its inverting terminal. IN , where I IN It is the sum of the current output from Ineu and the current shifter 4203, and generates the output V. OUTPUT , where V OUTPUT It is scaled (based on capacitor 4204) and shifted (based on Ishifter 4203) to compensate for data drift.

[0233] Figures 43 to 48 Various circuits are shown that can be used to provide the W value to be programmed or read into each selected cell during programming or reading operations.

[0234] Figure 43 The neuron output circuit 4300 is shown, which includes adjustable current source 4301 and adjustable current source 4302, which together generate I. OUT , where I OUT Equal to the current I generated by the adjustable current source 4301 W+ Subtract the current I generated by the adjustable current source 4302 W- The adjustable current Iw+ 4301 is a unit current or neuron current (such as bitline current) used to implement positive weight scaling. The adjustable current Iw- 4302 is a unit current or neuron current (such as bitline current) used to implement negative weight scaling. Current scaling is accomplished, for example, via an M:N ratio current mirror circuit, where Iout = (M / N). Iin.

[0235] Figure 44 The diagram illustrates a neuron output circuit 4400, which includes an adjustable capacitor 4401, a control transistor 4405, switches 4402 and 4403, and an adjustable current source 4404 Iw+, the current being a scaled output current of a cell current such as an M:N current mirror circuit or a (bit-line) neuron current. Transistor 4405 is used, for example, to apply a fixed bias voltage to current 4404. Circuit 4404 generates V OUT V OUT It is inversely proportional to capacitor 4401, directly proportional to the adjustable integration time (the time for switch 4403 to close and switch 4402 to open), and also proportional to the time from the adjustable current source 4404 I. W+The generated current is proportional to V. OUT Equals V+ -((Iw+) (integration time) / C 4401 ), where C 4401 This refers to the value of capacitor 4401. The positive terminal V+ of capacitor 4401 is connected to the positive power supply voltage, and the negative terminal V- of capacitor 4401 is connected to the output voltage V. OUT .

[0236] Figure 45 The diagram illustrates a neuron circuit 4500, which includes a capacitor 4401 and an adjustable current source 4502. This current is a scaled current, such as a cell current from an M:N current mirror or a (bitline) neuron current. Circuit 4500 generates V... OUT V OUT It is inversely proportional to capacitor 4401, directly proportional to the adjustable integral time (the time when switch 4501 is off), and also proportional to the time when adjustable current source 4502I is open. Wi The generated current is proportional. Capacitor 4401 is reused from neuron output circuit 44 after completing its integration of the current Iw+. The positive and negative terminals (V+ and V-) are then swapped in neuron output circuit 45, where the positive terminal is connected to the output voltage V. OUT The output voltage is integrated by the current Iw-. The negative terminal is held at the previous voltage value by a clamping circuit (not shown). In practice, output circuit 44 is used for positive weight implementation and circuit 45 is used for negative weight implementation, wherein the final charge on capacitor 4401 effectively represents the combined weight (Qw=Qw+-Qw-).

[0237] Figure 46 The diagram shows a neuron circuit 4600, which includes an adjustable capacitor 4601, a switch 4602, a control transistor 4604, and an adjustable current source 4603. Circuit 4600 generates V... OUT V OUT It is inversely proportional to capacitor 4601, directly proportional to the adjustable integration time (the time when switch 4602 is off), and also proportional to the time when adjustable current source 4603 I. W- The generated current is proportional. The negative terminal V- of capacitor 4601 is, for example, equal to ground. The positive terminal V+ of capacitor 4601 is initially pre-charged to a positive voltage, for example, before integrating the current Iw-. Neuron circuit 4600 can be used to replace neuron circuit 4500 and neuron circuit 4400 to achieve combined weights (Qw = Qw+ - Qw-).

[0238] Figure 47A neuron circuit 4700 is depicted, comprising: operational amplifiers 4703 and 4706; adjustable current sources Iw+ 4701 and Iw- 4702; and adjustable resistors 4704, 4705, and 4707. The neuron circuit 4700 generates V... OUT The voltage is equal to R. 4707 (Iw+ - Iw-). The adjustable resistor 4707 scales the output. Adjustable current sources Iw+ 4701 and Iw- 4702 also scale the output, for example, via an M:N ratio current mirror circuit (Iout = (M / N)). Iin).

[0239] Figure 48 A neuron circuit 4800 is depicted, comprising: operational amplifiers 4803 and 4806; switches 4808 and 4809; adjustable current sources Iw- 4802 and Iw+ 4801; and adjustable capacitors 4804, 4805, and 4807. The neuron circuit 4800 generates V... OUT This voltage is proportional to (Iw+ - Iw-), proportional to the integration time (the time switches 4808 and 4809 are off), and inversely proportional to the capacitance of capacitor 4807. Adjustable capacitor 4807 achieves output scaling. Adjustable current sources Iw+ 4801 and Iw- 4802 also achieve output scaling, for example, through an M:N ratio current mirror circuit (Iout = (M / N)). (Iin). The integration time can also be adjusted to scale the output.

[0240] Figure 49A , Figure 49B and Figure 49C The output circuit is shown as follows: Figure 33 The block diagram of the output circuit 3307 in the image.

[0241] exist Figure 49A In the output circuit 4901, there is an ADC circuit 4911, which is used to directly digitize the analog neuron output 4910 to provide a digital output bit 4912.

[0242] exist Figure 49B In this circuit, output circuit 4902 includes neuron output circuit 4921 and ADC 4911. Neuron output circuit 4921 receives neuron output 4920 and shapes it, which is then digitized by ADC circuit 4911 to generate output 4912. Neuron output circuit 4921 can be used for normalization, scaling, shifting, mapping, arithmetic operations, activation, and / or temperature compensation, as previously described. ADC circuit can be a serial (ramp-type, step-type, or counting-type) ADC, SAR ADC, pipelined ADC, Σ-Δ ADC, or any type of ADC.

[0243] exist Figure 49C In this design, the output circuitry includes a neuron output circuit 4921 and a converter circuit 4931. The neuron output circuit receives the neuron output 4930, and the converter circuit converts the output from the neuron output circuit 4921 into an output 4932. The converter 4931 may include an ADC, an AAC (similar to an analog-to-analog converter, such as a current-to-voltage converter), an APC (analog-to-pulse converter), or any other type of converter. The ADC 4911 or the converter 4931 may be used to implement the activation function through, for example, bit mapping (e.g., quantization) or clipping (e.g., clipped ReLU). The ADC 4911 and the converter 4931 may be configurable, such as for lower or higher accuracy (e.g., lower or higher bit depth), lower or higher performance (e.g., slower or faster speed), etc.

[0244] Another implementation for scaling and shifting involves configuring an ADC (analog-to-digital) conversion circuit (such as a serial ADC, SAR ADC, pipelined ADC, ramp ADC, etc.) to convert the array (bit line) output into digital bits with lower or higher bit precision, and then manipulating the digital output bits according to some function (e.g., linear or nonlinear, compression, nonlinear activation, etc.), such as through normalization (e.g., 12-bit to 8-bit), shifting, or remapping. Embodiments of ADC conversion circuitry are described in U.S. Provisional Patent Application No. 62 / 933,809, filed November 11, 2019, by the same assignee as this application, entitled “Precise Programming Method and Apparatus for Analog Neural Memory in a Deep Learning Artificial Neural Network,” which is incorporated herein by reference.

[0245] Table 9 shows alternative methods for performing read, erase, and program operations: Table 9: Operation of Flash Memory Cells

[0246] Read and erase operations are similar to those of the previous table. However, the two methods used for programming are implemented using the Fowler-Nordheim (FN) tunneling mechanism.

[0247] Implementations for scaling the input can be done, for example, by enabling a certain number of rows of the VMM at once and then combining the results completely.

[0248] Another implementation involves scaling the input voltage and appropriately rescaling the output to achieve normalization.

[0249] Another implementation for scaling pulse width modulation inputs is through the timing of pulse width modulation. Embodiments of this technique are described in U.S. Patent Application No. 16 / 449,201, filed June 21, 2019, by the same assignee as this application, entitled “Configurable Input Blocks and Output Blocks and Physical Layout for Analog Neural Memory in Deep Learning Artificial Neural Network,” which is incorporated herein by reference.

[0250] Another implementation for scaling the input is to enable one input bit at a time, for example, for an 8-bit input IN7:0, evaluating IN0, IN1, ..., IN7 sequentially, and then combining the output with the appropriate bit weights. Embodiments of this technique are described in U.S. Patent Application No. 16 / 449,201, filed June 21, 2019, by the same assignee as this application, entitled “Configurable Input Blocks and Output Blocks and Physical Layout for Analog Neural Memory in Deep Learning Artificial Neural Network,” which is incorporated herein by reference.

[0251] Optionally, in the above embodiments, the unit current can be measured in average or multiple times for the purpose of verification or reading the current, for example, 8 to 32 measurements, to reduce the influence of noise (such as RTN or any random noise) and / or to detect defective bits and any outlier bits that need to be replaced by redundant bits.

[0252] It should be noted that, as used herein, the terms “above” and “on” both encompass “directly on” (without intermediate material, elements, or space between) and “indirectly on” (with intermediate material, elements, or space between). Similarly, the term “adjacent” includes “directly adjacent” (without intermediate material, elements, or space between) and “indirectly adjacent” (with intermediate material, elements, or space between), “mounted to” includes “directly mounted to” (without intermediate material, elements, or space between) and “indirectly mounted to” (with intermediate material, elements, or space between), and “electrically coupled to” includes “directly electrically coupled to” (without intermediate material or elements electrically connecting the elements together) and “indirectly electrically coupled to” (with intermediate material or elements electrically connecting the elements together). For example, forming an element “above a substrate” can include forming an element directly on the substrate without intermediate material / elements between them, and forming an element indirectly on the substrate with one or more intermediate materials / elements between them.

Claims

1. A method for programming, verifying, and reading zero values ​​in difference pairs of non-volatile memory cells in a vector-matrix multiplication array, the method comprising: The first unit w+ in the differential pair is programmed to the first current value; The first unit is verified by applying a voltage equal to the first voltage plus the bias voltage to the control gate terminal of the first unit; The second unit w- in the differential pair is programmed to the first current value; The second unit is verified by applying a voltage equal to the first voltage plus the bias voltage to the control gate terminal of the second unit; The first unit is read by applying a voltage equal to the first voltage to the control gate terminal of the first unit; The second unit is read by applying a voltage equal to the first voltage to the control gate terminal of the second unit; The value w is calculated using the formula w=(w+)-(w-).

2. The method of claim 1, wherein the first unit is tuned by one or more of a coarse, fine, or ultra-fine tuning algorithm, and the second unit is tuned by one or more of a fine or ultra-fine tuning algorithm.

3. A method for programming, verifying, and reading zero values ​​in difference pairs of non-volatile memory cells in a vector-matrix multiplication array, the method comprising: The first unit w+ in the differential pair is programmed to the first current value; The first unit is verified by applying a voltage equal to the first voltage plus the bias voltage to the control gate terminal of the first unit; The second unit w- in the differential pair is programmed to the first current value; The second unit is verified by applying a voltage equal to the first voltage plus the bias voltage to the control gate terminal of the second unit; The first unit is read by applying a voltage equal to the first voltage to the control gate terminal of the first unit; The second unit is read by applying a voltage equal to the first voltage to the control gate terminal of the second unit; The value w is calculated using the formula w=(w+)-(w-).

4. The method according to claim 3, wherein the first current value is a positive value.

5. The method according to claim 3, wherein the second current value is negative.

6. The method of claim 3, wherein the first unit is tuned to w+ by one or more of a coarse, fine, or ultrafine tuning algorithm, and the second unit is tuned to w- by one or more of a fine or ultrafine tuning algorithm.

7. A method for reading non-volatile memory cells in a vector-matrix multiplication array, the method comprising: Reading the weights stored in selected cells of the array includes: Apply a zero-voltage bias to the control gate terminal of the selected cell; and Sensing includes the neuronal output current, which includes the current output from the selected unit.

8. The method of claim 7, wherein the reading step further comprises: Apply voltage to the word line terminals of the selected cell.

9. A method for operating on non-volatile memory cells in a vector multiplication matrix multiplication array, the method comprising: The non-volatile memory cell is read by applying a first bias voltage to the control gate of the non-volatile memory cell; as well as A second bias voltage is applied to the control gate of the non-volatile memory cell during one or more of standby operation, deep power-down operation, or test operation.

10. The method of claim 9, further comprising: Modulate the background data pattern or zero-weight or non-user cells in the array.

Images