Interface circuit for a stack comprising a plurality of vector-matrix multiplication arrays

Abstract

In one example, a system includes: a first device; a first interface coupled to the first device and including one or more device control lines to identify a second device within a plurality of devices that is intended to be communicated to, the plurality of devices respectively containing vector-matrix multiplication arrays; and a stack including an interface circuit coupled to the first interface, the plurality of devices, and a second interface including lines that conform to a legacy standard, the second interface coupled between the interface circuit and the plurality of devices.

Description

[0001] Priority Statement

[0002] This application claims priority to U.S. Provisional Patent Application No. 63 / 625,200, filed January 25, 2024, entitled “Interface for Artificial Neural Network Comprising a Three-Dimensional Integrated Circuit”, and U.S. Patent Application No. 18 / 623,985, filed April 1, 2024, entitled “Interface Circuit for Stack Comprising a Plurality of Vector-by-Matrix Multiplication Arrays”. Technical Field

[0003] Numerous examples of interface circuits for artificial neural networks including three-dimensional integrated circuits are disclosed. Background Technology

[0004] Artificial neural networks mimic biological neural networks (the central nervous system of animals, particularly the brain) and are used to estimate or approximate functions that can depend on a large number of inputs and are often unknown. Artificial neural networks typically consist of interconnected layers of "neurons" that exchange messages with each other.

[0005] Figure 1 An artificial neural network is illustrated, where circles represent the inputs or layers of neurons. Connections (called synapses) are indicated by arrows and have numerical weights that can be tuned empirically. This allows the neural network to adapt to its inputs and learn. Typically, a neural network consists of layers with multiple inputs. There are usually one or more intermediate layers of neurons, and an output layer of neurons that provide the output of the neural network. Neurons at each level make decisions individually or collectively based on data received from the synapses.

[0006] One of the major challenges in developing artificial neural networks for high-performance information processing is the lack of sufficient hardware technology. Real-world neural networks rely on a large number of synapses to achieve high connectivity between neurons, i.e., very high computational parallelism. In principle, this complexity can be achieved using digital supercomputers or dedicated clusters of graphics processing units. However, compared to biological networks, these methods are generally energy inefficient, in addition to being costly, as biological networks consume less energy primarily due to their ability to perform low-precision analog calculations. CMOS analog circuits have been used in artificial neural networks, but given the large number of neurons and synapses, most CMOS-implemented synapses are excessively large.

[0007] The applicant previously disclosed an artificial (simulated) neural network utilizing one or more non-volatile memory arrays as synapses in U.S. Patent Application Publication 2017 / 0337466A1, which is incorporated herein by reference. The non-volatile memory array operates as a simulated neural memory and includes non-volatile memory cells arranged in rows and columns. The neural network includes a plurality of first synapses configured to receive a first plurality of inputs and generate a first plurality of outputs therefrom, and a plurality of first neurons configured to receive the first plurality of outputs. The first plurality of synapses includes a plurality of memory cells, wherein each memory cell includes: spaced-apart source and drain regions formed in a semiconductor substrate, wherein a channel region extends between the source and drain regions; a floating gate disposed over and insulated from a first portion of the channel region; and a non-floating gate disposed over and insulated from a second portion of the channel region. Each memory cell stores a weight value corresponding to the number of electrons on the floating gate. The plurality of memory cells multiply the first plurality of inputs by the stored weight values ​​to generate the first plurality of outputs.

[0008] Non-volatile memory cells

[0009] Non-volatile memory is well known. For example, U.S. Patent 5,029,130 ​​(“the '130 Patent”), which is incorporated herein by reference, discloses an array of split-gate non-volatile memory cells, which is a type of flash memory cell. Figure 2The image shows such a memory cell 210. Each memory cell 210 includes a source region 14 and a drain region 16 formed in a semiconductor substrate 12, with a channel region 18 between the source and drain regions. A floating gate 20 is formed over and insulated from (and controls the conductivity of) a first portion of the channel region 18, and is formed over a portion of the source region 14. A word line terminal 22 (which is typically coupled to a word line) has a first portion disposed over and insulated from (and controlling the conductivity of) a second portion of the channel region 18, and a second portion extending upward and located over the floating gate 20. The floating gate 20 and the word line terminal 22 are insulated from the substrate 12 by a gate oxide. A bit line 24 is coupled to the drain region 16.

[0010] The memory cell 210 is erased by applying a high positive voltage to the word line terminal 22 (where electrons are removed from the floating gate), which causes electrons on the floating gate 20 to tunnel from the floating gate 20 to the word line terminal 22 through the intermediate insulator via the Fowler-Nordheim (FN).

[0011] Memory cell 210 is programmed via source-side injection (SSI) using hot electrons (where electrons are placed on the floating gate) by applying a positive voltage to word line terminal 22 and a positive voltage to source region 14. Electron current flows from drain region 16 to source region 14. When electrons reach the gap between word line terminal 22 and floating gate 20, they accelerate and become hot. Due to electrostatic attraction from floating gate 20, some heated electrons are injected onto floating gate 20 through gate oxide.

[0012] Memory cell 210 is read by applying a positive read voltage to the drain region 16 and word line terminal 22 (which conducts the portion of channel region 18 below the word line terminal). If the floating gate 20 is positively charged (i.e., electrons are erased), the portion of channel region 18 below the floating gate 20 is also conducted, and current flows through channel region 18, which is sensed as an erased state or a "1" state. If the floating gate 20 is negatively charged (i.e., programmed electronically), the portion of channel region below the floating gate 20 is mostly or completely turned off, and current does not flow (or very little current) through channel region 18, which is sensed as a programmed state or a "0" state.

[0013] Table 1 depicts the typical voltage and current ranges that can be applied to the terminals of memory cell 210 to perform read, erase, and program operations:

[0014] Table 1: Figure 2 Operation of flash memory cell 210

[0015]

[0016] Other split-gate memory cell configurations, used as other types of flash memory cells, are known. For example, Figure 3 A four-gate memory cell 310 is depicted, comprising a source region 14, a drain region 16, a floating gate 20 over a first portion of a channel region 18, a select gate 22 (typically coupled to a word line WL) over a second portion of the channel region 18, a control gate 28 over the floating gate 20, and an erase gate 30 over the source region 14. This configuration is described in U.S. Patent 6,747,310, which is incorporated herein by reference for all purposes. Here, all gates except the floating gate 20 are non-floating gates, meaning they are electrically connected to or can be electrically connected to a voltage source. Programming is performed by hot electrons from the channel region 18 injecting themselves into the floating gate 20. Erasing is performed by electrons tunneling from the floating gate 20 to the erase gate 30.

[0017] Table 2 depicts the typical voltage and current ranges that can be applied to the terminals of memory cell 310 to perform read, erase, and program operations:

[0018] Table 2: Figure 3 Operation of flash memory cell 310

[0019]

[0020] Figure 4 A tri-gate memory cell 410 is depicted, which is another type of flash memory cell. Memory cell 410 and... Figure 3 The memory cell 310 is the same as the memory cell 410, except that the memory cell 410 does not have a separate control gate. Except that no control gate bias is applied, the erase operation (thus erasing is performed using an erase gate) and read operation are the same as... Figure 3 The operation is similar. Programming is also performed without a control gate bias, and therefore, a higher voltage is applied to the source line during programming to compensate for the lack of a control gate bias.

[0021] Table 3 depicts the typical voltage and current ranges that can be applied to the terminals of memory cell 410 to perform read, erase, and program operations:

[0022] Table 3: Figure 4 Operation of flash memory cell 410

[0023]

[0024] Figure 5 A stacked-gate memory cell 510 is depicted, which is another type of flash memory cell. Except that the floating gate 20 extends over the entire channel region 18 and the control gate 22 (which here will be coupled to the word line) extends over the floating gate 20 and is separated by an insulating layer (not shown), the memory cell 510... Figure 2 The memory cell 210 is similar. Erasure is performed by electron tunneling from FG to FN in the substrate, and programming is performed by channel hot electron (CHE) injection in the region between the channel 18 and the drain region 16, by electron flow from the source region 14 to the drain region 16, and by a read operation similar to that used for a read operation for a memory cell 210 with a higher control gate voltage.

[0025] Table 4 depicts the typical voltage range that can be applied to the terminals of memory cell 510 and substrate 12 to perform read, erase, and program operations:

[0026] Table 4: Figure 5 Operation of flash memory cell 510

[0027]

[0028] The methods and apparatus described herein can be applied to other non-volatile memory technologies, such as, but not limited to, FINFET split-gate flash or stacked-gate flash memory, NAND flash memory, SONOS (silicon-oxide-nitride-oxide-silicon with charge trapped in nitride), MONOS (metal-oxide-nitride-oxide-silicon with metal charge trapped in nitride), ReRAM (resistive random access memory), PCM (phase change memory), MRAM (magnetic RAM), FeRAM (ferroelectric RAM), CT (charge trapped) memory, CN (carbon nanotube) memory, OTP (two-level or multi-level one-time programmable) and CeRAM (associated electron RAM), etc.

[0029] To utilize memory arrays comprising one of the aforementioned types of non-volatile memory cells in artificial neural networks, two modifications were made. First, the lines were configured such that each memory cell could be programmed, erased, and read individually without adversely affecting the memory state of other memory cells in the array, as explained further below. Second, sequential (simulated) programming of the memory cells was provided.

[0030] Specifically, the memory state (i.e., the charge on the floating gate) of each memory cell in the array can be continuously changed from a fully erased state to a fully programmed state and vice versa, independently and with minimal interference to other memory cells. This means that the cell storage device is essentially analog, or at least can store one discrete value from many discrete values ​​(such as 16 or 64 different values). This allows for very precise and individual tuning of all memory cells in the memory array, making the memory array ideal for both storage and fine-tuning of synaptic weights in neural networks.

[0031] Neural networks using non-volatile memory cell arrays

[0032] Figure 6 This conceptual example illustrates a non-limiting example of a neural network utilizing a non-volatile memory array. This example uses a non-volatile memory array neural network for a facial recognition application, but any other suitable application can also be implemented using a neural network based on a non-volatile memory array.

[0033] In this example, S0 is the input layer, which is a 32x32 pixel RGB image with 5-bit precision (i.e., three 32x32 pixel arrays, one for each color R, G, and B, with 5-bit precision per pixel). The synapse CB1 from the input layer S0 to layer C1 applies different sets of weights in some cases and shared weights in others, and scans the input image with a 3x3 pixel overlapping filter (kernel), shifting the filter by one pixel (or more than one pixel as indicated by the model). Specifically, the values ​​of nine pixels in a 3x3 portion of the image (i.e., called the filter or kernel) are provided to synapse CB1, where these nine input values ​​are multiplied by appropriate weights, and after summing the output of this multiplication, a single output value is determined by the first synapse of CB1 to be used to generate one of the pixels in the feature map of layer C1. The 3x3 filter is then shifted one pixel to the right within the input layer S0 (i.e., adding a column of three pixels to the right and releasing a column of three pixels to the left), thereby providing the nine pixel values ​​from this newly positioned filter to synapse CB1, where they are multiplied by the same weights and a second single output value is determined by the associated synapse. This process continues until the 3x3 filter scans all three colors and all bits (precision values) across the entire 32x32 pixel image of the input layer S0. This process is then repeated using different sets of weights to generate different feature maps for layer C1 until all feature maps for layer C1 are computed.

[0034] At layer C1, in this example, there are 16 feature maps, each with 30x30 pixels. Each pixel is a new feature pixel extracted from the product of the input and the kernel, and therefore each feature map is a two-dimensional array, and thus in this example, layer C1 consists of a 16-layer two-dimensional array (remember that the layers and arrays referred to in this article are logical relationships, which may or may not correspond to physical relationships—that is, the array may not be oriented in a physical two-dimensional array). Each of the 16 feature maps in layer C1 is generated by one set of sixteen different groups of synaptic weights applied to the filter scan. The C1 feature maps may all relate to different aspects of the same image features, such as boundary markings. For example, the first map (generated using the first weight reassembly, shared for all scans used to generate the first map) may mark circular edges, the second map (generated using the second weight reassembly, different from the first weight reassembly) may mark rectangular edges, or the aspect ratio of certain features, and so on.

[0035] Before transitioning from layer C1 to layer S1, activation function P1 (pooling) is applied, which pools the values ​​from consecutive non-overlapping 2x2 regions in each feature map. The purpose of pooling function P1 is to average the neighboring locations (or, alternatively, use a max function) to reduce, for example, the dependence on edge locations and to reduce the data size before moving to the next stage. At layer S1, there are 16 15x15 feature maps (i.e., sixteen distinct arrays, each with 15x15 pixels). The synapse CB2 from layer S1 to layer C2 scans the map in layer S1 using a 4x4 filter, where the filter is shifted by 1 pixel. At layer C2, there are 22 12x12 feature maps. Before transitioning from layer C2 to layer S2, activation function P2 (pooling) is applied, which pools the values ​​from consecutive non-overlapping 2x2 regions in each feature map. At layer S2, there are 22 6x6 feature maps. An activation function (pooling) is applied to synapse CB3 from layer S2 to layer C3, where each neuron in layer C3 is connected to each graph in layer S2 via a corresponding synapse in CB3. There are 64 neurons in layer C3. Synapse CB4 from layer C3 to the output layer S3 completely connects C3 to S3, meaning each neuron in layer C3 is connected to every neuron in layer S3. The output at S3 comprises 10 neurons, with the highest-output neuron determining the class. For example, this output could indicate an identification or classification of the content of the original image.

[0036] Synapses for each layer are implemented using an array or a portion of an array of non-volatile memory cells.

[0037] Figure 7 This is a block diagram of an array that could be used for this purpose. The vector-matrix multiplication (VMM) array 32 includes non-volatile memory cells and serves as synapses between layers (such as...) Figure 6 (CB1, CB2, CB3, and CB4 in the original text). Specifically, the VMM array 32 includes an array 33 of non-volatile memory cells, an erase gate and word line gate decoder 34, a control gate decoder 35, a bit line decoder 36, and a source line decoder 37, which decode the corresponding inputs of the non-volatile memory cell array 33. The inputs to the VMM array 32 may come from the erase gate and word line gate decoder 34 or from the control gate decoder 35. In this example, the source line decoder 37 also decodes the outputs of the non-volatile memory cell array 33. Alternatively, the bit line decoder 36 may decode the outputs of the non-volatile memory cell array 33.

[0038] The non-volatile memory cell array 33 serves two purposes. First, it stores weights that will be used by the VMM array 32. Second, the non-volatile memory cell array 33 efficiently multiplies the inputs with the weights stored in the non-volatile memory cell array 33 and adds them together on each output line (source line or bit line) to produce an output that will be used as the input to the next layer or the final layer. By performing multiplication and addition functions, the non-volatile memory cell array 33 eliminates the need for separate multiplication and addition logic circuits and is also highly efficient due to its in-situ memory computation.

[0039] The output of the non-volatile memory cell array 33 is provided to a differential summer (such as a summing operational amplifier or a summing current mirror) 38, which sums the output of the non-volatile memory cell array 33 to create a single value for the convolution. The differential summer 38 is arranged to perform the summation of positive and negative weights.

[0040] The summed output of the difference summer 38 is then provided to the activation function block 39, which modifies the output. Activation function block 39 can provide a sigmoid, tanh, or ReLU function. The modified output of activation function block 39 becomes the next layer's output (e.g., ...). Figure 6 Elements of the feature map of layer C1 are then applied to the next synapse to produce the next feature layer or the final layer. Thus, in this example, the non-volatile memory cell array 33 constitutes multiple synapses (which receive their input from existing neuron layers or from input layers such as an image database), and the summing operational amplifier 38 and the activation function block 39 constitute multiple neurons.

[0041] Figure 7 The inputs to the VMM array 32 (WLx, Egx, CGx and optional BLx and SLx) can be analog, binary or digital (in which case a DAC is provided to convert the digital bits to the appropriate input analog level), and the output can be analog, binary or digital (in which case an output ADC is provided to convert the output analog level to digital bits).

[0042] Figure 8 A block diagram illustrating the use of the multilayer VMM array 32, labeled VMM arrays 32a, 32b, 32c, 32d, and 32e. (See diagram for reference.) Figure 8As shown, the input (denoted as Inputx) is converted from digital to analog by a digital-to-analog converter 31 and provided to the input VMM array 32a. The converted analog input can be voltage or current. The first-level input D / A conversion can be accomplished by using a function or LUT (lookup table) of appropriate analog level to map Inputx to the input VMM array 32a matrix multiplier. Input conversion can also be accomplished by an analog-to-analog (A / A) converter to convert the external analog input into a mapped analog input to the input VMM array 32a.

[0043] The output generated by the input VMM array 32a is provided as input to the next VMM array (hidden level 1) 32b, which in turn generates the output provided as input to the next VMM array (hidden level 2) 32c, and so on. The various layers of the VMM array 32 serve as different layers of synapses and neurons in a convolutional neural network (CNN). Each VMM array 32a, 32b, 32c, 32d, and 32e can be an independent physical non-volatile memory array, or multiple VMM arrays can utilize different portions of the same physical non-volatile memory array, or multiple VMM arrays can utilize overlapping portions of the same physical non-volatile memory array. Figure 8 The example shown contains five layers (32a, 32b, 32c, 32d, 32e): one input layer (32a), two hidden layers (32b, 32c), and two fully connected layers (32d, 32e). Those skilled in the art will understand that this is merely an example, and conversely, a system may include more than two hidden layers and more than two fully connected layers.

[0044] Vector-Matrix Multiplication (VMM) Array

[0045] Figure 9 A neuronal VMM array 900 is depicted, which is particularly suitable for... Figure 3 The memory cell 310 shown serves as a synapse and partial neuron between the input layer and the next layer. The VMM array 900 includes a memory array 901 of non-volatile memory cells and a reference array 902 of non-volatile reference memory cells (at the top of the array). Alternatively, another reference array may be placed at the bottom.

[0046] In the VMM array 900, control gate lines (such as control gate line 903) extend vertically (therefore, reference array 902 is orthogonal to control gate line 903 in the row direction), and erase gate lines (such as erase gate line 904) extend horizontally. Here, the inputs to the VMM array 900 are set on control gate lines (CG0, CG1, CG2, CG3), and the outputs of the VMM array 900 appear on source lines (SL0, SL1). In one example, only even rows are used, and in another example, only odd rows are used. The currents placed on each source line (SL0, SL1, respectively) perform a summation function of all currents from the memory cells connected to that particular source line.

[0047] As described herein with respect to neural networks, the non-volatile memory cells of the VMM array 900 (i.e., memory cells 310 of the VMM array 900) can be configured to operate in a region below a threshold.

[0048] Bias the non-volatile reference memory cell and non-volatile memory cell described in this paper in the weak inversion (below the threshold region):

[0049] Ids = Io e (Vg- Vth) / nVt = w Io e (Vg) / nVt ,

[0050] Where w = e (- Vth) / nVt

[0051] Where Ids is the drain-to-source current; Vg is the gate voltage on the memory cell; Vth is the threshold voltage of the memory cell; and Vt is the thermal voltage = k T / q, where k is Boltzmann's constant, T is the temperature in Kelvin, and q is the electron charge; n is the slope factor = 1 + (Cdep / Cox), where Cdep = the capacitance of the depletion layer, and Cox is the capacitance of the gate oxide layer; Io is the memory cell current at the gate voltage equal to the threshold voltage, and Io is related to (Wt / L). u Cox (n-1) Vt 2 Proportional, where u is the carrier mobility, and Wt and L are the width and length of the memory cell, respectively.

[0052] For I-to-V logarithmic converters that use memory cells (such as reference memory cells or peripheral memory cells) or transistors to convert input current into input voltage:

[0053] Vg= n Vt log [Ids / wp Io]

[0054] Where wp represents the w in the reference memory cell or the peripheral memory cell.

[0055] For a memory array used as a vector matrix multiplier (VMM) array with current input, the output current is:

[0056] Iout = wa Io e (Vg) / nVt ,Right now

[0057] Iout = (wa / wp) Iin = W Iin

[0058] W = e (Vthp - Vtha) / nVt

[0059] Here, wa = w for each memory cell in the memory array.

[0060] Vthp is the effective threshold voltage of the peripheral memory cell, and Vtha is the effective threshold voltage of the primary (data) memory cell. Note that the threshold voltage of the transistor is a function of the substrate bulk bias voltage, and the substrate bulk bias voltage, denoted as Vsb, can be modulated to compensate for various conditions at this temperature. The threshold voltage Vth can be expressed as:

[0061] Vth = Vth0 + γ (SQRT |Vsb - 2 F) - SQRT |2 F |)

[0062] Where Vth0 is the threshold voltage with zero substrate bias. F is the surface potential, and γ is the host effect parameter.

[0063] Word lines or control gates can be used as inputs to memory cells that accept input voltages.

[0064] Alternatively, the flash memory cells of the VMM array described herein can be configured to operate in a linear region:

[0065] Ids = β (Vgs-Vth) Vds;β = u Cox Wt / L

[0066] W = α (Vgs-Vth)

[0067] This means that the weight W in the linear region is proportional to (Vgs-Vth).

[0068] Word lines, control gates, bit lines, or source lines can be used as inputs to memory cells operating in a linear region. Bit lines or source lines can be used as outputs to memory cells.

[0069] For an IV linear converter, memory cells (such as reference memory cells or peripheral memory cells) or transistors operating in the linear region can be used to linearly convert input / output current into input / output voltage.

[0070] Alternatively, the memory cells of the VMM array described herein can be configured to operate in a saturation region:

[0071] Ids = ½ β (Vgs-Vth) 2 ;β=u Cox Wt / L

[0072] Wα (Vgs-Vth) 2 This means that the weight W is related to (Vgs-Vth). 2 proportional

[0073] Word lines, control gates, or erase gates can be used as inputs to memory cells operating in saturation regions. Bit lines or source lines can be used as outputs of output neurons.

[0074] Alternatively, the memory cells of the VMM array described herein can be used in all regions or combinations thereof (below threshold, linear or saturated regions) of each or more layers of a neural network.

[0075] It is described in U.S. Patent No. 10,748,630 Figure 7 Other examples of VMM arrays 32 are described herein by reference. Source lines or bit lines can be used as neuron outputs (current summation outputs).

[0076] Figure 10 A neuronal VMM array 1000 is depicted, which is particularly suitable for... Figure 2The memory cell 210 shown serves as a synapse between the input layer and the next layer. The VMM array 1000 includes a memory array 1003 of non-volatile memory cells, a reference array 1001 of first non-volatile reference memory cells, and a reference array 1002 of second non-volatile reference memory cells. The reference arrays 1001 and 1002, arranged in the column direction of the array, are used to convert current inputs flowing into terminals BLR0, BLR1, BLR2, and BLR3 into voltage inputs WL0, WL1, WL2, and WL3. In practice, the first and second non-volatile reference memory cells are diode-connected via a multiplexer 1014 (partially depicted), into which current inputs flow. The reference cells are tuned (e.g., programmed) to a target reference level. The target reference level is provided by a reference microarray matrix (not shown).

[0077] The memory array 1003 serves two purposes. First, it stores the weights used by the VMM array 1000 on their respective memory cells. Second, the memory array 1003 efficiently multiplies the inputs (i.e., the current inputs provided in terminals BLR0, BLR1, BLR2, and BLR3, which reference arrays 1001 and 1002 convert into input voltages to supply to word lines WL0, WL1, WL2, and WL3) by the weights stored in the memory array 1003, and then adds all the results (memory cell currents) to produce an output on the corresponding bit lines (BL0 to BLN), which will be the input to the next layer or to the final layer. By performing multiplication and addition functions, the memory array 1003 eliminates the need for separate multiplication and addition logic circuits and is also highly efficient. Here, voltage inputs are provided on word lines WL0, WL1, WL2, and WL3, and the outputs appear on the corresponding bit lines BL0 to BLN during read (inference) operations. The current placed on each of the bit lines BL0 to BLN performs a summation function of the currents from all non-volatile memory cells connected to that particular bit line.

[0078] Table 5 depicts the operating voltages and currents used for the VMM array 1000. The columns in the table indicate the voltages applied to the word lines for selected cells, word lines for unselected cells, bit lines for selected cells, bit lines for unselected cells, source lines for selected cells, and source lines for unselected cells. The rows indicate read, erase, and program operations.

[0079] Table 5: Figure 10 Operation of VMM array 1000 :

[0080]

[0081] Figure 11A neuronal VMM array 1100 is depicted, which is particularly suitable for... Figure 2 The memory cell 210 shown serves as a synapse and partial neuron between the input layer and the next layer. The VMM array 1100 includes a memory array 1103 of non-volatile memory cells, a reference array 1101 of first non-volatile reference memory cells, and a reference array 1102 of second non-volatile reference memory cells. Reference arrays 1101 and 1102 extend in the row direction of the VMM array 1100. The VMM array is similar to VMM 1000, except that in VMM array 1100, word lines extend in the vertical direction. Here, inputs are set on word lines (WLA0, WLB0, WLA1, WLB2, WLA2, WLB2, WLA3, WLB3), and outputs appear on source lines (SL0, SL1) during read operations. The currents placed on each source line perform a summation function of all currents from the memory cells connected to that particular source line.

[0082] Table 6 depicts the operating voltages and currents used for the VMM array 1100. The columns in the table indicate the voltages applied to the word lines for selected cells, word lines for unselected cells, bit lines for selected cells, bit lines for unselected cells, source lines for selected cells, and source lines for unselected cells. The rows indicate read, erase, and program operations.

[0083] Table 6: Figure 11 Operation of VMM array 1100

[0084]

[0085] Figure 12 A neuronal VMM array 1200 was described, which is particularly suitable for... Figure 3The memory cell 310 shown serves as a synapse and partial neuron between the input layer and the next layer. The VMM array 1200 includes a memory array 1203 of non-volatile memory cells, a reference array 1201 of first non-volatile reference memory cells, and a reference array 1202 of second non-volatile reference memory cells. Reference arrays 1201 and 1202 are used to convert current inputs flowing into terminals BLR0, BLR1, BLR2, and BLR3 into voltage inputs CG0, CG1, CG2, and CG3. In practice, the first and second non-volatile reference memory cells are diode-connected via a multiplexer 1212 (partially shown), through which current inputs flow via BLR0, BLR1, BLR2, and BLR3. Each multiplexer 1212 includes a corresponding multiplexer 1205 and a common-source cascode transistor 1204 to ensure that the voltage on the bit lines (such as BLR0) of each of the first and second non-volatile reference memory cells remains constant during read operations. The reference cells are tuned to a target reference level.

[0086] Memory array 1203 serves two purposes. First, it stores weights that will be used by VMM array 1200. Second, memory array 1203 efficiently multiplies the inputs (current inputs provided to terminals BLR0, BLR1, BLR2, and BLR3, which reference arrays 1201 and 1202 convert into input voltages to supply to control gates (CG0, CG1, CG2, and CG3)) by the weights stored in the memory array, and then sums all the results (cell currents) to produce an output that appears on BL0 to BLN and will be the input to the next layer or the final layer. By performing multiplication and addition functions, the memory array eliminates the need for separate multiplication and addition logic circuits and is also highly efficient. Here, the inputs are provided on control gate lines (CG0, CG1, CG2, and CG3), and the outputs appear on bit lines (BL0-BLN) during read operations. The currents placed on each bit line perform a summation function of all currents from the memory cells connected to that particular bit line.

[0087] The VMM array 1200 performs unidirectional tuning of the non-volatile memory cells in the memory array 1203. That is, each non-volatile memory cell is erased and then partially programmed until the desired charge is reached on the floating gate. If too much charge is placed on the floating gate (causing an incorrect value to be stored in the cell), the cell is erased, and the sequence of partial programming operations restarts. As shown, two rows sharing the same erase gate (such as EG0 or EG1) are erased together (this is called page erasure), and thereafter, each cell is partially programmed until the desired charge is reached on the floating gate.

[0088] Table 7 depicts the operating voltages and currents used for the VMM array 1200. The columns in the table indicate the voltages applied to the word lines for selected cells, word lines for unselected cells, bit lines for selected cells, bit lines for unselected cells, control gates for selected cells, control gates for unselected cells in the same sector as the selected cell, control gates for unselected cells in different sectors from the selected cell, erase gates for selected cells, erase gates for unselected cells, source lines for selected cells, and source lines for unselected cells. Rows indicate read, erase, and program operations.

[0089] Table 7: Figure 12 Operation of VMM array 1200

[0090]

[0091] Figure 13 A neuronal VMM array 1300 is depicted, which is particularly suitable for applications such as... Figure 3 The memory cell 310 shown serves as a synapse and partial neuron between the input layer and the next layer. The VMM array 1300 includes a memory array 1303 of non-volatile memory cells, a reference array 1301 of first non-volatile reference memory cells, and a reference array 1302 of second non-volatile reference memory cells. EG lines EGR0, EG0, EG1, and EGR1 extend vertically, while CG lines CG0, CG1, CG2, and CG3 and SL lines WL0, WL1, WL2, and WL3 extend horizontally. The VMM array 1300 is similar to the VMM array 1400, except that the VMM array 1300 implements bidirectional tuning, where each individual cell can be fully erased, partially programmed, and partially erased as needed to achieve a desired amount of charge on the floating gate due to the use of separate EG lines. As shown, reference arrays 1301 and 1302 convert the input currents in terminals BLR0, BLR1, BLR2, and BLR3 into control gate voltages CG0, CG1, CG2, and CG3 to be applied to memory cells in the row direction (through the operation of reference cells connected via diodes via multiplexer 1314). The current outputs (neurons) are in bit lines BL0-BLN, where each bit line sums all currents from non-volatile memory cells connected to that particular bit line.

[0092] Table 8 depicts the operating voltages and currents used for the VMM array 1300. The columns in the table indicate the voltages applied to the word lines for selected cells, word lines for unselected cells, bit lines for selected cells, bit lines for unselected cells, control gates for selected cells, control gates for unselected cells in the same sector as the selected cell, control gates for unselected cells in different sectors from the selected cell, erase gates for selected cells, erase gates for unselected cells, source lines for selected cells, and source lines for unselected cells. Rows indicate read, erase, and program operations.

[0093] Table 8: Figure 13 Operation of VMM array 1300

[0094]

[0095] Figure 22 A neuronal VMM array 2200 was depicted, which is particularly suitable for... Figure 2 The memory unit 210 shown serves as a synapse and partial neuron between the input layer and the next layer. In the VMM array 2200, inputs INPUT0, ..., INPUT... N On bit lines BL0, ..., BL respectively N The data is received and outputs OUTPUT1, OUTPUT2, OUTPUT3 and OUTPUT4 are generated on source lines SL0, SL1, SL2 and SL3 respectively.

[0096] Figure 23 A neuronal VMM array 2300 was depicted, which is particularly suitable for... Figure 2 The memory unit 210 shown serves as a synapse and partial neuron between the input layer and the next layer. In this example, inputs INPUT0, INPUT1, INPUT2, and INPUT3 are received on source lines SL0, SL1, SL2, and SL3, respectively, and outputs OUTPUT0, ..., OUTPUT... N In position lines BL0, ..., BL N Generate above.

[0097] Figure 24 A neuronal VMM array 2400 was depicted, which is particularly suitable for applications such as... Figure 2 The memory unit 210 shown serves as a synapse and partial neuron between the input layer and the next layer. In this example, the inputs are INPUT0, ..., INPUT0. M On the word lines WL0, ..., WL respectively M The data is received and outputs OUTPUT0, ..., OUTPUT. N In position lines BL0, ..., BLN Generate above.

[0098] Figure 25 A neuronal VMM array 2500 was depicted, which is particularly suitable for applications such as... Figure 3 The memory unit 310 shown serves as a synapse and partial neuron between the input layer and the next layer. In this example, the inputs are INPUT0, ..., INPUT0. M On the word lines WL0, ..., WL respectively M The data is received and outputs OUTPUT0, ..., OUTPUT. N In position lines BL0, ..., BL N Generate above.

[0099] Figure 26 A neuronal VMM array 2600 is depicted, which is particularly suitable for applications such as... Figure 4 The memory unit 410 shown serves as a synapse and partial neuron between the input layer and the next layer. In this example, the inputs are INPUT0, ..., INPUT0. n On the vertical control grid lines CG0, ..., CG respectively N The data is received, and outputs OUTPUT1 and OUTPUT2 are generated on source lines SL0 and SL1.

[0100] Figure 27 The neuronal VMM array 2700 is depicted, which is particularly suitable for... Figure 4 The memory unit 410 shown serves as a synapse and partial neuron between the input layer and the next layer. In this example, the inputs are INPUT0, ..., INPUT0. N They are received on the gates of bit line control gates 2701-1, 2701-2, ..., 2701-(N-1) and 2701-N, respectively, and these gates are coupled to bit lines BL0, ..., BL0, respectively. N Example outputs OUTPUT1 and OUTPUT2 are generated on source lines SL0 and SL1.

[0101] Figure 28 A neuronal VMM array 2800 is depicted, which is particularly suitable for applications such as... Figure 3 The memory unit 310 shown is as follows: Figure 5 The memory cell 510 shown and as follows Figure 7 The memory unit 710 shown serves as a synapse and partial neuron between the input layer and the next layer. In this example, the inputs are INPUT0, ..., INPUT0. M In the word lines WL0, ..., WL MThe data is received and outputs OUTPUT0, ..., OUTPUT. N On bit lines BL0, ..., BL respectively N Generate above.

[0102] Figure 29 A neuronal VMM array 2900 is depicted, which is particularly suitable for applications such as... Figure 3 The memory unit 310 shown is as follows: Figure 5 The memory cell 510 shown and as follows Figure 7 The memory unit 710 shown serves as a synapse and partial neuron between the input layer and the next layer. In this example, the inputs are INPUT0, ..., INPUT0. M In the control grid lines CG0, ..., CG M The data is received. Outputs are OUTPUT0, ..., OUTPUT. N On the vertical source lines SL0, ..., SL respectively N The above is generated, where each source line SL i The source line coupled to all memory cells in column i.

[0103] Figure 30 A neuronal VMM array 3000 is depicted, which is particularly suitable for applications such as... Figure 3 The memory unit 310 shown is as follows: Figure 5 The memory cell 510 shown and as follows Figure 7 The memory unit 710 shown serves as a synapse and partial neuron between the input layer and the next layer. In this example, the inputs are INPUT0, ..., INPUT0. M In the control grid lines CG0, ..., CG M The data is received. Outputs are OUTPUT0, ..., OUTPUT. N On the vertical lines BL0, ..., BL respectively N Generate on, where each bit line BL i Bit lines coupled to all memory cells in column i.

[0104] Long Short-Term Memory

[0105] Existing technologies include a concept known as Long Short-Term Memory (LSTM). LSTM units are commonly used in neural networks. LSTM allows neural networks to remember information for a predetermined arbitrary time interval and use that information in subsequent operations. A typical LSTM unit includes a cell, an input gate, an output gate, and a forget gate. The three gates regulate the flow of information into and out of the cell and the time interval at which information is remembered in the LSTM. VMMs are particularly useful in LSTM units.

[0106] Figure 14 An example LSTM 1400 is depicted. This example LSTM 1400 includes cells 1401, 1402, 1403, and 1404. Cell 1401 receives the input vector x0 and generates the output vector h0 and the cell state vector c0. Cell 1402 receives the input vector x1, the output vector (hidden state) h0 from cell 1401, and the cell state c0 from cell 1401, and generates the output vector h1 and the cell state vector c1. Cell 1403 receives the input vector x2, the output vector (hidden state) h1 from cell 1402, and the cell state c1 from cell 1402, and generates the output vector h2 and the cell state vector c2. Cell 1404 receives the input vector x3, the output vector (hidden state) h2 from cell 1403, and the cell state c2 from cell 1403, and generates the output vector h3. Additional cells can be used, and this four-cell LSTM is merely an example.

[0107] Figure 15 Depicting what can be used Figure 14 The following is a specific example implementation of LSTM cell 1500, comprising cells 1401, 1402, 1403, and 1404. LSTM cell 1500 receives an input vector x(t), a cell state vector c(t-1) from the previous cell, and an output vector h(t-1) from the previous cell, and generates the cell state vector c(t) and the output vector h(t).

[0108] LSTM unit 1500 includes sigmoid function devices 1501, 1502, and 1503, each applying a number between 0 and 1 to control how much of each component in the input vector is allowed to pass through to the output vector. LSTM unit 1500 also includes tanh devices 1504 and 1505 for applying a hyperbolic tangent function to the input vector, multiplier devices 1506, 1507, and 1508 for multiplying two vectors together, and adder device 1509 for adding two vectors together. The output vector h(t) can be provided to the next LSTM unit in the system, or it can be accessed for other purposes.

[0109] Figure 16LSTM unit 1600 is depicted, which is an example of a specific implementation of LSTM unit 1500. For the reader's convenience, the same numbering is used in LSTM unit 1600 as in LSTM unit 1500. Sigmoid function devices 1501, 1502, and 1503, and tanh device 1504 each include multiple VMM arrays 1601 and activation function blocks 1602. Thus, it can be seen that VMM arrays are particularly useful in LSTM units used in certain neural network systems. Multiplier devices 1506, 1507, and 1508, and adder device 1509 are implemented digitally or analogically. Activation function blocks 1602 can be implemented digitally or analogously.

[0110] Alternative forms of the LSTM unit 1600 (and another example of a specific implementation of the LSTM unit 1500) are in Figure 17 As shown in [the image]. Figure 17 In this LSTM unit, sigmoid function devices 1501, 1502, and 1503, and tanh device 1504 share the same physical hardware (VMM array 1701 and activation function block 1702) in a time-division multiplexing manner. The LSTM unit 1700 also includes a multiplier device 1703 for multiplying two vectors together, an adder device 1708 for adding two vectors together, a tanh device 1505 (which includes activation function block 1702), a register 1707 for storing the value i(t) when it is output from sigmoid function block 1702, and a register 1707 for storing the value f(t). When c(t-1) is output from multiplier device 1703 via multiplexer 1710, register 1704 stores the value of i(t). When u(t) is output from multiplier device 1703 via multiplexer 1710, register 1705 stores the value of u(t), and register 1705 is used to store the value of o(t). When c~(t) is output from multiplier device 1703 via multiplexer 1710, register 1706 and multiplexer 1709 store the value.

[0111] LSTM unit 1600 contains multiple sets of VMM arrays 1601 and corresponding activation function blocks 1602, while LSTM unit 1700 contains a set of VMM arrays 1701 and activation function blocks 1702, which are used to represent multiple layers in the example of LSTM unit 1700. LSTM unit 1700 will require less space than LSTM 1600 because LSTM unit 1700 will require 1 / 4 of its space for VMMs and activation function blocks compared to LSTM unit 1600.

[0112] It is also understood that an LSTM cell will typically comprise multiple VMM arrays, each using functionality provided by certain circuit blocks outside the VMM array itself, such as summer and activation function blocks, and high-voltage generation blocks. Providing a separate circuit block for each VMM array would require a significant amount of space within the semiconductor device and would be inefficient to some extent. Therefore, the examples described below reduce the amount of circuitry present outside the VMM arrays themselves.

[0113] Gate control recursive unit

[0114] A simulated VMM implementation can be used in gated recurrent unit (GRU) systems. A GRU is a gated mechanism in a recurrent neural network. A GRU is similar to an LSTM, but a GRU unit generally contains fewer components than an LSTM unit.

[0115] Figure 18 An example GRU 1800 is depicted. This example GRU 1800 includes units 1801, 1802, 1803, and 1804. Unit 1801 receives an input vector x0 and generates an output vector h0. Unit 1802 receives an input vector x1, an output vector h0 from unit 1801, and generates an output vector h1. Unit 1803 receives an input vector x2 and an output vector (in a hidden state) h1 from unit 1802 and generates an output vector h2. Unit 1804 receives an input vector x3 and an output vector (in a hidden state) h2 from unit 1803 and generates an output vector h3. Additional units can be used, and this four-unit GRU is merely an example.

[0116] Figure 19 Depicting what can be used Figure 18 Examples of specific implementations of GRU unit 1900 are provided for units 1801, 1802, 1803, and 1804. GRU unit 1900 receives an input vector x(t) and an output vector h(t-1) from a previous GRU unit, and generates an output vector h(t). GRU unit 1900 includes sigmoid function devices 1901 and 1902, each applying a number between 0 and 1 to the components from the output vector h(t-1) and the input vector x(t). GRU unit 1900 also includes a tanh device 1903 for applying a hyperbolic tangent function to the input vector, multiple multiplier devices 1904, 1905, and 1906 for multiplying two vectors together, an adder device 1907 for adding two vectors together, and a complementary device 1908 for subtracting the input from 1 to generate the output.

[0117] Figure 20GRU unit 2000 is depicted, which is an example of a specific implementation of GRU unit 1900. For the reader's convenience, the same numbering is used in GRU unit 2000 as in GRU unit 1900. Figure 20 As can be seen, sigmoid function devices 1901 and 1902, and tanh device 1903 each include multiple VMM arrays 2001 and activation function blocks 2002. Therefore, it can be seen that VMM arrays are particularly useful in GRU units used in some neural network systems. Multiplier devices 1904, 1905, and 1906, adder device 1907, and complement device 1908 are implemented digitally or analogically. Activation function blocks 2002 can be implemented digitally or analogously.

[0118] Alternative forms of the GRU unit 2000 (and another example of a specific implementation of the GRU unit 1900) in Figure 21 As shown in [the image]. Figure 21 In this configuration, the GRU unit 2100 utilizes a VMM array 2101 and an activation function block 2102, which, when configured as a sigmoid function, applies numbers between 0 and 1 to control how much of each component in the input vector is allowed to pass through to the output vector. Figure 21 In this configuration, sigmoid function devices 1901 and 1902 and tanh device 1903 share the same physical hardware (VMM array 2101 and activation function block 2102) in a time-division multiplexing manner. The GRU unit 2100 also includes a multiplier device 2103 for multiplying two vectors together, an adder device 2105 for adding two vectors together, a complementary device 2109 for subtracting the input from 1 to generate the output, a multiplexer 2104, and a multiplexer for... (the sentence is incomplete and requires further context to translate accurately). The register 2106, which holds the value of r(t) when it is output from the multiplier device 2103 via the multiplexer 2104, is used when the value h(t-1) is... The register 2107 holds the value of z(t) when it is output from the multiplier device 2103 via the multiplexer 2104, and is used when the value h^(t) is... (1-z(t)) is output from the multiplier device 2103 via the multiplexer 2104 and is held in register 2108.

[0119] GRU unit 2000 contains multiple sets of VMM arrays 2001 and activation function blocks 2002, while GRU unit 2100 contains a set of VMM arrays 2101 and activation function blocks 2102, which are used to represent multiple layers in the example of GRU unit 2100. GRU unit 2100 will require less space than GRU unit 2000 because GRU unit 2100 will require 1 / 3 of its space for VMMs and activation function blocks compared to GRU unit 2000.

[0120] It is also understandable that a GRU system would typically include multiple VMM arrays, each using functionality provided by certain circuit blocks outside the VMM array itself, such as summer and activation function blocks, and high-voltage generation blocks. Providing a separate circuit block for each VMM array would require a significant amount of space within the semiconductor device and would be inefficient to some extent. Therefore, the examples described below reduce the amount of circuitry present outside the VMM arrays themselves.

[0121] converter

[0122] Existing technologies include a concept called a transformer neural network, which is an alternative architecture for neural networks. Figure 63 An example of a Transformer Neural Network 6300 is depicted. Transformer neural networks are discussed in "Attention is All You Need" by Ashish Vaswani et al. The transformer connects the encoder and decoder through an attention function. An example of the attention function is:

[0123] Attention(Q,K,V) = softmax(QK) T / V

[0124] It is worth noting that transformer neural networks can be implemented using VMMs. Specifically, VMMs can be used to perform multiplication (Q... K T ) V, where Q is the query matrix, K is the key matrix, and V is the value matrix, where each value forms a key-value pair with a key in K. Optionally, VMM is a dynamic matrix multiplier.

[0125] Other architectural aspects

[0126] The input to a VMM array can be analog level, binary level, pulse, time-modulated pulse, or digital bit (in which case a DAC is used to convert the digital bit to the appropriate input analog level), and the output can be analog level, binary level, timed pulse, pulse, or digital bit (in which case an output ADC is used to convert the output analog level to digital bit).

[0127] Generally, for each memory cell in a VMM array, each weight W can be implemented by a single memory cell, a differential cell, or a hybrid memory cell (the average of two cells). In the case of differential cells, two memory cells are used to implement the weight W as a differential weight (W = W+ - W-). In the case of two hybrid memory cells, two memory cells are used to implement the weight W as the average of the two cells.

[0128] Figure 31 A VMM system 3100 is depicted. In some examples, the weights W stored in the VMM array are stored as differential pairs W+ (positive weights) and W- (negative weights), where W = (W+) - (W-). In VMM system 3100, half of the bit lines are designated as W+ lines, i.e., bit lines connected to memory cells that will store positive weights W+, and the other half of the bit lines are designated as W- lines, i.e., bit lines connected to memory cells that implement negative weights W-. The W- lines are distributed alternately among the W+ lines. The subtraction operation is performed by summing circuits (such as summing circuits 3101 and 3102), which receive current from the W+ and W- lines. The outputs of the W+ and W- lines are combined to effectively give W = W+ - W- for each (W+, W-) cell pair of all (W+, W-) line pairs. Although the W- lines, which are alternately distributed between the W+ lines, have been described above, in other examples, the W+ and W- lines can be located anywhere in the array.

[0129] Figure 32 Another example is depicted. In the VMM system 3210, positive weights W+ are implemented in a first array 3211 and negative weights W- are implemented in a second array 3212, which is separate from the first array, and the resulting weights are appropriately combined by a summing circuit 3213.

[0130] Figure 33A VMM system 3300 is depicted. Weights W stored in the VMM array are stored as differential pairs W+ (positive weights) and W- (negative weights), where W = (W+) - (W-). The VMM system 3300 includes arrays 3301 and 3302. Half of the bit lines in each of arrays 3301 and 3302 are designated as W+ lines, i.e., bit lines connected to memory cells that will store positive weights W+, and the other half of the bit lines in each of arrays 3301 and 3302 are designated as W- lines, i.e., bit lines connected to memory cells that implement negative weights W-. The W- lines are distributed alternately among the W+ lines. Subtraction operations are performed by summing circuits (such as summing circuits 3303, 3304, 3305, and 3306) that receive current from the W+ and W- lines. The outputs of the W+ and W- lines from each array 3301, 3302 are combined to effectively give W = W+ - W- for each (W+, W-) cell pair of all (W+, W-) line pairs. Furthermore, the W values ​​from each array 3301 and 3302 can be further combined by summing circuits 3307 and 3308 such that each W value is the result of subtracting the W value from array 3302 from the W value from array 3301. This means that the final result from summing circuits 3307 and 3308 is the difference between two differences.

[0131] Artificial neural networks typically operate on large amounts of data and perform numerous computations involving vector-matrix multiplication. This necessitates providing many arrays for vector-matrix multiplication within a relatively small space. The applicant previously filed U.S. Patent Application No. 17 / 848,371, filed June 23, 2022, entitled "Artificial Neural Network Comprising a Three-Dimensional Integrated Circuit," which is incorporated herein by reference. This application describes aspects of three-dimensional integrated circuits that can be used to implement artificial neural networks. Some of these aspects will be repeated herein.

[0132] Figure 34A block diagram of a VMM system 3400 is depicted. The VMM system 3400 includes a VMM array 3401, a row decoder 3402, a high-voltage decoder 3403, a column decoder 3404, a bitline driver 3405, input circuitry 3406, output circuitry 3407, control logic 3408, and a bias generator 3409. The VMM system 3400 further includes a high-voltage generation block 3410, which includes a charge pump 3411, a charge pump regulator 3412, and a high-voltage analog precision level generator 3413. The VMM system 3400 further includes a (programming / erasing, or weighted tuning) algorithm controller 3414, analog circuitry 3415, a control engine 3416 (which may include, but is not limited to, functions such as arithmetic functions, activation functions, embedded microcontroller logic, etc.), and test control logic 3417.

[0133] Input circuitry 3406 may include circuitry such as a DAC (digital-to-analog converter), a DPC (digital-to-pulse converter, digital-to-time modulated pulse converter), an AAC (analog-to-analog converter, such as a current-to-voltage converter, logarithmic converter), a PAC (pulse-to-analog level converter), or any other type of converter. Input circuitry 3406 may implement one or more of a normalization, linear, or nonlinear up / down scaling function or arithmetic function. Input circuitry 3406 may implement a temperature compensation function for the input level. Input circuitry 3406 may implement an activation function such as ReLU or sigmoid.

[0134] Output circuitry 3407 may include circuitry such as an ADC (analog-to-digital converter, used to convert analog neuron outputs into digital bits), an AAC (analog-to-analog converter, such as a current-to-voltage converter, a logarithmic converter), an APC (analog-to-pulse converter, analog-to-time-modulated pulse converter), or any other type of converter. Output circuitry 3407 may implement activation functions such as a modified linear activation function (ReLU) or a sigmoid function. Output circuitry 3407 may implement one or more of the following for neuron outputs: statistical normalization, regularization, up / down scaling / gain functions, statistical rounding, or arithmetic functions (e.g., addition, subtraction, division, multiplication, shifting, logarithmic). Output circuitry 3407 may implement temperature compensation functions for neuron outputs or array outputs (such as bitline outputs) to keep the array's power consumption approximately constant, or to improve the accuracy of the array (neuron) outputs, such as by keeping the IV slope approximately the same.

[0135] 3D VMM System Architecture

[0136] Figure 35A 3D VMM system 3500 is depicted, comprising multiple dies such as dies 3501, 3502, 3503, 3504, 3505, and 3506, which are vertically stacked within a package 3522 to form a packaged integrated circuit. The 3D VMM system 3500 includes certain functional blocks that are functionally similar to those contained within... Figure 34 The blocks are located in the VMM system 3400, but these blocks may be located on different dies. Here, the components contained in dies 3501 and 3502 share the components contained in dies 3503, 3504, 3505 and 3506.

[0137] In this example, die 3501 contains a corresponding VMM array 3507 (functionally similar to...) Figure 34 The VMM 3401, the corresponding input multiplexer 3509, the corresponding row buffer 3523 (which can, for example, provide sample-and-hold buffer voltage to the array input), and the corresponding high-voltage decoder 3508 (functionally similar to...) Figure 34 The system includes a high-voltage decoder 3403 and a corresponding neuron circuit 3510 (which may, for example, perform scaling functions, minimum / maximum limit functions, differential output conversion, buffering, etc., for the array output current). An input multiplexer 3509 receives analog input signals and applies them to the VMM array 3507, and the neuron circuit 3510 receives analog output signals representing the neuron outputs from the VMM array 3507. The output signals can be transmitted to other blocks within the 3D VMM system 3500.

[0138] The bare die 3502 also includes a corresponding VMM array 3507, a corresponding input multiplexer 3509, a corresponding row buffer 3522, a corresponding high-voltage decoder 3508, and a corresponding neuron circuit 3510.

[0139] In this example, the two dies (dies 3501 and 3502) contain the corresponding VMM array 3507, but it should be understood that additional dies containing the corresponding VMM array may be included.

[0140] The bare die 3503 includes a high-voltage generator 3511 (functionally similar to...) Figure 34 The high-voltage generation block 3410 and the analog circuit 3512 (functionally similar to) are mentioned. Figure 34The 3D VMM system 3500 includes analog circuitry 3415 and temperature compensation circuitry 3513. The 3D VMM system 3500 may have thermal challenges not present in the 2D VMM system 3400. Because the dies 3501, 3502, 3503, 3504, 3505, and 3506 are stacked in a vertical configuration and contain different types of circuitry, each die may experience different thermal operating conditions during operation. For example, one die may become hotter than others, and the rate of temperature increase may vary between dies. This introduces the possibility of inaccuracies due to thermal variations. The temperature compensation circuitry 3513 compensates for the temperature variations experienced between the individual dies. Optionally, one or more thermal sensors are located on each respective die to provide temperature data to the temperature compensation circuitry 3513. The temperature compensation circuitry 3513 then adjusts the tuning or configuration settings to compensate for any temperature variations. The temperature compensation circuitry 3513 also compensates for temperature variations in each die, for example, to compensate for variations in cell current with temperature, such as making the resulting bitline (neuron) current approximately the same with temperature. Temperature compensation circuit 3513 is also used in neuron circuit 3510, DAC and ADC circuits so that the operating dynamic range (e.g., the output range of the DAC, the input range of the ADC, and the output range of the neuron circuit 3510) is approximately the same with temperature.

[0141] The die 3504 includes input circuitry 3514 (functionally similar to input circuitry 3406), which includes address decoding circuitry 3524, row register 3525 (holding the active input value of the array row), and digital-to-analog converter (DAC) 3515. DAC 3515 receives digital signals from row register 3525 and converts them into analog signals.

[0142] The die 3505 contains an analog-to-digital converter (ADC) 3516. The ADC 3516 receives analog signals and converts them into digital signals.

[0143] Die 3506 includes digital circuitry 3517, static random access memory (SRAM) 3518, registers 3519, physical I / O connections 3520, a digital accelerator 3531, and a network on-chip (NOC) 3715. Die 3506 provides control functions for other dies. Digital circuitry 3517 may include digital logic, a microcontroller, a SIMD (Single Instruction Multiple Data) processor, and a processor. SRAM 3518 and registers 3519 can be used to store system and configuration information used by digital circuitry 3517 or other circuitry or blocks within the 3D VMM system 3500. Physical I / O connections 3520 provide I / O interfaces to devices outside the VMM system 3500, such as external processing units, or another package 3522. Digital accelerator 3531 is used for certain neural networks or layers within neural networks that may require additional processing, such as when small activation sizes exist, where the weights stored in the cells are dynamic rather than fixed, where MAC operations need to be performed, but is not limited to this. NOC 3715 provides network routing functionality within the 3D VMM system 3500, for example, by generating control signals to route signals from one block to another.

[0144] A corresponding die among a plurality of dies is connected to one or more other dies within the plurality of dies via a vertical direct port 3521, which connects two or more dies together respectively. In one example, the vertical direct port 3521 is implemented as a through-silicon via (TSV).

[0145] During readout operations of the 3D VMM system 3500, digital input is received by input circuitry 3514. A digital input enable row register 3525 stores the active input and, in response to the digital input, applies a selected active input to a DAC 3515, which converts the digital output of register 3525 into a corresponding analog signal. The analog signal generated by DAC 3515 is provided by input circuitry 3514 to one or more vertical ports 3521 on an input multiplexer 3509 and a row buffer 3523 on dies 3501, 3502. The row buffer then applies the signal to one or more rows in the corresponding VMM array 3507, resulting in an output generated by the VMM array 3507. The output from the corresponding VMM array 3507 is received by a corresponding neural circuit 3510, which provides buffering to drive the parasitic capacitances of the one or more vertical ports 3521 to which it is connected. The neuron circuit 3510 provides analog signals to the ADC 3516 on the die 3505 via one or more vertical direct ports 3521, which converts the analog signals into digital signals. Alternatively, the analog signals may bypass the ADC 3516 and remain in analog form. The output of the ADC 3516 is provided to devices outside the 3D VMM system 3500 (such as processing units or graphics processing units) via physical I / O 3520, or applied as input to the corresponding VMM array 3507 (representing another layer in the artificial neural network). Alternatively, the analog signals from the neuron circuit 3510 may bypass the ADC 3516 and remain in analog form and be applied as input to the corresponding VMM array 3507.

[0146] Figure 36 A 3D VMM system 3600, including package 3622, is described. The 3D VMM system 3600 is similar to the 3D VMM system 3500 and contains many of the same components, except for some differences in the arrangement of components and certain additional components. Figure 35 The same project contains the same Figure 36 The same project number. Here, the components contained in dies 3601 and 3602 share the components contained in dies 3603, 3604, 3605 and 3606.

[0147] The 3D VMM system 3600 includes multiple dies, such as dies 3601, 3602, 3603, 3604, 3605 and 3606, which are vertically stacked within a common package 3522 to form a packaged integrated circuit.

[0148] In this example, die 3601 includes a corresponding VMM array 3507, a corresponding input multiplexer 3509, a corresponding register 3524 (holding the active input value of the array row), a corresponding row buffer 3523, a corresponding high-voltage multiplexer 3608, and a corresponding column multiplexer 3610.

[0149] The bare die 3602 also includes a corresponding VMM array 3507, a corresponding input multiplexer 3509, a corresponding register 3524, a corresponding row buffer 3523, a corresponding high-voltage multiplexer 3608, and a column multiplexer 3610.

[0150] In this example, the two dies (dies 3601 and 3602) contain the corresponding VMM array 3507, but it should be understood that additional dies containing VMM arrays may be included.

[0151] The bare die 3603 includes a high voltage generator 3511, an analog circuit 3512, a temperature compensation circuit 3513, and a high voltage multiplexer 3608.

[0152] The bare die 3604 includes input circuitry 3614, which includes address decoding 3524 and DAC 3515.

[0153] The bare die 3605 includes a neuron circuit 3510 and an ADC 3516.

[0154] The die 3606 includes digital circuitry 3517, SRAM 3518, registers 3519, and physical I / O connections 3520.

[0155] A corresponding die among a plurality of dies is connected to one or more other dies within the plurality of dies via a corresponding vertical direct port 3521.

[0156] During the read operation of the 3D VMM system 3600, input circuitry 3614 receives input and an address. Address decoder 3524 decodes the address and provides input to the corresponding row register 3525 corresponding to the decoded address via one or more vertical ports 3521. The output of the corresponding row register 3525 is coupled to DAC 3515 via one or more vertical ports 3521, which converts the digital output bits received from the row register 3525 into analog signals and provides analog signals to the corresponding input multiplexer 3509 and row buffer 3523 via one or more vertical ports 3521. Row buffer 3523 applies the buffered analog signals to the row inputs (such as array control gates or word lines) of the corresponding VMM array 3507 for the selected row. Outputs from the respective VMM array 3507 (such as from array bit lines) are received by the respective column multiplexer 3610, and the output of the respective column multiplexer 3610 provides a signal to the ADC 3516 on die 3605 via one or more vertical ports 3521, which converts analog signals into digital signals. Alternatively, the signal may bypass the ADC 3516 and remain in analog form. The output of the ADC 3516 is provided to digital circuitry 3517 on die 3606 (which may perform activation functions, pooling functions, or other network functions) via one or more vertical ports 3521, and the output of digital circuitry 3517 may be provided to devices outside the 3D VMM system 3600 (such as processing units or graphics processing units) via physical I / O 3520 on die 3606, or provided to input circuitry 3614 on die 3604 as input to another respective VMM array 3507 (representing another layer in an artificial neural network) via the respective vertical port 3521.

[0157] Figure 37AA 3D VMM system 3700 is depicted, comprising multiple dies in two or more vertical stacks. In this example, a first vertical stack includes dies 3701, 3702, 3703, 3704, 3705, and 3706, and a second vertical stack includes dies 3707, 3708, 3709, 3710, 3711, and 3712, all of which are contained in a common package 3522 to form a single packaged integrated circuit. In one example, the dies in the first vertical stack are physically separate from the dies in the second vertical stack. In another example, the dies in the first and second vertical stacks are the same physical dies (meaning, for example, dies 3701 and 3707 are the same dies). Here, the components contained in dies 3701 and 3702 share the components contained in dies 3703, 3704, 3705 and 3706, and the components contained in dies 3707 and 3708 share the components contained in dies 3709, 3710, 3711 and 3712.

[0158] In the example shown, die 3701 and die 3707 each include a corresponding VMM array 3507, a corresponding array input 3729 (which includes a corresponding input multiplexer 3509, a corresponding address decoder 3713, a corresponding row register 3525 and a corresponding row buffer 3523), a corresponding high-voltage multiplexer 3608 and a corresponding neuron circuit 3510.

[0159] Die 3702 and die 3708 also each include a corresponding VMM array 3507, a corresponding array input 3729 (which includes a corresponding input multiplexer 3509, a corresponding address decoder 3713, a corresponding row register 3525 and a corresponding row buffer 3523), a corresponding high-voltage multiplexer 3608 and a corresponding neuron circuit 3510.

[0160] In this example, the four dies (dies 3701, 3702, 3707 and 3708) contain the corresponding VMM array 3507, but it should be understood that additional dies containing VMM arrays may be included.

[0161] The bare die 3703 and bare die 3709 each contain a corresponding high voltage decoder 3714, a corresponding high voltage generator 3511, a corresponding analog circuit 3512, and a corresponding temperature compensation circuit 3513.

[0162] Both die 3704 and die 3710 contain corresponding input circuits 3514, which include corresponding DACs 3515.

[0163] Both the bare die 3705 and bare die 3711 contain a corresponding ADC 3516.

[0164] Die 3706 and die 3712 each include a corresponding digital circuit 3517, a corresponding SRAM 3518, a corresponding register 3519, a corresponding physical I / O connection 3520, and a corresponding NOC (Network on Chip) connection 3715.

[0165] A corresponding die among a plurality of dies is connected to one or more other dies within the plurality of dies via one or more vertical direct ports 3521 or horizontal interfaces 3716, which respectively connect two or more dies together. In one example, the vertical direct port 3521 is correspondingly a through-silicon via (TSV). In one example, the horizontal interface 3716 is correspondingly a redistribution layer (RDL) connector.

[0166] During readout operations of the 3D VMM system 3700, digital inputs are received by corresponding input circuitry 3514, which converts the digital inputs into analog signals via its corresponding DAC 3515. The output of the corresponding DAC 3515 is coupled to the row register 3525 of the corresponding array input 3729 via a corresponding vertical direct port 3521 and / or horizontal interface 3716. The output of the row register 3525 of the corresponding array input 3729 is provided to an input multiplexer 3509 and a row buffer 3523, which then applies the signal to one or more rows in the VMM array 3507. The output from the corresponding VMM array 3507 is received by corresponding neural circuitry 3510, which provides buffering to drive the parasitic capacitances of the one or more vertical direct ports 3521 or horizontal interfaces 3716 to which it is connected. The neuron circuit 3510 provides a buffered analog signal to the corresponding ADC 3516 via one or more vertical direct ports 3521 or horizontal interfaces 3716, which converts the analog signal into a digital signal. The output of the ADC 3516 is provided to the corresponding digital circuit 3517 (which executes an activation function, pooling function, or network function), and the output of the corresponding digital circuit 3517 can be provided to a device external to the 3D VMM system 3700 (such as a processing unit or graphics processing unit) via the corresponding physical I / O 3520, or to the corresponding input circuit 3514 for conversion by the corresponding DAC 3515, and the output of the corresponding input circuit 3514 is coupled to another VMM array 3507 (representing another layer in the artificial neural network) or coupled to another package 3522 via the corresponding physical I / O 3520.

[0167] Figure 37BA 3D VMM system 3750 is depicted, which is similar to the 3D VMM system 3750, except that it has another type of VMM array shown as VMM array 3557 on die 3758, which includes static RAM cells or dynamic RAM cells.

[0168] Figure 38 A 3D VMM system 3800 is depicted, comprising multiple dies in two vertical stacks. More than two stacks may be present. In this example, the first vertical stack includes dies 3801, 3802, 3803, 3804, 3805, and 3806, and the second vertical stack includes dies 3807, 3808, 3809, 3810, 3811, and 3812. All these dies are contained within a common package 3522 to form a single packaged integrated circuit. In one example, the dies in the first vertical stack are physically separate from the dies in the second vertical stack. In another example, the dies in the first and second vertical stacks are the same physical dies (meaning, for example, dies 3801 and 3807 are the same dies). Here, the components contained in dies 3801 and 3802 share the components contained in dies 3803, 3804, 3805 and 3806, and the components contained in dies 3807 and 3808 share the components contained in dies 3809, 3810, 3811 and 3812.

[0169] In the example shown, dies 3801, 3802, 3807, and 3808 each include a corresponding VMM array 3507, a corresponding array input 3729, a corresponding high-voltage multiplexer 3608, a corresponding column multiplexer 3610, and a corresponding array input circuit 3729. The corresponding array input 3729 includes an input multiplexer 3509, an address decoder 3713, a row register 3524, and a row buffer 3523.

[0170] In this example, the four dies (dies 3801, 3802, 3807 and 3808) contain VMM arrays, but it should be understood that additional dies containing VMM arrays may be included.

[0171] The bare die 3803 and bare die 3809 respectively include a high voltage decoder 3714, a high voltage generator 3511, an analog circuit 3512, and a temperature compensation circuit 3513.

[0172] Both die 3804 and die 3810 contain input circuitry 3514, including DAC 3515.

[0173] The bare die 3805 and bare die 3811 respectively contain an ADC 3516 and a neuron circuit 3510.

[0174] Die 3806 and die 3812 each include digital circuit 3517, SRAM 3518, register 3519, physical I / O connection 3520 and NOC connection 3715.

[0175] Multiple dies are connected to one or more other dies within the plurality of dies via one or more vertical direct ports 3521 or horizontal interfaces 3716, which respectively connect two or more dies together. In one example, the corresponding vertical direct port 3521 is implemented as a through-silicon via (TSV). In one example, the corresponding horizontal interface 3716 is implemented as a redistribution layer (RDL) connector.

[0176] During read operations of the 3D VMM system 3800, digital input is received by input circuitry 3514, which uses DAC 3515 to convert the digital input into analog form and provides analog signals to corresponding array input circuitry 3729 via respective vertical ports 3521 and / or horizontal interfaces 3716. Array input circuitry 3729 receives the analog signals and an address from the input circuitry and then applies the analog signals to selected rows in the VMM array 3507 in response to the address. Output from the VMM array 3507 is received by column multiplexer 3610, which provides analog signals to ADC 3516 via one or more vertical ports 3521 or horizontal interfaces 3716, which converts the analog signals into digital signals. Alternatively, the analog signals may bypass ADC 3516 and remain in analog form. The output of the ADC 3516 is provided to the digital circuit 3517 (which can perform activation functions, pooling functions, or other network functions) through one or more vertical direct ports 3521 or horizontal interfaces 3716, and the output of the digital circuit 3517 is provided to a device outside the 3D VMM system 3800 (such as a processing unit or graphics processing unit) through physical I / O 3520, or to the input circuit 3514 of another VMM array 3507 (representing another layer in an artificial neural network), or to another package 3522 through physical I / O 3520.

[0177] Figure 39AA 3D VMM system 3900 is depicted, comprising multiple dies in two vertical stacks. In this example, the first vertical stack includes dies 3901, 3902, 3903, and 3904, and the second vertical stack includes dies 3905, 3906, 3907, and 3908. All these dies are contained in a common package 3522 to form a single packaged integrated circuit. In one example, the dies in the first vertical stack are physically separate from the dies in the second vertical stack. In another example, the dies in the first and second vertical stacks are the same physical dies (meaning, for example, dies 3901 and 3905 are the same dies). Here, components contained in dies 3901 and 3902 share components contained in dies 3903 and 3904, and components contained in dies 3905 and 3906 share components contained in dies 3907 and 3908.

[0178] In the example shown, dies 3901, 3902, 3905 and 3906 respectively include a VMM array 3507, an array input 3729, a high-voltage multiplexer 3608 and a column multiplexer 3610.

[0179] In this example, the four dies (dies 3901, 3902, 3903 and 3904) contain VMM arrays, but it should be understood that additional dies containing VMM arrays may be included.

[0180] Diesel 3903 and die 3907 respectively include a high-voltage decoder 3714, a high-voltage generator 3511, an analog circuit 3512, a temperature compensation circuit 3513, an input circuit 3514 including a DAC 3515, a neuron circuit 3510, and an ADC 3516.

[0181] Die 3904 and die 3908 respectively include digital circuit 3517, SRAM 3518, register 3519, physical I / O connection 3520 and NOC connection 3715.

[0182] Multiple bare dies are connected to one or more other bare dies within the multiple bare dies via one or more vertical direct ports 3521 or horizontal interfaces 3716, wherein the one or more vertical direct ports or horizontal interfaces connect two or more bare dies together.

[0183] During read operations of the 3D VMM system 3900, digital input is received by input circuitry 3514, which uses DAC 3515 to convert the digital input into analog form and provides analog signals to corresponding array input circuitry 3729 via respective vertical ports 3521 and / or horizontal interfaces 3716. Array input circuitry 3729 receives analog signals and addresses from the input circuitry and applies analog signals to selected rows in the VMM array 3507 in response to the received address. Output from the VMM array 3507 is received by column multiplexer 3610, which provides analog signals to ADC 3516 via one or more vertical ports 3521 or horizontal interfaces 3716, which converts the analog signals into digital signals. Alternatively, the analog signals may bypass ADC 3516 and remain in analog form. The output of the ADC 3516 is provided to the digital circuitry 3517 via the corresponding vertical direct port 3521 and / or horizontal interface 3716, and is then provided to devices outside the 3D VMM system 3900 (such as processing units or graphics processing units) via physical I / O 3520, or to the input circuitry 3514 as input to the VMM array 3507 (representing another layer in the artificial neural network), or to another package 3522 via physical I / O 3520.

[0184] Figure 39B Depicting something similar to Figure 39A The 3D VMM system 3950 differs in that the dies 3951, 3952, 3955, and 3956 now have both input circuitry 3514 and array input 3729. The 3D VMM system 3950 includes a package 3522 and dies 3951, 3952, 3953, 3954, 3955, 3956, 3957, and 3958. Multiple dies are connected to one or more other dies within the multiple dies via one or more vertical direct ports 3521 or horizontal interfaces 3716, which respectively connect two or more dies together.

[0185] Figure 39CA 3D VMM system 3980 is depicted, comprising multiple dies in two vertical stacks. In this example, the first vertical stack includes dies 3981, 3982, and 3983, and the second vertical stack includes dies 3984, 3985, and 3986, all of which are contained in a common package 3522 to form a single packaged integrated circuit. In one example, the dies in the first vertical stack are physically separate from the dies in the second vertical stack. In another example, the dies in the first and second vertical stacks are the same physical dies (meaning, for example, dies 3981 and 3984 are the same dies). Here, components contained in dies 3981 and 3982 share components contained in die 3983, and components contained in dies 3984 and 3985 share components contained in die 3986.

[0186] In the example shown, dies 3981, 3982, 3984, and 3985 each include a VMM array 3507, a high-voltage block 3991, an input block 3990, an output block 3992, and an analog block 3993. Input block 3990 may include input circuitry 3514, a DAC 3515, and an array input circuitry 3729. High-voltage block 3991 may include a high-voltage multiplexer 3608 and a high-voltage decoder 3714. Output block 3992 may include a column multiplexer 3610, a neural network circuit 3510, and an ADC 3516. Analog block 3993 may include a high-voltage generator 3511, analog circuitry 3512, and temperature compensation circuitry 3513.

[0187] Die 3983 and die 3986 each include digital circuitry 3517, SRAM 3518, register 3519, physical I / O connection 3520, digital accelerator 3521 (for digital multiplex and accumulation (MAC) function) and NOC connection 3715.

[0188] Multiple dies are connected to one or more other dies within the multiple dies via one or more vertical direct ports 3521 or horizontal interfaces 3716, each of which connects two or more dies together.

[0189] Figure 40A 3D VMM system 4000 is depicted, comprising multiple dies in two vertical stacks. In this example, the first vertical stack includes dies 4001, 4002, 4003, and 4004, and the second vertical stack includes dies 4005, 4006, 4007, and 4008. All these dies are contained in a common package 3522 to form a single packaged integrated circuit. In one example, the dies in the first vertical stack are physically separate from the dies in the second vertical stack. In another example, the dies in the first and second vertical stacks are the same physical dies (meaning, for example, dies 4001 and 4005 are the same dies). Here, components contained in dies 4001 and 4002 share components contained in dies 4003 and 4004, and components contained in dies 4005 and 4006 share components contained in dies 4007 and 4008.

[0190] In the example shown, dies 4001, 4002, 4005 and 4006 respectively include a VMM array 3507, an array input 4029 (which includes an input multiplexer 3509 and / or a decoder 3713 - not shown), a high-voltage multiplexer 3608 and a column multiplexer 3610.

[0191] In this example, the four dies (dies 4001, 4002, 4003 and 4004) contain VMM arrays, but it should be understood that additional dies containing VMM arrays may be included.

[0192] The bare die 4003 and bare die 4007 respectively include a high voltage decoder 3714, a high voltage generator 3511, an analog circuit 3512, a temperature compensation circuit 3513, and a neuron circuit 3510.

[0193] Die 4004 and die 4008 respectively include digital circuit 3517, SRAM 3518, register 3519, physical I / O connection 3520 and NOC connection 3715.

[0194] Unlike the VMM System 3900, the VMM System 4000 does not include the DAC 3515 and ADC 3516 because the inputs and outputs remain in analog form and do not convert between analog and digital forms.

[0195] Multiple bare dies are connected to one or more other bare dies within the multiple bare dies via one or more vertical direct ports 3521 or horizontal interfaces 3716, wherein the one or more vertical direct ports or horizontal interfaces connect two or more bare dies together.

[0196] During readout operations of the 3D VMM system 4000, analog inputs (such as voltage, current, or timing-based entities, such as pulse sequences) are received by corresponding array inputs 4029, which then apply signals to one or more rows in the corresponding VMM array 3507. Outputs from the corresponding VMM array 3507 are received by column multiplexers 3610, which provide analog signals (such as voltage, current, or timing-based entities) to corresponding neural circuits 3510 via one or more vertical direct ports 3521 or horizontal interfaces 3716. These neural circuits 3510 provide buffered signals via physical I / O 3520 to devices external to the 3D VMM system 4000 (such as processing units or graphics processing units) or another package 3522, or to the array inputs 4029 of other VMM arrays 3507 (representing another layer in an artificial neural network).

[0197] Figures 41 to 44 Additional details are provided regarding an example configuration of the VMM array 3507. Figures 41 to 44 3D VMM systems 4100, 4200, 4300, and 4400 are depicted respectively. Each of the 3DD VMM systems 4100, 4200, 4300, and 4400 includes a VMM array 3507-1 on a first die and a VMM array 3507-2 on a second die in a common package (not shown). VMM arrays 3507-1 and 3507-2 each contain an array of non-volatile memory cells arranged in m+1 rows and n+1 columns, respectively. The corresponding rows are coupled to one of the control gate lines labeled CG0, ..., CGm, and the corresponding columns are coupled to one of the bit lines labeled BL0, ..., BLn. The cells are located at the intersection of the bit lines and the control gate lines. For example, cell 4101mn is located in row m and column n and coupled to CGm and BLn in VMM array 3507-1, and cell 4102mn is located in row m and column n and coupled to CGm and BLn in VMM array 3507-2. Since VMM arrays 3507-1 and 3507-2 are located on different dies, they can optionally be fabricated using different semiconductor processes. Regardless of whether the same or different semiconductor processes are used to fabricate VMM arrays 3507-1 and 3507-2, cells in VMM array 3507-1 can store a different number of bits than cells in VMM array 3507-2. For example, cells in VMM array 3507-1 (such as cell 4101mn) can store i bits, while cells in VMM array 3507-2 (such as cell 4102mn) can store j bits, where i and j are distinct integer values. For example, i can be 3 (meaning that the cells in VMM array 3507-1 store 3-bit values ​​accordingly), and j can be 5 (meaning that the cells in VMM array 3507-2 store 5-bit values ​​accordingly).

[0198] exist Figure 41 In this process, inputs are provided to VMM arrays 3507-1 and 3507-2 through control gate lines, and outputs are obtained on the bit lines respectively. Figure 41 Example units 4101mn and 4102mn are depicted. Alternatively, the outputs can be combined elsewhere if desired, such as by converting them into digital form using an ADC 3516 (as shown in the previous figures) and by adding them together using digital circuitry 3517 (as shown in the previous figures).

[0199] exist Figure 42 In this configuration, inputs are provided to VMM arrays 3507-1 and 3507-2 respectively via control gate lines. The outputs of VMM array 3507-1 are obtained on a first set of bit lines, and the outputs of VMM array 3507-2 are obtained on a second set of bit lines, wherein the first set of bit lines is coupled to the second set of bit lines via corresponding vertical ports 3521. This effectively adds the output signals together in analog form. Alternatively, the combined analog outputs can be digitized elsewhere if desired, such as by converting them to digital form using an ADC 3516 (as shown in the previous figures).

[0200] exist Figure 43 In this configuration, inputs are provided to VMM array 3507-1 via a first set of control gate lines, and to VMM array 3507-2 via a second set of control gate lines. The first set of control gate lines is coupled to the second set of control gate lines via corresponding vertical ports 3521, and outputs are obtained on the bit lines respectively. Optionally, the outputs can be combined elsewhere if desired, such as by converting them to digital form using an ADC 3516 (as shown in the previous figures) and adding them together using digital circuitry 3517 (as shown in the previous figures).

[0201] exist Figure 44 In this configuration, inputs are provided to VMM arrays 3507-1 and 3507-2 via control gate lines and vertical direct port 3521. Outputs are obtained on the bit lines, and these outputs are combined together via vertical direct port 3521, which effectively adds the signals together in analog form. Alternatively, the combined analog outputs can be digitized elsewhere if desired, such as by using ADC 3516 (as shown in the previous figures) to convert them into digital form.

[0202] Figures 45 to 48 Examples of structural layout options for bare die, vertical direct interface, and horizontal interface are depicted.

[0203] Figure 45A 3D VMM system 4500 is depicted, comprising a first set of dies (dies 4501, 4502, 4503, and 4504) arranged in a vertical configuration and connected via vertical direct ports 3521, and a second set of dies (dies 4505, 4506, 4507, and 4508) arranged in a vertical configuration and connected via vertical direct ports 3521. Here, a die can be connected to two other dies via corresponding vertical direct ports in the vertical direct ports 3521.

[0204] Figure 46 A 3D VMM system 4600 is depicted, comprising four layers of dies arranged in a vertically staggered configuration. The first layer includes dies 4601 and 4602, the second layer includes dies 4603, 4604, and 4605, the third layer includes dies 4606 and 4607, and the fourth layer includes dies 4608, 4609, and 4610. Dies in different layers are connected to dies in upper and lower layers via corresponding vertical direct ports 3521. Here, one die can be connected to four other dies via corresponding vertical direct ports 3521.

[0205] Figure 47 A 3D VMM system 4700 is depicted, comprising four layers of dies arranged in a vertically staggered configuration. The first layer includes dies 4701 and 4702, the second layer includes dies 4703, 4704, and 4705, the third layer includes dies 4706 and 4707, and the fourth layer includes dies 4708, 4709, and 4710. Dies in different layers are connected via corresponding vertical ports 3521, and dies in the same layer are connected via corresponding horizontal ports 3716. Here, one die can be connected to six other dies via a corresponding vertical port in the vertical port 3521 and a corresponding horizontal port in the horizontal port 3716.

[0206] Figure 48 An example of the physical layout of a 3D VMM system 4800 is depicted, showing a connector 4801. The connector 4801 is located in a die and connects to one or more vertical direct ports 3521 and horizontal interfaces 3716. As can be seen, the die and interfaces can be arranged such that the connector 4801 is positioned in a vertically staggered configuration.

[0207] As 3D integrated circuits become increasingly larger to meet the computational demands of artificial neural networks, a new interface circuit is needed to enable efficient communication between individual dies within a 3D structure and between dies and external devices such as servers or processors. Summary of the Invention

[0208] Numerous examples of interface circuits for artificial neural networks including three-dimensional integrated circuits are disclosed.

[0209] In one example, a system includes: a first device; a first interface coupled to the first device and including one or more device control lines to identify second devices within a plurality of devices intended for communication thereto, the plurality of devices each comprising a vector-matrix multiplication array; and a stack including interface circuitry coupled to the first interface, the plurality of devices, and a second interface including conventional standard-compliant lines, the second interface being coupled between the interface circuitry and the plurality of devices.

[0210] In another example, a method includes: receiving first communication from a first device via an interface circuit through a first interface, the first interface including one or more device control lines to identify a second device within a plurality of devices intended for communication thereto, the plurality of devices each comprising a vector-matrix multiplication array; converting the first communication into a second communication via the interface circuit; and transmitting the second communication via the interface circuit to one or more of the plurality of devices via a second interface including lines conforming to conventional standards. Attached Figure Description

[0211] Figure 1 This is a diagram illustrating an artificial neural network.

[0212] Figure 2 The prior art split-gate flash memory cell is described.

[0213] Figure 3 Another prior art split-gate flash memory cell is described.

[0214] Figure 4 Another prior art split-gate flash memory cell is described.

[0215] Figure 5 Another prior art split-gate flash memory cell is described.

[0216] Figure 6 This is a diagram illustrating different levels of an example artificial neural network utilizing one or more non-volatile memory arrays.

[0217] Figure 7 Here is a block diagram of an example VMM system.

[0218] Figure 8 Here is a block diagram illustrating an example artificial neural network utilizing one or more VMM systems.

[0219] Figure 9 Another example of a VMM system is described.

[0220] Figure 10 Another example of a VMM system is described.

[0221] Figure 11 Another example of a VMM system is described.

[0222] Figure 12 Another example of a VMM system is described.

[0223] Figure 13 Another example of a VMM system is described.

[0224] Figure 14 It describes the existing long short-term memory system.

[0225] Figure 15 An example unit used in a long short-term memory system is depicted.

[0226] Figure 16 Depicting Figure 15 The example implementation of the unit.

[0227] Figure 17 Depicting Figure 15 Another example of a specific implementation of the unit.

[0228] Figure 18 A prior art gate-controlled recursive cell system is described.

[0229] Figure 19 An example cell used in a gate-controlled recursive cell system is depicted.

[0230] Figure 20 Depicting Figure 19 The example implementation of the unit.

[0231] Figure 21 Depicting Figure 19 Another example of a specific implementation of the unit.

[0232] Figure 22 Another example of a VMM system is described.

[0233] Figure 23 Another example of a VMM system is described.

[0234] Figure 24 Another example of a VMM system is described.

[0235] Figure 25 Another example of a VMM system is described.

[0236] Figure 26 Another example of a VMM system is described.

[0237] Figure 27 Another example of a VMM system is described.

[0238] Figure 28 Another example of a VMM system is described.

[0239] Figure 29 Another example of a VMM system is described.

[0240] Figure 30 Another example of a VMM system is described.

[0241] Figure 31 Another example of a VMM system is described.

[0242] Figure 32 Another example of a VMM system is described.

[0243] Figure 33 Another example of a VMM system is described.

[0244] Figure 34 An example of a 2D VMM system is depicted.

[0245] Figure 35 An example of a 3D VMM system is depicted.

[0246] Figure 36 An example of a 3D VMM system is depicted.

[0247] Figure 37A and Figure 37B An example of a 3D VMM system is depicted.

[0248] Figure 38 An example of a 3D VMM system is depicted.

[0249] Figure 39A , Figure 39B and Figure 39C An example of a 3D VMM system is depicted.

[0250] Figure 40 An example of a 3D VMM system is depicted.

[0251] Figure 41 An example of a 3D VMM system is depicted.

[0252] Figure 42 An example of a 3D VMM system is depicted.

[0253] Figure 43 An example of a 3D VMM system is depicted.

[0254] Figure 44 An example of a 3D VMM system is depicted.

[0255] Figure 45An example of a 3D VMM system is depicted.

[0256] Figure 46 An example of a 3D VMM system is depicted.

[0257] Figure 47 An example of a 3D VMM system is depicted.

[0258] Figure 48 An example of a 3D VMM system is depicted.

[0259] Figure 49 A 2D VMM system is described.

[0260] Figure 50 The device is described as being coupled to multiple VMM systems via an interface.

[0261] Figure 51 A system comprising multiple stacks of VMM systems is described.

[0262] Figure 52 A system comprising multiple stacks of VMM systems is described.

[0263] Figure 53 An example of a stacked VMM system is depicted.

[0264] Figure 54A The memory system is described.

[0265] Figure 54B The VMM system is described.

[0266] Figure 55 The details of the interface are described.

[0267] Figure 56 The interfaces coupled to multiple VMM systems are described.

[0268] Figure 57 A VMM system coupled to a parallel interface is described.

[0269] Figure 58 A VMM system coupled to a serial interface is described.

[0270] Figure 59A A VMM system including a VMM array is described.

[0271] Figure 59B A VMM system comprising multiple VMM arrays is described.

[0272] Figure 59C A VMM system comprising multiple VMM arrays is described.

[0273] Figure 60 The system coupled to the AXI interface is described.

[0274] Figure 61 An array of VMM systems is described.

[0275] Figure 62 A system comprising multiple chips is described.

[0276] Figure 63 The converter of the prior art is described.

[0277] Figure 64 The method executed by the interface circuit is described. Detailed Implementation

[0278] Figure 49 A block diagram of a 2D VMM system 4900 is shown. The 2D VMM system 4900 includes many components related to... Figure 34 The 2D VMM system 4900 contains the same components as the 2D VMM system 3400, and those components will not be described again. The 2D VMM system 4900 also includes interface circuitry 4901. Interface circuitry 4901 couples the 2D VMM system 4900 to one or more other 2D VMM systems (not shown) and other devices (not shown), such as external processors or servers. Interface circuitry 4901 may include a physical layer (PHY) 4902, which may optionally include a network on-chip (NOC). The 2D VMM system 4900 includes a logic block 4903, which may include an embedded microcontroller (eMCU), an embedded NPU (eNPU - a neural processing unit for implementing vector processing operations in single-instruction multiple-data or SIMD format), a DMA (direct memory access) controller (e.g., which facilitates memory mapping from external addresses (such as from DRAM or SRAM) to internal addresses of the VMM array 3401), a BIST (memory-built-in self-test), an ACC (accumulator unit) or Act (activation function), and other circuitry.

[0279] Figure 50 System 5000 is depicted. System 5000 includes device 5001, interface 5002, and VMM systems 5003-1, ..., 5003-i, where i is an integer and represents the number of VMM systems present in system 5000. VMM systems 5003-1, ..., 5003-i are contained in one or more stacks 5004. Optionally, one or more of VMM systems 5003-1, ..., 5003-i may implement a static matrix multiplier, and one or more of VMM systems 5003-1, ..., 5003-i may implement a dynamic matrix multiplier. Optionally, such a dynamic matrix multiplier may perform multiplication Q. K TWhere Q is the query matrix, K is the key matrix, and V is the value matrix, where each value forms a key-value pair with a key in K. Optionally, such dynamic matrix multipliers can be implemented in transformer neural networks, such as... Figure 63 The converter neural network 6300 is shown.

[0280] Device 5001 is a device external to VMM system 5003, such as one or more processors or servers. In one example, device 5001 is a device that can request artificial neural network operations to be performed by one or more VMM systems in VMM system 5003. Device 5001 communicates with VMM system 5003 through interface 5002. Optionally, VMM systems 5003 can also communicate with each other through interface 5002.

[0281] VMM system 5003 can be Figure 49 The example instantiation of the 2D VMM system 4900 shown is illustrated. The VMM system 5003-1 includes interface circuitry 5004-1 (which is...) Figure 49 Example instantiation of interface circuit 4901 in the example) and VMM array 5005-1 (which is Figure 49 (Example instantiation of VMM array 3401 in the example). Similarly, VMM system 5003-i includes interface circuitry 5004-i (which is... Figure 49 Example instantiation of interface circuit 4901 in the example) and VMM array 5005-i (which is Figure 49 (Example instantiation of VMM array 3401 in the example). Other VMM systems 5003 include the same components.

[0282] Interface 5002 enables device 5001 to exchange commands, addresses, data, and other information with VMM system 5003. In one example, interface 5002 is directly coupled to all VMM systems 5003. In another example, interface 5002 is directly coupled to a subset of VMM systems 5003, and indirectly coupled to other VMM systems 5003 through one or more of the VMM systems 5003 directly coupled to by interface 5002.

[0283] Figure 51 An example physical layout of system 5100 is depicted. Figure 50An example instantiation of system 5000 is provided. System 5100 includes VMM stacks 5103-1 and 5103-2. VMM stack 5103-1 includes VMM systems 5003-1, 5003-2, 5003-3, 5003-4, and 5003-5 coupled together via TSVs (Through Silicon Vias) 5101. VMM stack 5103-2 includes VMM systems 5003-6, 5003-7, 5003-8, 5003-9, and 5003-10 coupled together via TSVs 5102. In one example, TSVs 5101 and 5102 utilize hybrid bonding. One type of hybrid bonding provides a permanent bond that combines a dielectric bond (SiOx) with an embedded metal (Cu) to form an interconnect. VMM systems 5003-5 and 5003-10 are directly coupled to interconnect 5105. Interconnect 5105 may include, for example, an RDL layer located in a silicon interposer or silicon bridge. All other VMM systems 5003 are indirectly coupled to interconnect 5105 and communicate with other VMM systems 5003 via VMM systems 5003-5 and 5003-10 using TSVs 5101 and / or 5102.

[0284] Figure 52 An example physical layout of System 5200 is depicted. Figure 50 An example instantiation of system 5000 is performed. System 5200 includes VMM stacks 5203-1, 5203-2, 5203-3, 5203-4, 5203-5, 5203-6, 5203-7, and 5203-8. Each VMM stack 5203 includes components that are compatible with... Figure 51 The VMM stacks 5103-1 and 5103-2 are multiple VMM systems coupled by TSVs in the same manner. Each VMM stack 5203 is coupled to interface 5002. For example, the bottom VMM system of the corresponding VMM stacks 5203-1, 5203-2, 5203-3, 5203-4, 5203-5, 5203-6, 5203-7, and 5203-8 is directly coupled to interface 5002.

[0285] Figure 53 An example of a VMM stack 5300 is depicted, which is a VMM stack that can be used as Figure 51 VMM stack 5103 and Figure 52An example of a VMM stack 5203 is shown in the image. The VMM system in the VMM stack 5300 may optionally implement an analog in-memory computing (CIM) system 5301, a digital CIM system 5302, a dynamic weight engine 5303, and an analog weight storage (AWS) 5304. The digital CIM system 5302 includes a VMM array operating in the digital domain, wherein the values ​​stored in the corresponding non-volatile memory cells of the VMM array are binary values. This means that the VMM array may include SRAM, DRAM, or other memory arrays that store binary values, or flash memory arrays, or other arrays that can store binary or analog values. The analog CIM system 5301 includes a VMM array operating in the analog domain, wherein the values ​​stored in each non-volatile memory cell are analog values, such as a VMM array comprising analog multi-level memory cells.

[0286] Figure 50 and Figure 52 The interface 5002 may be a parallel interface (such as a parallel memory interface) or a serial interface (such as a serial memory interface), and may conform in whole or in part to one or more traditional standards, such as AXI (Advanced Extensible Interface), DDR (Double Data Rate), MBMx (High Bandwidth Memory), PCIe (Fast Peripheral Component Interconnect), HBM (High Bandwidth Memory), UCIe (Universal Command and Control Interface), or other standards, but not limited thereto.

[0287] Figure 54A A memory system 5410 is depicted, comprising a memory macro 5411 (which includes a memory array 5412 and circuitry supporting the memory array 5412), a BIST circuitry 5413 (for performing tests during the calibration process), an automatic repair circuitry 5414 (for applying redundancy when a defective cell is detected), an ECC circuitry 5415 (for correcting errors encountered during the read process), and an interface circuitry 5416. The memory array 5412 serves as a memory device for storing digital data in corresponding memory cells and is not configured as a VMM array in this example. The memory macro 5411 includes circuitry for supporting the memory array 5412, such as row decoders, column decoders, and sense amplifiers.

[0288] Figure 54BA VMM system 5420 is depicted, comprising a VMM macro 5421 (which includes a VMM array 5422), an eMCU 5423 for control operation, a data stream DMA 5424 for facilitating data transfer between system inputs and VMM array inputs, a BIST circuit 5425, an auto-repair circuit 5426, an accumulator circuit 5427, an activation circuit 5428, and an interface circuit 5429. The VMM macro 5421 includes circuitry for supporting the VMM memory array 5422, such as row decoders, column decoders, and sense amplifiers.

[0289] Interface circuits 5416 and 5429 can be coupled to, for example, Figure 50 Interfaces such as interface 5002 or such Figure 51 Interconnectors such as the 5105 in the middle.

[0290] Figure 55 Details regarding a single instance of interface 5002 are described. At the physical layer, interface 5002 includes one or more electrical paths to carry control signals 5501, address 5502, and data 5503 in parallel, serial, or a combination of both.

[0291] Address 5502 includes address bits A0 to A10. n Each address consists of n+1 bits. Addresses can be used to read or write data from a specific location within a specific VMM array.

[0292] Data 5503 includes data bits D0 to D10. m (D or data input DIN, represented as DIN0 to DIN) m Or, the data output DOUT is represented as DOUT0 to DOUT0. m Each data item comprises m+1 bits. Data 5503 may include input data DIN (data transmitted to the VMM system) or output data DOUT (data received from the VMM system). In one example, interface 5002 provides data 5503 in parallel, for example, by simultaneously providing data bits D0 to D on different electrical paths. m In another example, interface 5002 provides data 5503 serially, for example, by providing data bits D0 to D50 on the same path in a time-division multiplexed manner. m On the other hand, in one example, interface 5002 may utilize the same path for both input and output data. In another example, interface 5002 includes separate paths for input and output data.

[0293] Control signal 5501 may include all or a subset of the control signals described in Table 9:

[0294] Table 9: Control signals used for interface 5002

[0295]

[0296] Optionally, interface 5002 may implement conventional interface standards, such as AXI (Advanced Extensible Interface), HBM (High Bandwidth Memory), UCI (Universal Command & Control Interface), or PCIex standards, but is not limited thereto. Optionally, interface 5002 may implement conventional interface standards and may include one or more additional lines. For example, one or more additional lines may be used to carry one or more control signals 5501 that are not yet carried by conventional interface standards, such as device control signals.

[0297] Figure 56 An interface 5002 coupled to VMM systems 5003-1 and 5003-2 is depicted. In this example, VMM systems 5003-1 and 5003-2 are contained in a stack 5005. Alternatively, VMM systems 5003-1 and 5003-2 may be contained in different stacks 5005. Interface circuits 5004-1 and 5004-2 receive control signals 5501 from interface 5002 and determine whether the control signals 5501 are intended for use with the VMM system associated with the respective interface circuit 5004. In one example, interface circuits 5004-1 and 5004-2 receive device control signals through interface 5002 and determine whether the device control signals enable or disable the device containing the VMM system associated with the interface circuit. Figure 56 In the example shown, the device control signal enables the device (e.g., a chip or die) containing VMM system 5003-1 and disables the device (e.g., a chip or die) containing VMM system 5003-2. VMM system 5003-1 then performs operations in conjunction with or following the device control signal. Optionally, interface circuits 5004-1 and 5004-2 can be coupled to VMM arrays 5005-1 and 5005-2 respectively via a standard memory interface 5601 (such as SPI (Serial Peripheral Interface)), because the respective interface circuit 5004 controls one or a limited number of VMM arrays that have the capabilities of a standard memory interface. This is because interface circuit 5004 interprets the device control signal and does not pass that information to the VMM arrays.

[0298] Figure 57A VMM system 5700 is depicted, including a parallel interface 5710 that can be used for interface 5002. The parallel interface 5710 is a modified version of the conventional parallel interface described in Table 10 below. The VMM system 5700 includes a VMM array 5701, a row decoder 5702, a column decoder 5703, a data buffer and data latch 5704, an address buffer and latch 5705, and control logic 5706. The parallel interface 5710 includes the following lines (which may include pins, solder balls, wires, traces, or other electrical paths): Address line A [N:0], which carries N+1 address bits in parallel; CE# line, which carries a chip enable signal (specifically designed to enable only a single chip); OE# line, which carries an output enable signal; WE# line, which carries a write enable signal; Data output line DOUT [M:0], which carries M+1 output data bits in parallel; and Data input line DIN [M:0], which carries M+1 input data bits in parallel. The parallel interface 5710 includes three control lines (CE#, OE#, and WE#) and carries a ratio of Figure 56 The interface 5002 shown has fewer control signals.

[0299] Table 10 contains line count information for various versions of the parallel interface 5710 (listed as parallel interfaces 5710a, 5710b, 5710c, 5710d, 5710e, and 5710f) and for various traditional parallel interfaces.

[0300] Table 10: Line Count of Parallel Interface

[0301]

[0302] In Table 10, the capacity column refers to the storage capacity of the VMM array 5701; the address lines column refers to the number of address lines, which is the value of N+1; the control lines column refers to the number of control lines (such as CE#, OE#, and WE#); and the I / O lines column refers to the total number of data output lines and data input lines, i.e., 2. The value of (M+1); and the total line count refers to the sum of the address line values, control line values, and I / O line values, which means the sum of the first three columns. Power lines are not included in the total line count. Parallel interfaces 5710a, 5710b, 5710c, 5710d, 5710e, and 5710f use more total lines than the conventional interfaces discussed in Table 10, but can perform I / O operations faster than conventional interfaces because they have more lines for output operations and more lines for input operations.

[0303] Figure 58A VMM system 5800 is depicted, including a serial interface 5810 that can be used for interface 5002. The serial interface 5810 is a modified version of the conventional serial interfaces known as QSPI (quad SPI) and OSPI (octa SPI) to provide a wider I / O interface with higher transmission bandwidth, as described below in Table 11. The VMM system 5800 includes a VMM array 5801; a row decoder 5802; a column decoder 5803; a page buffer, I / O buffer, and data latch 5804; an address buffer and latch 5805; and control logic 5806. The serial interface 5810 includes N+1 lines (which may include pins, solder balls, wires, traces, or other electrical paths) to carry address signals, I / O signals (which include time-division multiplexed input and output signals), and control signals. The QSPI-based version of the serial interface 5810 may include control pins carrying signals CE# (chip enable), WP# (write protect), HOLD# (hold), and SCK (clock), and four lines for I / O signals [3:0]. The OSPI-based version of the serial interface 5810 may include control signals CE# (chip enable), WP# (write protect), HOLD# (hold), RESET# (reset), INT# (interrupt), DQS (data strobe), SCK (clock), and eight lines for I / O signals [7:0].

[0304] Table 11 contains line count information for various versions of the 5810 serial interface (listed as serial interfaces 5810a, 5810b, and 5810c) and for various versions of the legacy QSPI serial interface.

[0305] Table 11: Line Count of Serial Interface

[0306]

[0307] In Table 11, the capacity column refers to the storage capacity of the VMM array 5701; the control lines column refers to the number of control lines (such as CE#, OE#, and WE#); the SIO lines column refers to the total number of serial I / O lines; and the total lines column refers to the sum of the control lines and SIO lines, which is the sum of the first two columns. Power lines are not included in the total lines value. Serial interfaces 5810a, 5810b, and 5810c use more total lines than the conventional interfaces discussed in Table 11, but can perform I / O bandwidth operations faster than conventional interfaces because they have more lines for serial I / O output operations. Additional SIO lines (such as 8 or 16) are possible.

[0308] Figure 59A , Figure 59B and Figure 59C Depicting the use of previous references Figure 58Various VMM system configurations for the 5810 serial interface are described.

[0309] Figure 59A A VMM system 5900 is described, which includes a VMM array 5901, a row decoder 5902, a column decoder 5903, a sense amplifier 5904, and a high-voltage decoder 5905.

[0310] Figure 59B A VMM system 5910 is described, which includes VMM arrays 5911 and 5921, row decoders 5912 and 5922, column decoders 5913 and 5923, sense amplifiers 5914 and 5924, and a shared high-voltage decoder 5915.

[0311] Figure 59C A VMM system 5930 is depicted, which includes VMM arrays 5931, 5941, 5951 and 5961; row decoders 5932, 5942, 5952 and 5962; column decoders 5933, 5943, 5953 and 5963; sense amplifiers 5934, 5944, 5954 and 5964; and shared high-voltage decoders 5935 and 5955.

[0312] Table 12 contains line count and sense amplifier count information for VMM systems 5900, 5910, and 5930 compared to equivalent conventional VMM systems.

[0313] Table 12: Line counts and sense amplifier counts for VMM systems

[0314]

[0315] In Table 12, the capacity column refers to the storage capacity of the VMM array; the SIO lines column refers to the number of serial I / O lines; the sense amplifier column refers to the number of sense amplifiers; and the I / O page width column refers to the I / O width used for data pages, where a page typically includes data in two or more rows of memory cells. VMM systems 5900, 5910, and 5930 can perform I / O operations faster than traditional OSPI systems because they have more lines for serial I / O output operations.

[0316] Figure 60 A system 6000 is described that uses a conventional interface 6001 (such as AXI, DDR, MBMx, PCIe, HBM, UCIe, or other interfaces) and an optional additional line 6008, which together can be used for Figure 50Interface 5002 is used to interact with components outside system 6000. For example, additional line 6008 can be used to carry one or more control signals 5501 (not shown) that are not carried by conventional interface 6001, such as device control signals discussed above in Table 9. System 6000 includes interface circuitry 6002. Interface circuitry 6002 includes circuitry 6003 for communicating with conventional interface 6001, FIFO 6004, memory interface circuitry and state machine 6005, memory interface 6006 (such as a serial peripheral interface), and VMM array 6007. Memory interface 6006 is a second conventional memory interface. Interface circuitry 6002 converts communication received on conventional interface 6001 into communication that it transmits on memory interface 6006, and vice versa.

[0317] Table 13 contains line counts for a conventional interface 6001, in which conventional interface 6001 includes examples of AXI interfaces with various read and write line sizes.

[0318] Table 13: Line counts for various R / W values ​​in System 600

[0319]

[0320] In Table 13, the Address-R column indicates the row count for reading address lines; the Address-W column indicates the line count for writing address lines; the Control column indicates the line count for control lines; the DATA-R column indicates the line count for reading data; the DATA-W column indicates the line count for writing data; and the Overall Lines column indicates the overall line count.

[0321] Figure 61 Die 6100 is depicted. Die 6100 includes an array 6101, which is an array of VMM systems arranged in rows and columns coupled by an interface 6104. Example VMM system 6102 is an illustration of a VMM system in array 6101. VMM system 6102 includes a PHY block 6103 (which optionally includes an on-chip network) connected to interface 6104. Die 6100 includes circuitry 6105, which may include SRAM, a microcontroller, an NPU, and DMA. Die 6100 includes a PHY block 6106, which is connected to an interface external to die 6100 via interface 5002 (not shown).

[0322] Figure 62System 6200 is depicted. System 6200 includes multiple chips, such as example chip 6201. Each chip includes a Nominal Object (NOC), such as NOC 6202 in chip 6201. The NOCs in each chip are interconnected via interface 6203, which may be a PCIex interface, a UCIe interface, a BOW interface, or any other interface. System 6200 also includes circuitry 6204, which may include SRAM, a microcontroller, an NPU, and DMA. System 6200 also includes a PHY block 6205, which is connected to interface 6203 and to an interface (not shown) external to system 6200 via interface 5002.

[0323] Figure 64 A method 6400 is described that can be performed by any of the previously described interface circuits 4901, 5004, 5416, 5429, or 6002. The method includes: receiving first communication from a device via a first interface, the first interface including one or more device control lines to identify a second device among a plurality of devices intended for communication thereto, the plurality of devices each including a vector-matrix multiplication array (6401); converting the first communication into second communication by the interface circuit (6402); and transmitting the second communication to one or more of the plurality of devices via the second interface including lines conforming to conventional standards (6403). Optionally, the second communication may be transmitted only to the second device. Optionally, the second communication may be transmitted to multiple devices, and all devices other than the second device will ignore the second communication.

[0324] It should be noted that, as used herein, the terms “above” and “on” both encompass “directly on” (without intermediate material, elements, or space between) and “indirectly on” (with intermediate material, elements, or space between). Similarly, the term “adjacent” includes “directly adjacent” (without intermediate material, elements, or space between) and “indirectly adjacent” (with intermediate material, elements, or space between), “mounted to” includes “directly mounted to” (without intermediate material, elements, or space between) and “indirectly mounted to” (with intermediate material, elements, or space between), and “electrically coupled to” includes “directly electrically coupled to” (without intermediate material or elements electrically connecting the elements together) and “indirectly electrically coupled to” (with intermediate material or elements electrically connecting the elements together). For example, forming an element “above a substrate” can include forming an element directly on the substrate without intermediate material / elements between them, and forming an element indirectly on the substrate with one or more intermediate materials / elements between them.

Claims

1. A system comprising: First equipment; A first interface, coupled to the first device and including one or more device control lines to identify a second device within a plurality of devices intended for communication thereto, the plurality of devices each comprising a vector-matrix multiplication array; and The stack includes an interface circuit coupled to the first interface, the plurality of devices, and a second interface including a conventional standard-compliant wire, the second interface being coupled between the interface circuit and the plurality of devices.

2. The system of claim 1, wherein the second device is a chip.

3. The system of claim 1, wherein the second device is a bare die.

4. The system according to claim 1, wherein the first interface is a serial interface.

5. The system according to claim 1, wherein the first interface is a parallel interface.

6. The system according to claim 1, wherein the second interface is a serial peripheral interface.

7. The system of claim 1, wherein the vector-matrix multiplication array comprises a first vector-matrix multiplication array providing computational functionality in digital memory and a second vector-matrix multiplication array providing computational functionality in analog memory.

8. The system according to claim 1, wherein the system comprises: The second stack includes a second interface circuit coupled to the first interface, a second plurality of devices, and a third interface including a line conforming to a second conventional standard, the third interface being coupled between the second interface circuit and the second plurality of devices.

9. The system according to claim 8, wherein the system comprises: An interconnect located between the stack and the second stack.

10. The system of claim 1, wherein the first device includes a processor.

11. The system of claim 1, wherein the first device comprises a server.

12. The system according to claim 1, One or more of the vector-matrix multiplication arrays in the vector-matrix multiplication arrays implement a static matrix multiplier, and one or more of the vector-matrix multiplication arrays in the vector-matrix multiplication arrays implement a dynamic matrix multiplier.

13. The system of claim 12, wherein the dynamic matrix multiplier performs multiplication Q. K T , where Q is the query matrix, K is the key matrix, and V is the value matrix, where each value forms a key-value pair with a key in K.

14. The system of claim 12, wherein the dynamic matrix multiplier performs multiplication (Q... K T ) V, where Q is the query matrix, K is the key matrix, and V is the value matrix, where each value forms a key-value pair with a key in K.

15. The system of claim 12, wherein the dynamic matrix multiplier implements a transformer neural network.

16. A method, the method comprising: The interface circuit receives first communication from a first device through a first interface, the first interface including one or more device control lines to identify a second device among a plurality of devices intended for communication thereto, the plurality of devices each comprising a vector-matrix multiplication array; The interface circuit converts the first communication into the second communication. as well as The second communication is transmitted by the interface circuit to one or more of the plurality of devices via a second interface including wires conforming to conventional standards.

17. The method of claim 16, wherein the first interface is a serial interface.

18. The method of claim 16, wherein the first interface is a parallel interface.

19. The method of claim 16, wherein the second interface is a serial peripheral interface.

Images