Memory device with configurable chip for multiplication

Abstract

This disclosure relates to systems, methods, and apparatus related to memory devices having configurable tiles for multiplication. In one method, a memory array includes tiles that can be dynamically configured during matrix vector multiplication of a neural network. Characteristics of the neural network are used to configure the tile differently for each processing step (e.g., for each layer) of the neural network as computations proceed. In one example, the characteristic is a characteristic of a current layer being processed and / or a size of a neuron. The tile is configured by deactivating a portion of an access line for accessing memory cells in the tile. In one example, bit lines that are not used for neurons of a currently computed tile are deactivated.

Description

[0001] Related Applications

[0002] This application claims priority to U.S. Provisional Patent Application Serial No. 63,496,180, filed on April 14, 2023, the entire disclosure of which is hereby incorporated by reference herein. Technical Field

[0003] At least some embodiments disclosed herein relate generally to memory devices, and more particularly, but not limited to, to memory devices having tiles of memory cell arrays that can be configured to perform multiplication and other operations. Background Art

[0004] Limited memory bandwidth is an important issue in machine learning systems. For example, DRAM devices used in current systems store a large number of weights and activations used in deep neural networks (DNNs).

[0005] In one example, a deep learning machine (e.g., a deep learning machine supporting convolutional neural network (CNN) processing) performs processing that determines a large number of calculations per second. For example, input / output data, deep learning network training parameters, and intermediate results are constantly extracted from and stored in one or more memory devices (e.g., DRAM). When large storage density (e.g., storage density greater than 100MB) is involved, DRAM type memory is often used due to its cost advantage. In one example of a deep learning hardware system, a computing unit (e.g., a system on a chip (SOC), FPGA, CPU, or GPU) is attached to a memory device (e.g., a DRAM device).

[0006] Existing computer architectures use processor chips specialized for serial processing and DRAM optimized for high-density memory. The interface between these two devices is a major bottleneck that imposes latency and bandwidth limitations and adds significant power consumption overhead. On-chip memory area is expensive and it is not feasible to add large amounts of memory to the CPU and GPU processors currently used to train and deploy DNNs.

[0007] Memory in a neural network is used to store input data, weight parameters, and activations as the input propagates through the network. In training, the activations from the forward pass must be kept until they can be used to calculate the error gradients in the backward pass. As an example, a network may have 26 million weight parameters and 16 million activations are calculated in the forward pass. If 32-bit floating point values ​​are used to store each weight and activation, this corresponds to a total storage requirement of 168MB.

[0008] GPUs and other machines require large amounts of memory for the weights and activations of neural networks. GPUs cannot directly and efficiently perform the small convolutions used in deep neural networks, so they require large amounts of activation or weight storage. Finally, memory is also needed to store input data, temporary values, and program instructions. For example, a high-performance GPU requires more than 7GB of local DRAM.

[0009] Large amounts of storage data cannot be kept on the GPU processor. In many cases, a high-performance GPU processor may have only 1KB of memory associated with each of the processor cores that can be read fast enough to saturate the floating point data path. Therefore, at each layer of the DNN, the GPU needs to save the state to external DRAM, load the next layer of the network, and then reload the data. Therefore, the off-chip memory interface bears the burden of constantly reloading weights and saving and retrieving activations. This significantly slows down training time and increases power consumption.

[0010] In one example, image and other sensors are used and generate large amounts of data. It is inefficient to transfer certain types of data from the sensor to a general purpose microprocessor (e.g., a central processing unit (CPU)) for processing in some applications. For example, it is inefficient to transfer image data from an image sensor to a microprocessor for image segmentation, object recognition, feature extraction, etc.

[0011] Some image processing may include intensive computations involving multiplying and accumulating columns or matrices of elements. Some specialized circuits have been developed to accelerate multiplication and accumulation operations. For example, a multiplier-accumulator (MAC unit) may be implemented using a set of parallel computational logic circuits to achieve computational performance higher than that of a general-purpose microprocessor. Summary of the invention

[0012] In some embodiments, a device is described that includes: a plurality of tiles; and a logic circuit system configured to: program a first memory cell of the tile to store a first weight; control a corresponding first portion of each tile based on a characteristic associated with a neural network; and sum output currents from the first memory cell.

[0013] In some embodiments, an apparatus is described. The apparatus includes a memory configured to store a culling map of a neural network, a controller configured to generate the culling map, and voltage drivers configured to drive a bit line coupled to a memory cell, wherein each of the voltage drivers is enabled or disabled based on the culling map.

[0014] In some embodiments, a system is described. The system includes: NAND flash memory cells organized in pillars extending vertically upward from a semiconductor substrate; analog circuitry configured to accumulate output currents from the memory cells during inference by a neural network; and sensing circuitry coupled to access lines and configured to enable or disable each access line during the inference. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] Embodiments are illustrated by way of example and not limitation in the accompanying figures in which reference numerals indicate similar elements.

[0016] Figure 1 An integrated circuit device having an array of image sensing pixels, an array of memory cells with configurable tiles, and circuitry for performing inference computations according to one embodiment is shown.

[0017] Figure 2 A computation is shown in which a column of weight bits is multiplied by a column of input bits to provide an accumulated result according to one embodiment.

[0018] Figure 3 A method of performing calculations in an integrated circuit device based on summing output currents from memory cells according to one embodiment is shown.

[0019] Figure 4 An analog weight-fixing architecture for matrix-vector multiplication (MVM) is presented according to one embodiment.

[0020] Figure 5 An analog weight fixing method using a select gate drain (SGD) cell architecture according to one embodiment is presented.

[0021] Figure 6 An analog weight fixing method using a two-cell architecture according to one embodiment is presented.

[0022] Figure 7 An analog weight fixing method using a bit line cell architecture according to one embodiment is presented.

[0023] Figure 8 An exemplary implementation of a deep neural network according to one embodiment is shown.

[0024] Fig. 9 An exemplary arrangement of memory cells for a tile of a NAND flash memory array is shown according to one embodiment.

[0025] Fig.10 Exemplary tiles of a memory device are shown, where each tile may be configured to be partially filled for performing multiplication or other operations of a neural network, according to one embodiment.

[0026] Fig.11 The bit lines of a tile are shown disabled based on a cull map according to one embodiment.

[0027] Fig.12 Sensing circuitry coupled to a bit line for accessing a NAND flash memory cell is shown according to one embodiment.

[0028] Fig.13 An architecture with resistive random access memory (RRAM) or NOR memory cells arranged in a parallel configuration to perform multiplication is shown according to one embodiment.

[0029] Fig.14 A method for performing multiplication using memory cells in a memory device having configurable tiles according to one embodiment is shown. DETAILED DESCRIPTION

[0030] The following disclosure describes various embodiments for configuring tiles of a memory cell array when performing multiplication in a memory device. The memory device may, for example, store data used by a host device such as a computing device of an autonomous vehicle or another computing device that accesses data stored in the memory device. In one example, the memory device is a solid-state drive installed in an electric vehicle.

[0031] In one embodiment, tiles are dynamically configured as computations of the neural network proceed (e.g., moving from one layer to another). For example, these computations include matrix-vector multiplications (MVMs) for each layer of the neural network. The weights of the neural network are stored in the memory cell array and multiplications using the weights are performed in the memory cell array itself based on the output currents from the memory cells in the array. The output currents are digitized and used by the controller to support the MVMs.

[0032] As part of the tile configuration, a portion of the access lines for each tile are disabled to save power. Improved power efficiency is particularly desirable for use of neural networks on mobile devices and automobiles. Storing the weights of the neural network in a memory device and performing multiplications in the memory device avoids or reduces the need to move the weights to a central processing unit or other processing device. This reduces the power consumption required to move data to and from memory and also reduces the memory bandwidth issues described herein.

[0033] More generally, neural networks are one of the most popular types of machine learning algorithms (e.g., modeled after our understanding of how the brain works). For example, a network has a large number of neurons, which by themselves perform fairly simple calculations but together can learn complex and nonlinear functions. For example, a neuron calculation is basically a number of input values ​​multiplied by a neuron weight (which represents how important each input is to the calculation) and summing the results. The weights are learned during network training. Each result is then passed through a nonlinear activation function to allow the neuron to learn complex relationships.

[0034] In terms of computational burden, multiplying all input values ​​by the neuron weights of all neurons in the network is the most demanding use of processing power. For example, this multiplication can be 90% or more of the computational requirements, depending on the network design. When extended to a full layer of a neural network, the computation is vectorized and becomes a matrix-vector multiplication problem. The computation is sometimes also referred to as a dot product or sum of products (SOP) computation.

[0035] Deep learning technology is an exemplary implementation of neural networks and has played an important role in various applications such as image classification, object detection, speech recognition, natural language processing, recommendation systems, auto-generation, and robotics. Many domain-specific deep learning accelerators (DLAs) such as GPUs, TPUs, and embedded NPUs have been introduced to provide the required efficient implementation of deep neural networks (DNNs) from the cloud to the edge. However, due to the frequent back-and-forth movement of data between the computing units and memory in deep learning, limited memory bandwidth remains a key challenge, especially for energy-constrained systems and applications such as edge AI.

[0036] Conventional Von-Neumann computer architectures have been developed with processor chips specialized for serial processing and DRAM optimized for high-density memory. The interface between these two devices is a major bottleneck, which imposes latency and bandwidth limitations and adds considerable power consumption overhead. As the demand for higher accuracy and higher speed in AI applications continues to grow, larger DNN models with massive weights and activations are being developed and implemented. The resulting bottlenecks on memory bandwidth and power consumption for data movement between chips are significant technical issues.

[0037] As time goes by, the complexity of neural networks continues to grow exponentially, which means more computation is required. This puts pressure on the performance of traditional computing architectures. For example, MVM operations require specially built computing blocks to meet the performance requirements (GPUs, digital accelerators). Moreover, neuron weights must be fetched from memory, which causes both performance bottlenecks and energy inefficiencies, as mentioned above.

[0038] In some cases, the precision of the computations may be reduced to address these concerns. For example, the choice of neural network training type may achieve approximately equivalent neural network accuracy at significantly lower precision. Lower precision may improve the performance and / or energy efficiency of the neural network implementation. Moreover, the use of lower precision may support storing weights in memory and performing multiplications in memory, as described herein.

[0039] For example, when using lower precision representations of weights and inputs (e.g., using a smaller number of bits for each weight or input), one key aspect to consider is the final answer, such as the classification of an image. In many cases, if the neural network model is correctly structured (e.g., the manner or method used to train the network), the accuracy of obtaining the correct final answer remains almost the same (e.g., only reduced by 2 to 5%) even with lower precision. For example, analog multiplication in the memory itself is even preferable because similar accuracy can be achieved as with traditional methods, but using this lower precision.

[0040] The neural network design itself usually dictates the size of the MVM operation at each layer of the network. Each layer may have a different number of features and neurons (see, for example, Figure 8 In one embodiment, the MVM calculation will occur in a portion of a NAND flash or other memory array. This portion is represented in the array as a tile.

[0041] For flexibility and parallelism, it is beneficial to make the size of the tile large, but there are usually some practical constraints. For example, the tile size dictates the inferential read performance because the word line, select gate drain (SGD), and bit line lengths are defined by the size of the tile. However, voltage drops caused by resistance (such as IR drops) can be a significant source of errors caused by high array currents, and this depends in part on the length of the bit line (or other access line). Therefore, variable-sized MVM operations (depending on the current layer being used) must be mapped to fixed-sized tiles, which will usually lead to inefficiencies. Since the MVM operation is mapped to the tile, the efficiency will be low in the case where the MVM operation does not completely fill all tiles.

[0042] To address these and other technical issues, a memory device integrates memory and processing. In one example, memory and inference computing processing are integrated into the same integrated circuit device. In some embodiments, the memory device is an integrated circuit device having an image sensing pixel array, a memory cell array, and one or more circuits that use the memory cell array to perform inference computing on image data from an image sensor. In some embodiments, the memory device includes other types of sensors (e.g., LIDAR, radar, sound) or is used with the other types of sensors.

[0043] Existing methods for matrix-vector multiplication use digital logic gates. Compared to the various embodiments described below, digital logic implementations are more complex, consume more silicon area, and dissipate more power. These embodiments effectively reduce multiplication to a memory access function that can be parallelized in an array. The accumulation function is implemented by wires connecting these memory elements, which can also be parallelized in an array. By combining these two features in an array, matrix-vector multiplication can be performed more efficiently than methods using digital logic gates.

[0044] In one embodiment, a cull map is generated to indicate portions of tiles that are disabled during computation of a current layer of a neural network. The cull map is provided by a host to a controller of a memory device. The controller uses the cull map to signal sensing or other circuitry that particular access lines are to be disabled (e.g., bit lines are not to be charged). For example, sensing activity associated with identifying bit lines for the current layer is turned off. In one example, the sensing circuitry includes an analog-to-digital converter (ADC). The disabled access lines are turned off prior to running the current layer. In other embodiments, the cull map is generated by a controller (e.g., based on configuration data from a host).

[0045] In one embodiment, an integrated circuit (IC) device (eg, Figure 1 101 of the integrated circuit device includes a host interface configured to communicate with a host. A tile including memory cells stores weights of a neural network used by the host. The logic circuit system of the integrated circuit device programs a first memory cell of the tile to store a first weight. The logic circuit system receives configuration data from the host via the host interface. The configuration data indicates characteristics associated with the neural network (e.g., the size of a layer in the neural network to be used next). The logic circuit system selects a corresponding first portion of each tile to be disabled based on the configuration data. This selection and disabling may be repeated for each different MVM operation.

[0046] The logic circuit system receives a first input from a host via a host interface. The logic circuit system performs a first weight multiplication by the first input to provide at least one first result. The first result is based on summing the output currents from the first memory cells and performing the multiplication without using second memory cells located in the disabled corresponding first portion of each tile. The logic circuit system sends the first result to the host via the host interface.

[0047] It should be noted that while various embodiments herein refer to disabling a portion of a tile, in other embodiments, a cull map or other data (e.g., configuration data from a host or data stored in a register of a controller) corresponding to or based on characteristics of a neural network may be used to disable selected portions of an array of memory cells. It is not required that a cull map or other data explicitly identify or reference a particular tile of the array itself. For example, a cull map may identify an access line having an address, location, or other characteristic as a basis for selecting that access line to disable during a computation.

[0048] In one embodiment, the image sensor is configured with analog capabilities to support inference computations, such as computations of artificial neural networks, by using matrix-vector multiplication. The image sensor may be implemented as an integrated circuit device having an image sensor chip and a memory chip. The memory chip may have a 3D memory array configured to support multiplication and accumulation operations. The integrated circuit device includes one or more logic circuits configured to process images from the image sensor chip and operate memory cells in the memory chip to perform multiplication and accumulation operations.

[0049] The memory chip may have multiple layers of memory cells. Each memory cell may be programmed to store bits of a binary representation of an integer weight. Each input line may be applied with a voltage according to the integer bit. A column of memory cells may be used to store bits of a weight matrix; and a set of input lines may be used to control a voltage driver to apply a read voltage to a row of memory cells according to the bits of an input vector.

[0050] The threshold voltage or state of the memory cell used for the multiplication and accumulation operation can be programmed so that the current through the memory cell subjected to a predetermined read voltage is a predetermined amount representing a 1 value stored in the memory cell or is negligible to represent a 0 value stored in the memory cell. When the predetermined read voltage is not applied, the current through the memory cell is negligible regardless of the value stored in the memory cell. Due to the configuration, the current through the memory cell corresponds to the result of a 1-bit weight (stored in the memory cell) multiplied by a 1-bit input (corresponding to the presence or absence of a predetermined read voltage driven by a voltage driver controlled by the 1-bit input).

[0051] The output current of the memory cell (representing the result of a column of 1-bit weights stored in the memory cell multiplied by a column of 1-bit inputs) is connected to a common line for summing. The summed current in the common line is a multiple of a predetermined amount; and the multiple can be digitized and determined using an analog / digital converter or other digital converter. Such results of 1-bit and 1-bit multiplication and accumulation can be performed on different significant bits of the weights and different significant bits of the inputs. The results of different significant bits can be shifted (e.g., left shifted) to apply the weights of the corresponding significant bits for summing to obtain the results of multiplication and accumulation of multi-bit weights and multi-bit inputs.

[0052] Using the ability to perform multiplication and accumulation operations implemented via the memory cell array, the logic circuits can be configured to perform inference computations, such as computations of artificial neural networks.

[0053] Various embodiments of memory devices that perform multiplication using logic states of memory cells are described below. The memory device typically has memory cells configured in an array, where each memory cell is programmed to, for example, allow a certain amount of current to pass when a voltage in a predetermined voltage region is applied to represent a first logic state (e.g., a first value stored in the memory cell) or to allow a negligible amount of current to represent a second logic state (e.g., a second value stored in the memory cell).

[0054] The memory device performs calculations based on applying voltages digitally, in the form of whether to apply an input voltage to produce a current for summing on a line (e.g., a bit line of a memory array). The total current on the line will be a multiple of the amount of current allowed by the cells programmed with the first value. In one example, an analog-to-digital converter is used to convert the current into a digital result of the sum of the bit-by-bit multiplications.

[0055] The memory cells in the array may generally be of various types. Examples include NAND or NOR flash memory cells and phase change memory (PCM) cells. In one example, the PCM cells are chalcogenide memory cells. In one example, a floating gate or charge trap memory device in a NAND or NOR memory configuration is used.

[0056] In various embodiments using chalcogenide memory cells, multiplication and other processing are performed by operating the chalcogenide memory cells in the subthreshold region. This will avoid any memory cell from exceeding threshold or snapping, which would normally prevent correct multiplication (e.g., due to undesirably large output currents associated with snapping).

[0057] In one embodiment, a memory device (e.g., integrated circuit device 101) includes a memory cell array having memory cells. Each memory cell is programmable to store a corresponding weight value for performing multiplication. The integrated circuit device also includes a voltage driver configured to apply an input voltage to the memory cell for performing multiplication. The input voltage represents an input that is multiplied by the corresponding weight value of each memory cell, and the voltage is applied so that the operation of the memory cell remains in a subthreshold mode during the multiplication.

[0058] The integrated circuit device has a bit line (or other common line) coupled to the memory cell. The bit line is configured to sum the output current from each of the memory cells resulting from the applied input voltage. The integrated circuit device has a digital converter configured to generate a result of the multiplication based on the summed output current.

[0059] In one embodiment, the memory device implements unsigned 1-bit by 1-bit multiplication using chalcogenide or other types of memory cells (e.g., NAND cells). Each memory cell can be programmed to a "1-state" such that when a voltage V is applied across the memory cell (e.g., across two terminals of a resistive memory cell), a predetermined amount of current can pass through the memory cell. Alternatively, the memory cell can be programmed to a "0-state" such that when the same voltage V is applied, only a negligible amount of current can pass through the memory cell. Thus, the memory cell can be programmed to a "1-state" or a "0-state" to represent a storage weight of "1" or "0", respectively.

[0060] An input voltage of V may be used to represent an input of "1"; and an input voltage of 0 may be used to represent an input of "0." Alternatively, a voltage may be used to represent an input of "0" when another voltage is lower than V but causes only a negligible amount of current to pass through the memory cell (regardless of the programming state of the memory cell).

[0061] When a voltage configured to represent the above-mentioned 1 or 0 input is applied to a memory cell programmed to the "1 state" or "0 state" to represent the weight 1 or 0 discussed above, the amount of current passing through the memory cell is a predetermined amount (representing an output of "1") or a negligible amount (representing an output of "0"). In addition, the input, weight, and output relationship satisfies the multiplication of a 1-bit input and a 1-bit weight to produce a 1-bit output in all possible variations of the input and weight.

[0062] Thus, the memory cell is used to perform unsigned 1-bit and multi-bit multiplication by being programmed to store a 1-bit weight, applying an input voltage to represent the 1-bit input, and determining a 1-bit output based on sensing whether the current through the memory cell (the output current from the memory cell) is a predetermined amount.

[0063] The summation of the results represented by the output currents from the memory cells can be implemented by connecting the currents to a common line, such as a bit line or a source SRC line. The summation of the results can be digitized to provide a digital output. In one example, an analog-to-digital converter is used to measure the sum as a multiple of a predetermined current amount and provide a digital output.

[0064] In one embodiment, the memory device implements unsigned 1-bit and multi-bit multiplication. The multi-bit weight can be implemented via multiple memory cells. Each of the memory cells is configured to store one of the bits of the multi-bit weight, as described above. The voltage represented by the 1-bit input can be applied separately to multiple memory cells to obtain the result of the unsigned 1-bit and 1-bit multiplication, as described above.

[0065] Each memory cell has a position corresponding to its storage bit in the binary representation of the multi-bit weight. Its digitized output (e.g., the sum of the output currents from the memory cells on a common bit line) can be left-shifted according to its position in the binary representation to obtain a shifted result. For example, the digitized output of the memory cell storing the least significant bit of the multi-bit weight is shifted by 0 bits; the digitized output of the memory cell storing the second least significant bit of the multi-bit weight is shifted by 1 bit; the digitized output of the memory cell storing the third least significant bit of the multi-bit weight is shifted by 2 bits; and so on. The shifted results can be summed to obtain the result of multiplying the 1-bit input by the multi-bit weight stored in multiple memory cells.

[0066] The sum of the results represented by the output currents from groups of memory cells (each group representing a separate multi-bit weight) can be summed bit by bit for different bit positions in the multi-bit weight via currents connected in common lines. For example, the current from the memory cell storing the least significant bit is connected to the first common line to form a summed output of the result derived from the least significant bit; the current from the memory cell storing the second least significant bit is connected to the second common line to form a summed output of the result derived from the second least significant bit; the current from the memory cell storing the third least significant bit is connected to the third common line to form a summed output of the result derived from the third least significant bit; and so on. The summed output can be converted into digital form and then shifted for summing in digital form. Alternatively, the corresponding currents can be scaled before digitization.

[0067] In one embodiment, the memory device implements time-slice unsigned multi-bit and multi-bit multiplication. An input represented by a binary number having a predetermined number of bits (e.g., 4 bits) may be applied one bit at a time (e.g., applied at times T0, T1, T2, etc.) over the same predetermined number of clock cycles. Each cycle produces an output as described above for unsigned 1-bit and multi-bit multiplication.

[0068] The result of an unsigned 1-bit by multi-bit multiplication (e.g., as discussed above) obtained within each clock cycle may be left shifted according to the position of the bits applied to the inputs in the clock cycle. For example, the result of the clock cycle to which the least significant bit of the input is applied is not shifted; the result for the second least significant bit is left shifted 1 bit; the result for the third least significant bit is left shifted 2 bits; etc. The shifted results from the clock cycles are summed in digital form.

[0069] Figure 1 An integrated circuit device 101 is shown having an image sensing pixel array 111, a memory cell array 113 having configurable tiles 141 and 142, and circuits for performing inference calculations according to one embodiment. Figure 1In the embodiment, the integrated circuit device 101 has an integrated circuit die 109 including logic circuits 121 and 123 , an integrated circuit die 103 including an image sensing pixel array 111 , and an integrated circuit die 105 including a memory cell array 113 .

[0070] In one example, integrated circuit die 109 with logic circuits 121 and 123 is a logic chip; integrated circuit die 103 with image sensing pixel array 111 is an image sensor chip; and integrated circuit die 105 with memory cell array 113 is a memory chip.

[0071] exist Figure 1 In the embodiment, the integrated circuit die 105 having the memory cell array 113 further includes a voltage driver 115 and a current to digital converter 117. The memory cell array 113 is connected so that the current generated by the memory cells in response to the voltage applied by the voltage driver 115 is summed for the memory cell columns in the array 113 (e.g., as Figure 2 and the summed current is digitized to produce the sum of the bitwise multiplication. The inference logic circuit 123 can be configured to instruct the voltage driver 115 to apply a read voltage according to a column input and perform shift and sum to produce a column or matrix weight multiplied by the input column and accumulated results.

[0072] In one embodiment, sensing circuitry 150 is coupled to memory cells in tiles 141, 142. Sensing circuitry 150 is used to sense one or more characteristics of the memory cells. In one embodiment, sensing circuitry 150 includes circuitry for precharging the bit lines of tiles 141, 142. Sensing circuitry 150 is configured to receive signals from controller 124 and / or read register 160 to determine which bit lines are to be disabled (e.g., as described below). In one embodiment, sensing circuitry 150 includes an ADC or other digital converter to convert the sum of output currents from the memory cells accumulated on enabled access lines (e.g., accumulated on enabled bit lines) to provide a digital result (e.g., an accumulated result).

[0073] The inference logic circuit 123 may be further configured to perform inference calculations based on weights stored in the memory cell array 113 (e.g., calculations of an artificial neural network) and inputs derived from image data generated by the image sensing pixel array 111. Optionally, the inference logic circuit 123 may include a programmable processor that can execute a set of instructions to control the inference calculations. Alternatively, the inference calculations are configured for a specific artificial neural network, certain aspects of which can be adjusted via weights stored in the memory cell array 113. Optionally, the inference logic circuit 123 is implemented via the core of an application specific integrated circuit (ASIC), a field programmable gate array (FPGA), or a programmable microprocessor.

[0074] In one embodiment, the inference logic circuit 123 includes a controller 124. In one example, the controller 124 manages communication with the host system via the interface 125. In one example, the controller 124 uses the memory cell array 113 to perform signed or unsigned multiplication. In one embodiment, the controller 124 selects either signed or unsigned multiplication to be performed based on the type of data used as input to the multiplication. In one example, the controller 124 selects signed multiplication in response to determining that the input to the multiplication is signed.

[0075] exist Figure 1 , the integrated circuit die 105 having the memory cell array 113 has a bottom surface 133; and the integrated circuit die 109 having the inference logic circuit 123 has a portion of the top surface 134. The two surfaces 133 and 134 can be connected via bonding (e.g., using hybrid bonding) to provide a portion of the interconnect 107 between the metal portions on the surfaces 133 and 134.

[0076] Similarly, the integrated circuit die 103 having the image sensing pixel array 111 has a bottom surface 131; and the integrated circuit die 109 having the inference logic circuit 123 has another portion of its top surface 132. The two surfaces 131 and 132 may be connected via bonding (e.g., using hybrid bonding) to provide a portion of the interconnect 107 between the metal portions on the surfaces 131 and 132.

[0077] Image sensing pixels in array 111 may include photosensitive elements configured to generate signals in response to the intensity of light received in the element. For example, image sensing pixels implemented using complementary metal oxide semiconductor (CMOS) technology or charge coupled device (CCD) technology may be used.

[0078] In some implementations, image processing logic 121 is configured to pre-process images from image sensing pixel array 111 to provide the processed images as input to inference computations controlled by inference logic 123. Optionally, image processing logic 121 may also use multiplication and accumulation functions provided via memory cell array 113.

[0079] In some implementations, interconnect 107 includes wires for writing image data from image sensing pixel array 111 to a portion of memory cell array 113 for further processing by image processing logic circuit 121 or inference logic circuit 123 or for retrieval via interface 125. Inference logic circuit 123 can buffer the results of the inference calculations in a portion of memory cell array 113.

[0080] The interface 125 of the integrated circuit device 101 may be configured to support the memory access protocol or the storage access protocol or any combination thereof. Thus, an external device (eg, a processor, a central processing unit) may send commands to the interface 125 to access the storage capacity provided by the memory cell array 113 .

[0081] For example, the interface 125 may be configured to support connections and communication protocols on a computer bus, such as a Peripheral Component Interconnect Express (PCIe) bus, a Serial Advanced Technology Attachment (SATA) bus, a Universal Serial Bus (USB) bus, a Compute Express link, etc. In some embodiments, the interface 125 may be configured to include an interface for a solid-state drive (SSD), such as a ball grid array (BGA) SSD. In some embodiments, the interface 125 is configured to include an interface for a memory module, such as a double data rate (DDR) memory module, a dual in-line memory module, etc. The interface 125 may be configured to support communication protocols, such as protocols according to the Non-Volatile Memory Express (NVMe), the Non-Volatile Memory Host Controller Interface Specification (NVMHCIS), etc.

[0082] From the perspective of devices communicating with the interface 125, the integrated circuit device 101 may be presented as a memory subsystem. Through the interface 125, an external device (e.g., a processor, a central processing unit) may access the storage capacity of the memory cell array 113. For example, the external device may store and update the weight matrix and instructions of the inference logic circuit 123, retrieve images generated by the image sensing pixel array 111 and processed by the image processing logic circuit 121, and retrieve the results of the inference calculations controlled by the inference logic circuit 123.

[0083] Integrated circuit die 105 includes registers 160. Integrated circuit die 109 includes memory 170 that stores a cull map 172 and includes registers 174. In one embodiment, configuration data from a host is received via interface 125. In one example, the configuration data is the cull map 172 or data for setting registers 174 and / or 160. The configuration data corresponds to a processing step being performed for the neural network. Processing includes MVM calculations mapped to tiles 141, 142. Portions of tiles 141, 142 are disabled differently for each processing step based on the configuration data (e.g., selected bit lines are not precharged).

[0084] exist Figure 1 In the embodiment of the present invention, for example, the interface 125 is positioned at the bottom side of the integrated circuit device 101, while the image sensor chip is positioned at the top side of the integrated device 101 to receive incident light for generating an image.

[0085] Figure 1The voltage driver 115 in the array 113 may be controlled to apply a voltage to program the threshold voltage of the memory cells in the array 113. The data stored in the memory cell may be represented by the level of the programmed threshold voltage of the memory cell.

[0086] In one example, the interface 125 is operable to enable the host system to write data into the memory cell array 113 and read data from the memory cell array 113. For example, the host system may send commands to the interface 125 to write a weight matrix of an artificial neural network into the memory cell array 113 and read the output of the artificial neural network, raw image data from the image sensing pixel array 111, or processed image data from the image processing logic circuit 121, or any combination thereof.

[0087] The reasoning logic circuit 123 may be programmable and include a programmable processor, an application specific integrated circuit (ASIC) or a field programmable gate array (FPGA) or any combination thereof. Instructions for implementing the calculation of the artificial neural network may also be written into the memory cell array 113 via the interface 125 for execution by the reasoning logic circuit 123.

[0088] Figure 2 A calculation of a column of weight bits multiplied by a column of input bits to provide an accumulated result is shown according to one embodiment. Figure 2 , 227 (eg, in the memory cell array 113 of the integrated circuit device 101) may be programmed to have a threshold voltage at a level representing a weight of storing one bit per memory cell.

[0089] , 223 (eg, in the voltage driver 115 of the integrated circuit device 101 ) is configured to apply voltages 205 , 215 , . . . , 225 to the memory cells 207 , 217 , . . . , 227 , respectively, according to the input bits 201 , 211 , . . . , 221 received therefrom.

[0090] For example, when the input bit 201 has a value of 1, the voltage driver 203 applies a predetermined read voltage as the voltage 205, thereby causing the memory cell 207 to output a predetermined current amount as its output current 209 to represent a storage weight of 1 when the memory cell 207 has a threshold voltage lower than the predetermined read voltage programmed at a lower level, or to output a negligible current amount as its output current 209 to represent a storage weight of 0 when the memory cell 207 has a threshold voltage higher than the predetermined read voltage programmed at a higher level.

[0091] However, when the input bit 201 has a value of 0, the voltage driver 203 applies a lower level voltage (e.g., 0) below the threshold voltage as the voltage 205 (e.g., no predetermined read voltage is applied), thereby causing the memory cell 207 to output a negligible amount of current at its output current 209, regardless of the weight value stored in the memory cell 207. Therefore, the output current 209, which is a multiple of the predetermined amount of current, represents the result of multiplying the weight value bit stored in the memory cell 207 by the input bit 201.

[0092] Similarly, current 219 passing through memory cell 217 as a multiple of a predetermined current amount represents the result of multiplying the weight bits stored in memory cell 217 by input bit 211; and current 229 passing through memory cell 227 as a multiple of a predetermined current amount represents the result of multiplying the weight bits stored in memory cell 227 by input bit 221.

[0093] The output currents 209, 219, ..., 229 of the memory cells 207, 217, ..., 227 are connected to a common line 241 (e.g., a bit line or source line in the tile 141) for summing. In one example, the common line 241 is a bit line. When summing the output currents, a constant voltage (e.g., ground or -1V) is maintained on the bit line.

[0094] The digital converter 233 of the analog-to-digital converter 245 compares the total current 231 with a unit current 232 equal to a predetermined current amount to determine a digital result 237 of the sum of the columns of weight bits stored in the memory cells 207, 217, ..., 227 respectively multiplied by the columns of input bits 201, 211, ..., 221 and the multiplication results.

[0095] The sum of the negligible current amounts from the memory cells connected to line 241 is less than the unit current 232 (eg, a predetermined current amount). Therefore, the presence of the negligible current amounts from the memory cells does not alter the result 237 and is negligible in the operation of the analog-to-digital converter 245.

[0096] exist Figure 2 , voltages 205, 215, ..., 225 applied to memory cells 207, 217, ..., 227 represent digitized input bits 201, 211, ..., 221; memory cells 207, 217, ..., 227 are programmed to store digitized weight bits; and currents 209, 219, ..., 229 represent digitized results.

[0097] The result 237 is an integer not greater than the count of memory cells 207, 217, ..., 227 connected to line 241. The digitized form of the output current 209, 219, ..., 229 can improve the accuracy and reliability of calculations performed using the memory cells 207, 217, ..., 227.

[0098] In general, the weights involved in the multiplication and accumulation operations may be more than one bit. Memory cells may be used to store different significant bits of the weights (e.g., Fig.13 ) to perform multiplication and accumulation operations. Figure 2 The circuit illustrated in can be viewed as a multiplier-accumulator unit configured to operate on a column of 1-bit weights and a column of 1-bit inputs. Multiple such circuits can be connected in parallel to implement a multiplier-accumulator unit to operate on a column of multi-bit weights and a column of 1-bit inputs.

[0099] Figure 2 The circuits described in 150 can also be used to read data stored in memory cells 207, 217, ..., 227. For example, the sensing circuit system 150 can be used to sense current associated with the memory cells. For example, to read the data or weight stored in the memory cell 207, the input bits 211, ..., 221 can be set to 0 to cause the memory cells 217, ..., 227 to output a negligible amount of current to the line 241 (e.g., as a bit line). The input bit 201 is set to 1 to cause the voltage driver 203 to apply a predetermined read voltage. Therefore, the result 237 from the digital converter 233 provides the data or weight stored in the memory cell 207. Similarly, the data or weight stored in the memory cell 217 can be read by applying 1 as the input bit 211 and applying 0 as the remaining input bits in the column; and the data or weight stored in the memory cell 227 can be read by applying 1 as the input bit 221 and applying 0 as the other input bits in the column.

[0100] Generally speaking, Figure 2 The circuits described in 200 may be used to select any of the memory cells 207, 217, ..., 227 for reading or writing. A voltage driver (e.g., 203) may apply a programming voltage pulse (e.g., one or more pulses or other waveforms appropriate for a memory cell type) to adjust the threshold voltage of a corresponding memory cell (e.g., 207) to erase data, store data or weights, etc.

[0101] In general, the inputs involved in multiplication and accumulation operations can be more than 1 bit. For example, the input bit columns can be applied to the weights stored in the memory cell array one column at a time to obtain the result of multiplying a column of weights by a column of inputs and accumulating the result.

[0102] Figure 2The multiplier-accumulator unit described in Figure 1 In the integrated circuit device 101.

[0103] In one implementation, a memory chip (e.g., integrated circuit die 105) includes circuits for voltage drivers, digitizers, shifters, and adders for performing multiplication and accumulation operations. The memory chip may further include circuits configured to control the operation execution of the drivers, digitizers, shifters, and adders. Figure 2 The control logic of the operations in .

[0104] The inference logic circuit 123 can be configured to use the computing power of the memory chip (such as the integrated circuit die 105) to perform inference calculations of applications, such as inference calculations of artificial neural networks. The inference results can be stored in a portion of the memory cell array 113 for external devices to retrieve via the interface 125 of the integrated circuit device 101.

[0105] Optionally, the voltage driver, digital converter, shifter, adder, and at least a portion of the control logic may be configured in the integrated circuit die 109 of the logic chip.

[0106] Memory cells (e.g., memory cells of array 113) may include volatile memory or non-volatile memory or both. Examples of non-volatile memory include flash memory, memory cells formed based on NAND logic gates, NOR logic gates, phase change memory (PCM), magnetic memory (MRAM), resistance random access memory, cross-point storage, and memory devices. Cross-point memory devices may use transistor-free memory elements, each of which has memory cells and selectors stacked together in a row. The memory element columns are connected via two wire layers running in a vertical direction, wherein the wires of one layer run in one direction, in a layer positioned above the memory element column, and the wires of the other layer run in another direction and in a layer positioned below the memory element column. Each memory element may be individually selected at the intersection of one wire on each of the two layers. Cross-point memory devices are fast and non-volatile and may be used as a unified memory pool for processing and storage. Other examples of nonvolatile memory include read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), and electronically erasable programmable read-only memory (EEPROM) memory, etc. Examples of volatile memory include dynamic random access memory (DRAM) and static random access memory (SRAM).

[0107] Integrated circuit die 105 and integrated circuit die 109 may include circuits for addressing memory cells in memory cell array 113, such as row decoders and column decoders for converting physical addresses into control signals to select a portion of memory cells for reading and writing. Thus, an external device may send commands to interface 125 to write weights into memory cell array 113 and read results from memory cell array 113.

[0108] In some implementations, image processing logic 121 may also send commands to interface 125 to write the image to memory cell array 113 for processing.

[0109] Figure 3 A method for performing calculations in an integrated circuit device based on summing output currents from memory cells according to one embodiment is shown. For example, Figure 3 The method can be used Figure 2 , 4 or 13 multiplication and accumulation techniques in Figure 1 is executed in the integrated circuit device 101.

[0110] Figure 3 The method may be performed by processing logic, which may include hardware (such as a processing device, a circuit system, a dedicated logic, a programmable logic, a microcode, hardware of a device, an integrated circuit, etc.), software (such as instructions running or executing on a processing device), or a combination thereof. In some embodiments, Figure 3 The method is at least partially performed by one or more processing devices (e.g. Figure 1 The inference logic circuit 123 is executed by the controller 124 or the local controller (not shown) of the integrated circuit die 105.

[0111] Although shown in a particular sequence or order, the order of the processes may be modified unless otherwise specified. Therefore, the illustrated embodiments should be understood as examples only, and the illustrated processes may be performed in different orders, and some processes may be performed in parallel. In addition, one or more processes may be omitted in various embodiments. Therefore, not every embodiment requires all processes. Other process flows are possible.

[0112] At block 301, a memory cell (or group of memory cells, such as a group of 4 cells storing bits of a signed weight) is programmed to a target weight for performing a multiplication. In one example, the memory cells of the memory cell array 113 are programmed. In one example, the memory cells 207, 206, 208 are programmed to store weights of different bit significance. The weights correspond to multi-bit weights (e.g. Fig.13 Weight1).

[0113] At block 303 , a voltage is applied to the memory cell. The voltage represents an input bit multiplied by a weight stored by the memory cell. In one example, a voltage driver applies the input voltage 205 , 215 , 225 .

[0114] At block 305, the output currents from the memory cells resulting from the applied voltages are summed. In one example, the output currents are collected and summed using line 241, as in Figure 2 middle.

[0115] At block 307, a digital result based on the summed output current is provided. In one example, the summed output current is used to generate Figure 2 The result is X 237.

[0116] In one embodiment, some of the memory cells have a first threshold voltage programmed to represent a value of 1, and the applied voltage is less than the first threshold voltage.

[0117] In one embodiment, the applied voltage is at least 0.5 volts less than the first threshold voltage.

[0118] In one embodiment, the device further includes an interface (eg, 125) operable to enable a host system to write data to and read data from the memory cell array.

[0119] In one embodiment, the memory cells include first and second memory cells; the corresponding weight stored by the first memory cell is the most significant bit (MSB) of the multi-bit weight; and the corresponding weight stored by the second memory cell is the least significant bit (LSB) of the multi-bit weight.

[0120] In one embodiment, the digitizer is configured in an analog-to-digital converter.

[0121] Figure 4 An analog weight fixed architecture for matrix-vector multiplication (MVM) is shown according to one embodiment. Because the computational burden is primarily on the MVM operations when executing a neural network, an analog weight fixed architecture focused on the MVM operations is used. Other computations / logic required can typically be implemented in the digital and / or analog space as it has relatively little impact on performance and energy efficiency.

[0122] In a weight-fixed architecture, computations are performed where the weights are stored (e.g., in a NAND flash memory device where the weights are stored). This removes or reduces the performance bottleneck and power inefficiency of moving weights out of memory for computation. MVM computations are performed in the analog domain. This typically results in some computational errors that are not present in the digital domain.

[0123] The weights are stored in a memory cell 405 (e.g., a memory cell) within a memory device (e.g., 101). An input is sent to an electrode 408 of the memory cell, resulting in the input being multiplied by the weight (the conductance of the memory cell based on the stored weight) (e.g., the weight of g12 multiplied by the input Vin1). A digital-to-analog converter (DAC) 402, 404 converts the digital input into a magnitude of an analog voltage for driving an electrode 408 (e.g., an access line, such as a select gate drain line).

[0124] The result is summed to another electrode (eg, 406) within the memory array (eg, Figure 2 The common line 241 of the memory cell 405 is connected to the memory cell 405 and detected by the ADC 420, 422. For example, the integrator 410, 412 accumulates the current I1, I2 from the memory cell 405 determined by the conductance of the cell and provides the accumulated current as the input of the ADC 420, 422.

[0125] Figures 5 to 7 Exemplary architectures that can be used to perform MVM on weights stored in memory cells of a three-dimensional (3D) NAND flash memory device are shown. All of these architectures store weights in flash memory cells. The memory cells extend vertically upward from a semiconductor substrate (not shown). The memory cells are arranged in vertical pillars of cells (sometimes referred to as strings). The cells in each pillar / string are connected in series. A bypass voltage is applied to the gates of non-selected memory cells during multiplication.

[0126] The threshold voltage (VT) of the memory cell is set (programmed) based on the expected weight. When the cell is read with a fixed word line voltage, the cell will sink some current that varies depending on the weight stored in the cell (based on the cell IV characteristic).

[0127] The input to the multiplied weight can be introduced to the pillar in various ways. For example, the input is applied as a gate voltage of another cell with a fixed threshold (VT). For example, the select gate is used as a digital input (e.g., by applying a digital time slice pulse stream). For example, the input is applied to the bit line.

[0128] In one example, summing of the multiplication results is done by summing the current at the bit lines. In one example, summing of the multiplication results is done by summing the current at the sources. This approach requires unique source routing, which is not part of conventional 3D NAND architectures.

[0129] Figure 5An analog weight fixing method using a select gate drain (SGD) cell architecture is shown according to one embodiment. For example, each weight (e.g., unsigned or signed bit) is stored in a cell or a group of cells (e.g., 510, 512) with a fixed word line (WL) voltage (e.g., W00, W01, W10, W11) applied to each selected cell. Input is applied to the select gate drain (SGD) lines 502, 504 (e.g., as a digital time slice pulse stream). Select transistors 530, 532 connect each pillar to the bit lines 506, 508. The output currents are summed on the bit lines 506, 508. A bypass voltage is applied to the non-selected cells 520, 522 during multiplication.

[0130] Figure 6 An analog weight fixing method using a two-cell architecture (e.g., cell pair 604, 606) is shown according to one embodiment. For example, each weight is stored in one cell 612 with a fixed WL voltage applied during multiplication. The input is applied to the gate of the other cell 610 in the pair. Cell 610 has a fixed threshold voltage VT applied during multiplication.

[0131] The output current is accumulated on the bit line 602. Sense amplifier 620 is used to sense the accumulated total current. Output circuitry 630 provides a digital output corresponding to the magnitude of the total current.

[0132] Figure 7 An analog weight fixing method using a bit line cell architecture according to one embodiment is shown. For example, each weight is stored in a selected cell 710, 712. A fixed WL voltage is applied to word lines 720, 722. Inputs are applied to bit lines 702, 704. The sum of currents is accumulated using source lines 730, 732.

[0133] Various memory cell implementations can be used to perform signed multiplication. In one embodiment, signed multiplication is performed in a so-called four-quadrant system, where each of the inputs and weights to be multiplied can have a positive or negative sign. For example, some neural network models utilize matrix-vector multiplication where the weights of the model are signed. In one example, resistive random access memory (RRAM) cells are used. In one example, NAND or NOR flash memory cells are used.

[0134] In one embodiment, matrix-vector multiplication is performed using stored weights. The input signal is multiplied by the weights to provide a result. In one example, the weights are determined by training a neural network model. The model uses both positive and negative values ​​of the weights. In one example, the weights are stored in Figure 1In a memory cell of the memory cell array 113 of . In one example, the model is trained using image data, and the trained model provides an inference result based on input from the image sensor.

[0135] In one embodiment, the multiplier accumulator unit uses signed multiplication. The weight can be represented by a multi-bit value (e.g., 8 to 64 bits). The extra bit is used to represent the sign of the weight. For example, the system can use an 8-bit signed weight, where the value of the weight is represented by 7 bits, and the eighth bit is used to represent the sign. The extra bit can be used in a similar manner to the signed input.

[0136] In one embodiment, a signed 1-bit number (e.g., input and / or weight) has one of three possible values: -1, 0, 1. For example, a signed weight may be represented by a 2-bit number, where a 2-bit value of 01 represents a signed 1-bit value of -1; a 2-bit value of 00 represents a signed 1-bit value of 0; and a 2-bit value of 10 represents a signed 1-bit value of +1. The 2-bit value 11 is not used. In other examples, various combinations of 2 bits may represent different signed values, as desired for a given implementation.

[0137] In one example, a controller that controls multiplication manages 2-bit values ​​by keeping track of the meaning (e.g., sign or magnitude) represented by each bit. In one example, the controller is Figure 1 Part of the inference logic circuit 123.

[0138] A 1-bit by 1-bit multiplication of a 2-bit number representing a signed 1-bit input and a signed 1-bit weight can be configured to produce a signed 1-bit by 1-bit multiplication result. In one example, the result has been generated in response to a 1-bit multiplication of a 2-bit number represented by a signed 1-bit input and a signed 1-bit weight. Figure 1 In one example, the signed input used to generate the result is based on the request from the host system through the interface 125. Figure 1 The image sensing pixel array 111 collects data.

[0139] In one embodiment, a two-cell implementation is used for signed 1-bit by 1-bit multiplication. A set of two memory cells is used to store two bits of a signed 1-bit weight in a two-bit representation.

[0140] The two input lines are used to apply two bits (two-bit representation) of a signed 1-bit input (sometimes referred to herein as the "positive version") at a first moment (e.g., the first clock cycle T0) and then apply the negative version of the input at a second moment (e.g., the second clock cycle T1).

[0141] In one embodiment, a four-cell implementation is used for signed 1-bit by 1-bit multiplication. A set of four memory cells is used to store two bits of a signed 1-bit weight in a two-bit representation (sometimes referred to herein as a "positive version") while storing two bits of a negative version of the two-bit representation. Two input lines are used to apply two bits of a signed 1-bit input (two-bit representation).

[0142] In one example, an input line provides a voltage to a group of memory cells. The group has four memory cells. In one example, the input line can be a word line, a bit line, or a select gate line (SL or SGD), depending on the type of memory cell and the specific group configuration (e.g., memory cells arranged in series for NAND flash versus memory cells arranged in parallel for RRAM or NOR).

[0143] The first pair of memory cells multiplies the signed input. The output currents are summed on the first line. The second pair of memory cells also multiplies the signed input. The output currents are summed on the second line.

[0144] The first line of bit results (e.g., 0 or 1) provides the first bit of a signed 1-bit by 1-bit multiplication (two-bit representation). The second line of bit results (e.g., 0 or 1) provides the second bit of a signed 1-bit by 1-bit multiplication (two-bit representation). In one example, these first and second bit results provide a 1-bit signed result.

[0145] Figure 8 An exemplary implementation of a deep neural network according to one embodiment is shown. An image 802 is provided as an input to the neural network. The neural network includes convolutional layers 810, 812, 814, etc. The size of each layer is different. For example, each layer has a different number of features and neurons. For example, layer 810 uses a smaller number of filters than layer 814.

[0146] The neural network provides a final result 820. In one example, the result 820 is a classification of the object represented by the image 802.

[0147] When performing calculations, matrix-vector multiplication operations are mapped to tiles in a memory cell array, such as 113. For example, this mapping involves identifying portions of the memory cell array that will be used during calculations for a particular layer. This mapping typically changes as calculations proceed from one layer to another.

[0148] In one embodiment, the host device sends configuration data identifying one or more characteristics associated with the computations performed on the next layer of the neural network to the controller. The controller uses the characteristics to determine which portions of the tiles in the memory cell array are to be enabled or disabled. In one example, the host device provides a cull map to the controller. The signal for controlling the disablement is sent before the MVM for the next layer.

[0149] In one example, image 802 is data obtained from image sensing pixel array 111. In one example, weights of the neural network have been programmed into memory cells of tiles 141, 142. In one example, a different culling map 172 is provided by the host device for each layer 810, 812, 814, etc. as computation proceeds from one layer to another.

[0150] Fig. 9 An exemplary arrangement of memory cells for a tile of a NAND flash memory array is shown according to one embodiment. The NAND flash memory array is an example of memory cell array 113. The tiles are examples of tiles 141, 142. The memory cells are arranged in vertical strings 910 (e.g., extending above a semiconductor substrate (not shown)).

[0151] For example, the illustrated tile has a size of 512 features and 512 neurons. The tile has 1,024 bit lines and 1,024 select gate drain (SGD) lines because the tile is configured to store signed weights for each of the 512 neurons. For example, group 902 includes four selected memory cells (indicated by W+, W-) that store bits (e.g., LSB bits or MSB bits) of the signed weights.

[0152] The input to the multiplication is provided on a select gate line 904. The select gate line is used to turn on or off a select transistor (e.g., 905) depending on the value of the input. For example, each bit position of the input feature vector (X0, X1, X2, etc.) runs serially. Each Xn is the same bit position for each of the 512 features. The output current from the selected memory cell is accumulated on a bit line (e.g., 906).

[0153] Fig.10 Exemplary tiles of a memory device are shown, where each tile can be configured to be partially filled for performing multiplication or other operations of a neural network, according to one embodiment. Tiles 1002, 1004, 1006, 1008, 1010, 1012, etc. are tiles in an array of memory cells (e.g., 113). In one example, the array includes about 1,500 NAND tiles. The tiles are filled (programmed) with weights for neurons to be used (e.g., for at least one layer 810, 812, 814). The specific weights effective for a given MVM operation will vary, as discussed above.

[0154] For example, portion 1030 of tile 1002 partially fills the tile because the number of neurons used for the corresponding MVM calculation is less than the fixed tile size of the memory cell array. In some cases, for a given layer of the neural network, all tiles are partially filled to the same extent, as indicated by portions 1030, 1032, 1034. In other cases, each tile may be partially filled in a different manner than other tiles, as indicated by portions 1040, 1050.

[0155] For example, different bit lines and / or other access lines may be disabled for each respective tile (eg, in different relative positions of the tile).The cull map may indicate a unique disabling configuration for each tile of the array.

[0156] Fig.11 The bit lines of a tile that is disabled based on a cull map according to one embodiment are shown. In one example, the tile corresponds to Fig. 9 or Fig.10 . The tile includes neurons 1104 and features 1102. In one example, each of the neurons 1104 corresponds to a bit line or source line for accumulating the output current of the memory cell, as described above. In one example, each of the features 1102 corresponds to a select gate drain line 1120, 1122 for providing one or more input bits to be multiplied with the weight stored in the memory cell.

[0157] In preparation for the matrix-vector multiplication operation, the controller causes various access lines of the tile to be disabled. These access lines may include bit lines or source lines and / or select gate drain lines 1120, 1122, as illustrated. These access lines may further include Fig.11 Word lines and / or other lines of the memory cell array not illustrated.

[0158] In one embodiment, based on the cull map (e.g., 172), bit lines 1106 to 1108 are enabled. These bit lines will be precharged and used during the multiplication operation. Other bit lines 1110 to 1112 are disabled. These bit lines will not be precharged and are not used during the current multiplication operation.

[0159] In one embodiment, each bit line is connected to an analog to digital converter (ADC) 1130, 1132. ADC 1130 will be charged and used during the multiplication operation. ADC 1132 will be disabled and not used during the operation. This reduces power consumption of unused neurons.

[0160] In one embodiment, if the number of features is less than the tile allows, then the features are set to remain inactive because their currents will sum to the bit line or source. Disabling a feature can be accomplished by turning off the select gate (SGD) of that particular feature. This is accomplished, for example, by forcing the input value of all unused features to be 0.

[0161] When a neuron is not used in an MVM operation, its results are completely negligible. Since power efficiency is desired in an analog weight fixed architecture, a cull map (or other configuration data) is used to disable portions of the sensing circuitry (e.g., page buffers / ADCs) to avoid unnecessary bit line / cell currents. The deactivation according to the cull map is set before the MVM operation. This is done, for example, to determine those page buffers / ADCs that contain valid neurons.

[0162] Fig.12 Sense circuitry is shown coupled to a bit line 1204 for accessing a NAND flash memory cell according to one embodiment. The memory cells are located in a string 1202. Select gate drain and source transistors 1206, 1208 are used to control access to the string 1202. The select gate transistor 1206 is coupled to the bit line 1204.

[0163] The sensing circuitry includes a current source 1218 for precharging the bit line 1204 in preparation for sensing the current (e.g., summing the output current) and / or the state of a selected memory cell in the string 1202. The sensing circuitry includes a transistor 1216 that is turned on or off to enable or disable charging of the bit line 1204. The sensing circuitry is connected to the bit line 1204 through a transistor 1210.

[0164] In one embodiment, a control signal is applied by a controller (eg, 124) to the gate of transistor 1216 based on configuration data received from a host device. In one example, transistor 1216 is turned on or off based on a cull map (eg, 172).

[0165] When the sensing circuitry of the bit line 1204 is disabled using transistor 1216, power consumption is reduced by avoiding unnecessary charging of node 1212, which corresponds to capacitance 1214 (eg, parasitic capacitance of the sensing circuitry). Also, unnecessary charging of the bit line 1204 is avoided.

[0166] In one embodiment, based on the culling map, the sensing circuitry does not charge the source lines (SRC) of unused neurons. This avoids wasting energy to charge nodes that are meaningless in the desired MVM calculation. It also avoids unused neuron pillars from drawing current.

[0167] The data for disabling the bit lines and / or other access lines can be provided in various ways. In one embodiment, the host writes a pattern of 1s and 0s to the sensing circuitry (e.g., page buffers / ADCs) that specifies those neurons to be culled (disabled). The default condition may be active to reduce I / O traffic to / from the host.

[0168] In one embodiment, the most common neuron sizes (e.g., 64, 128, etc.) are provided as features / trims that can be set by the host (e.g., using registers 174). The page buffer / ADC is loaded with 1s and 0s (e.g., to turn transistor 1216 on or off) based on the settings provided by the host, removing I / O traffic requirements.

[0169] In one embodiment, latches are used to store a knockout map or other configuration data. The host can directly write patterns 1 and 0. This pattern is an example of a knockout map.

[0170] In one embodiment, rather than using configuration data from a host, configuration data may be provided from a local controller of the NAND memory device.

[0171] In one embodiment, the culling map or pattern is determined by the characteristics of the neural network, such as its structure and / or the current computational step (e.g., the current layer being used). This determination can be done when mapping the current neural network layer to the tile size.

[0172] In one embodiment, the NAND memory device has registers that are exposed to the host interface. The host can set the registers to configure the NAND device (e.g., the parameters can be defined by the host). For example, the NAND device can provide fixed options to the host of certain predefined neuron sizes. The host can select the neuron size that is closest to the size of the current calculation among the predefined neuron sizes. The NAND device uses logic circuitry to set the culling configuration based on the definition made by the host of the predefined neuron sizes.

[0173] In one embodiment, the host or local controller communicates the neuron size in a register. In one embodiment, the NAND device selects one of the above predefined neuron sizes based on the neuron size stored in the register.

[0174] In one embodiment, the memory device uses an array of memory cells organized into groups of memory cells. In one example, resistive random access memory (RRAM) cells are used. In one example, NAND or NOR flash memory cells are used.

[0175] Each group can be programmed to store a multi-bit signed weight. After programming, a voltage driver applies a voltage to the memory cells in each group. The voltage represents the multi-bit signed input multiplied by the multi-bit signed weight.

[0176] One or more common lines are coupled to each group. The lines receive one or more output currents from the memory cells in each group (eg, similar to the discussion above for groups of two or four cells). Each common line accumulates currents to sum the output currents from the group.

[0177] In one example, the line is a bit line extending vertically above the semiconductor substrate. As an example, 512 memory cell groups are coupled to the line. Input is provided using 512 pairs of select lines (e.g., SL+, SL-), with one pair used for each group. Output current from each of the 512 groups is collected on the line, and then one or more total current magnitudes are digitized to provide first and second digital values.

[0178] In one example, the memory device includes one or more digitizers.The digitizers provide a signed result based on summing the output currents from each of the 512 sets on the first and second digit lines (eg, as described above).

[0179] A first digital value (e.g., an integer) representing a current on a first digit line is determined as a multiple of a predetermined current (e.g., as described above) representing 1. A second digital value representing a current on a second digit line is determined as a multiple of the predetermined current. For example, the first and second digital values ​​are outputs from a digitizer.

[0180] In one embodiment, a memory device includes a memory cell array having groups of NAND flash memory cells (eg, using Fig. 9 ). Each group is programmable to store a multi-bit signed weight. A voltage driver applies a voltage to each group. The voltage corresponds to a multi-bit signed input multiplied by the multi-bit signed weight of each group. Two common lines are coupled to each group. Each common line sums the corresponding output current from each group. The digital converter on each common line provides a signed result based on summing the output current from the group. Each signed result corresponds to the bit significance of the input and the bit significance of the weight, such as described above. The signed results are added together in view of the corresponding bit significance to provide first and second digital values ​​representing the signed accumulation results from the multi-bit and multi-bit multiplication.

[0181] In one embodiment, a signed input is applied to a group of memory cells on two wires (e.g., two select lines), each carrying a signal. Whether the input is positive or negative depends on where the magnitude of the signal is provided. In other words, the sign depends on which wire carries the signal. The other wire carries a constant value signal (e.g., a constant voltage corresponding to 0).

[0182] Each signed input applied to a group is considered to have a positive magnitude. One of the two wires is always biased to 0 (more generally, the bias is a constant signal). The other wire carries the magnitude of the input pattern.

[0183] In one embodiment, a multi-bit input is represented as a serial or time-slice input provided on two wires. For example, an input pattern is a number of bits (e.g., 1101011) whose corresponding voltages are applied serially to the wires, one bit per time slice. In one example, the input bits are applied serially one at a time.

[0184] In one embodiment, the weighting of the output current from each of the memory cells in the common line changes corresponding to the MSB, MID, or LSB significance of the bit stored by the memory cell (e.g., for 3 bits stored in a group of 3 memory cells as described above). The MSB significance (e.g., 100 nA) is weighted twice as much as the MID significance (e.g., 50 nA). The MID significance is weighted twice as much as the LSB significance (e.g., 25 nA).

[0185] When the output current ratio considers the bit significance, no left shift is required when adding the signed results (such as the first, second, third and fourth signed results) to obtain the signed accumulation result. Instead, the signed results can be added directly without left shift.

[0186] In one embodiment, a memory device performs an analog summation of 1-bit resultant currents with different bit significances implemented via different bias voltage levels. A memory cell (e.g., an RRAM cell or a NAND flash memory cell) can be programmed to have an exponentially increasing (e.g., increasing by powers of 2) current for different bias voltage levels.

[0187] In one embodiment, the memory cell may be programmed to have a threshold and exponentially increasing current for higher bias / applied voltages.A first voltage may be applied to the memory cell to allow a predetermined amount of current (indicated as 1X) to pass to represent a bit value of 1 for the least significant bit.

[0188] To represent a bit value of 1 for the second least significant bit, a second voltage may be applied to the memory cell to allow twice (indicated as 2X) the predetermined amount of current to pass, which is equal to the predetermined amount of current multiplied by the bit significance of the second least significant bit.

[0189] The memory cell may similarly be biased to have a higher current amount when the bit value is one that is equal to a predetermined current amount multiplied by the bit significance of the bit.

[0190] When different voltages are applied to memory cells that each represent a bit in a number so that the corresponding bit significance of each cell is built into the output current as described above, the results of multiplications involving the memory cells can be summed by connecting the memory cells to the line without having to individually convert the current of the bits for summing.

[0191] For example, a 3-bit resolution weight can be implemented using three memory cells. Each memory cell stores 1 bit of the 3-bit weight. Each memory cell is biased at a separate voltage level so that if it is programmed in a state representing 1, the current through the cell is the base unit multiplied by the bit significance of the cell. For example, the current through the cell storing the least significant bit (LSB) is 25nA per base unit, the cell storing the middle bit (MID) is 2x (50nA) per base unit, and the most significant bit (MSB) is 4x (100nA) per base unit.

[0192] In one embodiment, a solid state drive (SSD) or other storage device uses a memory cell array having memory cells. In one example, resistive random access memory (RRAM) cells are used. In one example, NAND or NOR flash memory cells are used.

[0193] In one embodiment, each memory cell can be programmed to store one bit of a multi-bit weight value. After programming, a voltage driver applies different voltages to bias the memory cell for performing multiplication. The input to be multiplied by the multi-bit weight value can be represented by a corresponding input pattern (e.g., as described above) applied to a select gate of a select transistor coupled to the memory cell or by varying different voltages between a fixed voltage state representing an input bit 1 and a zero state representing an input bit 0.

[0194] One or more common lines are coupled to the memory cells. The lines receive one or more output currents from the memory cells (eg, as described above). Each common line (eg, a digital or bit line) is used to add currents to sum the output currents.

[0195] In one embodiment, three memory cells store three-bit values ​​representing the storage weight. One bit is for the MSB, one bit is for the middle significant bit (sometimes indicated as "MID" herein), and one bit is for the LSB. This provides a multi-bit representation of the storage weight.

[0196] Fig.13An architecture with resistive random access memory (RRAM) or NOR memory cells arranged in a parallel configuration in a memory cell array 1302 to perform multiplication (e.g., MVM) according to one embodiment is shown. For example, memory cells 1330, 1331, 1332 store respective significant bits of a multi-bit weight (indicated as Weight1). Simple 3-bit weights are illustrated, but more bits may be stored for each weight. When performing multiplication, each of the memory cells 1330, 1331, 1332 may be accessed in parallel. In one example, the memory cell array 1302 includes a plurality of memory cells 1330, 1331, 1332, each of which may be accessed in parallel. Fig. 9 Memory cells arranged as described in the figure.

[0197] Each memory cell provides an output current corresponding to the validity of the bit stored by the memory cell. The memory cells 1330, 1331, 1332 are connected to a common line 1310 to add the output currents. In one example, the line 1310 is a bit line.

[0198] Different voltages V1, V2, V3 are applied to memory cells 1330, 1331, 1332 using word lines 1320, 1321, 1322. The voltages are selected so that the output current varies by powers of 2 based on the bit significance, such as described above.

[0199] In one embodiment, input signal I1 is applied to the gate of select transistor 1340. Select transistor 1340 is coupled to common line 1310. The output of select transistor 1340 provides the sum of output currents. In one embodiment, when the input signal is applied to the gate of select transistor 1340, different voltages V1, V2, V3 are maintained at a constant voltage level.

[0200] In an alternative embodiment, the input pattern multiplied by Weight1 may be applied to word lines 1320 , 1321 , 1322 by varying different voltages V1 , V2 , V3 between a fixed voltage and a zero voltage similar to the above description to represent input bits 1 or 0, respectively.

[0201] The memory cell array 1302 is formed over a semiconductor substrate 1304. In one embodiment, the memory cell array 1302 and the semiconductor substrate 1304 are positioned on different chips or wafers before being assembled (eg, combined by bonding).

[0202] Similarly, as described above for Weight1, multi-bit weights Weight2 and Weight3 may be stored in other memory cells of the memory cell array 1302, and the output currents are accumulated on common lines 1311, 1312, as illustrated. These other memory cells may be accessed using word lines 1320, 1321, 1322. Common lines 1311, 1312 are coupled to select transistors 1341, 1342, each of which provides the sum of the output currents as an output. Input patterns I2, I3 may be applied to the gates of the select transistors. Additional weights may be stored in the memory cell array 1302.

[0203] The output currents from the common lines 1310, 1311, 1312 are summed by summing circuitry 1350. In one embodiment, summing circuitry 1350 is formed in semiconductor substrate 1304 (eg, at the top surface).

[0204] In one embodiment, voltage driver 1306 and bias circuitry 1305 are formed in semiconductor substrate 1304. Logic circuitry (not shown) formed in semiconductor substrate 1304 is used to implement controller 1303. Controller 1303 controls voltage driver 1306 and bias circuitry 1305.

[0205] In one embodiment, voltage driver 1306 provides different voltages V1, V2, V3. Bias circuitry 1305 applies inputs I1, I2, I3.

[0206] Fig.14 A method for performing multiplication using memory cells in a memory device having configurable tiles according to one embodiment is shown. For example, Fig.14 The method can be found in Figure 1 The multiplication is performed in the integrated circuit device 101 similarly to that performed for the multiplication (eg, as described in the various embodiments above).

[0207] Fig.14 The method may be performed by processing logic, which may include hardware (such as a processing device, a circuit system, a dedicated logic, a programmable logic, a microcode, hardware of a device, an integrated circuit, etc.), software (such as instructions running or executing on a processing device), or a combination thereof. In some embodiments, Fig.14 The method is at least partially performed by one or more processing devices (e.g. Figure 1 Controller 124) executes.

[0208] Although shown in a particular sequence or order, the order of the processes may be modified unless otherwise specified. Therefore, the illustrated embodiments should be understood as examples only, and the illustrated processes may be performed in different orders, and some processes may be performed in parallel. In addition, one or more processes may be omitted in various embodiments. Therefore, not every embodiment requires all processes. Other process flows are possible.

[0209] At block 1401, memory cells of one or more tiles in a memory array are programmed to store weights of a neural network. In one example, memory cells in tiles 141, 142 are programmed to store weights. In one example, the weights correspond to a neural network used by a host device to perform matrix-vector multiplication. In one example, the weights are received by interface 125 of integrated circuit device 101. In one example, the weights correspond to Figure 8 In one example, the memory array is Fig.13 Memory cell array 1302.

[0210] At block 1403, configuration data indicating characteristics associated with the neural network is received. In one example, the configuration data is a culling map 172. In one example, the configuration data is stored in registers 174. In one example, the configuration data is generated by a local controller of the integrated circuit die 105.

[0211] At block 1405, a respective portion of each tile is selected to be disabled. The selection is based on the received configuration data. In one example, portion 1030 of tile 1002 is enabled and the remainder of tile 1002 is disabled. In one example, bit lines 1110-1112 are disabled based on a mode or other signal received from a host device operating the neural network.

[0212] At block 1407, the weights are multiplied by the inputs to provide at least one result. The result is based on summing the output currents from the memory cells. In one example, the inputs are features derived from the image 802. In one example, the inputs are data derived from sensors of the integrated circuit device 101. In one example, the result is Figure 2 The result is 237.

[0213] In one embodiment, a device includes: a host interface (e.g., 125) configured to communicate with a host; a plurality of tiles (e.g., 141, 142) configured to store weights (e.g., corresponding to Figure 8weights of a neural network); and a logic circuit system (e.g., 123, controller 124) configured to: program a first memory cell of a tile to store a first weight; receive configuration data from a host via a host interface, the configuration data indicating characteristics associated with the neural network; select a corresponding first portion of each tile to be disabled based on the configuration data; receive a first input from the host via the host interface; perform multiplication of the first weight by the first input to provide at least one first result, wherein the first result is based on summing output currents from the first memory cells, and wherein the multiplication is performed without using second memory cells located in the disabled corresponding first portion of each tile; and send the first result to the host via the host interface.

[0214] In one embodiment, the neural network includes a plurality of layers, and the characteristic is a layer for generating an inference result using a first input.

[0215] In one embodiment, the configuration data includes a culling map (eg, 172) indicating a respective first portion of each tile to be disabled.

[0216] In one embodiment, the logic circuitry is further configured to disable charging of access lines coupled to memory cells that will not be used in the multiplication.

[0217] In one embodiment, the access line includes at least one of a bit line, a select gate line, a source line, a word line, or a digit line.

[0218] In one embodiment, the logic circuitry is further configured to map layers of the neural network to the tiles, and the configuration data is selected by the host based on one of the layers for performing the multiplication of the first weight.

[0219] In one embodiment, the first memory cell is programmed by changing the respective threshold voltage of each memory cell.

[0220] In one embodiment, the multiplication is a matrix-vector multiplication for producing an inference output from a neural network, and the first input is a first input vector corresponding to data collected by at least one sensor (eg, 111).

[0221] In one embodiment, a device includes: a semiconductor substrate (eg, 1304); a memory cell array (eg, 1302) formed above the semiconductor substrate; Fig. 9A memory array of the present invention includes NAND flash memory cells organized in pillars extending vertically upward from a semiconductor substrate; a controller (e.g., 124, 1303) formed on the semiconductor substrate and configured to generate a culling map (e.g., 172) based on a first calculation performed on a neural network and to perform a first calculation based on accumulating output currents from first memory cells in the array; and voltage drivers (e.g., 115, 1306) formed on the semiconductor substrate and configured to drive bit lines coupled to the NAND flash memory cells, wherein for the first calculation, each of the voltage drivers is enabled or disabled based on the culling map.

[0222] In one embodiment, prior to performing the first computation, the first memory cell is programmed to a respective threshold voltage corresponding to a value of a weight of the neural network.

[0223] In one embodiment, the output current is accumulated on those bit lines connected to the first memory cell.

[0224] In one embodiment, the output current is accumulated on a source line connected to the first memory cell.

[0225] In one embodiment, NAND flash memory cells are organized into groups of memory cells, and each group of memory cells is programmable to store bits of a multi-bit weight value (e.g., 128 bits of a multi-bit signed weight value are stored in the array, with each bit being stored by a group of two or four memory cells).

[0226] In one embodiment, NAND flash memory cells are organized in horizontal levels, with the levels stacked vertically above a semiconductor substrate.

[0227] In one embodiment, a system includes: registers (e.g., 160, 174) (e.g., latches) configured to store configuration data associated with a neural network; a plurality of access lines; memory cells coupled to the access lines, wherein the memory cells are configured to store weights of the neural network; analog circuitry (e.g., integrators 410, 412) (e.g., accumulation circuitry 1350) configured to accumulate output currents from the memory cells during multiplication operations used to make inferences by the neural network; and sensing circuitry (e.g., sensing circuitry 150) (e.g., sense amplifier 620) coupled to each of the access lines and configured to enable or disable each access line based on the configuration data, wherein the output currents are accumulated using at least one of the enabled access lines.

[0228] In one embodiment, the number of access lines enabled corresponds to a characteristic (e.g., size) of the neural network (e.g., number of features and / or number of neurons) (e.g., current layer of the network used for computation) (e.g., type of neural network used) (e.g., type of training data used to train the neural network).

[0229] In one embodiment, the sensing circuit system (see, for example, Fig.12 ) includes a respective transistor (e.g., 1216) coupling each access line to a voltage driver for charging the access line (e.g., bit line 1204), and wherein the transistor is turned off for those corresponding access lines that are disabled.

[0230] In one embodiment, the configuration data indicates a neuron size.

[0231] In one embodiment, the memory cells are arranged in strings of memory cells connected in series, and each access line is a bit line connected to a respective one or more of the strings.

[0232] In one embodiment, the configuration data is a pattern received from a host, and the pattern corresponds to a portion of a neural network for multiplication operations.

[0233] Integrated circuit device 101 (e.g., as in Figure 1 ) can be configured as a storage device, a memory module, or a mixture of a storage device and a memory module. Examples of storage devices include solid-state drives (SSDs), flash drives, universal serial bus (USB) flash drives, embedded multimedia controller (eMMC) drives, universal flash storage (UFS) drives, secure digital (SD) cards, and hard disk drives (HDDs). Examples of memory modules include dual in-line memory modules (DIMMs), small form factor DIMMs (SO-DIMMs), and various types of non-volatile dual in-line memory modules (NVDIMMs).

[0234] Integrated circuit device 101 (e.g., as in Figure 1 The image sensor (in the embodiment of the present invention) can be installed in a computing system as a memory subsystem with embedded image sensor and inference computing capability. Such computing system can be a computing device, such as a desktop computer, a laptop computer, a network server, a mobile device, a part of a vehicle (such as an airplane, a drone, a train, a car or other transportation vehicle), an Internet of Things (IoT) enabled device, an embedded computer (such as an embedded computer included in a vehicle, industrial equipment or a networked commercial device), or such a computing device including a memory and a processing device.

[0235] Generally speaking, a computing system may include a computer coupled to one or more memory subsystems (e.g., Figure 1In one example, the host system is coupled to a memory subsystem.

[0236] As used herein, "coupled to" or "coupled with..." generally refers to a connection between components, which can be an indirect communication connection or a direct communication connection (e.g., without intervening components), whether wired or wireless, including connections such as electrical, optical, magnetic, etc.

[0237] For example, the host system may include a processor chipset (e.g., a processing device) and a software stack executed by the processor chipset. The processor chipset may include one or more cores, one or more caches, a memory controller (e.g., an NVDIMM controller), and a storage protocol controller (e.g., a PCIe controller, a SATA controller). The host system uses the memory subsystem, for example, to write data to the memory subsystem and read data from the memory subsystem.

[0238] The host system may be coupled to the memory subsystem via a physical host interface. Examples of the physical host interface include, but are not limited to, a serial advanced technology attachment (SATA) interface, a peripheral component interconnect express (PCIe) interface, a universal serial bus (USB) interface, a fiber channel, a serial attached SCSI (SAS) interface, a double data rate (DDR) memory bus interface, a small computer system interface (SCSI), a dual in-line memory module (DIMM) interface (e.g., a DIMM slot interface supporting double data rate (DDR)), an open NAND flash interface (ONFI), a double data rate (DDR) interface, a low power double data rate (LPDDR) interface, a compute express link (CXL) interface, or any other interface. The physical host interface may be used to transfer data between the host system and the memory subsystem. When the memory subsystem is coupled to the host system through a PCIe interface, the host system may further utilize an NVM express (NVMe) interface to access components (e.g., memory devices). The physical host interface may provide an interface for transferring control, address, data, and other signals between the memory subsystem and the host system. In general, a host system may access multiple memory subsystems via the same communication connection, multiple separate communication connections, or a combination of communication connections.

[0239] The processing device of the host system may be, for example, a microprocessor, a central processing unit (CPU), a processing core of a processor, an execution unit, etc. In some examples, the controller may be referred to as a memory controller, a memory management unit, or an initiator. In one example, the controller controls communication via a bus coupled between the host system and the memory subsystem. In general, the controller may send commands or requests to the memory subsystem to obtain desired access to the memory device. The controller may further include an interface circuit system that communicates with the memory subsystem. The interface circuit system may convert a response received from the memory subsystem into information for the host system.

[0240] The controller of the host system may communicate with the controller of the memory subsystem to perform operations such as reading data, writing data, or erasing data at the memory device and other such operations. In some examples, the controller is integrated into the same package of the processing device. In other examples, the controller is separated from the package of the processing device. The controller or processing device may include hardware, such as one or more integrated circuits (ICs), discrete components, buffer memory or cache memory, or a combination thereof. The controller or processing device may be a microcontroller, a dedicated logic circuit system (such as a field programmable gate array (FPGA), an application specific integrated circuit (ASIC), etc.), or another suitable processor.

[0241] A memory device may include any combination of different types of non-volatile memory components and volatile memory components. A volatile memory device may be, but is not limited to, a random access memory (RAM) such as dynamic random access memory (DRAM) and synchronous dynamic random access memory (SDRAM).

[0242] Some examples of non-volatile memory components include NAND (or NOT AND) type flash memory and write-in-place memory, such as three-dimensional cross-point ("3D cross-point") memory. Cross-point non-volatile memory arrays can perform bit storage based on bulk resistance changes in combination with a stacked cross-grid data access array. In addition, compared to many flash-based memories, cross-point non-volatile memories can perform write-in-place operations, where non-volatile memory cells can be programmed without first erasing the non-volatile memory cells. NAND-type flash memory includes, for example, two-dimensional NAND (2D NAND) and three-dimensional NAND (3D NAND).

[0243] Each of the memory devices may include one or more memory cell arrays. One type of memory cell, such as a single-level cell (SLC), may store one bit per cell. Other types of memory cells, such as multi-level cells (MLC), three-level cells (TLC), four-level cells (QLC), and five-level cells (PLC), may store multiple bits per cell. In some embodiments, each of the memory devices may include one or more memory cell arrays, such as SLC, MLC, TLC, QLC, PLC, or any combination thereof. In some embodiments, a particular memory device may include an SLC portion, an MLC portion, a TLC portion, a QLC portion, or a PLC portion of a memory cell, or any combination thereof. The memory cells of a memory device may be grouped into pages, which may refer to a logical unit of a memory device for storing data. With respect to some types of memory, such as NAND, pages may be grouped to form blocks.

[0244] Although non-volatile memory devices such as 3D cross-point type and NAND type memory (e.g., 2D NAND, 3D NAND) are described, the memory device may be based on any other type of non-volatile memory, such as read-only memory (ROM), phase change memory (PCM), self-select memory, other chalcogenide-based memory, ferroelectric transistor random access memory (FeTRAM), ferroelectric random access memory (FeRAM), magnetic random access memory (MRAM), spin transfer torque (STT)-MRAM, conductive bridging RAM (CBRAM), resistive random access memory (RRAM), oxide-based RRAM (OxRAM), NOR (NOR) flash memory, and electrically erasable programmable read-only memory (EEPROM).

[0245] A memory subsystem controller (or simply controller) may communicate with a memory device to perform operations such as reading data, writing data, or erasing data at the memory device and other such operations (e.g., in response to commands dispatched by the controller on a command bus). The controller may include hardware such as one or more integrated circuits (ICs), discrete components, or buffer memory, or a combination thereof. The hardware may include digital circuitry with dedicated (i.e., hard-coded) logic to perform the operations described herein. The controller may be a microcontroller, dedicated logic circuitry (e.g., a field programmable gate array (FPGA), an application specific integrated circuit (ASIC), etc.), or another suitable processor.

[0246] The controller may include a processing device (processor) configured to execute instructions stored in a local memory. In the illustrated example, the local memory of the controller includes an embedded memory configured to store instructions for executing various processes, operations, logic flows, and routines that control the operation of the memory subsystem, including handling communications between the memory subsystem and the host system.

[0247] In some embodiments, the local memory may include memory registers that store memory pointers, fetch data, etc. The local memory may also include a read-only memory (ROM) for storing microcode. While the example memory subsystem includes a controller, in another embodiment of the present disclosure, the memory subsystem does not include a controller, but may instead rely on external control (e.g., provided by an external host or by a processor or controller separate from the memory subsystem).

[0248] In general, the controller may receive commands or operations from the host system and may convert the commands or operations into instructions or appropriate commands to achieve the desired access to the memory device. The controller may be responsible for other operations such as wear leveling operations, obsolete item collection operations, error detection and error correction code (ECC) operations, encryption operations, cache operations, and address translation between logical addresses (e.g., logical block addresses (LBAs), namespaces) and physical addresses (e.g., physical block addresses) associated with the memory device. The controller may further include a host interface circuit system that communicates with the host system via a physical host interface. The host interface circuit system may convert commands received from the host system into command instructions to access the memory device and also convert responses associated with the memory device into information for the host system.

[0249] The memory subsystem may also include additional circuitry or components not illustrated. In some embodiments, the memory subsystem may include a cache or buffer (such as DRAM) and address circuitry (such as row decoders and column decoders) that can receive addresses from a controller and decode the addresses to access memory devices.

[0250] In some embodiments, a memory device includes a local media controller that operates in conjunction with a memory subsystem controller to perform operations on one or more memory cells of the memory device. An external controller (e.g., a memory subsystem controller) can externally manage the memory device (e.g., perform media management operations on the memory device). In some embodiments, the memory device is a managed memory device, which is a raw memory device combined with a local media controller for media management within the same memory device package. An example of a managed memory device is a managed NAND (MNAND) device.

[0251] The controller or memory device may include a storage manager configured to implement the storage functions discussed above. In some embodiments, the controller in the memory subsystem includes at least a portion of the storage manager. In other embodiments or in combination, the controller or processing device in the host system includes at least a portion of the storage manager. For example, the controller or processing device may include logic circuitry that implements the storage manager. For example, the controller or processing device (processor) of the host system may be configured to execute instructions stored in the memory for performing the operations of the storage manager described herein. In some embodiments, the storage manager is implemented in an integrated circuit chip disposed in the memory subsystem. In other embodiments, the storage manager may be part of the firmware of the memory subsystem, the operating system of the host system, a device driver or application, or any combination thereof.

[0252] In one embodiment, an example machine of a computer system in which a set of instructions for causing the machine to perform any one or more of the methodologies discussed herein may be executed. In some embodiments, the computer system may correspond to a host system that includes, is coupled to, or utilizes a memory subsystem or may be used to perform the operations described above. In alternative embodiments, the machine may be connected (e.g., networked) to other machines in a LAN, an intranet, an extranet, or the Internet, or any combination thereof. The machine may operate as a server or a client machine in a client-server network environment, as a peer machine in a peer-to-peer (or distributed) network environment, or as a server or a client machine in a cloud computing infrastructure or environment.

[0253] The machine may be a personal computer (PC), a tablet PC, a set-top box (STB), a personal digital assistant (PDA), a cellular phone, a network appliance, a server, a network router, a switch or a bridge, a network attached storage facility, or any machine capable of executing (sequentially or otherwise) a set of instructions that specify actions to be taken by the machine. Furthermore, while a single machine is described, the term "machine" shall also be taken to include any collection of machines that individually or jointly execute a set (or multiple sets) of instructions to perform any one or more of the methodologies discussed herein.

[0254] An example computer system includes a processing device, a main memory (e.g., read-only memory (ROM), flash memory, dynamic random access memory (DRAM) (e.g., synchronous DRAM (SDRAM) or Rambus DRAM (RDRAM)), static random access memory (SRAM), etc.), and a data storage system, which communicate with each other via a bus (which may include multiple buses).

[0255] The processing device may be one or more general purpose processing devices, such as a microprocessor, a central processing unit, or the like. More specifically, the processing device may be a complex instruction set computing (CISC) microprocessor, a reduced instruction set computing (RISC) microprocessor, a very long instruction word (VLIW) microprocessor, or a processor implementing other instruction sets or several processors implementing a combination of instruction sets. The processing device may also be one or more special purpose processing devices, such as an application specific integrated circuit (ASIC), a field programmable gate array (FPGA), a digital signal processor (DSP), a network processor, or the like. The processing device is configured to execute instructions for performing the operations and steps discussed herein. The computer system may further include a network interface device to communicate via a network.

[0256] The data storage system may include a machine-readable medium (also referred to as a computer-readable medium) on which is stored one or more sets of instructions or software embodying any one or more of the methodologies or functions described herein. The instructions may also reside completely or at least partially within the main memory and / or within the processing device during their execution by the computer system, the main memory and the processing device also constituting machine-readable storage media. The machine-readable medium, data storage system, or main memory may correspond to a memory subsystem.

[0257] In one embodiment, the instructions include instructions for implementing functionality corresponding to the above-described operations. Although the machine-readable medium is shown as a single medium in the example embodiment, the term "machine-readable storage medium" should be deemed to include a single medium or multiple media storing one or more sets of instructions. The term "machine-readable storage medium" should also be deemed to include any medium capable of storing or encoding a set of instructions executed by a machine and causing the machine to perform any one or more of the methodologies of the present disclosure. The term "machine-readable storage medium" should accordingly be deemed to include (but not limited to) solid-state memory, optical media, and magnetic media.

[0258] Some portions of the foregoing detailed description have been presented in terms of algorithms and symbolic representations of operations on data bits within a computer memory. These algorithmic descriptions and representations are the means used by those skilled in the data processing arts to most effectively convey the substance of their work to others skilled in the art. An algorithm is generally conceived here to be a self-consistent sequence of operations leading to a desired result. Operations are those requiring physical manipulation of physical quantities. Typically, but not necessarily, these quantities take the form of electrical or magnetic signals capable of being stored, combined, compared, and otherwise manipulated. It has proven convenient at times, principally for reasons of common usage, to refer to these signals as bits, values, elements, symbols, characters, terms, numbers, or the like.

[0259] It should be remembered, however, that all of these and similar terms should be associated with the appropriate physical quantities and are merely convenient labels applied to these quantities. The present disclosure may involve the actions and processes of computer systems or similar electronic computing devices that manipulate and transform data represented as physical (electronic) quantities within the computer system's registers and memories into other data similarly represented as physical quantities within the computer system's memories or registers or other such information storage systems.

[0260] The present disclosure also relates to an apparatus for performing the operations herein. This apparatus may be specially constructed for the intended purpose, or it may comprise a general purpose computer selectively activated or reconfigured by a computer program stored in the computer. This computer program may be stored in a computer-readable storage medium, such as (but not limited to) any type of disk (including floppy disks, optical disks, CD-ROMs, and magneto-optical disks), read-only memory (ROM), random access memory (RAM), EPROM, EEPROM, magnetic or optical cards, or any type of medium suitable for storing electronic instructions, each coupled to a computer system bus.

[0261] The algorithms and displays presented herein are not inherently related to any particular computer or other device. Various general purpose systems may be used with the program according to the teachings herein, or it may prove convenient to construct more specialized equipment to perform the method. The structures of various these systems will appear as set forth in the description below. In addition, the present disclosure is not described with reference to any particular programming language. It should be appreciated that various programming languages ​​may be used to implement the teachings of the present disclosure described herein.

[0262] In one embodiment, the memory device includes a voltage driver (eg, Figure 2 203, 213, 223) and / or other components. The controller is instructed by firmware or other software. The software may be stored on a machine-readable medium as instructions that may be used to program the control. A machine-readable medium includes any mechanism for storing information in a form that can be read by a machine (e.g., a computer). In some embodiments, a machine-readable (e.g., computer-readable) medium includes a machine (e.g., computer) readable storage medium, such as a read-only memory ("ROM"), a random access memory ("RAM"), a magnetic disk storage medium, an optical storage medium, a flash memory component, etc.

[0263] In this description, various functions and operations may be described as being performed or caused by computer instructions to simplify the description. However, those skilled in the art will recognize that such expressions mean that the functions are generated by one or more controllers or processors (such as microprocessors) executing computer instructions. Alternatively or in combination, the functions and operations may be implemented using dedicated circuit systems with or without software instructions, such as using application specific integrated circuits (ASICs) or field programmable gate arrays (FPGAs). Embodiments may be implemented using hard-wired circuit systems without or in combination with software instructions. Therefore, the technology is neither limited to any specific combination of hardware circuit systems and software, nor to any specific source of instructions executed by a data processing system.

[0264] Unless expressly stated otherwise, disjunctive language such as the phrase "at least one of X, Y, or Z" should be understood as generally used to indicate that an item, term, etc. may be X, Y, or Z, or any combination thereof (e.g., X, Y, and / or Z), depending on the context. Thus, such disjunctive language is generally not intended to, and should not, imply that certain embodiments require the presence of at least one X, at least one Y, or at least one Z, respectively.

[0265] In the above description, embodiments of the present disclosure have been described with reference to specific example embodiments of the present disclosure. It should be understood that various modifications may be made to the present disclosure without departing from the broader spirit and scope of the embodiments of the present disclosure as set forth in the appended claims. The description and drawings should accordingly be regarded as intended to be illustrative rather than restrictive.

Claims

1. A device comprising: Multiple pieces; and A logic circuit system configured to: programming a first memory cell of the tile to store a first weight; controlling a respective first portion of each patch based on a characteristic associated with the neural network; and The output currents from the first memory cells are summed.

2. The device according to claim 1, wherein: The neural network comprises a plurality of layers, and the characteristic is the layer to be used; The characteristic indicates the corresponding first portion of each tile to be controlled; The logic circuitry is further configured to control charging of access lines coupled to memory cells that are not to be used. The apparatus of claim 1 , wherein the characteristic is selected based on a layer for the first weight.

4. The device of claim 1 , wherein the first memory cells are programmed by changing a respective threshold voltage of each memory cell, and wherein an output is generated from the neural network based on a first input vector corresponding to data collected by at least one sensor.

5. A device comprising: a memory configured to store a culling map of the neural network; a controller configured to generate the culling map; and Voltage drivers are configured to drive bit lines coupled to memory cells, wherein each of the voltage drivers is enabled or disabled based on the cull map.

6. The apparatus according to claim 5, wherein: Prior to performing the first computation, the memory cells are programmed to respective threshold voltages corresponding to values ​​of weights of the neural network; The output current is accumulated on the bit lines connected to the memory cells or the source lines connected to the memory cells; The memory cells are organized into groups of memory cells, and each group of memory cells is programmable to store bits of a multi-bit weight value.

7. A system comprising: NAND flash memory cells organized in pillars extending vertically upward from a semiconductor substrate; analog circuitry configured to accumulate output currents from the memory cells during inference by the neural network; and Sensing circuitry is coupled to the access lines and configured to enable or disable each access line during the inference period.

8. The system of claim 7, wherein the number of enabled access lines corresponds to a characteristic of the neural network.

9. The system of claim 7, wherein: The sensing circuitry includes a respective transistor coupling each access line to a voltage driver for charging the access line; switching off the transistors for those corresponding access lines that are disabled; The memory cells are arranged in strings of memory cells connected in series, and each access line is a bit line connected to a respective one or more of the strings.

10. The system of claim 7, further comprising a register configured to store configuration data associated with a neural network, wherein the configuration data indicates a neuron size, or the configuration data is a pattern received from a host, the pattern corresponding to a portion of the neural network for multiplication operations.

Images