Configurable in-memory compute engines, platforms, bit cells and their layout

Abstract

Various embodiments include systems, methods, architectures, mechanisms or devices for providing programmable or preprogrammed in-memory computation operations. An in-memory computation architecture includes: a shaping buffer configured to shape a sequence of received data words to form a massively parallel bitwise input signal; a plurality of CIM channel output signals to thereby provide a sequence of multi-bit output words; a control circuit configured to cause the CIM array to perform a multi-bit computation operation on the input signal and the accumulated signal using single-bit internal circuits and signals; and a near-memory computation path configured to provide a sequence of multi-bit output words as a computation result.

Description

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS

[0002] This application claims the benefit of U.S. Provisional Patent Application Serial No. 62 / 686,296 filed on June 18, 2018, U.S. Provisional Patent Application Serial No. 62 / 702,629 filed on July 24, 2018, U.S. Provisional Patent Application Serial No. 62 / 754,805 filed on November 2, 2018, and 62 / 756,951 filed on November 7, 2018, which are incorporated herein by reference in their entirety. Technical Field

[0003] The present invention relates to the fields of in-memory computing and matrix-vector multiplication. Background Art

[0004] Charge domain in-memory computing has recently emerged as a robust and scalable way to perform in-memory computing. Here, computational operations within a memory bit cell provide results as charges, typically using voltage-to-charge conversion via capacitors. Thus, the bit cell circuit involves appropriate switching of local capacitors in a given bit cell, where the local capacitors are also appropriately coupled to other bit cell capacitors to produce an aggregated computational result across the coupled bit cells. In-memory computing is well suited to implementing matrix-vector multiplication, where the matrix elements are stored in a memory array, and the vector elements are broadcasted over the memory array in a parallel manner. Summary of the invention

[0005] Various deficiencies in the prior art are addressed by providing a system, method, architecture, mechanism or device for programmable or pre-programmed in-memory computing operations.

[0006] One embodiment provides an in-memory computation architecture, comprising: a reshape buffer configured to reshape a sequence of received data words to form a massively parallel bitwise input signal; a computation in memory (CIM) array of bit cells configured to receive the massively parallel bitwise input signal through a first CIM array dimension and receive one or more accumulation signals through a second CIM array dimension, wherein each of a plurality of bit cells associated with a common accumulation signal forms a respective CIM channel configured to provide a respective output signal; an analog-to-digital converter (ADC) circuit configured to process the plurality of CIM channel output signals to thereby provide a sequence of multi-bit output words; a control circuit configured to cause the CIM array to perform a multi-bit computation operation on the input signal and the accumulation signal using single bit internal circuits and signals; and a near memory computation path configured to provide a sequence of multi-bit output words as a computation result.

[0007] Other purposes, advantages and novel features of the present invention will be described in part in the following description, and will become apparent to those skilled in the art after examining the following content, or may be understood through practice of the present invention. The purposes and advantages of the present invention may be realized and achieved through the means and combinations particularly pointed out in the appended claims. BRIEF DESCRIPTION OF THE DRAWINGS

[0008] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments of the invention and, together with the general description of the invention given above and the detailed description of the embodiments given below, serve to explain the principles of the invention.

[0009] Figure 1 Describes a typical structure of an in-memory computing architecture;

[0010] Figure 2 depicts a high-level block diagram of an exemplary architecture according to an embodiment;

[0011] Figure 3 Describes the suitable Figure 2 A high-level block diagram of an exemplary in-memory compute unit (CIMU) used in the architecture of;

[0012] Figure 4 Describes an embodiment and suitable for Figure 2 A high-level block diagram of the input activation vector shaping buffer (IA BUFF) used in the architecture of ;

[0013] Figure 5 Describes an embodiment and suitable for Figure 2A high-level block diagram of the CIMA read / write buffer used in the architecture;

[0014] Figure 6 Describes an embodiment and suitable for Figure 2 A high-level block diagram of the Near Memory Datapath (NMD) module used in the architecture of

[0015] Figure 7 Describes an embodiment and suitable for Figure 2 A high-level block diagram of the direct memory access (DMA) module used in the architecture of

[0016] FIG. 8A to FIG. 8B Describes the suitable Figure 2 A high-level block diagram of different implementations of CIMA channel digitization / weighting used in the architecture;

[0017] Fig. 9 depicts a flow chart of a method according to an embodiment;

[0018] Fig.10 Depicted is the circuit diagram of the multiplication bit unit;

[0019] Fig.11 depicts a circuit diagram of three M-BCs configured to perform an XNOR function;

[0020] Fig.12 depicts a circuit diagram of an M-BC according to an embodiment;

[0021] Fig.13 Depicted Fig.12 Exemplary IC of M-BC;

[0022] FIG. 14A to FIG. 14B depicts a block diagram of a bit cell having a switch-based coupling structure and a switch-free coupling structure, respectively;

[0023] Fig.15 A circuit diagram depicting a bit cell circuit having a switchless coupling structure according to an embodiment; and

[0024] Fig.16 A circuit diagram depicting a 2-way interleave of a bit cell layout according to an embodiment is depicted.

[0025] It should be understood that the drawings are not necessarily drawn to scale, and present somewhat simplified representations of different features that illustrate the basic principles of the invention. The specific design features of the order of operations disclosed herein, including, for example, the specific size, orientation, location, and shape of the various illustrated components, will be determined in part by the specific intended application and use environment. Certain features of the illustrated embodiments have been enlarged or deformed relative to other features for ease of visualization and clear understanding. In particular, for example, thin features may be thickened for clarity or illustration. DETAILED DESCRIPTION

[0026] Before describing the present invention in further detail, it should be understood that the present invention is not limited to the specific embodiments described, as such embodiments may of course vary. It should also be understood that the terminology used herein is only for the purpose of describing specific embodiments, and is not intended to be limiting, as the scope of the present invention will be limited only by the appended claims.

[0027] Where a numerical range is provided, it is to be understood that each intervening value between the upper and lower limits of that range and any other stated range, to one-tenth of the unit of the lower limit unless the context clearly dictates otherwise, or an intervening value within that stated range is included within the present invention. The upper and lower limits of these smaller ranges may independently be included within the smaller ranges, which are also included within the present invention, subject to any explicitly excluded limits in the stated range. Where a stated range includes one or both of the limits, ranges excluding one or both of these included limits are also included within the present invention.

[0028] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which the invention belongs. Although any methods and materials similar or equivalent to those described herein can also be used in the practice or testing of the present invention, a limited number of exemplary methods and materials are described herein. It should be noted that as used herein and in the appended claims, the singular forms "a", "an", and "the" include plural references unless the context clearly dictates otherwise.

[0029] The disclosed embodiments include systems and methods for achieving programmability of a highly efficient linear algebra computing architecture for modem VLSI implementations. The computing architecture disclosed herein is referred to as "in-memory computing," and the programmability of the architecture will enable its widespread use in a range of applications, particularly for machine learning and artificial intelligence. The disclosed methods introduce a series of configurability features around the in-memory computing array, which also enables its integration in programmable platforms to address current energy and throughput challenges for a wide range of machine learning and artificial intelligence applications.

[0030] Linear algebra operations - most notably matrix-vector multiplication - have become very prominent in emerging workloads, especially those from machine learning (ML) and artificial intelligence (AI). Often the dimensions of matrices and vectors can be quite large (typically >100). Due to the way elements are reused in such operations, the elements are often stored in embedded or off-chip memory (depending on how many elements must be stored). Implementations in modem VLSI technology have shown that, in fact, the energy and latency of accessing data from such memory greatly exceeds the energy and latency of performing actual computations on the data. This limits the energy / latency reductions that can be achieved with conventional accelerators, which separate memory and computation, and has led to the paradigm of in-memory computation. In the context of in-memory computation systems, instead of accessing raw data from memory, the result of a computation over many raw data bits is accessed, thereby amortizing the access energy and latency.

[0031] Thus, the disclosed systems, methods, and portions thereof enable configurability and programmability of a highly efficient linear algebraic computing architecture for modem VLSI implementations, etc., resulting in integrated circuit implementations. The computing architecture disclosed herein may be broadly referred to as "in-memory computing," and the programmability of the architecture enables widespread use in a range of applications, including, for example, matrix-vector computations used in machine learning, artificial intelligence, and other applications. In various embodiments, the disclosed systems, methods, and portions thereof enable configurability and programmability of the in-memory computing architecture using a hybrid analog / digital computing scheme employing parallel and serial operations.

[0032] While the disclosed method creates certain forms of quantization noise in the way it performs calculations, this is controlled through several architectural features disclosed herein, in one case achieving operations that behave like quantization noise in standard integer fixed-point precision calculations.

[0033] The disclosed system and portions thereof have been implemented as integrated circuits, and various features thereof have been implemented as field programmable gate arrays (FPGAs) and simulated using Verilog / transistor level simulation.

[0034] The disclosed methods can be applied to a range of in-memory computing architectures, including the architecture disclosed in provisional patent application serial number 62 / 555,959, “Analog Switched-Capacitor Neural Network,” filed on September 8, 2017, which is incorporated herein by reference in its entirety as if fully set forth herein.

[0035] The disclosed method enables configurability and programmability of in-memory computations that are proven to be energy-efficient methods for linear algebra computations. Such configurability and programmability enable the disclosed architecture to be used in a wide range of applications.

[0036] Further disclosed embodiments provide a bit-parallel / bit-serial method for multi-bit in-memory computation. Specifically, an in-memory computation architecture is also disclosed herein, wherein memory bit cells perform computations on lb operands, and the hardware is extended to implement operations on multi-bit operands using a bit-parallel / bit-serial (BP / BS) scheme, wherein multiple bits of one operand are mapped to parallel bit cells, and multiple bits of another operand are input serially. The disclosed method allows: the following in-memory computing, in which the computing output bit cells are digitized and then fed into further computing, possibly across different bit cell output calculations in time and space; the following in-memory computing, in which multi-bit operand computing is performed in a bit-parallel / bit-serial (BP / BS) manner on the digitized output; the following in-memory computing, in which the BP / BS method uses a two's complement representation, for example, by using bitwise AND (AND: "and" logical relationship) calculations through the bit cells; and the following in-memory computing, in which the BP / BS method uses different digital representations, for example, where the 1 / 0 bits mathematically take on the values ​​of +1 / -1, enabling the use of XNOR (XNOR: "exclusive or not" logical relationship) calculations through the bit cells.

[0037] Configurable in-memory compute engine and platform

[0038] Various embodiments are directed to integrated configurability and hardware support around in-memory compute accelerators to enable programmability and virtualization required to broaden to practical applications. Generally, in-memory compute implements matrix-vector multiplication, where matrix elements are stored in a memory array and vector elements are broadcasted across the memory array in a parallel manner. Several aspects of the embodiments are directed to programmability and configurability to achieve such an architecture.

[0039] In-memory computations typically involve lb representations of matrix elements, vector elements, or both. This is because the memory stores data in independent bit cells for which broadcasting is done in a parallel homogeneous manner without providing different binary weighted couplings between bits required for multi-bit computations. In the present invention, expansion of multi-bit matrix and vector elements is achieved through a bit-parallel / bit-serial (BPBS) scheme.

[0040] In order to implement common computational operations that often revolve around matrix-vector multiplication, a highly configurable / programmable near-memory computational data path is included. This not only implements the expansion of bit-by-bit computations from in-memory computations to the computations required for multi-bit computations, but also generally supports multi-bit computations and is no longer subject to the lb representation inherent in in-memory computations. Since programmable / configurable and multi-bit computations are more efficient in the digital domain, in the present invention, analog-to-digital conversion is performed after in-memory computations, and in a specific embodiment, the configurable data path is multiplexed between eight ADC / in-memory computation channels, but other multiplexing ratios may be used. This is also well aligned with the BPBS scheme for multi-bit matrix element support, where support for up to 8-b operands is provided in an embodiment.

[0041] Since input vector sparsity is common in many linear algebra applications, the present invention integrates support to achieve energy proportional sparsity control. This is achieved by masking the broadcast of bits corresponding to zero-valued elements from the input vector (all bits in the bit serial process are masked in this way). This not only saves broadcast energy and computational energy in the memory array.

[0042] Given the internal bit-wise computational architecture for in-memory computations and the external digital word architecture of a typical microprocessor, data shaping hardware is used both for the computational interface through which input vectors are provided and for the memory interface through which matrix elements are written and read.

[0043] Figure 1 Depicts a typical structure of an in-memory computing architecture. In-memory computing, which consists of a memory array (which can be based on standard bit cells or improved bit cells), involves two additional "vertical" sets of signals; namely, (1) input lines; and (2) accumulation lines. Figure 1 , it can be seen that a two-dimensional array of bit cells is depicted, wherein each of the plurality of in-memory computing channels 110 comprises a respective column of bit cells, wherein each of the bit cells, channels are associated with a common accumulation line and bit line (column) and a respective input line and word line (row). It is noted that the columns and rows of signals are herein represented as being "perpendicular" relative to one another to simply indicate that in a bit cell array such as Figure 1 The row / column relationship in the context of a two-dimensional array of bit cells depicted in FIG. The term "vertical" as used herein is not intended to convey any particular geometric relationship.

[0044] The input / bit group and accumulation / bit group of signals can be physically combined with existing signals (e.g., word lines, bit lines) within the memory or can be separate. For implementing matrix-vector multiplication, the matrix elements are first loaded in the memory cells. Then, multiple input vector elements (possibly all) are applied at once through the input lines. This allows local computation operations to occur at each memory bit cell, usually some form of multiplication. The results of the computation operations are then driven onto the shared accumulation line. In this way, the accumulation line represents the results of the computations on multiple bit cells activated by the input vector elements. This is in contrast to standard memory access, in which bit cells are accessed one at a time through bit lines and activated by a single word line.

[0045] The in-memory computation described has several important properties. First, the computation is typically analog. This is because the constrained structure of memory and bit cells requires a richer computational model than a simple digital switch-based abstraction enables. Second, local operations at the bit cells typically involve computations using the lb representation stored in the bit cells. This is because the bit cells in a standard memory array are not coupled to each other in any binary-weighted manner; any such coupling must be achieved through methods of accessing / reading the bit cells from peripheral devices. Below, the extensions proposed by the present invention regarding in-memory computation are described.

[0046] Extensions to near-memory computing and multi-bit computing

[0047] While in-memory computation has the potential to solve matrix-vector multiplication in a way that conventional digital acceleration does not do well enough, a typical computation pipeline will involve a series of other operations surrounding the matrix-vector multiplication. Typically, such operations are well handled by conventional digital acceleration; however, it may be of great value to place such acceleration hardware near the in-memory computation hardware, in an appropriate architecture to handle the parallel nature, high throughput (and therefore the need for high communication bandwidth in / out), and general computational patterns associated with in-memory computation. Since many of the surrounding operations will preferably be performed in the digital domain, analog-to-digital conversion through the ADC is included after each of the in-memory computation accumulation lines, so we refer to them as in-memory computation channels. The main challenge is to integrate the ADC hardware into the spacing of each in-memory computation channel, but the appropriate layout approach taken in the present invention achieves this.

[0048] The introduction of ADC after each computational pass implements an efficient way to extend in-memory computation to support multi-bit matrix and vector elements through bit-parallel / bit-serial (BPBS) computation, respectively. Bit-parallel computation involves loading different matrix element bits in different in-memory computation columns. The ADC outputs from different columns are then appropriately bit-shifted to represent the corresponding bit weights, and digital accumulation is performed on all columns to produce multi-bit matrix element computation results. On the other hand, bit-serial computation involves applying each bit of the vector element one at a time, storing the ADC output each time, and appropriately bit-shifting the stored output before digitally accumulating the next output corresponding to the subsequent input vector bit. This BPBS approach, which implements a hybrid of analog and digital computation, is highly efficient because it exploits the high efficiency and low precision flow regime of analog (lb) and the high efficiency and high precision flow regime of digital (multi-bit), while overcoming the access costs associated with conventional memory operations.

[0049] While a range of near-memory computing hardware is contemplated, the details of the hardware integrated in the current embodiment of the invention are described below. To facilitate the physical layout of such multi-bit digital hardware, eight in-memory computing channels are multiplexed into each near-memory computing channel. We note that this allows the high parallelism of in-memory computing to match the high-frequency computing throughput of digital near-memory computing (the highly parallel analog in-memory computing runs at a clock frequency lower than the digital near-memory computing). Each near-memory computing channel then includes a digital barrel shifter, a multiplier, a multiplier, and a look-up table (LUT) and fixed nonlinear function implementation. In addition, a configurable finite state machine (FSM) associated with the near-memory computing hardware is integrated to control the computation through hardware.

[0050] Input interface and bit scalability control

[0051] In order to integrate in-memory computation with a programmable microprocessor, it is necessary to properly interface the internal bitwise operations and representations with the external multi-bit representations employed in typical microprocessor architectures. Therefore, a data shaping buffer is included at both the input vector interface and the memory read / write interface, through which the matrix elements are stored in the memory array. The details of the design employed in the embodiments of the present invention are described below. The data shaping buffer enables bit-width scalability of the input vector elements while maintaining the maximum bandwidth of data transfer to the in-memory computation hardware - between the in-memory computation hardware and external memory and other architectural blocks. The data shaping buffer consists of a register file that acts as a line buffer that receives incoming parallel multi-bit data element by element for the input vector and provides outgoing parallel single-bit data for all vector elements.

[0052] In addition to word-by-word / bit-by-bit joining, hardware support for convolution operations applied to input vectors is also included. Such operations are very prominent in convolutional neural networks (CNNs). In this case, matrix-vector multiplication is performed using only a subset of the new vector elements that need to be provided (the other input vector elements are stored in the buffer and simply shifted appropriately). This alleviates the bandwidth limitations of acquiring data to the computing hardware in the high-throughput memory. In an embodiment of the present invention, the convolution support hardware that must perform appropriate bit-serial ordering on multi-bit input vector elements is implemented in a dedicated buffer, and its output readout is appropriately shifted for the data used for the configurable convolution stride.

[0053] Dimensionality and sparsity control

[0054] For programmability, the hardware must address two additional considerations: (1) the matrix / vector sizes may be variable in different applications; and (2) in many applications, the vectors will be sparse.

[0055] Regarding dimensionality, in-memory computation hardware often integrates controls to enable / disable sliced ​​portions of the array to consume energy only for the dimensionality levels desired in the application. However, in the adopted BPBS approach, the input vector dimensionality has a significant impact on computational energy and SNR. Regarding SNR, in the case of bitwise computation in each in-memory computation channel, assuming that the computation between each input (provided on the input line) and the data stored in the bit cell produces a one-bit output, the number of possible different levels on the accumulation line is equal to N+l, where N is the input vector dimensionality. This suggests that a log2(N+l)-bit ADC is required. However, the energy cost of the ADC is closely related to the number of bits. Therefore, it is beneficial to support very large N but less than log2(N+l) bits in the ADC to reduce the relative contribution of the ADC energy. As a result of this, the signal-to-quantization-noise ratio (SQNR) of the computation operation is different from that of standard fixed-precision computation and decreases with the number of ADC bits. Therefore, hardware support for configurable input vector dimensionality is essential to support different application-level dimensionality and SQNR requirements, as well as the corresponding energy consumption. For example, if a reduced SQNR can be tolerated, large dimensional input vector segments should be supported; on the other hand, if a high SQNR must be maintained, lower dimensional input vector segments should be supported, where the inner product results of multiple input vector segments can be combined from different in-memory calculation libraries (in particular, the input vector dimensionality can therefore be reduced to a level set by the number of ADC bits to ensure that the calculation is ideally matched to standard fixed-precision operations). The hybrid analog / digital approach taken in the present invention achieves this. That is, the input vector elements can be masked to filter the broadcast to only the desired dimensionality. This saves broadcast energy and bit cell computation energy proportional to the input vector dimensionality.

[0056] Regarding sparsity, the same masking approach can be applied throughout the bit-serial operation to prevent broadcasting of all input vector element bits corresponding to zero-valued elements. We note that the adopted BPBS approach is particularly advantageous for this. This is because, while the expected number of non-zero elements is often known in sparse linear algebra applications, the input vector dimensionality can be large. Therefore, the BPBS approach allows us to increase the dimensionality of the input vector while still ensuring that the number of levels that need to be supported on the accumulation lines is within the ADC resolution, thereby ensuring a high computational SQNR. Although the number of expected non-zero elements is known, a variable number of actual non-zero elements must still be supported, which may vary depending on the input vector. This is easily implemented in a hybrid analog / digital approach because in the digital domain after the BPBS operation, the masking hardware only needs to calculate the number of zero-valued elements for a given vector and then apply the corresponding offset to the final inner product result.

[0057] Exemplary Integrated Circuit Architecture

[0058] Figure 2 A high-level block diagram of an exemplary architecture according to one embodiment is depicted. Specifically, Figure 2 The exemplary architecture of is implemented as an integrated circuit using specific components and functional elements using VLSI manufacturing technology to test different embodiments of the present invention. It will be understood that the inventors contemplate other embodiments with different components (e.g., larger or more powerful CPUs, memory elements, processing elements, etc.) within the scope of the present disclosure.

[0059] like Figure 2 As shown, the architecture 200 includes: a central processing unit (CPU) 210 (e.g., a 32-bit RISC-VCPU); a program memory (PMEM) 220 (e.g., a 128KB program memory); a data memory (DMEM) 230 (e.g., a 128KB data memory); an external memory interface 235 (e.g., configured to illustratively access one or more 32-bit external memory devices (not shown) to thereby expand the accessible memory); a boot loader module 240 (e.g., configured to access an 8KB off-chip EEPROM (not shown)); an in-memory computing unit (CIMU) 300, which includes different configuration registers 255 and is configured to perform in-memory computing and different other functions according to the embodiments described herein; a direct memory access (DMA) module 260, which includes different configuration registers 265; and different support / peripheral modules, such as a universal asynchronous receiver / transmitter (UART) module 271 for receiving / transmitting data, a general purpose input / output (GPIO) module 273, different timers 274, etc. Other elements not depicted herein may also be included Figure 2 In the architecture 200, such as a SoC configuration module (not shown) and the like.

[0060] CIMU 300 is well suited for matrix-vector multiplication, etc.; however, other types of calculations / operations may be more appropriately performed by non-CIMU computing devices. Therefore, in various embodiments, a very close coupling between CIMU 300 and near memory is provided so that the selection of computing devices to undertake specific calculations and / or functions can be controlled to provide more efficient computing functions.

[0061] Figure 3 Describes the suitable Figure 2 A high-level block diagram of an exemplary in-memory compute unit (CIMU) 300 used in the architecture of FIG. Figure 2 The architecture 200 and an exemplary CIMU 300 suitable for use in the context of the architecture 200 are described herein.

[0062] In general, CIMU 300 includes various structural elements including a compute in memory array (CIMA) of bit cells illustratively configured via various configuration registers to provide programmable in-memory compute functions such as matrix-vector multiplication, etc. In particular, the exemplary CIMU 300 is configured as a 590 kb, 16-bank CIMU whose task is to multiply an input matrix A by an input vector A to produce an output matrix Y.

[0063] Reference Figure 3 , CIMU300 is depicted as including an in-memory compute array (CIMA) 310, an input activation vector shaping buffer (IABUFF) 320, a sparsity / AND logic controller 330, a memory read / write interface 340, a row decoder / WL driver 350, multiple A / D converters 360, and a multiply-shift-accumulate data path (NMD) 370 for near-memory computation.

[0064] The exemplary compute in-memory array (CIMA) 310 includes a 256x (3x3x256) compute in-memory array arranged as a 4x4 clock-gable 64x (3x3x64) compute in-memory array, thus having a total of 256 compute in-memory channels (e.g., memory columns), which also includes 256 ADCs 360 to support the compute in-memory channels.

[0065] IA BUFF 320 operates to receive a sequence of illustrative 32-bit data words and reshape these 32-bit data words into a sequence of high-dimensional vectors suitable for processing by CIMA 310. It is noted that 32-bit, 64-bit, or any other width of data words can be shaped to fit the available or selected size of the in-memory computation array 310, which itself is configured to operate on high-dimensional vectors and includes elements that can be 2-8 bits, 1-8 bits, or some other size, and apply them in parallel across the array. It is also noted that the matrix-vector multiplication operations described herein are depicted as utilizing the entirety of CIMA 310; however, in various embodiments, only a portion of CIMA 310 is used. Additionally, in various other embodiments, CIMA 310 and associated logic circuits are adapted to provide interleaved matrix-vector multiplication operations, wherein parallel portions of matrices are processed simultaneously by corresponding portions of CIMA 310.

[0066] In particular, the IA BUFF 320 shapes a sequence of 32-bit data words into a highly parallel data structure that can be added to the CIMA 310 at once (or at least in larger chunks) and properly ordered in a bit-serial manner. For example, a four-bit calculation with eight vector elements can be associated with a high-dimensional vector of more than 2000 n-bit data elements. The IA BUFF 320 forms this data structure.

[0067] As described herein, IA BUFF 320 is configured to receive input matrix A as a sequence of illustrative 32-bit data words and adjust the size / position of the received sequence of data words according to the size of CIMA 310, illustratively to provide a data structure including 2303 n-bit data elements. Each of these 2303 n-bit data elements and corresponding masking bits are communicated from IA BUFF 320 to sparsity / AND logic controller 330.

[0068] The sparsity / AND logic controller 330 is configured to receive the exemplary 2303 n-bit data elements and corresponding mask bits and responsively invoke a sparsity function wherein zero-valued data elements (such as indicated by corresponding mask bits) are not propagated to the CIMA 310 for processing. In this manner, energy that would otherwise be required by the CIMA 310 to process these bits is conserved.

[0069] In operation, the CPU 210 reads the PMEM 220 and the boot loader 240 through a direct data path implemented in a standard manner. The CPU 210 can access the PMEM 220, the IA BUFF 320, and the memory read / write buffer 340 through a direct data path implemented in a standard manner. All of these memory modules / buffers, the CPU 210, and the DMA module 260 are connected through the AXI bus 281. The chip configuration module and other peripheral modules are grouped by the APB bus 282, which is attached to the AXI bus 281 as a slave. The CPU 210 is configured to write to the PMEM 220 through the AXI bus 281. The DMA module 260 is configured to access the PMEM 220, the IA BUFF 320, the memory read / write buffer 340, and the NMD 370 through a dedicated data path, and access all other accessible memory spaces through the AXI / APB bus, such as each DMA controller 265. The CIMU 300 performs the BPBS matrix-vector multiplication described above. Further details of these and other embodiments are provided below.

[0070] Thus, in various embodiments, the CIMA operates in a bit-serial bit-parallel (BSBP) manner to receive vector information, perform matrix-vector multiplication, and provide a digitized output signal (i.e., Y=AX), which can be further processed appropriately by another computational function to provide a composite matrix-vector multiplication function.

[0071] Generally speaking, embodiments described herein provide an in-memory computation architecture comprising: a shaping buffer configured to shape a sequence of received data words to form a massively parallel bitwise input signal; a computation in memory (CIM) array of bit cells configured to receive the massively parallel bitwise input signal through a first CIM array dimension and receive one or more accumulation signals through a second CIM array dimension, wherein each of a plurality of bit cells associated with a common accumulation signal forms a respective CIM channel configured to provide a respective output signal; an analog-to-digital converter (ADC) circuit configured to process a plurality of CIM channel output signals to thereby provide a sequence of multi-bit output words; a control circuit configured to cause the CIM array to perform a multi-bit computation operation on the input signal and the accumulation signal using single bit internal circuits and signals; and a near memory computation path configured to provide a sequence of multi-bit output words as a computation result.

[0072] Memory Map and Programming Model

[0073] Since the CPU 210 is configured to directly access the IA BUFF 320 and the memory read / write buffer 340, these two memory spaces appear similar to the PMEM 220 from the user program's perspective and in terms of latency and energy, especially for structured data such as array / matrix data, etc. In various embodiments, when the in-memory computation feature is not activated or partially activated, the memory read / write buffer 340 and CIMA 310 can be used as normal data memory.

[0074] Figure 4 Describes an embodiment and suitable for Figure 2A high-level block diagram of an input activation vector shaping buffer (IA BUFF) 320 used in the architecture of FIG. The depicted IA BUFF 320 supports input activation vectors with element precision ranging from 1 bit to 8 bits; other precisions may also be accommodated in different embodiments. According to the bit-serial streaming mechanism discussed herein, specific bits of all elements in the input activation vector are broadcast to CIMA 310 at a time for matrix-vector multiplication operations. However, the highly parallel nature of this operation requires maximum bandwidth and minimum energy to be provided for the elements of the high-dimensional input activation vector, otherwise the throughput and energy efficiency advantages of in-memory computing cannot be utilized. To achieve this, the input activation shaping buffer (IA BUFF) 320 can be constructed as follows so that the in-memory computing can be integrated into the 32-bit (or other bit width) architecture of the microprocessor, whereby the hardware used for the corresponding 32-bit data transfer is maximized for the highly parallel internal organization of the in-memory computing.

[0075] Reference Figure 4 IA BUFF 320 receives a 32-bit input signal, which may include input vector elements with bit precision from 1 to 8 bits. Therefore, the 32-bit input signal is first stored in 4x8-b registers 410, of which there are 24 in total (represented herein as registers 410-0 to 410-23). ​​These registers 410 provide their contents to 8 register files (represented as register files 420-0 to 420-8), each of which has 96 columns, and wherein the input vectors with dimensions up to 3x3x256=2304 are arranged in a manner in which their elements are parallel columns. This is accomplished by providing 24 4x8-b registers 410 with 96 parallel outputs across one of the register files 420 when the input element is 8-b, and by providing 24 4x8-b registers 410 with 1536 parallel outputs across all 8 register files 420 (or by intermediate configuration for other bit precisions) when the input element is 1b. Each register file column is 2x4x8-b in height, which allows each input vector (with up to 8 bits of element precision) to be stored in 4 segments and double buffering is achieved for the case where all input vector elements are to be loaded. On the other hand, for the case where only one-third of the input vector elements are to be loaded (i.e., CNN with stride 1), 1 out of every 4 register file columns acts as a buffer, which allows data from the other 3 columns to be propagated forward to the CIMU for computation.

[0076] Thus, of the 96 columns output by each register file 420, only 72 columns are selected by the corresponding ring barrel shift interface 430, giving a total of 576 outputs at a time on eight register files 420. These outputs correspond to one of the four input vector segments stored in the register file. Thus, four cycles are required to load all the input vector elements into the sparsity / AND logic controller 330 within 1 b registers.

[0077] To exploit the sparsity in the input activation vector, a masking bit is generated for each data element while the CPU 210 or DMA 260 writes to the shaping buffer 320. The masked input activations prevent charge-based computation operations in CIMA 310, which saves computational energy. The masking vector is also stored in an SRAM block that is organized similarly to the input activation vector, but with a one-bit representation.

[0078] The 4-to-3 barrel shifter 430 is used to support VGG style (3x3 filter) CNN calculations. When moving to the next filtering operation (convolution reuse), only one of the three input activation vectors needs to be updated, which saves energy and improves throughput.

[0079] Figure 5 Describes an embodiment and suitable for Figure 2 A high level block diagram of a CIMA read / write buffer 340 used in the architecture of FIG. The depicted CIMA read / write buffer 340 is organized as an exemplary 768-bit wide static random access memory (SRAM) block 510, while in this example, the depicted CPU has a word width of 32 bits; the read / write buffer 340 is used to interface therebetween.

[0080] The depicted read / write buffer 340 contains a 768-bit write register 511 and a 768-bit read register 512. The read / write buffer 340 generally acts as a cache for the wide SRAM block in the CIMA 310; however, some details are different. For example, the read / write buffer 340 only writes back to the CIMA 310 when the CPU 210 writes to a different line, and reading a different line does not trigger a write back. When the read address matches the tag of the write register, the modified byte in the write register 511 (indicated by the dirty bit) is bypassed to the read register 512 instead of being read from the CIMA 310.

[0081] Accumulation Line Analog-to-Digital Converters (ADCs). The accumulation lines from CIMA 310 each have an 8-bit SAR ADC that fits the spacing of the computation channels within the memory. To save area, the finite state machine (FSM) that controls the SAR ADC bit cycle is shared between the 64 ADCs required for the computation slice within each memory. The FSM control logic consists of 8+2 shift registers, generating pulses to cycle through the reset, sample, and then 8 bit decision stages. The shift register pulses are broadcast to the 64 ADCs, where they are buffered locally to trigger the local comparator decision, store the corresponding bit decision in the local ADC code register, and then trigger the next capacitor DAC configuration. The small size of the capacitor array for each ADC can be achieved using high-precision metal-oxide-metal (MOM) capacitors.

[0082] Figure 6 Describes an embodiment and suitable for Figure 2 A high-level block diagram of a near memory datapath (NMD) module 600 used in the architecture of FIG. 1 , although digital near memory computing with other features may be employed. Figure 6 The depicted NMD module 600 depicted in shows the digital computation data path after the ADC output, which supports multi-bit matrix multiplication through the BPBS scheme.

[0083] In a specific embodiment, for the digital computation flow, the 256 ADC outputs are organized into 8 groups. This enables support for matrix element configurations up to 8 bits. Therefore, the NMD module 600 contains 32 identical NMD units. Each NMD unit consists of: a multiplexer 610 / 620 to select the multiplicand 622 / 623, the shift number 624 from the 8 ADC outputs 610 and the corresponding bias 621; and an accumulation register; an adder 631 with an 8-bit unsigned input and a 9-bit signed input to subtract the global bias and masking number; a signed adder 632 to calculate the local bias for neural network tasks; a fixed-point multiplier 633 to perform scaling; a barrel shifter 634 to calculate the exponent of the multiplicand and perform shifts on different bits in the weighted elements; a 32-bit signed adder 635 to perform accumulation; eight 32-bit accumulation registers 640 to support weights with 1, 2, 4, and 8-bit configurations; and a ReLU unit 650 for neural network applications.

[0084] Figure 7 Describes an embodiment and suitable for Figure 2A high-level block diagram of a direct memory access (DMA) module 700 used in the architecture of FIG. The depicted DMA module 700 illustratively includes: two channels to support data transfers from / to different hardware resources at the same time, and five independent data paths from / to DMEM, IA BUFF, CIMU R / W BUFF, NMD result, and AXI4 bus, respectively.

[0085] Bit-Parallel / Bit-Serial (BPBS) Matrix-Vector Multiplication

[0086] For multi-bit MVM The BPBS regimen is as follows FIG. 8A to FIG. 8B As shown, B A Applies to matrix element a m,n The number of bits, B X Applies to the input vector element x n The number of bits in the input vector, and N corresponds to the dimension of the input vector, which can be up to 2304 (M n is a masking bit for sparsity and dimensionality control). m,n The multiple bits of x are mapped to parallel CIMA columns, and n Multiple bits are input serially. Multiplication and accumulation of multiple bits can then be achieved via in-memory calculations in a bitwise XNOR manner or in a bitwise AND manner, both of which are supported by the multiplication bit cell (M-BC) of this embodiment. Specifically, the difference between bitwise AND and bitwise XNOR is that when the input vector element bit is low, the output should remain low. The M-BC of this embodiment involves inputting the input vector element bits (one at a time) as differential signals. The M-BC implements XNOR, where the output of each logical '1' in the truth table is driven to V by the true signal and the complement signal of the input vector element bit, respectively. DD Therefore, AND is easy to implement. We only need to mask the complement signal so that the output remains low to obtain the truth table corresponding to AND.

[0087] Bitwise AND can support standard two's complement representation for multi-bit matrix and input vector elements. This involves applying the negative sign appropriately to the column calculations corresponding to the most significant bit (MSB) elements in the digital domain after the ADC, and then adding the digitized output to those in the other column calculations.

[0088] Bitwise XNOR requires a slight modification of the numeric representation. That is, the element bits are mapped to +1 / -1 instead of 1 / 0, requiring two bits with equivalent LSB weighting to correctly represent zero. This is done as follows. First, each B-bit operand (in standard two's complement representation) is decomposed into a B+1-bit signed integer. For example, y decomposes into B+1 plus / minus one bit. To get

[0089] Where the bits of the l / 0 values ​​are mapped to mathematical values ​​of +1 / -1, bitwise in-memory computation multiplication can be implemented by a logical XNOR operation. Thus, an M-BC that performs logical XNOR using differential signals for input vector elements can implement signed multi-bit multiplication by bit weighting and adding digitized outputs from column computations.

[0090] Although M-BC multiplication based on AND and M-BC multiplication based on XNOR propose two options, other options are also possible by using appropriate number representations that are possible in the M-BC logical operations. Such alternative selections are beneficial. For example, M-BC multiplication based on XNOR is preferred for binary (lb) calculations, while M-BC multiplication based on AND implements a more standard digital representation to facilitate integration within the digital architecture. Additionally, these two methods produce slightly different signal-to-quantization noise ratios (SQNRs), so they can be selected based on application requirements.

[0091] Heterogeneous computing architecture and interfaces

[0092] Different embodiments described herein contemplate different aspects of computation within a charge domain memory, where a bit cell (or multiplied bit cell, M-BC) drives an output voltage corresponding to the result of the computation onto a local capacitor. The capacitors from the computational channels (columns) within the memory are then coupled to produce accumulation by charge redistribution. As described above, such capacitors can be formed using very easily replicable specific geometries such as in VLSI processes such as by simply being close to each other and thus coupled by electric fields. Thus, a local bit cell formed as a capacitor stores a charge representing 1 or 0, and adding up all the charges in several of these capacitors or bit cells can locally implement the functions of multiplication and accumulation / summation, which is the core operation in matrix-vector multiplication.

[0093] The above different embodiments advantageously provide improved bit-cell based architectures, computing engines and platforms. Matrix-vector multiplication is an operation that is not effectively performed by standard, digital processing or digital acceleration. Therefore, performing this type of calculation in memory has a huge advantage over existing digital designs. However, different other types of operations are effectively performed using digital designs.

[0094] Different embodiments contemplate mechanisms for connecting / joining these bit-cell-based architectures, computing engines, platforms, etc. with more conventional digital computing architectures and platforms to form heterogeneous computing architectures. In this way, those computing operations that are well suited for bit-cell architecture processing (e.g., matrix-vector processing) are processed as described above, while those other computing operations that are well suited for conventional computer processing are processed by conventional computer architectures. That is, different embodiments provide a computing architecture including a high parallel processing mechanism as described herein, wherein the mechanism is connected to multiple interfaces so that it can be externally coupled to a more conventional digital computing architecture. In this way, the digital computing architecture can be directly and effectively aligned with the in-memory computing architecture, allowing the two to be placed very close to minimize the data movement overhead between them. For example, although machine learning applications may include 80% to 90% matrix-vector calculations, there are still 10% to 20% of other types of calculations / operations that need to be performed. By combining the in-memory computing discussed herein with the more conventional near-memory computing in architecture, the resulting system provides excellent configurability to perform many types of processing. Therefore, different embodiments contemplate combining near-memory digital computing with the in-memory computing described herein.

[0095] The in-memory computations discussed herein are massively parallel but single bit operations. For example, only one bit can be stored in a bit cell. One or zero. The signal driven into the bit cell is typically the input vector (i.e., in a 2D vector multiplication operation, each matrix element is multiplied by each vector element). The vector element is placed on the following signal, which is also digital and is only one bit, so that the vector element is also a bit.

[0096] Different implementations use bit-parallel / bit-serial approaches to expand matrices / vectors from one bit element to multiple bit elements.

[0097] FIG. 8A to FIG. 8B Describes the suitable Figure 2 A high-level block diagram of different implementations of CIMA channel digitization / weighting used in the architecture of FIG. Fig. 8A An implementation of digital binary weighting and summing similar to that described above with respect to different other figures is depicted. Figure 8B Depicts an implementation of analog binary weighting and summing in which various circuit elements are modified to enable the use of Fig. 8A Embodiments and / or other embodiments described herein having fewer analog-to-digital converters.

[0098] As previously discussed, various embodiments contemplate configuring a compute-in-memory (CIM) array of bit cells to receive a massively parallel bitwise input signal via a first CIM array dimension (e.g., a row of a 2D CIM array) and to receive one or more accumulation signals via a second CIM array dimension (e.g., a column of a 2D CIM array), wherein each of a plurality of bit cells (depicted as, for example, a column of bit cells) associated with a common accumulation signal forms a respective CIM channel configured to provide a respective output signal. Analog-to-digital converter (ADC) circuitry is configured to process the plurality of CIM channel output signals to thereby provide a sequence of multi-bit output words. Control circuitry is configured to cause the CIM array to perform multi-bit computation operations on the input and accumulation signals using single bit internal circuitry and signals, whereby operably engaged near-memory computation paths can thereby be configured to provide a sequence of multi-bit output words as computation results.

[0099] Reference Fig. 8A , depicting an implementation of a digital binary weighted and summed ADC circuit function. In particular, a two-dimensional CIMA 810A receives matrix input values ​​(i.e., via a plurality of buffers 805) at a first (row) dimension and vector input values ​​at a second (column) dimension, wherein the CIMA 810A operates according to a control circuit or the like (not shown) to provide different channel output signals CH-OUT.

[0100] Fig. 8A The ADC circuit provides for each CIM channel: a corresponding ADC 760, which is configured to digitize the CIM channel output signal CH-OUT; and a corresponding shift register 865, which is configured to assign corresponding binary weights to the digitized CIM channel output signal CH-OUT to thereby form corresponding parts of the multi-bit output word 870.

[0101] Reference Figure 8B , depicting an implementation of an analog binary weighted summation that performs the functions of an ADC circuit. In particular, a two-dimensional CIMA 810B receives matrix input values ​​(i.e., via a plurality of buffers 805) at a first (row) dimension and vector input values ​​at a second (column) dimension, wherein the CIMA 810B operates according to a control circuit or the like (not shown) to provide different channel output signals CH-OUT.

[0102] Figure 8BThe ADC circuitry of the invention provides four controllable (or preset) banks of switches 815-1, 815-2, etc. within the CIMA 810B, which operate to couple and / or decouple capacitors formed therein to thereby implement an analog binary weighting scheme for each of one or more subsets of channels, wherein each channel subset provides a single output signal, thereby requiring only one ADC 860B to digitize the weighted analog sum of the CIM channel output signals of the corresponding subset of CIM channels to thereby form a corresponding portion of a multi-bit output word.

[0103] Fig. 9 A flow chart of a method according to an embodiment is depicted. Specifically, Fig. 9 The method 900 involves different processing operations implemented by the architecture, system, etc. described herein, wherein an input matrix / vector is expanded to be calculated in a bit-parallel / bit-serial manner.

[0104] At step 910, matrix and vector data are loaded into the appropriate memory locations.

[0105] At step 920, each of the vector bits (MSB to LSB) is processed sequentially. Specifically, the MSB of the vector is multiplied by the MSB of the matrix, the MSB of the vector is multiplied by the MSB-1 of the matrix, the MSB of the vector is multiplied by the MSB-2 of the matrix, and so on until the MSB of the vector is multiplied by the LSB of the matrix. Then for each of the MSB to LSB vector multiplications, the resulting analog charge result is digitized to obtain a latched result. This process is repeated for vector MSB-1, vector MSB-2, etc. to vector LSB until each of the vector MSB-LSB has been multiplied by each of the MSB-LSB elements of the matrix.

[0106] At step 930, the bits are shifted to apply the appropriate weighting, and the results are added together. Note that in some implementations using analog weighting, the shift operation of step 930 is not necessary.

[0107] Various embodiments enable high stability and robustness computations to be performed within circuits for storing data in dense memory. Additionally, various embodiments advance the computational engines and platforms described herein by enabling higher densities of memory bit cell circuits. The density can be increased both due to a more compact layout and due to enhanced compatibility of the layout with highly aggressive design rules (i.e., push rules) for memory circuits. Various embodiments substantially enhance the performance of processors for machine learning and other linear algebra.

[0108] A bit cell circuit is disclosed that can be used within an in-memory computing architecture. The disclosed method enables high stability / robustness computations to be performed within a circuit for storing data in dense memory. The disclosed method for robust in-memory computing enables a higher density of memory bit cell circuits than known methods. The density can be higher due to both a more compact layout and due to enhanced compatibility of the layout with highly aggressive design rules (i.e., push rules) for memory circuits. The disclosed apparatus can be manufactured using standard CMOS integrated circuit processing.

[0109] Memory access dominates energy and latency in many computing workloads. Memory energy and latency arise because in standard memories, raw data is accessed row by row, incurring the communication cost of moving data from the storage point to the computation point outside the memory array. On the other hand, in-memory computing architectures access computation results on many data bits stored across rows, accessing many rows at once and amortizing the communication cost.

[0110] While such amortization reduces energy and latency by a multiplicative factor (i.e., the number of rows accessed approximately simultaneously), the main challenge is that the computational signal-to-noise ratio (SNR) is also reduced by a corresponding factor. This is because computation on a large number of bits typically increases the required dynamic range; however, fitting it into the limited swing range of the existing bit lines in the memory squeezes the SNR. Specifically, the computational noise of the in-memory architecture is dominated by the variation and nonlinearity of the computational operations performed by the bit cells. In standard memories, the bit cells provide an output current. This makes current-domain computation a natural choice for in-memory computation architectures, which aim to minimize the variation of the high-density bit cells used in the standard.

[0111] However, the bit cell current is affected by high level variations and nonlinearities affecting the bit cell transistors. This limits the SNR of in-memory computation, and thus limits the scalability of in-memory computation. Improvements are obtained according to different embodiments using charge domain computation. Here, the computation output from the bit cell is stored as a charge on a capacitor. For example, different embodiments contemplate the use of metal finger capacitors arranged above the bit cell transistors; such capacitors do not generate additional area, thereby enabling the bit cell to remain a compact structure. Importantly for computing the SNR, such capacitors also exhibit very good linearity, as well as high stability in the presence of process and temperature variations. This greatly increases the scalability of in-memory computation.

[0112] Although metal finger capacitors can be placed above the bit cell, some circuit changes inside the bit cell are necessary for switched capacitor charge domain computation. The present invention focuses on circuits and layouts to achieve high density charge domain bit cell computation in static random access memory (SRAM). In particular, circuits and layouts are described that enhance compatibility with aggressive push design rules for SRAM bit cells.

[0113] Fig.10 Depicted is the circuit diagram of the multiplication bit cell. Fig.10 The bit cell performs operations such as storage, writing and reading of 1b data and also implements multiplication between the stored 1b data and the 1b IA / IAb signal (differential). Therefore, this structure is called a multiplication bit cell (M-BC). It is noted that the 1b multiplication corresponds to a logical XNOR operation. To achieve this, a PMOS transistor is added, coupled to the bit cell storage node and driven by the IA / IAb signal, such as Fig.10 shown.

[0114] Fig.11 A circuit diagram of three M-BCs configured to perform an XNOR function is depicted. Specifically, for the three M-BCs, Fig.11 The operation of the circuit includes: (1) unconditionally discharging the M-BC capacitor by asserting the capacitor shorting switch (TSHORT / TSHORTb) and the discharge NMOS transistor (PRE); (2) de-asserting TSHORT / TSHORTb and driving IA / IAb to store the XNOR output of the M-BC on the local M-BC capacitor; (3) de-asserting IA / IAb and asserting TSHORT to accumulate the charge from all XNOR results to give a multiply-accumulate calculation. It should be noted that in addition to the 8 M-BC transistors, each M-BC requires two additional NMOS / PMOS transistors for TSHORT / TSHORTb implementation. In previous designs, these NMOS / PMOS transistors were placed outside each M-BC and used together for three M-BCs because this enabled efficient node sharing.

[0115] Note that the use of PMOS transistors for bit cell computation results in a balanced use of NMOS and PMOS transistors within the M-BC, resulting in an IC layout that is very different from that of a standard 6-transistor (6T) SRAM bit cell. This affects the potential for push rule compatibility.

[0116] Different embodiments contemplate new M-BC circuits that add NMOS transistors for bit cell computation. In this way, these embodiments provide an IC layout that is both denser and closer to a standard 6T bit cell and also enhances push rule compatibility.

[0117] Fig.12 A circuit diagram of an M-BC according to one embodiment is depicted. Specifically, Fig.12 The M-BC implements the lb charge domain multiplication (XNOR) using NMOS transistors. Here, the NMOS input IA / IAb is low during the unconditional discharge of the local capacitor and then driven differentially to perform the calculation.

[0118] Fig.13 Depicted Fig.12 An exemplary layout of an M-BC. Specifically, Fig.13 The layout is expected to include NMOS / PMOS T HORT switches compactly within a single M-BC (metal finger capacitors are placed above the bit cell). Fig.13 In the layout of , signals WL, IA / IAb run horizontally, while signals BL, Blab, PA, VDD, GND run vertically. This layout has roughly twice the area of ​​a standard 6T cell and takes advantage of sharing several nodes with the surrounding M-BC.

[0119] Although the disclosed method takes up more area than a standard memory bit cell circuit, a large portion of the bit cell has been demonstrated using push rules, and extensions to it are similar to other structures that have successfully used push rules. The disclosed method greatly enhances the performance of processors for machine learning and other linear algebra. Such gains have been experimentally demonstrated for early architectures, and the disclosed method greatly advances the architecture.

[0120] As previously described, computational operations within a memory bit cell provide their results as charge, typically using voltage-to-charge conversions through capacitors. Thus, a bit cell circuit involves appropriate switching of a local capacitor in a given bit cell, where the local capacitor is also appropriately coupled to other bit cell capacitors to produce an aggregate computational result across the coupled bit cells.

[0121] Disclosed herein are bit cells and bit cell layouts for charge injection robustness of computation in reconfigurable charge domain memories. The disclosed apparatus, and in particular the bit cell circuit, can be used in an in-memory computing architecture. The disclosed method enables high stability / robustness computations and reconfigurable computations to be performed in circuits for storing data in dense memories. The disclosed method achieves greater robustness and configurability of in-memory computations than previous methods. The disclosed apparatus can be manufactured using standard CMOS integrated circuit processing. The disclosed method will have significant utility to the semiconductor industry as it can significantly enhance the performance of processors used for machine learning and other linear algebra.

[0122] The methods disclosed herein involve two new aspects of bit cell circuits: (1) a configuration in which coupling across bit cell capacitors can be achieved without the need for explicit switches (switchless coupling structure); and (2) a physical layout in which coupled bit cells are interleaved with other coupled bit cells (interleaved layout).

[0123] A switchless coupling structure involves a bit cell providing its computational result on one of the capacitor plates, and wherein coupling across the capacitor is achieved through the other capacitor plate. This is in contrast to a switch-based coupling structure in which the bit cell circuit provides the computational result on the same capacitor plate, which is ultimately coupled to the other capacitor, typically through a switch.

[0124] Fig.14A depicts a block diagram of a bit cell having a switch-based coupling structure, and Fig. 14B Block diagrams of bit cells with switchless coupling structures are depicted. In both cases, the coupled capacitors first need to be reset (charge removed from them), such as by shorting the output nodes where the capacitors are coupled. The computational operation f(.) is then performed locally in the bit cell. This is illustrated for two operands, a & b, one of which is stored in the bit cell and one of which is provided externally from the periphery of the bit cell array; however, in general, structures with more operands are possible. The computational operation then drives the plates of the local capacitors, which are coupled to other capacitors via switches on the sampling plates (switch-based coupling structures), or to other capacitors without switches on other plates (switchless coupling structures). Advantageously, Fig. 14B The switchless coupling structure avoids the need for coupling switches in the bit cells and also has the potential to reduce the effects of charge injection errors, such as switches, when implemented by MOSFETs, can cause variable amounts of charge to be absorbed / released by the MOSFETs (depending on the voltage level), thereby slightly corrupting charge domain calculations.

[0125] Fig.15 A circuit diagram of a bit cell circuit having a switchless coupling structure according to an embodiment is depicted. Note that other variations of this circuit are possible within the context of the disclosed embodiments. Fig.15 The bit cell can implement an XNOR or AND operation between the storage data W / Wb (within the 6-transistor cross-coupled circuit formed by MN1-3 / MP1-2) and the input data IA / IAb. For example, for the XNOR operation, after reset, IA / IAb can be driven in a complementary manner so that the bottom plate of the local capacitor is pulled up / down according to IAXNOR W. On the other hand, for the AND operation, after reset, only IA can be driven (and IAb is kept low) so that the bottom plate of the local capacitor is pulled up / down according to IA AND W. Advantageously, this structure enables the total switching energy of the capacitor to be reduced due to the generation of a series of serial pull-up / pull-down charging structures between all coupling capacitors, and the impact of switch charge injection errors can be reduced due to the elimination of coupling switches at the output nodes.

[0126] Fig.16 A circuit diagram of a 2-way interleaved layout of a bit cell layout according to an embodiment is depicted. Specifically, the interleaved layout of the bit cells refers to a layout in which capacitors can be coupled together in two or more groups. Although Fig.16 A layout illustration is shown for the case of a 2-way interleave, but higher interleaves are also contemplated in different embodiments. Additionally, while the capacitors are shown as being laid out on the column side, it is noted that in practice, they may also be laid out above the bit cell transistors and / or at other locations near the bit cell transistors. The benefit of this structure is, among other things, enhanced configurability. That is, couplings A and B may be used to implement separate computations because the outputs are provided at two different nodes. Alternatively, couplings A and B may be used to implement joint computations by appropriately combining the outputs at different nodes, such as through appropriate peripheral circuitry.

[0127] It will be understood that the functions depicted and described herein can be implemented in hardware or in a combination of software and hardware, for example, using a general-purpose computer, one or more application-specific integrated circuits (ASICs), or any other hardware equivalents. It is contemplated that some of the steps discussed herein can be implemented in hardware, for example, as circuits that collaborate with a processor to perform different method steps. Some of the functions / elements described herein can be implemented as a computer program product, in which computer instructions adapt the operation of a computing device when processed by a computing device so that the methods or techniques described herein are called or otherwise provided. Instructions for calling the method of the present invention can be stored in a tangible and non-temporary computer-readable medium, such as a fixed or removable medium or memory, or in a memory within a computing device that runs according to the instructions.

[0128] Different modifications may be made to the systems, methods, devices, mechanisms, techniques, and parts thereof described herein with respect to different drawings, and these modifications are expected to be within the scope of the present invention. For example, although a specific order of steps or an arrangement of functional elements is proposed in the different embodiments described herein, different other orders / arrangements of steps or functional elements may be utilized in the context of different embodiments. Additionally, although modifications to the embodiments may be discussed separately, different embodiments may use multiple modifications, use composite modifications, etc. simultaneously or sequentially.

[0129] Although specific systems, devices, methods, mechanisms, etc. have been disclosed as discussed above, it should be clear to those skilled in the art that, in addition to those already described, more modifications may be made without departing from the inventive concepts herein. Therefore, in addition to the spirit of the present disclosure, the inventive subject matter should not be limited. In addition, when interpreting the disclosure, all terms should be interpreted in the broadest sense consistent with the context. In particular, the terms "include" and "comprise" should be interpreted as referring to elements, parts or steps in a non-exclusive manner, indicating that the referenced elements, parts or steps may exist, or be utilized, or be combined with other elements, parts or steps that are not explicitly referenced. In addition, the references listed herein are also a part of this application and are incorporated by reference as a whole, as fully set forth herein.

[0130] Aspects of the different embodiments are specified in the claims and / or in the following numbered clauses:

[0131] 1. A bit cell circuit configuration comprising: a bit cell storage circuit coupled to at least one bit cell computing device; and a bit cell capacitor coupled to the bit cell computing device, the bit cell capacitor also coupled to one or more additional capacitors without a switch between the bit cell capacitor and the additional capacitors.

[0132] 2. The bit cell circuit arrangement of clause 1, wherein a cathode plate of the bit cell capacitor is coupled to the bit cell computing device.

[0133] 3. The bit cell circuit arrangement of clause 1, wherein an anode plate of the bit cell capacitor is coupled to the additional capacitor.

[0134] 4. A bit cell circuit arrangement according to clause 1, wherein the bit cell computation means is arranged to perform a computation operation having two operands.

[0135] 5. As Fig.11 The bit cell circuit configuration is shown.

[0136] 6. The bit cell circuit arrangement of clause 5, wherein the arrangement enables an XNOR or AND operation between stored data and input data.

[0137] 7. An interleaved layout for at least two bitcell configurations according to clause 1, wherein the bitcell capacitors are coupled together in at least two groups.

[0138] 8. The staggered layout of clause 7, wherein the set of coupled bit cell capacitors is laid out over one or more bit cell transistors.

[0139] 9. A charge domain memory in-memory computing bit cell driving one plate of a local capacitor and wherein coupling to other bit cell capacitors is achieved on the other plate.

[0140] 10. A computational bit cell in a charge domain memory that can implement an XNOR or AND operation between stored data and input data.

[0141] 11. An interleaved layout for computing bit cells within a charge domain memory, wherein the bit cell capacitors are coupled in a plurality of different groups.

[0142] 12. An interleaved layout for computing bit cells in a charge domain memory, wherein the bit cell capacitors are coupled in a plurality of different groups such that there are X groups interleaved X-ways, X being an integer greater than 1.

[0143] 13. A layout for a computing bit cell in a charge domain memory, wherein different sets of coupling capacitors are arranged above the bit cell transistors.

[0144] 14. A multiplication bit cell (M-BC) configured to perform a charge domain calculation between data stored in the bit cell and an Ib input signal.

[0145] 15. The M-BC of clause 14, wherein the charge domain computation is performed using one or more NMOS transistors.

[0146] 16. The M-BC of clause 14, further comprising a capacitor, the capacitor being a metal structure located above the bit cell.

[0147] 17. The M-BC of clause 14, wherein the M-BC is configured to implement logical operations.

[0148] 18. The M-BC according to clause 17, wherein the logic operation includes XNOR, NAND (NAND: "Negative AND" logical relationship), AND and other logic operations.

[0149] 19. The M-BC according to clause 14, further comprising: Fig.12 Layout shown.

[0150] 20. The M-BC of clause 14, further comprising an extended layout of 6T cells.

[0151] 21. The M-BC of clause 20, further comprising a transistor having a regular polycrystalline structure.

[0152] 22. The M-BC according to clause 14, further comprising: Fig.13 Layout shown.

[0153] Although different embodiments of the teachings of the present invention have been shown and described in detail herein, those skilled in the art can easily design many other variations of the teachings that still include these teachings. Therefore, although the foregoing is directed to different embodiments of the present invention, other and additional embodiments of the present invention can be designed without departing from the basic scope of the present invention.

Claims

1. An in-memory computing architecture, comprising: an in-memory computation array of bit cells, the in-memory computation array being configured to receive parallel bitwise input signals via a first in-memory computation array dimension and to receive one or more accumulation signals via a second in-memory computation array dimension, wherein each of the plurality of bit cells associated with a common accumulation signal forms a respective in-memory computation channel configured to provide a respective output signal; and a control circuit configured to cause the in-memory computation array to perform a multi-bit computation operation on the input signal and the accumulated signal using single-bit internal circuits and signals; Each bit cell in the in-memory computation array of the bit cells has a bit cell circuit configuration, and the bit cell circuit configuration includes: a bit cell storage circuit coupled to at least one bit cell computing device; and A bit cell capacitor is coupled to the bit cell computing device, the bit cell capacitor being further coupled to one or more additional capacitors corresponding to other bit cell capacitors within the bit cell column.

2. The in-memory computing architecture of claim 1 , further comprising a shaping buffer configured to shape a sequence of received data words to form the parallel bitwise input signal.

3. The in-memory computing architecture of claim 1 further comprising an analog-to-digital converter circuit configured to process a plurality of in-memory computing channel output signals to thereby provide a sequence of multi-bit output words. 4 . The in-memory computing architecture of claim 3 , further comprising a near memory computing path configured to provide the sequence of multi-bit output words as a computation result.

5. The in-memory computing architecture of claim 3, wherein: For each in-memory computing channel, the analog-to-digital converter circuit includes: a corresponding analog-to-digital converter, which is configured to digitize the in-memory computing channel output signal; and a corresponding shift register, which is configured to assign corresponding binary weights to the digitized in-memory computing channel output signal to thereby form corresponding parts of the multi-bit output word.

6. The in-memory computing architecture of claim 3, wherein: For each subset of the plurality of subsets of the in-memory computing channels, the analog-to-digital converter circuit includes a corresponding analog-to-digital converter configured to digitize a weighted analog sum of the in-memory computing channel output signals of the corresponding subset of the in-memory computing channels to thereby form a corresponding portion of a multi-bit output word.

7. The in-memory computing architecture of claim 1 further comprises a sparsity controller configured to mask zero-valued elements of the parallel bitwise input signals so that the multi-bit computing operation avoids processing zero-valued elements of the parallel bitwise input signals.

8. The in-memory computing architecture of claim 1, wherein: The input signal and the accumulated signal are combined with existing signals in memory.

9. The in-memory computing architecture of claim 1, wherein: The input signal and the accumulated signal are separated from existing signals in memory.

10. The in-memory computing architecture of claim 3, wherein: Each analog-to-digital converter and the corresponding accumulated signal form a computation channel in the memory.

11. The in-memory computing architecture of claim 4, wherein: The near memory computation path includes one or more of: a digital barrel shifter, a multiplexer, an accumulator, a lookup table, and a non-linear function element.

12. The in-memory computing architecture of claim 10, wherein: The multi-bit computing operations of the computing array in the memory include bit parallel computing / bit serial computing, Wherein, the bit-parallel computing includes: Load different matrix element bits into corresponding memory calculation channels; barrel-shifting the digitized outputs from the computational channels using corresponding barrel shifters to thereby achieve corresponding bit weightings; and Using corresponding accumulators, performing digital accumulation on all of the computation channels to generate multi-bit matrix element calculation results; and Wherein, the bit serial calculation includes: applying each bit of the vector element individually to the stored matrix element bits and storing each resulting digitized output; and Using a corresponding barrel shifter, the stored digitized outputs associated with each vector element bit are barrel shifted prior to digitally accumulating the stored digitized outputs corresponding to subsequent input vector bits.

13. The in-memory computing architecture of claim 4, wherein: The near-memory compute path is physically aligned with the in-memory compute architecture to increase throughput therethrough.

14. The in-memory computing architecture of claim 1, further comprising one or more configurable finite state machines configured to control computing operations of the in-memory computing architecture.

15. The in-memory computing architecture of claim 14, wherein: The finite state machine that controls the computational operations is configured to control parallel computational hardware utilized by some or all of the plurality of in-memory computational channels.

16. The in-memory computing architecture of claim 14, wherein: The finite state machine controls computations according to software instructions loaded into local memory.

17. The in-memory computing architecture of claim 2, wherein: The shaping buffer is configured to convert an external digital word of a first precision into a high-dimensional input vector.

18. The in-memory computing architecture of claim 17, wherein: The shaping buffer is configured to provide bits of input vector elements to the in-memory computation array in a sequential and parallel manner.

19. The in-memory computing architecture of claim 2, wherein: The shaping buffer is configured to align input data to ensure desired utilization and throughput of in-memory computation operations.

20. The in-memory computing architecture of claim 2, wherein: The shaping buffer is configured to implement reuse and shifting of input data according to a convolutional neural network operation.

21. The in-memory computing architecture of claim 1, wherein: The bitcell circuit configuration also includes a cathode plate of the bitcell capacitor coupled to the bitcell computing device.

22. The in-memory computing architecture of claim 1, wherein: The bit cell circuit configuration also includes an anode plate of the bit cell capacitor coupled to the additional capacitor.

23. The in-memory computing architecture of claim 1, wherein: The bitcell circuit arrangement further comprises the bitcell computation means configured to perform a computation operation having two operands.

24. The in-memory computing architecture of claim 1, wherein: The bit cell circuit configuration enables an XNOR or AND operation between storage data and input data.

25. The in-memory computing architecture of claim 1, wherein: The bitcell circuit configuration includes a staggered layout for at least two bitcells, wherein corresponding bitcell capacitors are coupled together in at least two independent groups to thereby implement corresponding independent computations.

26. The in-memory computing architecture of claim 25, wherein: A bank of coupled bit cell capacitors is laid out over one or more bit cell transistors.

27. The in-memory computing architecture of claim 25, wherein: The bit cell comprises an intra-charge domain memory computation bit cell driving one plate of a corresponding local bit cell capacitor and wherein coupling with other bit cell capacitors is achieved on another plate of the corresponding local bit cell capacitor.

28. The in-memory computing architecture of claim 27, wherein: The computational bit cells in the charge domain memory are configured to implement an XNOR or AND operation between the stored data and the input data.

29. The in-memory computing architecture of claim 25, wherein: A staggered layout for computing bit cells within a charge domain memory is provided such that bit cell capacitors are coupled in a plurality of different groups configured to be selectively coupled through one or more peripheral switches.

30. The in-memory computing architecture of claim 25, wherein: A staggered layout for computing bit cells in a charge domain memory is provided such that the bit cell capacitors are coupled in a plurality of different groups such that there are X groups for an X-way stagger, where X is an integer greater than one.

31. The in-memory computing architecture of claim 26, wherein: A layout for a computational bit cell within a charge domain memory is provided such that different sets of coupled capacitors are laid out over the bit cell transistors.

Images