In-memory computing device and related methods

Abstract

The invention provides a device and method for performing MAC operations using a memory array as a compute-in-memory (CIM) device that enables higher compute throughput, higher performance, and lower power consumption compared to computation using a processor located outside the memory array. In some embodiments, an activation architecture is provided using an array of bitcells arranged in rows and columns to store charges representing weight values in a weight matrix. A read word line (RWL) can also be used to provide input activation values to bitcells within a row of bitcells, while a read bit line (RBL) is configured to receive a product from a column of bitcells. Some embodiments provide multiple subarrays or blocks of the array of bitcells.

Description

TECHNICAL FIELD

[0001] The present disclosure relates generally to compute-in-memory (CIM) devices, and in particular, to a CIM device capable of performing multiply and accumulate (MAC) operations in neural network computations. BACKGROUND

[0002] Deep learning, machine learning, neural networks, and other matrix-based differentiable programs are used to solve a variety of problems, including natural language processing and object recognition in images. Solving these problems often involves performing computations based on matrix vector multiplication. For example, in a neural network having a plurality of neurons, the activation values input to the neurons can be viewed as an input activation vector, and the output activation values from the neurons can be viewed as an output activation vector. Computation of such a neural network often involves matrix vector multiplication of the input activation vector with a weight matrix to compute the output activation vector. Typically, the weight matrix can be rectangular, and the length of the output activation vector is not necessarily the same as the length of the input activation vector. Computation of the neural network often involves computation of multiply and accumulate (MAC) operations performed on data values, including the input / output activation vectors and the weights. A MAC operation refers to a multiplication operation between two values, and, subsequently, accumulation of a series of multiplication results to provide an output MAC value.

[0003] Computation of large deep neural networks involves large-scale parallel computation of many different data values. Computation operations (e.g., arithmetic or logical operations) are typically performed by a processor based on data transfers between the processor and a storage device (e.g., a memory array). As the data throughput requirements of large-scale parallel neural networks increase, the slower data transfers of fast processors and memories sometimes become a bottleneck for machine learning applications. SUMMARY

[0004] In view of the above, it is one of the objects of the present disclosure to provide a compute-in-memory (CIM) device and method for performing multiply and accumulate (MAC) operations, which is capable of achieving higher computation throughput.

[0005] In a first aspect, the present disclosure provides an in-memory computing device, comprising a memory array and one or more adders, wherein the memory array comprises a plurality of bitcell arrays, each bitcell array comprising: a plurality of bitcells arranged in a plurality of rows and a plurality of columns, each bitcell configured to store a weight according to a weight matrix and to multiply the weight with an input activation of an input activation vector to produce a product; a plurality of read word lines (RWLs), each RWL extending along a row direction and each RWL configured to provide an input activation of the input activation vector to bitcells of a respective row; and a plurality of read bit lines (RBLs), each RBL extending along a column direction and each RBL configured to receive a product from bitcells of a respective column; wherein the one or more adders are coupled to the plurality of RBLs of each bitcell array and configured to generate a multiply-accumulate (MAC) value based on some or all of the products received from the plurality of RBLs of each bitcell array.

[0006] In some embodiments, the one or more adders comprise an adder tree.

[0007] In some embodiments, the in-memory computing device further comprises a plurality of inverters, wherein each RBL of the plurality of RBLs is coupled to the one or more adders via a respective inverter of the plurality of inverters.

[0008] In some embodiments, the plurality of bitcells within the memory array are 8-transistor bitcells.

[0009] In some embodiments, the memory array is configured to receive an input activation vector and a weight matrix.

[0010] In some embodiments, each RWL is configured to provide an input activation of the input activation vector to bitcells of a respective row, and the MAC value is based on a vector product between the input activation vector and the weight matrix.

[0011] In some embodiments, the in-memory computing device further comprises a routing circuit having a routing input configured to receive an input activation and a plurality of routing outputs coupled to some or all of the plurality of RWLs, the routing circuit configured to route the input activation to some or all of the plurality of RWLs.

[0012] In some embodiments, the routing circuit comprises a demultiplexer.

[0013] In some embodiments, the in-memory computing device further comprises a static random access memory (SRAM) edge cell disposed alongside a row of the plurality of rows of bitcells.

[0014] In some embodiments, the memory array includes a first subarray and a second subarray aligned along a column direction with the first subarray, and the in-memory computing device further includes: a first routing circuit having a routing input and a plurality of routing outputs coupled to some or all of a plurality of RWLs in the first subarray; a second routing circuit having a routing input and a plurality of routing outputs coupled to some or all of a plurality of RWLs in the second subarray.

[0015] In some embodiments, the in-memory computing device further includes: one or more input multiplexers controllable to route an input activation to the routing input of the first routing circuit or the routing input of the second routing circuit.

[0016] In some embodiments, the in-memory computing device further includes: an interconnect circuit coupling a plurality of RBLs in the first subarray and the second subarray to the adder tree, wherein the interconnect circuit includes one or more output multiplexers.

[0017] In some embodiments, the one or more output multiplexers and the one or more input multiplexers are configured to receive a same address signal and address based on the address signal.

[0018] In some embodiments, the memory array includes a first subarray and a second subarray, and the in-memory computing device further includes: a first set of switches, each switch in the first set of switches switchably coupling a corresponding RBL in the first subarray to the adder tree; and a second set of switches, each switch in the second set of switches switchably coupling a corresponding RBL in the second subarray to the adder tree; wherein the first set of switches is configured to turn on or off at an opposite phase relative to the second set of switches.

[0019] In some embodiments, the first subarray and the second subarray are aligned along a column direction or along a row direction.

[0020] In some embodiments, the first set of switches is switched based on an address signal, and the second set of switches is switched based on an inverted version of the address signal.

[0021] In some embodiments, the in-memory computing device further includes a decoder coupled to the first set of switches and the second set of switches, the decoder configured to receive an address signal and control the first set of switches and the second set of switches to switch based on the address signal.

[0022] In some embodiments, the memory array includes a first subarray and a second subarray aligned along a row direction with the first subarray, and the adder tree is located between the first subarray and the second subarray and coupled to some or all RBLs of the first subarray and some or all RBLs of the second subarray.

[0023] In some embodiments, the memory array includes a plurality of subarrays arranged into a plurality of rows and a plurality of columns, the plurality of subarrays within each row being contiguous with one another.

[0024] In some embodiments, the one or more adders include an adder located between two adjacent rows of the plurality of subarrays, and an adder tree coupling the adder via conductors extending along a row direction.

[0025] In some embodiments, the one or more adders are coupled to the plurality of RBLs via one or more conductors extending along a row direction.

[0026] In some embodiments, the one or more conductors are arranged in a first metal layer that is parallel to and offset from a second metal layer in which the plurality of RBLs are arranged.

[0027] In a second aspect, the disclosure provides a method for operating a memory array to perform a multiply-accumulate (MAC) operation of an input activation vector with a weight matrix, wherein the memory array includes a plurality of bitcell arrays each arranged into a plurality of rows and a plurality of columns of bitcells, the method comprising: storing a plurality of weights in bitcells of the memory array according to the weight matrix; for each row of the plurality of rows, providing an input activation of the input activation vector to bitcells within the row using a corresponding read word line (RWL) extending along a row direction; for each bitcell, multiplying the input activation with a weight stored therein to produce a product; for each column of the plurality of columns, receiving products from corresponding bitcells within the column using a read bit line (RBL) extending along a column direction; and producing a MAC value by summing the received products using an adder tree of RBLs coupled to the plurality of bitcell arrays.

[0028] In some embodiments, producing the MAC value includes producing a vector product between the input activation vector and the weight matrix.

[0029] In some embodiments, the method further includes routing an input activation to one of the plurality of RWLs using a demultiplexer.

[0030] In some embodiments, the memory array includes a first subarray and a second subarray, and the method further includes switchably coupling one of a first set of RBLs in the first subarray or a second set of RBLs in the second subarray to the adder tree based on an address signal.

[0031] In some embodiments, the method further includes selectively routing an input activation to one of the first subarray or the second subarray based on the address signal.

[0032] These and other objects of the present application will in no way be doubted by those skilled in the art upon reading the following detailed description of the preferred embodiments illustrated in the accompanying drawings. The detailed description will be given in the following examples with reference to the drawings. BRIEF DESCRIPTION OF DRAWINGS

[0033] The present application can be more fully understood by reading the subsequent detailed description and examples given with reference to the accompanying drawings, of which:

[0034] Figure 1 is a schematic diagram illustrating an exemplary neural network representation of MAC operations (which are performed using the CIM devices disclosed herein).

[0035] Figure 2A is a schematic diagram of an exemplary CIM device (for computing a single layer neural network as shown in Figure 1 ) having a memory array, shown in accordance with some embodiments.

[0036] Figure 2B is a schematic diagram of an exemplary CIM device (for applying a filter to compute weights W0,0 of a weight matrix used in Figure 2A ) having a memory array, shown in accordance with some embodiments.

[0037] Figure 3A is a schematic diagram of an exemplary 8-transistor (8-T) memory bitcell that can be used in a CIM device for performing MAC operations, shown in accordance with some embodiments.

[0038] Figure 3B is a schematic diagram illustrating the application of a filter using a multiplier for a single bit (e.g., W F(0,0) ).

[0039] Figure 4 is a schematic diagram of a CIM device architecture using an array of memory bitcells, shown in accordance with a first embodiment.

[0040] Figure 5 is a schematic diagram of a CIM device architecture (which is a variation of the CIM device shown in Figure 4 ) shown in accordance with a second embodiment.

[0041] Figure 6 is a schematic diagram of a CIM device architecture (which is a variation of the CIM device shown in Figure 5 ) shown in accordance with a third embodiment.

[0042] Figure 7 is a schematic diagram of a CIM device architecture having sub-arrays or tiles, shown in accordance with a fourth embodiment.

[0043] Figure 8 is a schematic diagram of a CIM device architecture with subarrays having multiplexed activation inputs according to a fifth embodiment.

[0044] Figure 9 is a schematic diagram of a CIM device architecture with interconnect circuitry according to a sixth embodiment.

[0045] Figure 10 is a schematic diagram of a CIM device subarray architecture with adder multiplexing between rows of subarrays according to a seventh embodiment.

[0046] Figure 11 is a schematic diagram of a CIM subarray architecture with adder multiplexing between columns of subarrays according to an eighth embodiment.

[0047] Figure 12 is a schematic diagram of a CIM device subarray architecture with adder multiplexing in rows and columns of subarrays according to a ninth embodiment.

[0048] Figure 13 is a schematic diagram of a CIM device subarray architecture with multiple columns of subarrays and adjacent to each other according to a tenth embodiment.

[0049] Figure 14 is a high level block diagram illustrating an illustrative computing device 1400 in which some embodiments can be practiced.

[0050] In the following detailed description, for the purposes of explanation, numerous specific details are set forth in order to facilitate a thorough understanding of the embodiments of the application. However, it will be apparent to one skilled in the art that embodiments of the application can be practiced without these specific details, different embodiments can be combined according to the needs of the situation, and that not all of the depicted embodiments are necessary for the practice of the embodiments of the present application. DETAILED DESCRIPTION

[0051] The following description is provided in relation to preferred embodiments of the present application, which are merely used to illustrate technical features of the present application, and are not used to limit the scope of the present application. Certain words are used throughout the description and claims to indicate certain features of the application. Those skilled in the art will understand that the manufacturer can use different names to refer to the same features. Therefore, the description and claims of the present application do not distinguish the features by the difference in names, but by the difference in functions. The terms "element", "system" and "device" used in the present application can be computer-related entities, which can be hardware, software, or a combination of hardware and software. In the following description and claims, the terms "comprise" and "include" are open terms, which should be interpreted as "comprising, but not limited to". In addition, the term "coupled" means indirect or direct electrical connection. Therefore, if a device is described as being coupled to another device, it means that the device can be directly electrically connected to the other device, or indirectly electrically connected to the other device through other devices or connection means.

[0052] In the drawings, corresponding reference numerals and symbols in different drawings generally refer to corresponding parts unless otherwise indicated. The drawings clearly illustrate the relevant parts of the embodiments and are not necessarily drawn to scale.

[0053] The term "substantially" or "approximately" used herein means that within an acceptable range, those skilled in the art can solve the technical problems to be solved and substantially achieve the technical effects to be achieved. For example, "substantially equal" means that within an acceptable range, the skilled person can accept a certain error from "exactly equal" without affecting the correctness of the result.

[0054] Disclosed herein are devices and methods for performing MAC operations using a memory array as a compute-in-memory (CIM) device. As shown in Figure 2A The memory array includes a plurality of bitcell arrays (e.g., bitcell array 110), which can be arranged in a plurality of rows and a plurality of columns, with bitcell arrays within each row sharing the same input activation, each bitcell array can include a plurality of bitcells (e.g., bitcell 110_0) arranged in a plurality of rows and a plurality of columns, as shown in Figure 2B The plurality of bitcells (e.g., bitcell 110_0) arranged in a plurality of rows and a plurality of columns, as shown in Figure 2A And Figure 2B As shown in the array, those skilled in the art can understand that Figure 2A The weights (e.g., W0,0) corresponding to the bitcell arrays can be a representation of a corresponding multi-dimensional weight matrix, for example, corresponding to Figure 2BA 4*4 weight matrix is shown. CIM (compute-in-memory) or in-memory computation (or interchangeably referred to as “in-memory computing”) is a technique that performs computations on data using internal circuitry located within a memory array without sending such data to a processor, which can achieve higher computation throughput and higher performance compared to computations using a processor located outside of the memory array. CIM can also reduce energy consumption due to reduced data movement between an external processor and a memory.

[0055] Some embodiments are directed to a CIM device with an activation architecture that can be used in MAC operations with a weight matrix in neural network computation. The weight matrix can be a filter matrix used in deep neural network computation. The size (or interchangeably described as “dimension”) of the weight matrix can be scaled to accommodate different numbers of neurons and different numbers of layers in a neural network. In some embodiments, a memory array (e.g., a bit cell array) has a programmable number of filters, and one or more subarrays of the memory array can be selectively activated or deactivated to program the CIM device to perform computation of different filter dimensions given the same activation input. Such programmable design can increase the flexibility of the CIM device. The ability to deactivate subarrays when not in use can also reduce the power consumption of the CIM device.

[0056] Some embodiments include a memory array arranged in rows and columns. Each bit cell has a static random-access memory (SRAM) that stores a charge Q, which is used to represent a weight value in a weight matrix. Each bit cell can operate as a multiplier that performs multiplication between the stored weight and an input activation value applied on a read word line (RWL) in the bit cell array. In such embodiments, the read word line RWL can be repurposed to provide the input activation value to the bit cells in a row of bit cells corresponding to the RWL.

[0057] In some embodiments, in one bitcell array, the products from multiple bitcells can be read using read-bit lines (RBLs) that extend along the column direction. In one bitcell array, each RBL is coupled to bitcells within a column of bitcells and is configured to receive products from the bitcells. For multiple bitcell arrays, each RBL is connected to one or more adders, e.g., an adder tree that accumulates the products to produce a MAC value. The number of bitcells within a column can be designed to accommodate input activation vectors of different sizes.

[0058] Some aspects relate to routing input activation signals to different RWLs, e.g., in Figure 5 In an example of FIG. 4, input activation signal a0 is routed to one of the 4 RWLs according to control signal CIM_MUXSEL<1:0>. In some embodiments, a routing circuit can be provided near the edge of the memory array. The routing circuit has a routing input configured to receive an input activation, and multiple routing outputs coupled to the RWLs of different rows of bitcells. In some embodiments, the routing circuit can include a demultiplexer.

[0059] Some embodiments provide multiple sub-arrays or tiles of bitcell arrays within a memory array. Some aspects relate to routing of input activation signals and output products from multiple sub-arrays. In one embodiment, two routing circuits are provided to route input activation signals to the RWLs in two sub-arrays. In one embodiment, an input multiplexer is used to route one input activation (e.g., via one of the two routing circuits) to one of the sub-arrays. By using two isolated sub-arrays (which have longer columns of bitcells coupled to RBLs) instead of a single array, device loading effects on the RBLs can be reduced, which can improve power consumption of the CIM device.

[0060] In some embodiments, the output products from multiple RBLs can be multiplexed using interconnect circuitry before being provided to the adder tree. Thus, fewer adder tree connections are needed compared to the number of RBLs, which can reduce RBL capacitance and lower the power consumption of the CIM device. In some embodiments, a set of switches is provided to switchably disconnect the RBLs in a subarray from the adders when the subarray is inactive, which can also reduce RBL capacitance and power consumption of the CIM device. The set of switches can be addressed by using address signals configured to control input multiplexers (e.g., to route input activations to different subarrays).

[0061] Some aspects can use an existing foundry design for a bitcell memory array (bitcell array), such as an 8-transistor (8-T) memory array, without or with only minimal hardware changes, which can reduce the design and manufacturing costs of the CIM device of the present disclosure.

[0062] Embodiments disclosed herein can be used for the computation of neural networks in artificial intelligence (AI) applications, some examples of which are described in detail below. It should be appreciated that embodiments of the present disclosure can also be used for other applications, such as but not limited to image processing, image classification, and face recognition using a camera to take an image. In some embodiments, the CIM device is part of a mobile device.

[0063] The above aspects and embodiments, as well as additional aspects and embodiments, are further described below. These aspects and / or embodiments can be used individually, together, or in any arbitrary combination of two or more, as the present application is not limited in this respect.

[0064] Figure 1 is a schematic diagram of an exemplary neural network representation showing a MAC operation (also interchangeably described as a “MAC operation”) performed using a CIM device disclosed herein. Figure 1 An input activation vector (a0, a1, a2, a3) is shown, which can represent four input activation values from a previous layer. The neural network applies a weight matrix where each weight w i,jThese represent the weights of the corresponding input activations and neurons, respectively. The neural network generates an output activation vector (N0, N1, N2, N3) with four output values ​​N0...N3 based on a vector-matrix multiplication of the input activation vector and the weight matrix. This output activation vector is used for the four corresponding neurons in the next layer (for ease of understanding and simplification, the four neurons can be represented as N0, N1, N2, N3). This vector-matrix multiplication can be performed using MAC operations. Those skilled in the art will understand that the weights w... i,j It is a descriptive method for ease of explanation and understanding, and it can usually be represented as a multidimensional weight matrix.

[0065] It should be understood that, although Figure 1 A 4x4 square weight matrix is ​​shown, but this is for illustrative purposes only. The number of neurons in the previous layer can differ from the number of neurons in the next layer, and the input and output activation vectors can have any suitable size and / or dimensions. Furthermore, although... Figure 1 The illustration shows a single layer of neurons N0-N3, but various aspects of the invention can also be applied to MAC operations for neural network computations involving multiple layers. For example, in Figure 1 In the single-layer neural network shown, preceding neurons N0-N3, there can be multiple possible weights between the input activation vector (a0, a1, a2, a3) and neurons N0-N3. Multiple filters can be selected to represent the intermediate neuron layers, and... Figure 1 Each weight in the weight matrix shown can be computed using a convolution operation on the selected filter (implemented using the MAC operation of the CIM device disclosed herein). In such a MAC operation, the selected filter can be formed with... Figure 1 The weight matrix shown is similar to the weight matrix shown below. The following uses... Figure 2A and Figure 2B Describe an example of MAC operation on a single-layer weight matrix and filter convolution.

[0066] Figure 2A It is according to some embodiments having a memory array (in Figure 2A The example includes an exemplary CIM device (for calculating such as a 16-bit cell array 110). Figure 1 A schematic diagram of a single-layer neural network (shown). Figure 2AA CIM device 100 is shown having an array of bit cells (also interchangeably described as a "memory array," comprising multiple bit cells or memories) 110 arranged in four rows and four columns. The bit cell array in each row is used to receive one of four input activations a0, a1, a2, a3 (responders). Each bit cell in the array is programmed with a charge (e.g., representing a weight in a 4×4 weight matrix) and configured to perform a multiplication operation between the input activation received in its column and the weight stored therein. The product of the multiplication operations is collected along the bit cell array within each column and based on... Figure 1 The formulas shown are accumulated (for example, the accumulated multiplication value collected for row 0 is: a0*W0,0+a1*W1,0+a2*W2,0+a3*W3,0; the accumulated multiplication value collected for row 1 is: a0*W0,1+a1*W1,1+a2*W2,1+a3*W3,1; and the accumulated multiplication value collected for row 2 is: a0*W0,2+a1*W1,2+a2*W2,2+a3*W3,2) to generate four output activations N0, N1, N2, and N3 for the neurons, respectively. Those skilled in the art will understand how MAC values ​​are generated based on the product of bit cells along the column direction of the bit cell array.

[0067] Figure 2B This is an exemplary bit cell array according to some embodiments (for applying filters to calculate in Figure 2A The weights W0,0 of the weight matrix used, or, for execution Figure 2A The diagram shows the relevant calculations for the bit cell array 110. It can be understood that... Figure 2B The bit cell array shown can also be understood as a memory array. Figure 2B A bit cell array 110 is shown, having an array of bit cells 110_0 arranged in four rows and four columns. Each row receives input activation from a corresponding one of the read word lines RWL0, RWL1, RWL2, RWL3. Each bit cell is programmed to have a filter value (also interchangeably referred to as a "weight") W. F(X,Y) Where X is the filter identifier and Y is the filter bit number. For example, W F(2,1) This indicates the first bit (bit 1) of filter 2, used to calculate the weight 0 (W0,0) of neuron 0 or for correlation calculations of the weight 0 (W0,0) of neuron 0. Figure 2B The example shown provides a 4×4 matrix of filter values, where each of the four rows represents one of four 4-bit filters X = (0, 1, 2, 3), i.e., inFigure 2B The example shows 4 groups / filters, each with 4 bits. RWL is used to select one of the four filters.

[0068] Still refer to Figure 2B Using four read-bit lines RBL0, RBL1, RBL2, and RBL3, the output products of each bit unit are collected column-wise. For example... Figure 2B As shown, using four conductors (e.g., wires, metal wiring in a metal layer) 120, the output values ​​MULT0...MULT3 corresponding to each respective RBL are routed for further processing, such as accumulating MULT0...MULT3 of the corresponding RBLs in other bit cell arrays located in the corresponding column of the same bit cell array 110. For example, conductor 120 can route the output product to an adder tree (not shown).

[0069] Figure 2B This is merely an example illustrating the choice of a 4-bit filter to compute W0,0 using a 4×4 bit cell array. It can be understood that this can be achieved by using a bit cell array 110 with a different number of columns. Figure 3A The design can be extended to filters of any bit size. By having different numbers of rows, the bit cell array 110 can also be used to compute any number of filters. For example, the bit cell array 110 can store 4, 8, 16, 32, or any suitable number of filters. Each filter can be 4, 8, 16, 32, or any suitable number of bits.

[0070] Figure 3A This is a schematic diagram of an exemplary 8-transistor (8-T) memory bit cell (used in a CIM device to perform MAC operations) according to some embodiments. Figure 2B Bit unit 310 is shown, which can be used as Figure 3A Bit cell 110_0 is included. Bit cell 310 includes 6-T SRAM (typically, an inverter consists of two transistors). It is understood that any foundry design of bit cells known now or developed in the future can be used for bit cell 310, and this invention is not limited thereto. In this invention, the SRAM within bit cell 310 can operate as an independent multiplier of the input value and the stored charge Q. For example, the inverted output node MULT is coupled to RBL 103 via inverter 1301. Figure 2BThe value of the inverting output node MULT represents the product of the input value provided by RWL0 and the weight Q stored in the SRAM of the bitcell 310, as shown in the truth table 1011. In some embodiments, the stored charge Q can be used to represent a weight value (such as a filter value described above). For example, MULT is 1 only when both RWL0 and weight Q0 are 1. The output of the product can be provided by MULT or routed by RBL along the row direction.

[0071] The bitcell 310 is used to perform MAC operations to compute filter convolutions of the design shown in Figure 3B For example, the bitcell 310 can be used as a multiplier to apply one bit of the filter, as shown in the schematic diagram in Figure 4 The W F(0,0) .

[0072] Figure 4 A schematic diagram of a CIM device architecture (showing a portion thereof) using an array of memory (bitcells) according to a first embodiment. Figure 2A The CIM device 400 shown includes an array of bitcells 410 arranged in four rows 412-0, 412-1, 412-2, 412-3 and four columns 413-0, 413-1, 413-2, 413-3. In the first row 412-0, the bitcells are interconnected to RWL0 extending along the row direction. During operation, each of the read word lines RWL0, RWL1, RWL2 and RWL3 can be used as an activation line for the respective row 412-0, 412-1, 412-2, 412-3 to provide an input activation value to the bitcells within the respective row. In an embodiment, the input activation values can be provided sequentially, for example, one in one RWL while the remaining RWLs are unused or deactivated. Other usage scenarios are also possible, for example, using two or more RWLs in parallel. The activation timing in the four RWLs can be controlled using any suitable means, for example, by internal or external clock signals. According to an aspect, the RWLs can perform two functions simultaneously: carrying the input activation (e.g., a0) and selecting the filtering selection information, for example, one of the read word lines RWL0, RWL1, RWL2 and RWL3 is active / activated to select the bitcell (or filter) coupled thereto to perform the multiplication operation, and represents the input activation value a0, while the other read word lines are not active, e.g., 0.

[0073] In the first column 413-0, four bit cells can store four-bit values, e.g., values of a four-bit filter. The bit cells within the first column are interconnected by RBL0 that extends along the column direction. The output activation line 420-0 couples RBL0 to the adder tree 140 through an inverter and provides an inverted output MULT0 (MULT0 and MULTB0 are inverses of each other) to the adder tree 140. Similarly, each of the output activation lines 420-1, 420-2, and 420-3 interconnects the respective RBL1, RBL3, and RBL4 to the adder tree 140. The products on each RBL can be driven in any suitable manner to coordinate with the timing of the input activation in the four RWLs. The adder tree 140 is used to accumulate the received products to produce MAC values, e.g., N0= a0*W0,0+ a1*W1,0+ a2*W2,0+ a3*W3,0, N1= a0*W0,1+ a1*W1,1+ a2*W2,1+ a3*W3,1, and so on, where the calculations of the weights W0,0, W1,0, etc. can be referenced to Figure 2B and Figure 4 For purposes of understanding, the present application will not be described in further detail. The adder tree 140 can be implemented in any suitable manner now known or hereafter developed. Although not shown in the figure, one or more adders (e.g., 4-bit adders) can be provided to couple MULT0 to the adder tree 140.

[0074] In some embodiments, each of the output activation lines 420-0, 420-1, 420-2, and 420-3 extends along the row direction to route signals to the adder tree. To avoid shorting between the activation lines 420 (including the output activation lines 420-0, 420-1, 420-2, and 420-3) and the RBLs (including RBL0, RBL1, RBL3, and RBL4), the output activation lines 420 can be implemented as conductors extending in a metal layer that is a different layer / plane (e.g., metal wiring / traces, etc.) offset from the metal layer in which the RBLs are located and parallel to the surface of the semiconductor substrate of the bit cells 410 (or the bit cell array in which the bit cells 410 are located). In one non-limiting example, the output activation lines 420 are provided in the M3 metal layer, while interlayer interconnect structures (such as vias) are provided to couple the output activation lines 420 with the respective RBLs. In embodiments where the bit cells 410 reuse an existing foundry memory bit cell design, adding the output activation lines 420 does not significantly increase the cost or complexity of the circuit design for the overall memory array.

[0075] In other embodiments, output activation lines 420-0, 420-1, 420-2, and 420-3 may be absent, thus eliminating the need for additional routing in the metal layer on the bit cell to route the RBL output. In such embodiments, signals from the RBL can be routed along the column direction rather than the row direction. For example, for this purpose, routing circuitry can be placed below the bit cell row.

[0076] The memory-based CIM device 400 offers several advantages. The CIM array architecture is highly scalable, providing flexibility for applications across different technology generations with varying matrix and vector size requirements. For example, although... Figure 5 The MAC operation of a 4x4 weight matrix is ​​illustrated, but the CIM device 400 can be extended to accommodate weight matrices of any size. For example, by adding more 4 rows of bit cells, computation can be performed using an 8×4 weight matrix, e.g., 8 filters. Similarly, by expanding each of the four rows to have 8 columns, an 8-bit filter can be computed using a 4×8 weight matrix. A large number of multiplier cells can be provided based on any suitable SRAM read / write circuit architecture now known or developed in the future to benefit from the high memory density of the bit cell array. For example, scaling can be provided by using more rows and columns of bit cells within the array, and / or by using a set of subarrays or blocks as described below.

[0077] Figure 4 This is a schematic diagram of the CIM device architecture according to the second embodiment. Figure 5 A variant of the CIM device shown. Figure 5 A CIM device 500, similar in many respects to CIM device 400, is shown, wherein similar components are indicated by the same reference numerals. The difference between CIM device 500 and CIM device 400 lies in the provision of a routing circuit 551, which has a routing input 5511 for receiving input activation a0, and four routing outputs 5512, each coupled to one of RWL0, RWL1, RWL2, and RWL3 to selectively route input activation a0 to one of the four RWLs (i.e., RWL0, RWL1, RWL2, and RWL3) based on the control signal CIM_MUXSEL<1:0>. Figure 5 In the example shown, routing circuit 551 is a demultiplexer.

[0078] Figure 5The loading of the 4-bit weights of the weight matrix stored in each bitcell is also shown. During operation, one of the four RWLs is activated at a time based on the output of the demultiplexer 551. In an example, the bitcells of each row are used to store the weights from one filter or the weights from one layer. Thus, the CIM device 500 can be used to compute four filters. Each filter for a row has four bits, which are stored in the bitcells of each of the four columns. Figure 5 The architecture in FIG. 6 can be scaled by adding more rows and / or columns in order to provide computation for a larger number of filters and / or filters with more number of bits.

[0079] The CIM device 500 also includes a plurality of static random-access memory (SRAM) edge cells 130, which are located below the fourth row (bottom-most row) of bitcells and above the first row (top-most row) of bitcells. The edge cells 130 can provide Figure 6 electrical isolation between the four rows of bitcells shown and additional circuitry (e.g., located below or above the different blocks of additional rows of bitcells shown). Figure 5

[0080] Figure 6 is a schematic diagram of a CIM device architecture according to a third embodiment, which is a variation of the CIM device shown in FIG. 6. Figure 7 Figure 7 A CIM device 600 is shown, which is similar in many respects to the CIM device 500, with similar components denoted by the same reference numerals. The CIM device 600 differs from the CIM device 400 in that eight rows of bitcells are provided instead of four rows. The additional rows can be used to scale the number of filters or weights used in the computation. Each row can store 4-bit weights of one filter or one layer. A demultiplexer 651 activates one of the 8 RWLs to route the input activation a0 according to a control signal CIM_MUXSEL<2:0>.

[0081] Figure 2A is a schematic diagram of a CIM device architecture with subarrays or blocks according to a fourth embodiment. As shown in FIG. 7, Figure 7 The CIM device 700 includes subarrays 701 and 702 adjacent to each other and aligned along the column direction. The CIM device 700 also includes subarrays 703 and 704 aligned along the row direction with the subarrays 701 and 702, respectively.

[0082] ​​As shown, each of the sub-arrays 701 and 702 is loaded with an independent weight matrix having 4-bit weights. It can be appreciated that the sub-arrays 701, 702, 703, 704 can be 4-bit cell arrays, that is, each of the sub-arrays 701, 702, 703, 704 is a single 4-bit cell array. Figure 6 Figure 5 In the CIM device 500, the sub-arrays 701, 702, 703, 704 are each a single 4-bit cell array. Two demultiplexers (referred to as "DEMUX" in the figure) 751, 752 are provided in the CIM device 700. The first demultiplexer 751 is coupled to the four RWLs in the first sub-array 701, and the second demultiplexer 752 is coupled to the four RWLs in the second sub-array 702. The first demultiplexer 751 is configured to route a first activation input a0 to one of the four RWLs in the first sub-array 701 according to an address signal CIM_ADDR<1:0>. The second demultiplexer 752 is configured to route a second activation input a1 to one of the four RWLs in the second sub-array 702 according to the address signal CIM_ADDR<1:0>. It can be appreciated that for the sub-arrays 701 and 702, the same RWL is activated, for example, RWL0 in the sub-array 701 and RWL0 in the sub-array 702 are active for carrying the input activation values a0 and a1, respectively, while the other RWLs are not active / inactive.

[0083] The adder tree 740 is located between the first sub-array 701 and the third sub-array 703. As shown, the input activation in the third sub-array 703 is provided by the same RWLs coupled to the first sub-array 701. Since the adder tree 740 is shared by two sub-arrays (two bit cell arrays) 701 and 703, fewer adder components are needed to perform the CIM operation for the two sub-arrays, which can increase the area utilization efficiency of the circuit on the semiconductor substrate. In some embodiments, the adder tree can be shared by using time-multiplexing or time-sharing.

[0084] Referring back to Figure 8 Each RBL connects 8 bit cells to the adder tree. When the 8 bit cells in a column are switched during operation, the capacitance of the RBL is changed from 8 bit cells to 0 bit cells, and vice versa. Figure 8 to Figure 10 The CIM device 600 consumes more power compared to the CIM device 500 in

[0085] In embodiments of the present application, by splitting a plurality of weights or filters of a column into isolated sub-arrays, the device load can be reduced and the power consumption can be reduced.

[0086] Figure 2A ​is a schematic diagram of a CIM device architecture according to a fifth embodiment (with subarrays of multiplexed activation inputs). In Figure 8 to Figure 10 the example, subarray 701 and subarray 702 together constitute Figure 8 one / a single bitcell array as shown in Figure 2A the example, the first bitcell array includes subarray 701 and subarray 702), and similarly, in Figure 8 the example, subarray 703 and subarray 704 constitute Figure 9 another bitcell array as shown in Figure 10 the example, the second bitcell array includes subarray 703 and subarray 704), it is understood that Figure 8 and Figure 8 can be considered as a variant implementation of Figure 6 the left half. Figure 8 CIM device 800 is similar to CIM device 700 in many respects, with like components denoted by like reference numerals. CIM device 800 differs from CIM device 700 in that input multiplexer 854 is controllable according to a second address signal CIM_ADDR<2> to route input activation a0 to the routed input of one of two demultiplexers 751, 752. That is, the second address signal CIM_ADDR<2> specifies whether the input activation is to be routed to the top four rows of bitcells or the bottom four rows of bitcells. 4x4 subarrays 701 and 702 are loaded with weights or filters representing 8 filters or 8 weights. First address signals CIM_ADDR<1:0> can be used to route the input activation to one of the four RWLs within the two subarrays. Subarray 701 is isolated from subarray 702 by the provision of SRAM edge cell 731 between the bottom row of subarray 701 and the top row of subarray 702. Thus, each RBL of either of subarrays 701 or 702 is coupled to only 4 bitcells in a column, as opposed to Figure 8 the scheme for eight rows or eight weights / filters, in Figure 9 the device load on the RBLs is reduced.

[0087] By adding additional subarrays and demultiplexers, Figure 9 the architecture of can be further extended to activate one of 16 or 32 filters, and the present invention is not limited in this regard.

[0088] Figure 8 is a schematic diagram of a CIM device architecture according to a sixth embodiment with interconnect circuitry. Figure 9A CIM device 900, which is similar in some respects to CIM device 800, is shown, wherein similar components are indicated by the same reference numerals. CIM device 900 includes interconnect circuitry 930 comprising a plurality of output multiplexers 931 that route eight MULT signals received at inputs 9311 of the output multiplexers 931 to one of four outputs 9312 for supply to an adder tree 940. Therefore, only half the number of inputs need to be connected to the adder tree 940. For example, the adder tree 940 may include four adders, each corresponding to a multiplexer output 9312, instead of... Figure 10 The adder tree 740 requires eight adders to receive the product from subarrays 701 and 702. For example... Figure 10 As shown, the output multiplexer 931 can be provided to the input multiplexer 854 to selectively activate the same address signal CIM_ADDR of one of the two subarrays 701, 702. <2> It should be understood that any suitable multiplexing scheme can be used to reduce the number of adder inputs required for the adder tree. By multiplexing the output products, the complexity and circuit area efficiency of the adder tree 940 can be advantageously improved.

[0089] Figure 10 This is a schematic diagram of the CIM device subarray architecture (which has adder multiplexing between rows of the subarray) according to the seventh embodiment. Figure 10 A CIM device 1000, which is similar in many respects to CIM device 900, is shown, wherein similar components are indicated by the same reference numerals. The difference between CIM device 1000 and CIM device 900 is that CIM device 1000 provides a first set of switches 1041 and a second set of switches 1042, each switch in the first set of switches 1041 switchably coupling a corresponding RBL in the first subarray 701 to the adder tree 940, and each switch in the second set of switches 1042 switchably coupling a corresponding RBL in the second subarray 702 to the adder tree. As shown... Figure 9 As shown, each switch in the first group of switches 1041 is positioned within the circuit path from the corresponding RBL of the first subarray 701 to the adder tree 940. It is understood that the switches can be implemented in any suitable manner, such as, but not limited to, metal-oxide-semiconductor field-effect transistors (MOSFETs). In a non-limiting example, the switch could be an NMOS.

[0090] The first set of switches 1041 and the second set of switches 1042 are only one set of switches open or turned off at a time, while the other set of switches is closed or turned on. As shown in Figure 10 , this can be achieved by controlling the switching of the second set of switches 1042 according to the address signal CIM_ADDR<2> and by controlling the switching of the first set of switches 1041 according to the inverted CIM_ADDR<2> via inverter 1043, such that the two sets of switches are opened or closed in opposite phases to each other (e.g., the first set of switches is on while the second set of switches is off, or the second set of switches is on while the first set of switches is off). Thus, the two sets of switches act as an alternative multiplexing scheme, which can save circuit area compared to the interconnection circuit 930 in Figure 11 . For example, by removing the interconnection circuit 930 in Figure 11 , the multiplexer 931 is not needed.

[0091] As another advantage, when a set of switches is turned off, e.g., when the corresponding subarray is not selected by CIM_ADDR<2>, the switches disconnect the RBL in the subarray from the adder tree, which can reduce the power consumption of the RBL capacitance and the CIM device.

[0092] Figure 11 is a schematic diagram of a CIM subarray architecture according to an eighth embodiment (which has adder multiplexing between columns of subarrays). Figure 10 CIM device 1100 is shown, which is similar to CIM device 1000 in many respects, with similar components denoted by the same reference numerals. Like CIM device 1000, CIM device 1100 is loaded with 8 4-bit filters (it is understood that subarrays 1101 and 1103 constitute one / a single bitcell array, i.e., the single bitcell array includes subarrays 1101 and 1103, Figure 10 , and Figure 11 are different embodiments implemented based on similar ideas, Figure 4 is a bitcell array implemented with subarrays utilizing up (U) down (D) structure, Figure 11 is a bitcell array implemented with subarrays utilizing left-right structure), CIM device 1100 differs from CIM device 1000 in that the two subarrays 1101, 1103 are arranged side-by-side along the row direction and share the same RWL for input activation.

[0093] , and Figure 11In contrast to the CIM device 400, the output RBLs of the CIM device 1100 are not routed to the adder tree using horizontal conductors 420 that overlap the memory array. Instead, the RBLs of the subarrays 1101 and 1103 are routed under the subarrays in the column direction before interconnecting to the adder tree 940. The CIM device 1100 can thus be implemented without adding vertical vias and conductors 420 in different metal layers to provide the interconnections.

[0094] The CIM device 1100 provides a different demultiplexing scheme than the CIM device 1000 to select one of the eight filters. As shown in FIG. 11, the first address signals CIM_ADDR<1:0> control the demultiplexer 1151 to route the input activation a0 to the RWL of one of the four rows. The selection between the two subarrays is further performed by the second address signal CIM_ADDR<2> that controls the opening and closing of the first set of switches 1141 and the second set of switches 1143. Each switch in the first set of switches 1141 switchably couples a corresponding RBL in the first subarray 1101 to the adder tree 940; each switch in the second set of switches 1142 switchably couples a corresponding RBL in the second subarray 1103 to the adder tree. As shown in FIG. 11, each switch in the first set of switches 1141 is disposed within the circuit path from the corresponding RBL in the first subarray 1101 to the adder tree 940. The switches can be implemented in any suitable manner, such as but not limited to metal-oxide-silicon field-effect transistors (MOSFETs). In a non-limiting example, the switches are NMOS. Figure 11 Figure 12 As shown in FIG. 11, each switch in the first set of switches 1141 is disposed within the circuit path from the corresponding RBL in the first subarray 1101 to the adder tree 940. The switches can be implemented in any suitable manner, such as but not limited to metal-oxide-silicon field-effect transistors (MOSFETs). In a non-limiting example, the switches are NMOS.

[0095] The first set of switches 1141 and the second set of switches 1143 have only one set of switches open at a time while the other set of switches is closed. This can be achieved, as shown in FIG. 11, by controlling the switching of the second set of switches 1143 based on the address signal CIM_ADDR<2> and, by controlling the switching of the first set of switches 1141 based on the CIM_ADDR<2> inverted via the inverter, such that the two sets of switches open or close in opposite phases to each other. The two sets of switches thus act as a multiplexing scheme for selectively disconnecting the RBLs in the subarrays from the adder tree. As a result, the RBL capacitance can be reduced and the power consumption of the CIM device 1100 can be reduced. Figure 12

[0096] Figure 12 is a schematic diagram of a CIM device subarray architecture according to a ninth embodiment that has adder multiplexing in the rows and columns of the subarrays. Figure 12 ​​A CIM device 1200 is shown that is similar to the CIM device 1100 in many respects, with similar components denoted by the same reference numerals. The CIM device 1200 differs from the CIM device 1100 in that four sub-arrays 1201, 1202, 1203, and 1204 are provided (understandably, Figure 13 The four sub-arrays 1201, 1202, 1203, and 1204 in the illustrated embodiment constitute one / a single bitcell array, i.e., the bitcell array comprises the four sub-arrays 1201, 1202, 1203, and 1204). The decoder 1256 controls switching of the four sets of switches 1241, 1242, 1243, 1244 based on the address signal CIM_ADDR<3:2> to select one of the four sub-arrays and to route the output product of the RBLs in the selected sub-array to the adder tree 940. Figure 13 The architecture of the CIM device 1200 is an extension of the CIM device 1100 to provide computation with 16 filters, and can be further extended to 32 or 64 filters by further extension to include more sub-arrays.

[0097] Figure 12 is a schematic diagram of a CIM device sub-array architecture according to the tenth embodiment (which has multiple rows of sub-arrays that are adjacent to each other). Figure 13 A CIM device 1300 is shown that has four columns 1313-0, 1313-1, 1313-2, 1313-3 that are adjacent to each other along the row direction. Each sub-array can represent a bitcell array, e.g., each sub-array in each column for each input activation (such as a0, a1, … a63) is a 4x4 dimensional bitcell array (which is constituted by 4x4 bitcells) that shares a de-multiplexed RWL to provide the input activation, similar to Figure 13 the CIM device 1200 in. A plurality of sub-arrays are provided with an adder between two adjacent rows, and the adder tree 1340 is coupled to the adder by conductors that extend along the row direction. In Figure 2A the example of, four 4-bit adders 1342 are provided below the first row 1312-0 and above the second row 1312-1. The adders 1342 are adjacent to each other along the row direction, e.g., provided in a sliced or interleaved manner to represent the products of four neurons N0, N1, N2, and N3. Since the adders 1342 separate two rows of sub-arrays (in Figure 14 the example of, one sub-array corresponds Figure 14 to one bitcell array in ), no SRAM edge cells are needed between adjacent sub-arrays within the same row or between sub-arrays representing adjacent neurons. Thus, the CIM device 1300 can provide better circuit area utilization efficiency on the semiconductor substrate on which the CIM device 1300 is implemented.

[0098] Figure 14 is a high-level block diagram illustrating an illustrative computing device 1400 in which some embodiments can be practiced. In Figure 14 , the computing device 1400 can be a desktop computer, a server, or a high-performance computing system such as a machine learning accelerator. The computing device 1400 can also be a portable, handheld, or wearable electronic device. In some embodiments, the computing device 1400 can be a smartphone, a personal digital assistance (PDA), a tablet computer, a smartwatch. The computing device 1400 can be powered by a battery such as a rechargeable battery. The computing device 1400 can also be a general-purpose computer, as aspects of the present application are not limited to portable or battery-powered devices. The computing device 1400 can be a CIM device capable of performing MAC operations according to any of the embodiments disclosed herein.

[0099] The computing device 1400 includes a central processing unit (CPU) 12 having one or more processors, a graphics processing unit (GPU) 14 having one or more graphics processors, and a memory (e.g., a non-transitory computer-readable storage medium) 16, e.g., including volatile and / or non-volatile memory. The memory 16 can store one or more instructions to program the CPU 12 and / or GPU 14 to perform any of the functions described herein. The memory 16 can include one or more TCAM bitcell arrays that can perform MAC operations according to the disclosure described herein. The memory 16 can also include routing circuitry and one or more memory controllers configured to route input activation values and adder tree programming operations.

[0100] The computing device 1400 can have one or more input devices and / or output devices, e.g., as Figure 14The user input interface 18 and output interface 17 are shown. These devices can be used to present a user interface. Examples of output interfaces that can be used to provide a user interface include a printer or display screen for visual presentation output, a speaker or other sound generating device for sound output, and a vibration or mechanical motion generator for tactile presentation output. Examples of input interfaces that can be used for a user interface include keyboards and pointing devices, such as mice, touchpads, or digitizers for pen, stylus, or finger touch input. As another example, input interface 18 may include one or more microphones for capturing audio signals, one or more cameras, and a light sensor for capturing visual signals, and output interface 17 may include a display screen and / or speakers for visual presentation to present audio images or text to user 30 of computing device 1400.

[0101] like ​ As shown, computing device 1400 may include one or more network interfaces 19 to enable communication via various networks (e.g., communication network 20). Examples of such networks include local area networks (LANs) or wide area networks (WANs), such as corporate networks or the Internet. Such networks can be based on any suitable technology and can operate according to any suitable protocol, and may include wireless networks, wired networks, or fiber optic networks. Examples of network interfaces include Wi-Fi, WiMAX, 3G, 4G, 5G NR, white space, 802.11x, satellite, Bluetooth, near field communication (NFC), LTE, GSM / WCDMA / HSPA, CDMA1x / EVDO, DSRC, GPS, etc. Although not shown in the figure, computing device 1400 may also include one or more high-speed data buses (which are connected to...) ​ The components shown are interconnected) and the power subsystem (which provides power to these components).

[0102] The aspects and embodiments described above can be used alone, together, or in any combination of two or more, as this application is not limited in this respect.

[0103] Having described some aspects of at least one embodiment of the invention, it should be understood that various changes, modifications, and improvements will readily occur to those skilled in the art. Such changes, modifications, and improvements are intended to be part of this disclosure and are intended to fall within the spirit and scope of the invention. Furthermore, while advantages of the invention have been pointed out, it should be understood that not every embodiment of the technology described herein will include every described advantage.

[0104] Furthermore, the present application can be embodied as a method, of which an example has been provided. The acts performed as part of the method can be ordered in any suitable way. Accordingly, embodiments can be constructed in which acts are performed in an order different than illustrated in the illustrative embodiments.

[0105] The use of ordinal terms such as "first", "second", "third", etc. to modify a claim element does not by itself connote any priority, precedence, or order of one claim element over another, or of an execution of one method act over another, but are used merely as labels to identify elements having the same name in different claims.

[0106] The present application can be presented in a variety of formats and can be used in other specific forms without departing from the spirit or essential characteristics thereof. The embodiments described are to be considered in all respects as illustrative only and not restrictive in nature. The scope of the present application is indicated by the appended claims rather than by the foregoing description. Those skilled in the art will readily recognize a number of means of modulating and / or varying the instant application, as applied to other related applications, while still remaining within the spirit and scope of the present application.

[0107] While the present application has been described by way of example and in terms of the preferred embodiments, it is to be understood that the application is not limited to the disclosed embodiments. To the contrary, it is intended to cover various modifications and equivalent structures (and functions) as would be apparent to one skilled in the art upon reading this description. For example, the various features of the different embodiments described above can be combined in any combination. Therefore, the scope of the appended claims should be accorded the broadest interpretation so as to encompass all such modifications and equivalent structures and functions.

Claims

1. An in-memory computing device, comprising: includes a memory array and one or more adders, wherein the memory array includes a plurality of bitcell arrays, each bitcell array including: a plurality of bitcells arranged in a plurality of rows and a plurality of columns, each bitcell configured to store a weight according to a weight matrix and to multiply the weight with an input activation of an input activation vector to produce a product; a plurality of read word lines (RWLs), each RWL extending along a row direction and each RWL configured to provide an input activation of the input activation vector to bitcells of a respective row; and a plurality of read bit lines (RBLs), each RBL extending along a column direction and each RBL configured to receive a product from bitcells of a respective column; wherein the one or more adders are coupled to the plurality of RBLs of each bitcell array and configured to generate a multiply-accumulate (MAC) value based on some or all of the products received from the plurality of RBLs of each bitcell array; wherein the one or more adders include an adder tree, the memory array includes a first subarray and a second subarray, and the memory array-in- compute device further includes: a first set of switches, each switch of the first set of switches switchably coupling a corresponding RBL in the first subarray to the adder tree; and a second set of switches, each switch of the second set of switches switchably coupling a corresponding RBL in the second subarray to the adder tree; wherein the first set of switches is configured to turn on or off at an opposite phase relative to the second set of switches. 2.The all-memory computing device of claim 1, wherein, The memory array-in-compute device further includes a plurality of inverters, wherein each RBL of the plurality of RBLs is coupled to the one or more adders via a respective inverter of the plurality of inverters. 3.The all-memory computing device of claim 1, wherein, The plurality of bitcells within the memory array are 8-transistor bitcells. 4.The all-memory computing device of claim 1, wherein, The memory array is configured to receive an input activation vector and a weight matrix. 5.The all-memory computing device of claim 1, wherein, Each RWL is configured to provide an input activation of the input activation vector to bitcells of a respective row, and the MAC value is based on a vector product between the input activation vector and the weight matrix. 6.The all-memory computing device of claim 1, wherein, The memory array-in-compute device further includes: routing circuitry having a routing input configured to receive an input activation and a plurality of routing outputs coupled to some or all of the plurality of RWLs, the routing circuitry configured to route the input activation to some or all of the plurality of RWLs. 7.The all-memory computing device of claim 6, wherein, The routing circuitry includes a demultiplexer. 8.The all-memory computing device of claim 1, wherein, The memory array-in-compute device further includes: a static random access memory (SRAM) edge cell disposed alongside a row of the plurality of rows of bitcells. 9.The all-memory computing device of claim 1, wherein, The memory array includes a first subarray and a second subarray aligned along a column direction with the first subarray, and the memory array-in-compute device further includes: first routing circuitry having a routing input and a plurality of routing outputs coupled to some or all of the plurality of RWLs in the first subarray; second routing circuitry having a routing input and a plurality of routing outputs coupled to some or all of the plurality of RWLs in the second subarray. 10.The all-memory computing device of claim 9, wherein, The memory array-in-compute device further includes: one or more input multiplexers controllable to route an input activation to the routing input of the first routing circuitry or the routing input of the second routing circuitry. 11.The all-memory computing device of claim 10, wherein, The in-memory computing device further includes: an interconnect circuit that couples the plurality of RBLs in the first subarray and the second subarray to the adder tree, wherein the interconnect circuit includes one or more output multiplexers. 12.The all-memory computing device of claim 11, wherein, The one or more output multiplexers and the one or more input multiplexers are configured to receive the same address signal and address based on the address signal. 13.The all-memory computing device of claim 1, wherein, The first subarray and the second subarray are aligned along a column direction or along a row direction. 14.The all-memory computing device of claim 13, wherein, The first set of switches are switched based on an address signal, and the second set of switches are switched based on an inverted version of the address signal. 15.The all-memory computing device of claim 13, wherein, The in-memory computing device further includes a decoder that couples the first set of switches and the second set of switches, the decoder configured to receive an address signal and control the first set of switches and the second set of switches to switch based on the address signal. 16.The all-memory computing device of claim 1, wherein, The memory array includes a first subarray and a second subarray aligned along a row direction with the first subarray, and the adder tree is located between the first subarray and the second subarray and coupled to some or all RBLs of the first subarray and some or all RBLs of the second subarray. 17.The all-memory computing device of claim 1, wherein, The memory array includes a plurality of subarrays arranged into a plurality of rows and a plurality of columns, the plurality of subarrays within each row abutting each other. 18.The all-memory computing device of claim 17, wherein, The one or more adders include: an adder located between two adjacent rows of the plurality of subarrays; and an adder tree that couples the adder via conductors extending along a row direction. 19.The all-memory computing device of claim 1, wherein, The one or more adders couple the plurality of RBLs via one or more conductors extending along a row direction.

20. The storage and computing integrated device of claim 19, wherein, The one or more conductors are arranged in a first metal layer that is parallel to and offset from a second metal layer in which the plurality of RBLs are arranged.

21. A method for operating a memory array of an in-memory computing device to perform a multiply-accumulate (MAC) operation of an input activation vector with a weight matrix, wherein, The memory array includes a plurality of bitcell arrays, each bitcell array arranged into a plurality of rows and a plurality of columns of bitcells, the method comprising: storing a plurality of weights in the bitcells of the memory array according to the weight matrix; for each row of the plurality of rows, providing input activations of the input activation vector to bitcells within the row using a corresponding read word line (RWL) extending along a row direction; for each bitcell, multiplying the input activation with the weight stored therein to produce a product; for each column of the plurality of columns, receiving products from corresponding bitcells within the column using a read bit line (RBL) extending along a column direction; and producing a MAC value by summing the received products using an adder tree that couples the plurality of RBLs. wherein the memory array includes a first subarray and a second subarray, and the in-memory computing device further includes: a first set of switches, each switch in the first set of switches switchably coupling a corresponding RBL in the first subarray to the adder tree; and a second set of switches, each switch in the second set of switches switchably coupling a corresponding RBL in the second subarray to the adder tree; wherein the first set of switches is configured to turn on or off at an opposite phase relative to the second set of switches.

22. The method of claim 21, wherein, Producing the MAC value includes producing a vector product between the input activation vector and the weight matrix.

23. The method of claim 21, wherein, The method further includes: The input activation is routed to one of the plurality of RWLs using a demultiplexer.

24. The method of claim 21, wherein, The memory array includes a first subarray and a second subarray, and the method further includes: switchably coupling one of a first set of RBLs in the first subarray or a second set of RBLs in the second subarray to the adder tree based on an address signal.

25. The method of claim 24, wherein, The method further includes: selectively routing an input activation to one of the first subarray or the second subarray based on the address signal.

Images