Memory computing

Abstract

This disclosure generally relates to in-memory computing. An in-memory computing (CIM) device has a storage array having a plurality of storage cells arranged in rows and columns. The plurality of storage cells includes a first storage cell in a first row and a first column of the storage array, and a second storage cell in a first row and a second column of the storage array. The first and second storage cells are configured to store respective first and second weight signals. An input driver provides a plurality of input signals. A first logic circuit is coupled to the first storage cell to provide a first output signal based on the first weight signal and the first input signal from the input driver. A second logic circuit is coupled to the second storage cell to provide a second output signal based on a second weight signal and the second input signal from the input driver.

Description

Technical Field

[0001] This disclosure generally relates to in-memory computing. Background Technology

[0002] This disclosure generally relates to in-memory computing or computational memory (“CIM”), and further to storage arrays used in data processing, such as multiply-accumulate (“MAC”) operations. Computational memory, or in-memory computing systems, stores information in a computer’s main random access memory (RAM) and performs computations at the storage cell level, rather than moving large amounts of data between main RAM and data storage devices for each computational step. Because stored data is accessed much faster when stored in RAM, computational memory allows for real-time data analysis, enabling faster reporting and decision-making in business and machine learning applications. Efforts are underway to improve the performance of computational memory systems. Summary of the Invention

[0003] According to one embodiment of this disclosure, a memory computing (CIM) device is provided, comprising: a storage array including a plurality of storage cells arranged in rows and columns, the plurality of storage cells including a first storage cell in a first row and a first column of the storage array, and a second storage cell in the first row and a second column of the storage array, the first storage cell and the second storage cell being configured to store respective first weighting signals and second weighting signals; an input driver configured to provide a plurality of input signals; a first logic circuit coupled to the first storage cell and configured to provide a first output signal based on the first weighting signal and the first input signal from the input driver; and a second logic circuit coupled to the second storage cell and configured to provide a second output signal based on the second weighting signal and the second input signal from the input driver.

[0004] According to another embodiment of this disclosure, a computing in memory (CIM) device is provided, comprising: a storage array including a plurality of storage cells arranged in rows and columns, the plurality of storage cells being configured to store a weight signal having w bits, wherein w is a positive integer, the storage array including w columns, each of the storage cells being configured to store one bit of the weight signal; a plurality of multiplication circuits, each of the storage cells being coupled to a corresponding multiplication circuit in the multiplication circuits, and the plurality of multiplication circuits being configured to provide a product signal based on a received input signal and the weight signal stored in the corresponding storage cell; and an adder circuit configured to add the product signals and output a partial sum signal.

[0005] According to another embodiment of this disclosure, a computation in memory (CIM) method is provided, comprising: storing a plurality of weight signals in a plurality of storage cells, wherein each of the weight signals has w bits, w being a positive integer, and wherein each of the storage cells stores one bit of the w-bit weight signal; providing a plurality of logic circuits connected to corresponding storage cells in the plurality of storage cells; providing an input signal to the plurality of logic circuits to multiply the weight signals by the input signal to provide a plurality of product signals; outputting the plurality of product signals from the plurality of logic circuits to an adder tree; providing a weight sign signal configured to indicate whether the weight signals are signed; and outputting a partial sum signal based on the product signals and the weight sign signal through the adder tree. Attached Figure Description

[0006] Various aspects of this disclosure can be best understood from the following detailed description when read in conjunction with the accompanying drawings. It should be noted that, according to industry standard practice, the various features are not drawn to scale. In fact, for clarity of discussion, the dimensions of the various features may be arbitrarily enlarged or reduced. Furthermore, the drawings are illustrative of embodiments of the invention and are not intended to be limiting.

[0007] Figure 1 This is a block diagram illustrating an example of a computational memory (CIM) device according to some embodiments.

[0008] Figure 2 This illustrates, according to some embodiments, in Figure 1 A schematic diagram of an example of an SRAM memory cell used in a CIM device.

[0009] Figure 3 This illustrates, according to some embodiments, in Figure 1 A schematic diagram illustrating an example of the memory cells and NOR gates used in a CIM device.

[0010] Figure 4 This illustrates coupling to according to some embodiments. Figure 1 A schematic diagram illustrating an example of SRAM memory cells and NOR gates in a CIM device.

[0011] Figure 5 This illustrates, according to some embodiments, in Figure 1 A schematic diagram illustrating examples of memory cells and AND gates used in CIM devices.

[0012] Figure 6 This illustrates coupling to according to some embodiments Figure 1 A schematic diagram illustrating an example of SRAM memory cells and AND gates in a CIM device.

[0013] Figure 7 This is a schematic diagram illustrating an example of a CIM storage array according to some embodiments.

[0014] Figure 8 This is a block diagram illustrating an example of an adder tree according to some embodiments.

[0015] Figure 9 This is a block diagram illustrating an example of an accumulator according to some embodiments.

[0016] Figure 10 This is a timing diagram illustrating the timing for CIM output calculation according to some embodiments.

[0017] Figure 11 This is a block diagram illustrating a portion of a CIM storage array in which multiple weight columns are combined, according to some embodiments.

[0018] Figure 12 This is a block diagram illustrating an adder circuit combining a signed / unsigned weighting format according to some embodiments.

[0019] Figure 13 This is a block diagram illustrating a binary adder circuit according to some embodiments.

[0020] Figure 14 This is a block diagram illustrating a binary two's complement adder circuit according to some embodiments.

[0021] Figure 15 This is a block diagram illustrating an adder circuit configuration according to some embodiments.

[0022] Figure 16 It is a truth table of a full adder circuit according to some embodiments.

[0023] Figure 17 This is a schematic diagram illustrating an example of a simplified adder circuit according to the disclosed embodiment.

[0024] Figure 18 This is a schematic diagram illustrating an example of a full adder according to the disclosed embodiments.

[0025] Figure 19 This is a schematic diagram illustrating an example of a modified full adder according to the disclosed embodiments.

[0026] Figure 20 This is a flowchart illustrating a CIM method according to a disclosed embodiment. Detailed Implementation

[0027] The following disclosure provides numerous different embodiments or examples for implementing various features of the provided subject matter. Specific examples of components and arrangements are described below to simplify this disclosure. Of course, these are merely examples and not intended to be limiting. For example, in the following description, forming a first feature above or on a second feature may include embodiments in which the first and second features are formed in direct contact, and may also include embodiments in which an additional feature may be formed between the first and second features such that the first and second features do not need to be in direct contact. Furthermore, reference numerals and / or letters may be repeated in various examples throughout this disclosure. Such repetition is for simplicity and clarity and does not, in itself, indicate a relationship between the various embodiments and / or configurations discussed.

[0028] In addition, spatially related terms (e.g., "below," "below," "lower than," "above," "upper") may be used in this document to facilitate the description of the relationship of one element or feature shown in the figure relative to another element(s) or feature(s). These spatially related terms are intended to cover different orientations of the device in use or operation other than those shown in the figure. The device may be oriented in other ways (rotated 90 degrees or in other orientations), and the spatially related descriptors used herein shall be interpreted accordingly.

[0029] This disclosure generally relates to computational memory (“CIM”). One example of a CIM application is the multiplication-accumulation (“MAC”) operation. Computer artificial intelligence (“AI”) uses deep learning techniques, where computational systems can be organized as neural networks. For example, a neural network refers to multiple interconnected processing nodes capable of analyzing data. Neural networks compute “weights” to perform computations on new input data. Neural networks use multiple layers of computational nodes, where deeper layers perform computations based on the results of computations performed by higher layers.

[0030] CIM circuits perform operations locally within memory, eliminating the need to send data to the host processor. This reduces the amount of data transferred between memory and the host processor, resulting in higher throughput and performance. Reduced data movement also lowers the overall energy consumption of data movement within the computing device.

[0031] According to some disclosed embodiments, the CIM device includes a memory array having memory cells arranged in rows and columns. The memory cells are configured to store weight signals, and an input driver provides input signals. Each memory cell is coupled to a corresponding first logic circuit, such as a multiplication circuit, which provides an output signal based on the input signal from the input driver and the weight signals stored in the corresponding memory cell. The outputs of the logic cells are accumulated or summed to obtain a system output value.

[0032] An example of the mathematical description of MAC operations is shown below.

[0033] O J (1)

[0034] In equation (1) above, A I is the i-th input, and WIJ is the weight corresponding to the i-th input and the j-th weight column. O J is the MAC output of the Jth weighted column, and h is the cumulative number.

[0035] Figure 1 This is a block diagram illustrating an example CIM device 100 according to the present disclosure. The CIM memory array 110 includes a plurality of memory cells 112 configured to store weight signals W. CIM can be implemented using various memory devices, including static random access memory (“SRAM”). In a typical SRAM device, data is written to and read from the SRAM cell via one or more bit lines (“BL”) when one or more access transistors in the SRAM cell are activated by an enable signal from one or more word lines (“WL”).

[0036] Figure 2 This is a circuit diagram illustrating an example memory cell 112 according to some embodiments. Memory cell 112 includes, but is not limited to, a six-transistor (6T) SRAM cell 212. In some embodiments, the SRAM cell 212 may be implemented using more or fewer six transistors. For example, in some embodiments, the SRAM cell 212 may use a 4T, 8T, or 10T SRAM structure, and in other embodiments, it may include memory-type bit cells or building blocks. The SRAM cell 212 includes: a first inverter formed by NMOS / PMOS transistor pairs M1 and M2, a second inverter formed by NMOS / PMOS transistor pairs M3 and M4, and access transistors / transmission gates M5 and M6.

[0037] Each inverter is powered; for example, the first terminal of each of transistors M2 and M4 is coupled to the power supply VDD, while the first terminal of each of transistors M1 and M3 is coupled to a reference voltage VSS, for example, ground. One bit of data is stored in SRAM cell 212 as the voltage level at node Q and can be read by the circuit via bit line BL. Access to node Q is controlled by transmission gate transistor M5. Node Qbar (QB) stores the two's complement of the value at Q; for example, if Q is "high", then QB will be "low", and can be read by the circuit via bit line BLbar (BLB). Access to QB is controlled by transmission gate transistor M6.

[0038] The gate of transmission gate transistor M5 is coupled to word line WL. The first source / drain (S / D) terminal of transmission gate transistor M5 is coupled to bit line BL, and the second S / D terminal of transmission gate transistor M5 is coupled to the second terminals of transistors M1 and M2 at node Q. Similarly, the gate of transmission gate transistor M6 is coupled to word line WL. The first S / D terminal of transmission gate transistor M6 is coupled to two's complement bit line BLB, and the second S / D terminal of transmission gate transistor M6 is coupled to the second terminals of transistors M3 and M4 at node QB.

[0039] return Figure 1 The CIM device 100 also includes an input activation driver 102 and a WL driver 104. The input activation driver 102 drives the input signal A to the logic circuit 114. The WL driver outputs a WL signal to activate the desired row i of the CIM memory array 110, thereby providing the stored weight signal W to the logic circuit 114. The memory controller 120 receives a control input and provides a control signal to the SRAM read / write circuit 122 connected to the bit lines BL, BLB of the CIM memory array 110 to select the appropriate bit lines BL, BLB (i.e., columns) corresponding to the stored weight signal W. In the example shown, a 4-bit weight signal W[0:3] is used, so four columns of the CIM memory array 110 and the logic circuit are used to store the individual weight values. Therefore, a weight value with w bits uses the w columns of the CIM memory array 110 and the corresponding logic circuit. The output signal from the logic circuit 114 is provided to an adder tree 116, which adds the outputs of the individual logic circuits 114. The accumulator circuit is coupled to adder tree 116 and configured to provide MAC output O J .

[0040] As described above, in some embodiments, logic circuit 114 is a multiplier circuit configured to calculate A in equation 1 above. I × WIJ section. Figure 3 An example is shown where logic circuit 114 is a NOR gate 214 that receives a weight signal W from its corresponding memory cell 112 and an input signal A in the form of an inverted selection signal SELB, to output the product P of the weight signal W and the selection signal SELB. Figure 4 Other aspects of the disclosed embodiments are shown, wherein the storage unit is as follows: Figure 2 The 6T SRAM cell 212 shown and discussed above, and the logic circuit 114 include a dual-input NOR gate 214. One input of the NOR gate 214 is coupled to node QB of the SRAM cell 212 to receive the inverted weight signal, while the other input of the NOR gate 214 receives the SELB signal.

[0041] Figure 5Another example is shown, in which logic circuit 114 is an AND gate 215 that receives a weight signal W and an input signal A in the form of a selection signal SEL from its corresponding memory cell 112, and outputs the product P of the weight signal W and the selection signal SEL. Figure 6 Other aspects of the disclosed embodiments are shown, wherein the storage unit is as follows: Figure 2 The 6T SRAM cell 212 shown and discussed above, and the logic circuit 114 include a two-input AND gate 215. One input of the AND gate 215 is coupled to node Q of the SRAM cell 212 to receive a weight signal, while the other input of the AND gate 215 receives the SEL signal.

[0042] Figure 7 Various aspects of an example CIM memory array 110 are shown. Memory cell 112 may include the SRAM cell 212 discussed above or other suitable memory circuitry. Logic circuitry 114 is implemented using NOR gate 214, but other logic circuitry may be used for multiplier circuitry, such as AND gate 215. Figure 7 In this example, each storage cell 112 stores one bit of weight signal W. Using 4-bit input A and 4-bit weight signal W, the four columns j of storage cell 112 and the associated NOR gate 214 (i.e., multiplier circuitry) are used to store 4 bits of weight in each input row i. In other words, the weight signal is w-bit weight, where w = 4. The illustrated portion of the CIM storage array 110 therefore includes a "multiplication" section 130, representing a weight column J of 4-bit weight values, where the weight signal W stored in storage cell 112 is multiplied by the received input A using NOR gate 214 (or other suitable logic circuitry). In other words, the weight column J has w columns j of storage cell 112 and corresponding NOR gates 214.

[0043] Note that in this disclosure, the columns of storage cell 112 and the associated logic circuit 114 are designated with lowercase "j". As described above, in the example shown with 4-bit weighted signals W[0:3], the "weighted column" (i.e., multiplication portion 130) of the four columns j with storage cells is designated with uppercase letter J. The "accumulation" portion 132 receives the product from the multiplication portion 130 to shift and add the received product, as will be discussed further below. Therefore, Figure 7 Implement the MAC operation as shown below:

[0044] O (2)

[0045] Where A IHere, is the i-th input, WI is the weight corresponding to the i-th input in the column shown, O is the output of adder tree 116, and h is the accumulated number. In the example shown, multi-cycle "bitwise" multiplication is used, where each bit of the 4-bit weight signal W[0:3] is multiplied sequentially by each bit of the input signal A to provide a partial sum.

[0046] In the example shown, the product output of the NOR gate 214 from the adjacent row i is coupled to the adder tree 116 via rows 218 and 219. More specifically, the product output of the NOR gate 214 in the first row i-1 is received by row 218 and passed to the adder tree 116, while the product output of the NOR gate 214 in the adjacent row i-2 is passed to the adder tree 116 in row 219.

[0047] Figure 8 Aspects of an example of adder tree 220 of adder tree 116 are shown. Adder tree 220 includes eight adder tree branches, including a "sum 2" branch 222 that adds the product outputs of two adjacent rows of CIM storage array 110. As described above, the product value includes the weight signal W[3:0] and the input multiplier provided by input activation driver 102. Figure 8 In this process, the input values ​​are labeled A0….A255 (i.e., h-1). The 5-bit adders 224 of the summation branch 222 each add the two product outputs of two adjacent rows of the NOR gate (i.e., multiplier circuit) 214. Therefore, the product of the first input A0 and the weight signal W[3:0] is added to the product of the second input A1 and the corresponding weight signal W[3:0] by the highest 5-bit adders 224 in the summation branch 222 of the adder tree 116.

[0048] The adder tree also includes a "sum 4" branch 226 with a 6-bit adder 228, a "sum 8" branch 230 with a 7-bit adder 232, a "sum 16" branch 234 with an 8-bit adder 236, a "sum 32" branch 238 with a 9-bit adder 240, a "sum 64" branch 242 with a 10-bit adder 244, a "sum 128" branch 246 with an 11-bit adder 248, and a "sum 256" branch 250 with a 12-bit adder circuit 252.

[0049] Each 6-bit adder 228 in summation branch 4 226 receives the output of two adjacent 5-bit adders 224 in summation branch 222; each 7-bit adder 232 in summation branch 8 230 receives the output of two adjacent 6-bit adders 228 in summation branch 4 226; each 8-bit adder 236 in summation branch 16 234 receives the output of two adjacent 7-bit adders 232 in summation branch 8 230; and each 9-bit adder 240 in summation branch 32 238 receives the output of summation branch 234 in summation branch 16 234. The outputs of two adjacent 8-bit adders 236 of summation 64, each 10-bit adder 244 of summation 64 branch 242 receives the outputs of two adjacent 9-bit adders 240 of summation 32 branch 238, each 11-bit adder 248 of summation 128 branch 246 receives the outputs of two adjacent 10-bit adders 244 of summation 64 branch 242, and the 12-bit adder circuit 252 of summation 256 branch 250 receives the outputs of adjacent 11-bit adders 248 of summation 128 branch 246.

[0050] The 12-bit partial sum PSUM[11:0] output by the 12-bit adder circuit 252 is provided to the accumulator 124, which in the example shown includes a partial sum adder and a shifter. As will be discussed further below, in addition to the product output, each adder circuit also receives a weight sign signal WS, which indicates whether the weights are signed (i.e., negative) or unsigned. Therefore, for example, a 4-bit weight signal W[3:0] is multiplied by the input A. I The corresponding weight symbol signal WS is received by the 5-bit adder 224. The weight symbol signal WS can be stored, for example, in a register associated with the CIM memory array 110.

[0051] In some publicly available examples, multi-cycle "bit-by-bit" multiplication is used for configurable n-bit signed / unsigned inputs. As will be discussed below, computation is achieved by shifting the input bit-by-bit. An example with a 4-bit input A and a 4-bit weighted signal W is shown below.

[0052] O J (3)

[0053] Equation 3 above can be restated as follows.

[0054]

[0055] For signed inputs, a two's complement adder configuration can be used. Therefore, if the weighted input is signed (WS=1), the most significant bit (MSB) is calculated to be negative (i.e., the result is inverted and incremented by 1).

[0056] Figure 9An example of a CIM device 100 is shown, including other aspects of the accumulator 124. As previously described, the CIM storage array 110 includes an array of storage cells 112 configured to store a weight signal W. Input signal A i The input register 312 is multiplied by the weight signal W[0:3] stored in the CIM storage array 110, and the adder tree 116 provides, as Figure 7 The 12-bit portion and PSUM[11:0] are shown. The WL driver 104 outputs a WL signal to activate the CIM memory array 110, and the memory controller 120 receives the control input and provides the control signal to the SRAM read / write circuitry 122 connected to the bit lines of the CIM memory array 110. The output signal from the CIM memory array 110 is provided to the adder tree 116, and the accumulator 124 is coupled to the adder tree 116 and configured to provide a MAC output O. J .

[0057] The partial sum from adder tree 116 is output to the first register 314 of the partial sum circuit. The first MUX 316 receives the partial sum output PSUM[11:0] and its inverse, and outputs the selected input based on the weight sign signal WS and the product SIGNED*MSB of the most significant weight bit. Therefore, if the MSB of the PSUM input is signed (i.e., WS=1), the MSB result is made negative by inverting the result and adding 1. This partial sum 318 is provided as one input to the 20-bit adder 310.

[0058] The output of the 20-bit adder 310 is received by the second part and the second register 320. The output of the second part and the second register 320 is received by the shifter 322 and shifted left by 1 bit (i.e., multiplied by 2). 1 The output register 330 receives the output of the 20-bit adder 310 at its D input and adds it to the next part 318. The output register 330 receives the output of the 20-bit adder 310 at its D input and provides the output Q[19:0] of the accumulator at its Q output.

[0059] Figure 10 This is a timing diagram illustrating the operations of adder tree 116 and accumulator 124. In the example discussed above, a 4-bit weight signal W[0:3] is shown, but other weight configurations are within the scope of this disclosure. Figure 10A 5-cycle operation is illustrated. The inputs include a clock signal CLK, an IMCEN signal marking the start and end of the 5-cycle operation, and input signals A[1023:0]. Each bit of the 4-bit input signal A[3:0] is multiplied by a weight signal W[3:0]. During the first operation cycle 350, the logic circuit 114 of the CIM memory array 110 multiplies the MSB of input A by each bit of the weight signal W[3:0] to produce partial products, which are then added by the adder tree 116 to obtain a 12-bit partial sum S3.

[0060] During the second operation cycle 352, the logic circuit 114 of the CIM storage array 110 multiplies the second MSB of input A by each bit of the weight signal W[3:0] to produce a partial product. These partial products are added by the adder tree 116 to obtain a 12-bit partial sum S2. During the third operation cycle 354, the logic circuit 114 of the CIM storage array 110 multiplies the third MSB of input A by each bit of the weight signal W[3:0] to produce a partial product. These partial products are added by the adder tree 116 to obtain a 12-bit partial sum S1. And during the fourth operation cycle 356, the logic circuit 114 of the CIM storage array 110 multiplies the LSB of input A by each bit of the weight signal W[3:0] to produce a partial product. These partial products are added by the adder tree 116 to obtain a 12-bit partial sum S0.

[0061] The partial outputs S3, S2, S1, and S0 of PSUM[11:0] are delayed by one cycle, and are therefore output to the first part and the first register 314 during the second operation cycle 352, the third operation cycle 354, the fourth operation cycle 356, and the fifth operation cycle 358, respectively. Therefore, in the first operation cycle 350, there is no partial input from the first part and the first register 314 to the 20-bit adder 310. There is also no output from the adder 310 to the second register 320 and the shifter 322, and therefore, in the second operation cycle 352, SUM[15:0] = 0.

[0062] As described above, in the second operation cycle 352, the S3 partial sum is provided to the first register 314 and input to the first input of adder 310. The previous output of adder 310 (i.e., sum = 0) is received by the second register 320, shifted left by shifter 322 by 1 bit, and input to the second input of adder 310. The SUM[15:0] output by adder 310 during the third operation cycle 354 is therefore the S3 partial sum. In the third operation cycle 354, the second part of the second MSB, S2, is output by adder tree 116 and received by the first register 314 and input to adder 310. The sum output by adder 310 during the third operation cycle 354 (i.e., the S3 partial sum) is output by the second register 320 and shifted left by shifter 322. During the fourth operation cycle 356, the shifted sum is therefore added to the S2 partial sum.

[0063] In the fourth operation cycle 356, the next part of the third MSB and S1 are output by adder tree 116 and received by first register 314 and input to adder 310. The sum output by adder 310 during the fourth operation cycle 356 is output by second register 320 and left-shifted by shifter 322. During the fifth operation cycle 358, the shifted sum is thus added to the S1 part sum.

[0064] In the fifth operation cycle 358, the next part of LSB and S0 are output by adder tree 116 and received by first register 314 and input to adder 310. The sum output by adder 310 during the fifth operation cycle 358 is output by second register 320 and shifted left by shifter 322. During the first operation cycle 360 ​​of the next operation, the shifted sum is thus added to the S0 part sum. The SRDY signal is asserted during this cycle, and in response, the final sum Q[19:0] is output by output register 330.

[0065] Configurable n-bit weights can be supported using the programmable signed / unsigned weights disclosed above. In other words, multiple CIMs or weight columns J can be combined to store the desired number of n weight bits. Return to Reference Figure 7 The CIM storage array 110 shown includes a 4-bit weight column J. Therefore, the weight column J has four columns of storage cells 112 and an associated NOR gate 214 to store each bit of the 4-bit weight value. If a larger weight value is required, multiple weight columns J can be combined. For example, two 4-bit weight columns J can be combined to provide an 8-bit weight value. An example of an 8-bit weight with a signed MSB is shown below.

[0066] W7 W6 W5 W4 W3 W2 W1 W0

[0067] -128 64 32 16 8 4 2 1

[0068] Such an 8-bit signed weight can be divided into 4-bit signed weights plus 4-bit unsigned weights, as shown below (factor out 16 from the first 4 weights).

[0069] 16*(W7 W6 W5 W4) + (W3 W2 W1 W0)

[0070] -8 4 2 1 8 4 2 1

[0071] The MAC operation can therefore be represented as follows.

[0072]

[0073] Figure 11 An example CIM storage array 111 is shown, in which four 4-bit weight columns J-1, J-2, J-3, J-4 are combined to provide a sixteen-bit signed weight value. Figure 11 Each of the weight columns J-1, J-2, J-3, and J-4 shown has four storage array columns j for storing... Figure 7 The example shown uses 4-bit weights. The first weight column J-1 is signed, while the remaining weight columns J-2, J-3, and J-4 are unsigned because only the MSB of the weight values ​​is signed. The input activation driver 102 controls the input signal A multiplied by the weight values. I The activation of the weight columns J-1, J-2, J-3, J-4 is achieved by summing the tree 116 parts associated with each of the weight columns J, and these partial sums are provided to the accumulator 124 for shift / add operations to calculate the final CIM output.

[0074] Figure 12 An example of a 5-bit adder 224 in the summation branch 222 of adder tree 220 is shown. As described above, in the example shown, the 4-bit product output from adjacent rows of the array is output to the 5-bit adder 224 in adder tree 116. Figure 8 In the example, the product output of adjacent rows 218 and 219 is received by a 5-bit adder 224. Figure 12 In the diagram, the inputs in adjacent rows 218 and 219 are labeled A and B, respectively. For example, the 4-bit output of the NOR gate 214 in row i-1 is... Figure 12 The 4-bit output of the NOR gate 214 in row i-2 is labeled A[3:0], while the 4-bit output of the NOR gate 214 in row i-2 is labeled B[3:0].

[0075] The 5-bit adder 224 includes a half adder HA0, which receives first output bits A[0] and B[0] and provides bit 0 and output S[0] and first carry output C[1]. The 5-bit adder 224 also includes four full adders FA1, FA2, FA3 and FA4. The first full adder FA1 receives the outputs A[1] and B[1] of the bit-1 NOR gate and the first carry output C[1] from the half adder HA0. The first full adder FA1 adds the inputs A[1] and B[1] and provides bit-1 and output S[1] and second carry output C[2]. The second full adder FA2 receives the bit-2 output of the bit-2 NOR gate and the second carry output C[2] from the first full adder FA1. The second full adder FA2 adds the inputs A[2] and B[2] and provides bit-2 and output S[2] and third carry output C[3]. The third full adder FA3 receives the outputs A[3] and B[3] of the bit-3 NOR gate 214, as well as the third carry output C[3] from the second full adder FA2. The third full adder FA3 adds the bit-3 inputs A[3] and B[3] and provides the bit-3 sum output S[3] and the fourth carry output C[4].

[0076] The fourth full adder FA4 receives the outputs of two AND gates 270 and 272, as well as the fourth carry output C[4] from the third full adder FA3. AND gate 270 receives the A[3] signal and the weight symbol WSA of input A at its input, while AND gate 272 receives the B[3] signal and the weight symbol WSB of input B at its input. By providing AND gates 270 and 272 configured to receive the weight symbol signal WS, adder 224 becomes “switchable” for signed and unsigned weight formats, as will be discussed further below.

[0077] More specifically, if the weights are unsigned, the 4-bit weight values ​​can be added using a 4-bit binary adder 224a, such as... Figure 13 As shown. The binary adder 224a includes a half adder HA0, which receives first output bits A[0] and B[0], and provides bit 0 and output S[0] and first carry output C[1]. However, the binary adder 224a uses only three full adders FA1, FA2 and FA3. The three full adders FA1, FA2, FA3 and the half adder HA0 are capable of adding the unsigned 4-bit input A[3:0] and the unsigned 4-bit input B[3:0].

[0078] For signed weighted inputs, a two's complement adder 224b is used to add the negative values ​​indicated by the weight sign signal WS, such as... Figure 14As shown. The binary two's complement adder 224b includes a half adder HA0, which receives the first output bits A[0] and B[0] and provides bit 0 and output S[0] and first carry output C[1]. The 5-bit adder 224 also includes four full adders FA1, FA2, FA3 and FA4. The first full adder FA1 receives the outputs A[1] and B[1] of the bit 1 NOR gate and the first carry output C[1] from the half adder HA0. The first full adder FA1 adds the inputs A[1] and B[1] and provides bit 1 and output S[1] and second carry output C[2]. The second full adder FA2 receives the bit 2 output of the bit 2 NOR gate and the second carry output C[2] from the first full adder FA1. The second full adder FA2 adds the inputs A[2] and B[2] and provides bit 2 and output S[2] and third carry output C[3]. The third full adder FA3 receives the outputs A[3] and B[3] of the bit-3 NOR gate 214, as well as the third carry output C[3] from the second full adder FA2. The third full adder FA3 adds the bit-3 inputs A[3] and B[3] and provides the bit-3 sum output S[3] and the fourth carry output C[4].

[0079] The fourth full adder FA4 receives the outputs A[3] and B[3] of the bit-3 NOR gate 214, and the fourth carry output C[4] from the third full adder FA3. The third full adder FA3 adds the bit-3 inputs A[3] and B[3] and provides the bit-3 sum output S[3] and the fourth carry output C[4].

[0080] return Figure 12 The addition of AND gates 270 and 272 makes adder 224 "switchable" so that if neither of the MSB weighted inputs A[3] nor B[3] is signed, both AND gates 270 and 272 output 0, and the fourth full adder FA4 is effectively removed from the circuit, with the fourth carry output C[4] providing the fifth sum bit S[4]. Alternatively, if either of the weighted inputs A[3] or B[3] is signed, AND gates 270 and 272 provide outputs based on the MSB weighted inputs A[3] and B[3]. The S[4] output of the fourth full adder FA4 provides a sign extension of the output S[3].

[0081] Figure 12 The 5-bit adder 224 shown is used for Figure 8 The summation branch 222 of the adder tree 220 shown. The 5-bit output S[4:0] of the 5-bit adder 224 is received by the 6-bit adder 228 of the summation branch 226 of the adder tree 220. Figure 15A general form 225 of the adders used in adder tree 220 is shown. Each adder receives its own product bits A[0:n] and B[0:n], and the MSB adder n also receives a weight sign signal WS. For example, 5-bit adder 224 receives 4-bit product signals A[0:3] and B[0:3], and a weight sign signal WS. Therefore, 5-bit adder 224 employs one half adder HA and four full adders FA1-FA4.

[0082] therefore, Figure 15 The general-purpose adder circuit 225 shown has n adders that provide n sum outputs. These n adders include a half-adder adder 0, which receives the product inputs A[0] and B[0] and outputs a sum output S[0]. The n adders also include n-1 full adders. The first full adder adder 1 receives inputs A1 and B1, and the carry output Co from the half-adder adder 0. Each full adder receives the appropriate A and B inputs, as well as the carry input bit Ci from the previous adder, and outputs the corresponding sum output and carry output bit Co.

[0083] The nth adder (full adder) receives the weight sign signal WS and the carry input bit from the n-1 adder, as well as the An and Bn inputs, and the adder n outputs the carry output bit Co and the S[n] bit, i.e., the WS bit. In the example shown, the adder n receives the inverse of the carry output bit of the adder n-1 at its carry input (i.e., the carry in bar CiB).

[0084] Figure 16 It shows Figure 15 The truth table of the nth adder in the adder circuit 225 shown. (See attached...) Figure 12 The nth adder (FA4) under discussion receives the C[4] carry input bit Ci and the outputs of AND gates 270 and 272. AND gates 270 and 272 receive the MSB product bits A[3] and B[3], respectively, and the corresponding weight sign extension bits WSA and WSB. Figure 16 The sum of the unsigned weights (WS=0) and signed weights (WS=1) corresponding to various input combinations of A, B, and Ci is shown.

[0085] If the WS bit is 1—that is, the weights are signed—then for the nth adder ( Figure 12 In FA4, the cases A=B=0, Ci=1 and A=B=1, Ci=0 will not occur because the outputs of AND gates 270 and 272 will follow the inputs A and B. For example, refer to Figure 12When WSA and WSB are extended to 1 (signed weight), both FA3 and FA4 full adders (i.e., the (n-1)th and nth adders) receive inputs A[3] and B[3]. If A[3] = B[3] = 0, the carry output bit Co of the FA3 adder will be 0. Therefore, for the FA4 adder, there will never be a case where A = B = 0 and Ci = 1. Similarly, if A[3] = B[3] = 1, the carry output bit Co of the FA3 adder will be 1, and for the FA4 adder, there will never be a case where A = B = 1 and Ci = 0. Furthermore, the overflow (i.e., carry) of the nth adder is discarded. Therefore, the circuitry of the nth adder can be simplified, thereby reducing the number of transistors used to implement the full adder.

[0086] Figure 17 An example of an adder circuit 280 for the nth adder is shown, for example... Figure 12 The 5-bit adder 224 shown is a full adder FA4. Compared to combining two AND gates 270, 272 with a conventional full adder circuit, in some examples... Figure 17 The circuit shown reduces the number of transistors by 22. Figure 17 The adder circuit 280 uses 10 transistors to implement the input circuit 282, which is configured to receive the A and B input bits and the CiB signal (i.e., the inverted Ci signal received from the n-1 adder). Based on these inputs, an intermediate signal is provided at node 284. Node 284 is the input to the output circuit 286, which also receives the CiB signal and the SIGN extended signal and its inverse SIGNB to provide Sn and the output.

[0087] As described above, the simplified adder circuit 280 receives the inverse of the carry output signal Co from the previous full adder (n-1 adder). Therefore, the n-1 adder is modified to output the inverse of the Co signal (i.e., the CiB signal). Figure 18 This is a schematic diagram illustrating an example of a full adder circuit, for example... Figure 15 The adder 1 shown is a full adder. The circuit of adder 1 shown includes an inverter 290, which provides a carry output signal Co at the carry output terminal 292. The carry output bit Co is output to the carry input terminal Ci of the subsequent adder.

[0088] However, the last full adder (adder n) receives the inverse of the carry output bit Co from the previous adder (adder n-1). Figure 19 An example of adder n-1 is shown, where inverter 290 is canceled, resulting in the output of the inverse of the carry output signal at carry output 292, which provides the inverted carry input CiB to the final full adder (adder n). As previously stated, Figure 17An example of the final adder (adder n) is shown in the simplified adder circuit 280.

[0089] Figure 20 This is a flowchart illustrating an example of a CIM method according to a disclosed embodiment. Further reference... Figure 7 and Figure 8 Method 400 includes operation 410, storing multiple weight signals W[3:0] in multiple memory cells 112. Each memory cell 112 stores one bit of the w-bit weight signal. In operation 412, multiple logic circuits 114 are provided, which are connected to corresponding memory cells in the multiple memory cells. In operation 414, input signals A are provided to the multiple logic circuits 114. I In operation 416, logic circuit 114 is configured to multiply the weighted signal W[3:0] by the input signal A. I Multiple product signals are provided. In operation 418, the multiple product signals are output from multiple logic circuits 114 to adder tree 220. In operation 420, a weight sign signal WS is provided. The weight sign signal WS is configured to indicate whether the weight signals are signed or unsigned. In operation 422, adder tree 220 outputs a partial sum signal based on the product signals and the weight sign signal.

[0090] Therefore, this disclosure provides a CIM system including storage cells for storing CIM weights and multiplication circuitry coupled to each storage cell. The disclosed system is configured to use multi-cycle inputs to multiply an input signal with column-based stored weights. The input signal and / or weight signal can be signed or unsigned, and the product output of the multiplication circuitry is accumulated by an adder tree and an accumulator.

[0091] The disclosed embodiments include a CIM device having a memory array having a plurality of memory cells arranged in rows and columns. The plurality of memory cells includes a first memory cell in a first row and a first column of the memory array, and a second memory cell in a first row and a second column of the memory array. The first and second memory cells are configured to store respective first weighting signals and second weighting signals. An input driver is configured to provide a plurality of input signals, and first logic circuitry is coupled to the first memory cell to provide a first output signal based on the first weighting signal and the first input signal from the input driver. Second logic circuitry is coupled to the second memory cell and provides a second output signal based on the second weighting signal and the second input signal from the input driver.

[0092] According to a further embodiment, a CIM device includes a memory array having a plurality of memory cells arranged in rows and columns. The plurality of memory cells store a weight signal having w bits, where w is a positive integer. The memory array has w columns, and each memory cell is configured to store one bit of the weight signal. Each memory cell is coupled to a corresponding multiplication circuit configured to provide a product signal based on a received input signal and the weight signal stored in the corresponding memory cell. An adder circuit is configured to add the product signals and output a partial sum signal.

[0093] According to a further embodiment, a CIM method includes storing multiple weight signals in multiple memory cells, wherein each weight signal has w bits (w is a positive integer). Each memory cell stores one bit of the w-bit weight signal. Multiple logic circuits are connected to corresponding memory cells in the multiple memory cells. Input signals are provided to the multiple logic circuits to multiply the weight signals by the input signals to provide multiple product signals. The multiple product signals are output from the multiple logic circuits to an adder tree. A weight sign signal indicates whether the weight signals are signed. The adder tree outputs a partial sum signal based on the product signals and the weight sign signal.

[0094] This disclosure outlines various embodiments to enable those skilled in the art to better understand various aspects of this disclosure. Those skilled in the art will understand that they can readily use this disclosure as a basis for designing or modifying other processes and structures to achieve the same purposes and / or the same advantages of the embodiments described herein. Those skilled in the art should also recognize that such equivalent constructions do not depart from the spirit and scope of this disclosure, and that various changes, substitutions, and alterations can be made herein without departing from the spirit and scope of this disclosure.

[0095] Example 1 is a memory computing (CIM) device comprising: a storage array including a plurality of storage cells arranged in rows and columns, the plurality of storage cells including a first storage cell in a first row and a first column of the storage array, and a second storage cell in the first row and a second column of the storage array, the first storage cell and the second storage cell being configured to store respective first weighting signals and second weighting signals; an input driver configured to provide a plurality of input signals; a first logic circuit coupled to the first storage cell and configured to provide a first output signal based on the first weighting signal and the first input signal from the input driver; and a second logic circuit coupled to the second storage cell and configured to provide a second output signal based on the second weighting signal and the second input signal from the input driver.

[0096] Example 2 is the device described in Example 1, wherein the first logic circuit and the second logic circuit each include a multiplication circuit.

[0097] Example 3 is the device described in Example 2, wherein the multiplication circuit includes a NOR gate.

[0098] Example 4 is the device described in Example 2, wherein the multiplication circuit includes an AND gate.

[0099] Example 5 is the device described in Example 1, wherein the first memory cell and the second memory cell each include an SRAM cell.

[0100] Example 6 is the device described in Example 1, wherein at least one of the first weight signal and the second weight signal is a signed weight.

[0101] Example 7 is the device described in Example 1, further comprising: a third memory cell in the second row and the first column of the memory array, and a fourth memory cell in the second row and the second column of the memory array, the third memory cell and the fourth memory cell being configured to store a respective third weight signal and a fourth weight signal; a third logic circuit coupled to the third memory cell and configured to provide a third output signal based on the third weight signal and a third input signal from the input driver; and a fourth logic circuit coupled to the fourth memory cell and configured to provide a fourth output signal based on the fourth weight signal and a fourth input signal from the input driver.

[0102] Example 8 is the device described in Example 7, further comprising: an adder circuit configured to add the first output signal, the second output signal, the third output signal, and the fourth output signal.

[0103] Example 9 is the device described in Example 8, wherein the adder circuit is configured to add weight sign signals associated with the first weight signal, the second weight signal, the third weight signal, and the fourth weight signal.

[0104] Example 10 is a computing in memory (CIM) device comprising: a storage array including a plurality of storage cells arranged in rows and columns, the plurality of storage cells being configured to store a weight signal having w bits, where w is a positive integer, the storage array including w columns, each of the storage cells being configured to store one bit of the weight signal; a plurality of multiplication circuits, each of the storage cells being coupled to a corresponding multiplication circuit in the multiplication circuits, and the plurality of multiplication circuits being configured to provide a product signal based on a received input signal and the weight signal stored in the corresponding storage cell; and an adder circuit configured to add the product signals and output a partial sum signal.

[0105] Example 11 is the device described in Example 10, wherein the storage cell storing the weight signal of the w bits is in one row of the storage array.

[0106] Example 12 is the device described in Example 10, wherein the storage cell is configured to store a plurality of the weight signals, and wherein the storage cell storing the w bits of the plurality of weight signals and the corresponding multiplication circuit are in the respective plurality of rows of the storage array.

[0107] Example 13 is the device described in Example 12, wherein each of the adder circuits is configured to receive the product signal from two rows of the memory array.

[0108] Example 14 is the device described in Example 13, wherein each of the adder circuits is configured to receive a weight symbol corresponding to the weight signal.

[0109] Example 15 is the device described in Example 13, wherein each of the adder circuits includes a half-adder circuit and w full-adder circuits.

[0110] Example 16 is the device described in Example 10, further comprising: an accumulation circuit configured to shift and add the portions and signals to generate a CIM output.

[0111] Example 17 is a computation in memory (CIM) method comprising: storing a plurality of weight signals in a plurality of storage cells, wherein each of the weight signals has w bits, w being a positive integer, and wherein each of the storage cells stores one bit of the w-bit weight signal; providing a plurality of logic circuits connected to corresponding storage cells in the plurality of storage cells; providing input signals to the plurality of logic circuits to multiply the weight signals by the input signals to provide a plurality of product signals; outputting the plurality of product signals from the plurality of logic circuits to an adder tree; providing a weight sign signal configured to indicate whether the weight signals are signed; and outputting a partial sum signal based on the product signals and the weight sign signal via the adder tree.

[0112] Example 18 is the method of Example 17, further comprising: shifting and adding the portion and signal by an accumulator to generate a CIM output.

[0113] Example 19 is the method of Example 17, wherein the storage cells are arranged in a matrix of rows and columns, wherein the adder tree includes a plurality of adder circuits, and wherein outputting the plurality of product signals from the plurality of logic circuits includes: outputting the product signals from adjacent rows to each of the adder circuits.

[0114] Example 20 is the method described in Example 17, further comprising: combining multiple w-bit weighted signals.

Claims

1. A computing in memory (CIM) device, comprising: A storage array includes a plurality of storage cells arranged in rows and columns, the plurality of storage cells including a first storage cell in a first row and a first column of the storage array, and a second storage cell in a first row and a second column of the storage array, the first storage cell and the second storage cell being configured to store a respective first weight signal and a second weight signal; The input driver is configured to provide multiple input signals; A first logic circuit is coupled to the first memory cell and configured to provide a first output signal based on the first weighting signal and a first input signal from the input driver; A second logic circuit is coupled to the second memory cell and configured to provide a second output signal based on the second weighting signal and a second input signal from the input driver; as well as The adder circuit is configured as follows: Receive a weight symbol signal associated with the first weight signal and the second weight signal; and The partial signal is output based on the first output signal, the second output signal, and the weight symbol signal.

2. The device according to claim 1, wherein, The first logic circuit and the second logic circuit each include a multiplication circuit.

3. The device according to claim 2, wherein, The multiplication circuit includes NOR gates.

4. The device according to claim 2, wherein, The multiplication circuit includes an AND gate.

5. The device according to claim 1, wherein, The first storage unit and the second storage unit each include an SRAM unit.

6. The device according to claim 1, wherein, At least one of the first weight signal and the second weight signal is a signed weight.

7. The device according to claim 1, further comprising: The third storage cell in the second row and the first column of the storage array, and the fourth storage cell in the second row and the second column of the storage array, are configured to store a third weight signal and a fourth weight signal, respectively. A third logic circuit is coupled to the third memory cell and configured to provide a third output signal based on the third weighting signal and a third input signal from the input driver; as well as A fourth logic circuit, coupled to the fourth memory cell and configured to provide a fourth output signal based on the fourth weighting signal and a fourth input signal from the input driver.

8. The device according to claim 7, wherein, The adder circuit is configured to add the first output signal, the second output signal, the third output signal, and the fourth output signal.

9. The device according to claim 8, wherein, The adder circuit is configured to add weight sign signals associated with the first weight signal, the second weight signal, the third weight signal, and the fourth weight signal.

10. A computing in memory (CIM) device, comprising: A storage array comprising a plurality of storage cells arranged in rows and columns, the plurality of storage cells being configured to store a weight signal having w bits, where w is a positive integer, the storage array comprising w columns, and each of the storage cells being configured to store one bit of the weight signal; A plurality of multiplication circuits, each of the memory cells being coupled to a corresponding multiplication circuit, and the plurality of multiplication circuits being configured to provide a product signal based on a received input signal and a weight signal stored in a corresponding memory cell; and The adder circuit is configured as follows: Receive the weight symbol signal associated with the weight signal; and The partial sum signal is output based on the product signal and the weight symbol signal.

11. The device according to claim 10, wherein, The storage cell storing the weight signal of the w bits is located in one row of the storage array.

12. The device according to claim 10, wherein, The storage unit is configured to store a plurality of the weight signals, and wherein the storage unit storing the w-bit plurality of the weight signals and the corresponding multiplication circuit are located in the respective plurality of rows of the storage array.

13. The device according to claim 12, wherein, Each of the adder circuits is configured to receive the product signal from two rows of the storage array.

14. The device according to claim 13, wherein, Each adder circuit includes one half-adder circuit and w full-adder circuits.

15. The device according to claim 10, further comprising: An accumulation circuit is configured to shift and add the portions and signals to produce a CIM output.

16. A method for computation in memory (CIM), comprising: Multiple weight signals are stored in multiple storage units, wherein each of the weight signals has w bits, where w is a positive integer, and wherein each of the storage units stores one bit of the w-bit weight signal; A plurality of logic circuits are provided, the plurality of logic circuits being connected to corresponding memory cells in the plurality of memory cells; The input signal is provided to the plurality of logic circuits to multiply the weight signal by the input signal to provide a plurality of product signals; The plurality of product signals are output from the plurality of logic circuits to the adder tree; Provide a weight sign signal, the weight sign signal being configured to indicate whether the weight signal is signed; and The adder tree outputs a partial sum signal based on the product signal and the weight sign signal.

17. The method of claim 16, further comprising: The CIM output is generated by shifting and adding the aforementioned components and signals using an accumulator.

18. The method according to claim 16, wherein, The storage cells are arranged in a matrix of rows and columns, wherein the adder tree includes a plurality of adder circuits, and wherein outputting the plurality of product signals from the plurality of logic circuits includes outputting the product signals from adjacent rows to each of the adder circuits.

19. The method of claim 16, further comprising: Combine multiple w-bit weighted signals.

Images