Product-sum iterative approximation device and method

Abstract

A multiply-accumulate successive approximation (MASAR) array is provided. The MASAR array includes a plurality of MASAR cells, each MASAR cell including a multiplier configured to perform a digital multiplication between an input activation received for an input and an operand to calculate a result, and a unit capacitor configured to store the result as an analog charge. The MASAR array further includes digital logic configured to perform an analog summation of the analog charges of the unit capacitors of the plurality of MASAR cells to determine a digital output of the multiplication.

Description

[Technical Field]

[0001] Aspects of the present disclosure generally relate to hardware and methods for improved implementation of multiplication and multiply-accumulate functions.

[0002] background Current machine learning (ML) models, particularly neural network (NN) models, can include a combination of multiple layers with different numbers of weights in each layer. Each layer can compute multiple multiply-accumulate (MAC) operations involving the stored weights and the inputs to each layer. NNs are very successful in classification tasks (inference), but increasing task difficulty requires larger networks with more layers and more weights per layer. As NN size increases, the memory required for the weights increases, as does the computational power required to implement the network. In typical digital hardware implementations, the large number of weights cannot all be stored in the same application-specific integrated circuit (ASIC) that performs the MAC operations, requiring significant data transfer along with off-chip memory. Both the MAC operations, which consist of multiple multiplication and accumulation steps, and the data transfers can be costly in terms of time and energy.

[0003] overview In one or more illustrative examples, a multiply-accumulate successive approximation (MASAR) sequence is provided. The MASAR sequence includes a plurality of MASAR cells, each of which includes a multiplier configured to perform a digital multiplication between an input activation received for an input and an operand to calculate a result, and a unit capacitor configured to store the result as an analog charge. The MASAR sequence further includes digital logic configured to perform an analog summation of the analog charges of the unit capacitors of the plurality of MASAR cells to determine a digital output of the multiplication by configuring the unit capacitors as capacitive digital-to-analog converters (CDACs) in a successive approximation register (SAR) analog-to-digital converter (ADC).

[0004] In one or more illustrative examples, the MASAR string includes a plurality of MASAR cells, each MASAR cell including a multiplier configured to perform digital multiplication between an input activation received for an input and an operand to calculate a result, a unit capacitor configured to store the result as an analog charge, and a multiplexer (MUX) having at least first and second inputs and an output, where the MUX is configured to receive the result on a first input, receive a bit guess input from the digital logic on a second input, and apply the output to the unit capacitor. The MASAR string further includes digital logic configured to perform analog summation of the analog charges of the unit capacitors of the plurality of MASAR cells by controlling the unit capacitance of the individual MASAR cells via the bit guess input to form a capacitive digital-to-analog converter (CDAC) and utilize a successive approximation register (SAR) algorithm to determine the digital output of the MAC. The MASAR string further includes a comparator having a comparator input and a comparator output, where each of the unit capacitors is connected to the comparator input via a common bit line, and digital logic is configured to receive the comparator output, where the common bit line is connected to a RESET switch controllable by a RESET line. The MUX is further configured to be controlled by an enable MAC control line to select between (i) storing the result in the unit capacitor and (ii) utilizing the unit capacitors to determine an analog sum of charge. The RESET switch is further configured to be controlled to select between (i) connecting the common bit line to a reference voltage and (ii) disconnecting the common bit line from the reference voltage.

[0005] In one or more illustrative examples, a method for performing multiplication and multiply-accumulate functions using multiple MASAR cells and digital logic includes: utilizing a multiplier of each of the plurality of MASAR cells to perform a digital multiplication between input activations received for the input of the respective MASAR cell and operands to calculate a result; storing the result of the digital multiplication as an analog charge in the unit capacitor of each MASAR cell; and performing an analog summation of the analog charges of the unit capacitors of the plurality of MASAR cells under control of digital logic to determine a digital output of the multiplication by configuring the unit capacitors as a capacitive digital-to-analog converter (CDAC) in a successive approximation register (SAR) analog-to-digital converter (ADC).

[0006] In one or more illustrative examples, a MASAR array for performing multiple parallel MAC computations is provided. The MASAR array includes a plurality of MASAR columns, each MASAR column including a plurality of MASAR cells, each of the MASAR cells including a multiplier configured to perform a digital multiplication between an input activation received for an input and an operand to calculate a result, and a unit capacitor configured to store the result as an analog charge. The MASAR array further includes general-purpose digital logic configured to control an analog summation of the analog charges of the unit capacitors of the plurality of MASAR cells to determine a digital output of the multiplication.

[0007] In one or more illustrative examples, a parallel multi-bit MASAR architecture for performing multi-bit multiplication is provided. The parallel architecture includes a two-dimensional array of MASAR cells configured to collectively multiply each digit of a multi-bit input activation by each digit of a multi-bit operand, the MASAR cells arranged in MASAR columns by bit significance, whereby summation is performed analogically via column-by-column charge summation to determine a multi-bit digital output of the multiplication for each MASAR column. The parallel architecture also includes a plurality of scalars, each configured to digitally scale the multi-bit digital output of each MASAR column by bit significance to generate a scaled digital output. The parallel architecture also includes an adder configured to add the scaled digital outputs to generate a multi-bit digital result of the multiplication.

[0008] In one or more illustrative examples, a serial multi-bit MASAR architecture for performing multi-bit multiplication is provided. The serial architecture includes a single row of MASAR cells configured to multiply a single bit of a multi-bit input activation by each digit of a multi-bit operand, the MASAR cells arranged in MASAR columns by bit significance such that summation is performed analogically via charge summation for each column to determine an intermediate result of the multiplication for each single bit of the multi-bit input activation. The serial architecture also includes a plurality of scalars, each configured to digitally scale the intermediate result of each MASAR column by bit significance to generate a scaled digital output. The serial architecture also includes a register and an adder configured to add the scaled digital output to the register. The serial architecture also includes control logic configured to cycle through the single row of MASAR cells through each bit of the multi-bit input activation and utilize the adder for summation of the multi-bit digital result of the multiplication using the register. [Brief explanation of the drawings]

[0009] [Figure 1] FIG. 1 illustrates an exemplary multiply-and-accumulate successive approximation (MASAR) sequence according to one embodiment of the present disclosure. [Figure 2A] FIG. 1 is a functional diagram illustrating a MASAR cell with embedded static random access memory (SRAM) supporting single-bit precision MAC computation. [Figure 2B] FIG. 1 is a functional diagram illustrating a MASAR cell supporting single-bit precision MAC computation without embedded SRAM. [Figure 2C] FIG. 1 is a functional diagram illustrating in simplified form a MASAR cell supporting single-bit precision MAC computation. [Figure 3A] FIG. 1 is a functional diagram illustrating a single-ended (one bit line) multi-bit MASAR cell. [Figure 3B] FIG. 1 is a functional diagram illustrating a differential multi-bit MASAR cell. [Figure 3C] FIG. 1 is a simplified functional diagram showing a MASAR cell in MAC mode. [Figure 3D] FIG. 1 is a simplified functional diagram illustrating a multi-bit MASAR cell that can be used for MAC or SAR mode. [Figure 4A] FIG. 10 illustrates the first part of the computation of a MASAR sequence in MAC mode. [Figure 4B] FIG. 10 illustrates the second part of the computation of a MASAR sequence in MAC mode. [Figure 4C] FIG. 10 illustrates the second part of the computation of a MASAR sequence in MAC mode. [Figure 5A] FIG. 1 illustrates a first part of the computation of a MASAR sequence in SAR mode. [Figure 5B] FIG. 10 illustrates a second part of the computation of a MASAR sequence in SAR mode. [Figure 6]A is a diagram showing a SAR ADC with a CDAC, B is a diagram showing a SAR ADC N-bit CDAC showing the binary scaling of the capacitors and the connection of the ADC guess bits BG[0:N-1], and C is a diagram showing a MASAR column with Nr MASAR cell rows. [Figure 7] FIG. 1 illustrates an exemplary MASAR column with 32 rows. [Figure 8A] FIG. 10 shows an example of ADC inference bit connections for quantization of a 5-bit MAC result to 4 bits for a MASAR column with 32 rows. [Figure 8B] FIG. 10 shows an example of ADC inference bit connections for quantization of a 5-bit MAC result to 3 bits for a MASAR column with 32 rows. [Figure 9] FIG. 10 illustrates an example of spatial distribution of ADC estimation bits for a MASAR column with 32 rows. [Figure 10] FIG. 1 illustrates an exemplary MASAR sequence of arbitrary size N. [Figure 11] FIG. 1 illustrates an exemplary MASAR sequence of size N=6. [Figure 12] FIG. 10 shows an example of a MASAR sequence combined with a graph of ADC output values. [Figure 13] FIG. 10 illustrates an exemplary coarse-to-grain mapping of output values ​​of a MASAR sequence. [Figure 14] FIG. 1 illustrates a first exemplary fine-precision mapping of a subset of output values ​​of a MASAR sequence. [Figure 15] FIG. 10 illustrates a second exemplary fine-precision mapping of a subset of output values ​​of a MASAR sequence. [Figure 16] FIG. 10 illustrates a third exemplary fine-precision mapping of a subset of output values ​​of a MASAR sequence. [Figure 17] FIG. 10 illustrates a fourth exemplary fine-precision mapping of a subset of output values ​​of a MASAR sequence. [Figure 18] FIG. 10 illustrates a fifth exemplary fine-precision mapping of a subset of output values ​​of a MASAR sequence. [Figure 19]FIG. 10 illustrates a sixth exemplary fine-precision mapping of a subset of output values ​​of a MASAR sequence. [Figure 20] FIG. 10 illustrates an exemplary medium-precision shift mapping of a subset of output values ​​of a MASAR sequence. [Figure 21] FIG. 10 illustrates an alternative example of an exemplary medium precision shift mapping of a subset of output values ​​of a MASAR sequence. [Figure 22] FIG. 1 illustrates an example MASAR array of a group of MASAR columns. [Figure 23] FIG. 1 illustrates an exemplary MASAR array in a series configuration with digital logic. [Figure 24A] FIG. 1 illustrates a first SAR ADS transformation of a first bit from a first MASAR column of a MASAR array. [Figure 24B] FIG. 10 illustrates a second SAR ADS transformation of a second bit from a second MASAR column of the MASAR array. [Figure 24C] FIG. 10 illustrates the conversion of the Nth bit from the Nth MASAR column of the MASAR array. [Figure 25] FIG. 1 illustrates an exemplary MASAR array in a parallel configuration with digital logic. [Figure 26] FIG. 10 illustrates an exemplary MASAR array in a parallel configuration showing the routing of the ADC inference signal. [Figure 27] FIG. 10 is a diagram illustrating an example of 4-bit signed integer multiplication in the parallel case. [Figure 28A] FIG. 1 illustrates a parallel 4-bit signed integer multiplier. [Figure 28B] FIG. 28B illustrates a MASAR array implementing the parallel 4-bit signed integer multiplier of FIG. 28A. [Figure 29A] FIG. 1 illustrates a parallel 4-bit signed integer multiplier. [Figure 29B] FIG. 29B shows a product representation of the diagram of FIG. 29A. [Figure 30] FIG. 10 shows an example of a single channel / kernel multi-bit MASAR array for computing NM 4-bit signed integer MACs. [Figure 31] FIG. 10 illustrates an example of multiple channels / kernels in parallel to accommodate greater parallel computation. [Figure 32] FIG. 1 illustrates an example of a single-channel 8-bit signed integer parallel MASAR array accelerator. [Figure 33A] FIG. 1 shows an example of a MASAR array implementing a serial arrangement for 4-bit signed integers. [Figure 33B] FIG. 1 shows a serial MASAR product cell for Np=4 bits with comparators and digital logic. [Figure 34] FIG. 33B illustrates exemplary operations performed for the case of a 4-bit signed integer using the MASAR array of FIG. 33A. [Figure 35] FIG. 1 illustrates an exemplary serial 4-bit precision MASAR array for MAC computation with NM products in K=NK channels or kernels. [Figure 36] FIG. 1 illustrates an exemplary two's complement bit-serial architecture using 3-bit activations and weights. [Figure 37] FIG. 37 illustrates an example of adding partial products of a serial calculation performed by the architecture of FIG. 36. [Figure 38] FIG. 38 is a diagram illustrating an example of a first step of the partial product calculation of FIG. 37. [Figure 39] FIG. 38 is a diagram illustrating an example of a second step of the partial product calculation of FIG. 37. [Figure 40] FIG. 38 is a diagram illustrating an example of the third step of the partial product calculation of FIG. 37.

[0010] Detailed Description Embodiments of the present disclosure are described herein. However, it should be understood that the disclosed embodiments are merely exemplary and that other embodiments may take various alternative forms. The drawings are not necessarily to scale, and some features may be exaggerated or minimized to show details of particular components. Therefore, specific structural and functional details disclosed herein should not be interpreted as limiting, but rather as a representative basis for teaching those skilled in the art how to utilize the embodiments in various ways. As will be understood by those skilled in the art, various features illustrated and described with reference to any one drawing can be combined with features shown in one or more other drawings to create embodiments not explicitly illustrated or described. The illustrated combinations of features provide representative embodiments for typical applications. However, various combinations and modifications of features consistent with the teachings of the present disclosure may be desirable for particular applications.

[0011] The computational workload of a convolutional neural network (CNN) can be dominated by multiplication and accumulation, or MAC, operations (also known as dot products). These operations essentially multiply the input activations A of the CNN. i and weight W ij Therefore, there is interest in hardware (HW)-based building blocks that can accelerate MAC operations while improving performance in terms of energy / MAC, area / MAC, and clock cycles / MAC.

[0012] Aspects of the present disclosure relate to new building blocks for implementing MAC functions in HW using both digital and analog circuit techniques. These approaches can enable architectures ranging from a single multiplier to large MAC arrays that enable parallel multiplication and accumulation of data and weights for artificial intelligence (AI) / ML applications. Such architectures may be adapted for weight and activation calculation precision from 1 bit to multi-bit (4-bit, 8-bit).

[0013] The array may be composed of modular processing elements or cells. These processing elements may be configured so that a column of cells can perform a digital-input / digital-output MAC calculation without the need for an additional analog-to-digital converter (ADC). A column of such cells can perform a mixed-signal MAC calculation that results in an analog charge proportional to the MAC calculation result. This analog result is then converted to digital using cells in the same column configured as a SAR ADC. These cells may be referred to as MAC+SAR cells or MASAR cells. These MASAR cells are the processing elements that enable all the functionality for the MAC calculation and analog-to-digital conversion. Thus, in the proposed approach, the same processing element array is used for digital multiplication and charge summation as well as for ADC conversion.

[0014] The MASAR modular processing element may be used to implement multi-bit precision multiplication and MAC calculations, which are useful for ML / AI HW acceleration. Each MASAR cell uses a unit capacitance to store the result of a 1-bit multiplication as a charge. A column of MASAR cells can be used to sum 1-bit products with the charge using charge redistribution. A column of MASAR cells also provides the ability to convert charge to a digital value by configuring the column of MASAR cells as a SAR ADC. The SAR is used to convert the sum of products back to a digital representation. These novel building blocks (MASAR cells) enable both MAC and SAR functionality when used in a column (MASAR column). MASAR columns can be arranged in parallel to form a MASAR array capable of performing multi-bit precision MAC calculations.

[0015] 1 illustrates an exemplary MASAR column 102 according to one embodiment of the present disclosure. These MASAR columns 102 include a plurality of MASAR cells 104. The MASAR columns 102 include a digital input 106 configured to receive various inputs. The input vectors are of size N y As described herein, i may refer to the row index of the MASAR cell 104, where i=1...N y where j can refer to the column index. In the illustrated example, the columns are N r =2 N Contains the MASAR cell 104 of the row.

[0016] In weight programming mode, weight w ij may be applied to the digital inputs 106 of the MASAR cells 104. These weights may be stored in the MASAR cells 104 and used in a two-mode runtime approach to perform MAC calculations from the digital inputs to the digital outputs. These weights may be stored in each MASAR cell, or only in some or none of them. In some examples, the weights may be stored outside the MASAR columns. Weight programming modes may be specific to when the weights are stored in the MASAR columns or cells. In this case, it may be advantageous to use the same input (e.g., a wire) to the cell to program the weights and apply the input activations. Also, note that in some examples, more than one weight may be stored in each MASAR cell (e.g., each MASAR cell may include multiple memory cells), which may be advantageous for computing ML algorithms.

[0017] This two-mode approach includes a MAC mode (multiplication + charge summation) followed by a SAR mode (charge to digital). Note that memory can be present in all MASAR cells 104, or only certain rows in the MASAR column 102 can have memory. For cases where not all MASAR cells 104 have memory, there are different options for how the memory can be distributed in the MASAR column 102. Some examples are described herein. Programming the memory can also be done in several ways. One way is to use weights w ij In this case, the weights w ij may be applied to the inputs of the MASAR column 102. Other options may include programming one row i at a time, programming individual MASAR cell 104 memories one at a time, or programming multiple MASAR cell 104 memories at once throughout the entire MASAR array.

[0018] In MAC mode, input activation a i or input bias b i may be applied to each of the cells. These values ​​may be applied to the digital input 106. In a first embodiment (MAC step 1), multiple 1-bit digital multiplications are performed digitally. In a second embodiment (MAC step 2), the results of the multiplications are stored in charges. In a third embodiment (MAC step 3), the results of the multiplications are summed using charge sharing / redistribution on the MASAR column 102. The total charge stored as a unit capacitance on the MASAR column 102 represents the analog value of the result of the MAC calculation.

[0019] In SAR mode, the conversion of charge back to digital is achieved by configuring the unit capacitors of the MASAR cells 104 in the column as a capacitive digital-to-analog converter (CDAC) 115. The column is used with a single comparator to perform a successive approximation analog-to-digital conversion of the charge stored in the column. In SAR mode, the ADC's guess bit BG i[0:N-1] may be used to facilitate conversion back to digital.

[0020] The MASAR sequence 102 is a function of input i and stored weights w ij The MASAR string 102 may generate digital outputs 108 representing the multiplication with B[N]. These digital outputs 108 may provide a single bit B[N] result, or in other examples, may include the complete output B[0:N-1]. The MASAR string 102 may further include bit line drivers 110, zero-input cells 112, comparators 114, and digital logic 116. These components are described in further detail below.

[0021] 2A shows a functional diagram of a MASAR cell 104 that supports single-bit precision MAC calculations with an embedded SRAM 202. As shown, the MASAR cell 104 includes a programming weight w for the cell. ij It should be noted that although many examples herein refer to SRAM, the memory in MASAR cell 104 may be other types of memory besides SRAM. MASAR cell 104 stores the weights w ij to digital input 106 (here I i ) by the received input. j I i to SRAM 202 or to multiplier 204. MASAR cell 104 may also include a unit capacitance capacitor 206 configured to store the value calculated by multiplier 204. MASAR cell 104 may further include digital logic 116 in the form of a multiplexer (MUX) 208, where MUX 208 is configured to select between applying the output of multiplier 204 to capacitor 206 or resetting capacitor 206 to a defined value.

[0022] The MASAR cell 104 is connected to the input (I i =a i ) and calculate the 1-bit multiplication between them and the unit capacity C u The output of the MASAR cell 104 can be configured to be stored as a charge on the unit capacitor 206 as j th As a bit line, BL j Setting the EM signal may store a unit capacitance in the capacitor 206, and resetting the EM signal may reset the capacitor 206. Additionally, the EM signal may be used to select between a MAC mode, in which the capacitance is determined by a multiple, and a SAR mode, in which the lumped capacitance across the MASAR cells 104 is measured.

[0023] 2B shows a functional diagram of a MASAR cell 104 supporting single-bit precision MAC calculations without an embedded SRAM 202. In comparison to the MASAR cell 104 shown in FIG. 2A, the MASAR cell 104 of FIG. 2B uses weights w ij are applied to the MASAR cell 104 for each use, as opposed to being retrieved from the local weight memory SRAM 202.

[0024] 2C illustrates, in simplified form, a functional diagram of a MASAR cell 104 supporting single-bit precision MAC calculations. This simplified diagram will be used throughout this disclosure to explain further aspects of the operation of the MASAR cell 104 and MASAR column 102.

[0025] Note that a sign bit is not required for single-bit calculations, since one-bit or single-bit multiplication does not involve a sign. For signed integer multiplication, a sign bit may be utilized. Note that in cases where signed integer arithmetic is not used, such as binary signed decimal, a sign bit is not required. Figures 3A-3D collectively illustrate an exemplary MASAR cell 104 for a multi-bit signed integer MAC. Here, (N p ≧2).

[0026] 3A shows a functional diagram of a single-ended (one bit line) multi-bit MASAR cell 104. In comparison with the MASAR cell 104 shown in FIGS. 2A-2C, in FIG. 3A, the sign bit S i Additionally, MUX 208 provides a sign bit S i 116 is replaced with digital logic 116 configured to process the additional signaling required for

[0027] FIG. 3B shows a functional diagram of a differential multi-bit MASAR cell 104. In this example, there is an additional line for a differential output (a bit-by-bit inverse of the existing output). FIG. 3C shows a simplified functional diagram of the MASAR cell 104 in MAC mode. As shown, the signs are now considered with respect to the sum of the capacitor 206 unit capacitances across the MASAR cell 104. FIG. 3D shows a simplified functional diagram of a multi-bit MASAR cell 104 that can be used for either MAC or SAR mode.

[0028] [Table 1]

[0029] [Table 2]

[0030] [Table 3-1] [Table 3-2]

[0031] 4A shows the first part of the operation of the MASAR column 102 in MAC mode (MAC step 1). The illustrated MASAR column 102 includes four MASAR cells 104, a comparator 114, and SAR digital logic 116. In this example, N BG= 2, and N r There are N = 4 rows of MASAR cells 104. These MASAR cells 104 are y = 3 active MASAR cells 104 + 1 zero-input cell 112. A zero-input cell 112 is always set to zero because no input is applied to that MASAR cell 104. Note that there are cases where a zero-input cell 112 is not needed, for example, when the MASAR column 102 is not using a full-resolution transform.

[0032] Here, the MASAR sequence 102 is shown in the first part of the MAC, where all the products a i ·w ij is applied to the MASAR column 102. At this point, the ADC guess bits are set to zero (BG0=BG1=0). Additionally, the signal EM is set to EM=1,

number

[0033] With more specific reference to aspects of MAC step 1, a 1-bit product a i ·w ij charges Q on the unit capacitor 206 of the MASAR cell 104. i It is stored as (Equation 1). Here, EM=1,

number

number

number

[0034] 2 N For this MASAR column 102 with cells, y There are at most N inputs, where y =2 N Note that one MASAR cell 104 (zero-input cell 112) has a zero input, and N y cells have inputs. This is to ensure a perfect analog-to-digital conversion of the MAC result on the MASAR array 102. The total capacity C TOT is given by Equation 3. If the MASAR sequence 102 does not perform a full resolution conversion, i.e., the output of the MASAR sequence is y Note again that if the byte has fewer than 10 bits, then the zero-input cell 112 is not required.

[0035] 4B illustrates the second part of the operation of the MASAR sequence 102 in MAC mode (MAC step 2). In this embodiment, the unit capacitors 206 or the common side of the bit lines are connected to the power supply V S Again, EM=1,

number

[0036] 4C shows the third part (MAC step 3) of the operation of the MASAR sequence 102 in MAC mode. In this embodiment, the cell side of the unit capacitor 206 is set to zero potential or 0V. This is done by first verifying that the guess bits (BG[0], BG[1]) are zero, followed by EM=0,

number

[0037] 5A shows the first part of the operation of the MASAR column 102 in SAR mode (SAR step 1), where two MASAR cell 104 inputs are connected to BG[1] and one to BG[0]. Using these connections, the MASAR column 102 is configured as a 2-bit CDAC 115, whose 2 N It is possible to perform a 2-bit (N=2) conversion or a full resolution conversion of the MASAR column 102 output of the cell. This diagram shows one possible connection of the ADC extrapolated bits to the MASAR cell 104 to allow for N=2-bit analog-to-digital conversion. y Any one of the MASAR cells 104 is connected to BG[0], and the MASAR cells 104 are connected to N MASAR cells 104 that are separate from the MASAR cell 104 connected to BG[0]. y Note that the MASAR cells 104 may be connected to BG[0] and BG[1] in any order, as long as two of them are connected to BG[1].

[0038] In this example, the bit line voltage is defined by Equation 4, and the input to comparator 114 is defined by Equation 5. Finally, comparator 114 calculates Equation 6. To illustrate how the SAR algorithm operates, the expected result of the MAC is defined by Equation 7. In this case, the output of MASAR column 102 is 2, or B[1]=1, B[0]=0 in this example.

number

number

number

number

[0039] Referring more specifically to the SAR conversion, this SAR conversion (SAR step 1) begins with the SAR logic inferring the most significant bit BG[1]=1 while maintaining the least significant bit BG[1]=0. This results in an input of zero in comparator 114 as shown in Equation 8, which in turn results in an output of zero in comparator 114 as shown in Equation 9. Here, the SAR logic assigns B[1]=1.

number

number

[0040] 5B shows the second part of the operation of the MASAR sequence 102 in SAR mode (SAR step 2), where the actual value of BG[0] is determined. BG[1] was already determined in SAR step 1.

[0041] This part of the SAR conversion begins with the SAR logic inferring the least significant bit BG[0]=1. Since the most significant bit BG[1] has already been determined to be 1, its value is not changed. With BG[0]=1 set, the input of comparator 114 becomes greater than zero, as shown in Equation 10. Therefore, the output of comparator 114 becomes 1, as shown in Equation 11. The SAR logic assigns B[0]=0. This is the last part of the SAR calculation for this example. Therefore, the final output of MASAR sequence 102 is B[1]=1, B[0]=0, which matches the expected MAC result 2.

number

number

[0042] Therefore, the simplified 4-cell MASAR column 102(N BG =2,N y =3) can accomplish the MAC calculation and SAR analog-to-digital conversion using the same capacitor 206 array. Note that this was done for 1-bit calculations, which do not require a sign for each MAC product. However, this approach may be extended to signed arithmetic for multi-bit MACs.

[0043] The previous example uses four cells, but the MAC mode uses N r In this case, the MASAR column 102 may be expanded to contain N rows. r The MASAR column 102, which consists of N rows, y =N r -1 1-bit MAC calculations can be performed. Therefore, N r The maximum digital value of the MAC output for a MASAR column 102 using row 1 as input is B MAX is.

number

number

number

[0044] The total charge stored on the capacitance of the MASAR string 102 is given by Equation 2 above. The bit line voltage (extending this example to the general case) is given by Equation 13. For the general case, j th The output of the MASAR sequence 102 is a digital output given by Equation 14.

[0045] As a variation, the addition of input bias and calibration may be performed in the MASAR train 102. In some cases, the input bias b j It may also be interesting to add to the MAC calculation. In this case, the desired output of the MASAR sequence 102 is given by Equation 15:

number

[0046] To add these biases, N b Rows can be dedicated to bias inputs. The number of inputs is N y Since the input activations are fixed at N, this means that the number of possible input activations is N. a =N y -N b For example, if it is desired to calibrate a SAR ADC, an additional N cA row may be dedicated to adding calibration of the ADC output. If desired, this can further reduce the amount of input for the MAC, as shown in Equation 16. Note that while adding bias to the MASAR cells 104 may be performed in some approaches, in other approaches the bias may be added to the output after the MASAR columns 102. This could occur, for example, in a digital summation stage (as shown in the figures herein).

[0047] Similarly, in the example above, four cells are used, but the SAR mode uses N r , the ADC guess bits can be distributed to form an N-bit CDAC 115.

[0048] 6A shows a diagram of a SAR ADC with a CDAC 115. Generally, a capacitor-based SAR ADC with N-bit resolution uses 2 N 206 are required.

[0049] 6B shows a diagram of a SAR ADC N-bit CDAC 115 showing the binary scaling of capacitors 206 and the connection of the ADC guess bits BG[0:N-1]. This shows a simplified binary weighted CDAC 115 commonly used in SAR ADCs. This CDAC 115 has N+1 weighted capacitors C DAC Each weight capacitor in the CDAC 115 can include C(n), as shown in Equation 17. u ~2 N-1 C u During ADC conversion, C DAC The binary weighted capacitors (n=0...N) are controlled by the corresponding ADC guess bit BG[n-1], where n>0.

[0050] Figure 6C shows the N r1 shows a diagram of a MASAR column 102 having MASAR cell rows 104. As shown, it shows how blocks of MASAR cells 104 are connected to ADC inference bits to enable ADC conversion of charge-based MAC results. It shows how rows of MASAR cells 104 in a MASAR column 102 can be arranged to form binary-weighted capacitors in an SAR ADC. For example, CDAC 115, C DAC (n=N), 2 is the maximum capacity. N-1 MAAR cell 104 of row 2 is required. N A MASAR array 102 having MASAR cells 104 can be configured to perform up to N-bit SAR ADC conversion.

number

[0051] Therefore, the MASAR column 102 determines the maximum ADC resolution or maximum number of bits N for signed integer calculations. BG = k (for two's complement) and N BG = k+1. Optionally, one row in every MASAR column 102 is assumed to be a row of zero-input cells 112. This ensures that the SAR ADC contains a row of MASAR cells 104 that can perform a full resolution conversion of the MAC result.

[0052] One key aspect of the SAR calculation and MASAR concept is the routing of the ADC guess bits BG[n] to the MASAR cell 104. The exemplary MASAR column 102 may be used to demonstrate different options for setting the SAR ADC conversion resolution by varying how the ADC guess bits in the MASAR column 102 are configured.

[0053] 7 shows an example MASAR column 102 with 32 rows. In this example, Nr = 32, k = 5, so the maximum SAR ADC resolution for this column is N BG = k = 5 bits. Here we show one possible configuration of the ADC guess bits BG[0:4] connections to allow for 5-bit SAR ADC conversion. In general, N yMAX N of inputs r =2 k A full ADC resolution conversion of the MAC result using a MASAR column 102 with rows requires k bits. This ensures (ideally) no loss of information. However, in some cases, the result does not need to be converted at full resolution, but with less than k bits. This can be done by rearranging the connections of the ADC guess bits in the MASAR column 102.

[0054] Figure 8A shows an example of ADC guess bit connections for quantization of a 5-bit MAC result to N=4 bits. Note that there is no BG[4] bit. Also, here there are now two input cells with zero input during SAR ADC conversion. One of these has input during MAC calculation mode, and the other is the zero-input cell 112.

[0055] Figure 8B shows the N of the 5-bit MAC result. BG 1 shows an example of an ADC guess bit connection for quantization to N = 3 bits. In general, the configuration of the MASAR column 102 ADC guess bit input is BG = k bits can be used to reduce the MAC result down to 1 bit so that it can be quantized.

[0056] FIG. 9 shows an example of a spatial distribution of ADC guess bits for a MASAR column 102 having 32 rows. As noted above, the configuration of the ADC guess bits may be used to change the ADC resolution. Another aspect to consider is how capacitor matching affects quantization accuracy. For SAR ADCs, these may be layout techniques that can be used to reduce the effects of capacitor mismatch by spatially distributing the capacitors of the SAR ADC, as listed in Equation 17. This approach may be extended to the MASAR column 102 by distributing the connections of the ADC guess bits. An example of this type of spatial distribution of ADC guess bits is shown, where the bits that make up different guess bits are spatially randomized across the MASAR column 102.

[0057] Note that these SAR ADC conversion modes assume that the MASAR array 102 is configured as a binary CDAC 115. In other words, the capacitors are sized to be binary weighted, as shown in Equation 17. The choice of binary weighting may dictate how the ADC inference bits are distributed to control the individual MASAR cells 104 in the preceding section. However, alternatives to binary weighting exist. For example, a SAR ADC may be developed using a non-binary split capacitor array, which may also be implemented in the MASAR array 102. Using the MASAR array 102 for such applications may provide a more compact architecture and / or even a lower-energy implementation compared to other SAR ADC designs.

[0058] Figure 10 shows an arbitrary number of rows N r =2 N 1 shows an example MASAR column 102. As shown, the MASAR column 102 is N -One activation input (digital input 106) and two N -1 weight (w i ) and MASAR column 102 is 0 to 2 N11 shows an example MASAR sequence 102 of size N=6. As shown, the MASAR sequence 102 can generate output values ​​ranging from 2 6 -1 = 63 activation inputs (digital input 106) and 63 weights (w i Such a MASAR sequence 102 can generate output values ​​in the range of 0 to 63.

[0059] 12 shows an example of a MASAR column 102 combined with a graph of ADC output values. As shown, the output values ​​of the MASAR column 102 are shown on the Y-axis for each possible result of the multiplication performed by the MASAR cells 104 of the MASAR column 102. As shown, each result is mapped to a corresponding output value. In this configuration of the ADC, the MASAR column 102 N It performs an M-bit transformation on the −1 value, where M=N. In other words, the MASAR sequence 102 performs a complete transformation.

[0060] However, in some applications, it may be desirable to utilize a subset of the possible values ​​that may be available through the use of the MASAR column 102. For example, in some cases, less precision may be desired. In such cases, the LSB may not be used. Alternatively, in other cases, conversion may be desired for a subset of the range of values, starting with values ​​below the range of interest set to the minimum value and values ​​above the range of interest set to the maximum value. As summarized above with respect to FIG. 8, a MASAR column 102 having a full resolution may be configured to use a lower resolution (as some examples, a 5-bit MASAR column 102 may be configured to use 4-bit or 3-bit resolution). FIG. 13 shows another example of the concept introduced in FIGS. 8A-8B. In this case, FIG. 13 shows in detail how a 6-bit full-resolution MASAR column 102 may be configured for 3-bit operations.

[0061] Figures 14 to 21 may be zoomed in to expand this concept. The A / D conversion of the subset of interest may be referred to as zoom ADC conversion. Depending on which bits are used, the value range is determined, and how the unused bits are connected to the SAR ADC may be used to affect where the value range may shift to fit. Advantageously, by configuring the mapping of the unit capacitors 206 to the SAR DAC, a configurable output mapping of the value offset and offset range may be performed.

[0062] Referring more specifically to FIG. 13, the drawing shows an exemplary coarse accuracy mapping of the output values of the MASAR column 102. Here, an approximate conversion is being performed. In this configuration of the ADC, the MASAR column 102 N performs an M-bit conversion for the -1 value, provided that M < N. In this example, N = 6 and M = 3. As shown, LSB = 2 N-M = 2 3 = 8 is the code value. Additionally, the MASAR column 102 is wired such that only the MSB M bits of the output of the MASAR column 102 are considered. The three least significant bits are simply not considered in the ADC conversion and may be discarded. Such an approach provides values over a range of 2 N but lacks accuracy.

[0063] FIG. 14 shows a first example fine-precision mapping of a subset of output values ​​of the MASAR column 102. Here, the M-bit conversion is again performed on three of the six bits, but instead of the LSB being three times larger than the minimum value (FIG. 13), the LSB remains the same size as in the example of FIG. 12. In this case, three bits are used (here, 4-, 2-, and 1-bit positions, for a range of 4+2+1=7). The three MSBs (here, 32-, 16-, and 8-bit positions) are always hardwired high, and the corresponding difference of the three MSBs is also hardwired high. Effectively, this shifts the output by an offset of (32+16+8) / 2=28. Therefore, the resulting mapping is from a low value of 28 to a high value of 28+7 or 35, based on the values ​​of the three LSBs.

[0064] 15 shows a second exemplary fine-precision mapping of a subset of the output values ​​of the MASAR column 102. Here, the M-bit conversion is again performed on three of the six bits, again with the three least significant bits (LSBs). However, the three most significant bits are always wired low, and the corresponding differential of the three most significant bits is wired high. This effectively leaves the conversion range unshifted. Therefore, the resulting mapping is from a low of 0 to a high of 7, based on the values ​​of the three least significant bits.

[0065] 16 shows a third exemplary fine-precision mapping of a subset of the output values ​​of the MASAR column 102. Here, the M-bit conversion is again performed on three of the six bits, again with the three LSBs. However, the two MSBs are always wired low and the corresponding differential of the three MSBs is wired high, while the third MSB (8 bits) is always wired high and its corresponding differential is also always wired high. Effectively, this shifts the output by an offset of 8 / 2=4. Therefore, the resulting mapping goes from a low value of 4 to a high value of 4+7=11, based on the values ​​of the three LSBs.

[0066] 17 shows a fourth exemplary fine-precision mapping of a subset of the output values ​​of the MASAR column 102. Here, the M-bit conversion is again performed on three of the six bits, again with the three LSBs. However, the two MSBs are always wired low and the corresponding differential of the three MSBs is wired high, while the third MSB (8 bits) is always wired high and its corresponding differential is always wired low. This effectively shifts the output by an offset of 8. Therefore, the resulting mapping is from a low value of 8 to a high value of 8 + 7 = 15, based on the values ​​of the three LSBs.

[0067] 18 shows a fifth exemplary fine-precision mapping of a subset of the output values ​​of the MASAR column 102. Here, the M-bit conversion is again performed on three of the six bits, again with the three LSBs. However, the MSBs of the 32 and 8 bits are always wired low and their corresponding differentials are wired high, while the second MSB (16 bits) is always wired high and its corresponding differential is always wired low. This effectively shifts the output by an offset of 16. Therefore, the resulting mapping is from a low value of 16 to a high value of 16 + 7 = 23, based on the values ​​of the three LSBs.

[0068] 19 shows a sixth exemplary fine-precision mapping of a subset of the output values ​​of the MASAR column 102. Here, the M-bit conversion is again performed on three of the six bits, again with the three LSBs. The 32 MSBs are always wired high and the corresponding differential is wired low, while the second and third MSBs (16 and 8 bits) are always wired low, with the corresponding differential always wired high. This effectively shifts the output by an offset of 32. Therefore, the resulting mapping is based on the values ​​of the three LSBs, from a low of 32 to a high of 32 + 7 = 39.

[0069] 20 shows an exemplary medium-precision shift mapping of a subset of the output values ​​of the MASAR column 102. Here, the M-bit conversion is again performed on 3 of the 6 bits, but this time on 8, 4, and 2 bits (a range of 8 + 4 + 2 = 14). The 32, 16, and 1 bits are always hardwired high, and the corresponding differentials are also hardwired high. Effectively, this shifts the output by an offset of (32 + 16 + 1) / 2 = 24. Therefore, the resulting mapping is from a low of 24 to a high of 24 + 14 = 46, with a step size of 2, based on the three utilized (8, 4, and 2) values.

[0070] 21 shows an alternative example of an exemplary medium-precision shift mapping of a subset of the output values ​​of the MASAR column 102. Here, the M-bit translation is again performed on 3 of the 6 bits, but this time on 16, 8, and 4 bits (a range of 16+8+4=28). The 32, 2, and 1 bits are always hardwired high, and the corresponding differentials are also hardwired high. Effectively, this shifts the output by an offset of 16. Therefore, the resulting mapping is based on the three utilized (16, 8, and 4) values, with a step size of 4, from a low value of 16 to a high value of 47.

[0071] Therefore, by configuring the mapping of unit capacitors 206 to SAR DACs, a configurable output mapping of a range of offset values ​​may be performed. N Note also that we are not limited to needing to go all the way to -1. In fact, we can use any number of inputs N>2 M However, this may be possible by an approximate transformation.

[0072] 22 shows an exemplary MASAR array 150 of a group of MASAR columns 102. The illustrated MASAR array 150 has N x102. Each column may contain j elements, as described in the preceding MASAR column 102 example. This MASAR array 150 may further include bit line drivers 110 and digital logic 116, as well as row drivers 152, as described above. MASAR array 150 may be used to accelerate large-scale multi-bit precision parallel MAC computations.

[0073] Serial and parallel SAR architectures may be utilized for the MASAR train 102 and the MASAR array 150. Figures 23, 24A, 24B, and 24C illustrate a serially configured MASAR array 150. Figures 25-26 illustrate a parallelly configured MASAR array 150.

[0074] For a serial SAR MASAR array 150, the MAC calculations are performed in parallel, but the SAR ADC conversion of the MAC results is performed in a serial fashion. The ADC conversions are performed one at a time in each MASAR column 102. The advantage of this architecture is that the SAR logic can be generalized and does not need to be placed in each MASAR column 102, resulting in area savings for the MASAR array 150. The disadvantage is a reduction in throughput or speed of the MAC calculations. However, in some applications, the tradeoff between area and speed is advantageous.

[0075] 23 shows an exemplary MASAR array 150 in a serial configuration with digital logic 116. Here, general-purpose digital logic 154 is used for each column j=1, 2, ... N x , which may be used to adjust the SAR transformation for N for each column. th Bit B j [0:N BG −1] output is determined. In this case, the MASAR sequence 102 determines the comparator 114 output V CO,jThe general purpose digital logic 154 has a 1-bit output coming from BG, which the general purpose digital logic 154 can use as input for the SAR ADC algorithm. The general purpose digital logic 154 generates the ADC's estimated signal, BG, during SAR mode. j [0:N BG −1] may also be applied to row drivers 152. Finally, digital logic 116 provides a digital MAC output B for each j row in the serial array. j [0:N BG -1] can be provided.

[0076] Additionally, general-purpose digital logic 154 can provide control signals for different modes of MASAR array 150. These modes are described in Table 2. For example, in MAC mode, digital logic 116 provides input activation a to row driver 152. i In weight programming mode, weight values ​​w can be applied to program the weight memory in SRAM 202. ij can be applied.

[0077] Figures 24A-24C collectively show how each column output is converted to digital in a serial fashion. Figure 24A shows a first SAR ADC conversion of a first bit from a first MASAR column 102 of a MASAR array 150. Figure 24B shows a second SAR ADC conversion of a second bit from a second MASAR column 102 of a MASAR array 150. This process can continue sequentially until the conversion of the last MASAR column 102. Figure 24C shows a second SAR ADC conversion of a second bit from a second MASAR column 102 of a MASAR array 150. th N from MASAR column 102 th Indicates the conversion of bits.

[0078] 25 shows an exemplary MASAR array 150 in a parallel configuration with digital logic 116. In a parallel SAR ADC array 150, the SAR ADC conversions are performed in parallel. Therefore, the outputs of all of the MASAR columns 102 may be available simultaneously. Each MASAR column 102 may have N BG Bit Output B j [0:NBG −1]. The advantage of this architecture is in throughput or speed, but at the cost of the additional circuitry and area required to place the SAR ADC digital logic 116 within each MASAR column 102. However, if throughput is important, the trade-off of increased area for increased throughput may be advantageous.

[0079] General-purpose digital logic 154 may be used to coordinate top-level functions of the parallel array to provide control signals for different modes of MASAR array 150, as described in Table 2. For example, in MAC mode, general-purpose digital logic 154 may provide input activation a to row driver 152. i and in weight programming mode, weight memory values ​​w for programming the SRAM 202. ij The digital logic 116 can also control the timing of the array signals.

[0080] However, unlike the serial MASAR array 150, the general purpose digital logic 154 in the parallel MASAR array 150 generates the ADC estimated signal BG during SAR mode. j [0:N BG −1] may not be applied to the row drivers 152. Instead, this may be done by local SAR logic 156 in each MASAR column 102 routed through the MASAR column 102 to each MASAR cell 104.

[0081] Figure 26 shows the ADC estimated signal BG i [0:N BG 1 illustrates an exemplary MASAR array 150 in a parallel configuration showing routing of

[0041] . As shown, the estimated signal is routed through the MASAR columns 102 from the local SAR logic 156 to the MASAR cells 104 within each MASAR column 102.

[0082] Thus, MASAR column 102 and MASAR array 150 performing 1-bit MAC calculations may be utilized in a serial or parallel configuration. These calculations may include sum-of-products of 1-bit weights and activations. Additionally, MASAR column 102 and MASAR array 150 may be used to perform multi-bit MAC calculations. In such instances, the weights and activations may have a precision of >1 bit.

[0083] The multi-bit digital multiplication may be broken down into individual units, which may be implemented using the MASAR sequence 102. Thus, N p The product of the weights and activations may be achieved with 4-bit signed integers (N p An example of activations and weights for a matrix of 1 = 4 bits is defined as shown in Equation 19 and Equation 20. Multi-bit activations and weights may be represented by single bits with different significances l. For example, A i is a 1-bit value a il and W il is a 1-bit value w il The most significant bit is the sign bit a i3 ,w i3 These may be used to calculate the sign bit for the overall product as given by Equation 21. Note that for simplicity of notation, the column index j for the weights has been omitted in these examples.

number

number

number

[0084] Figure 27 shows an example of 4-bit signed integer multiplication in the parallel case. As shown, the product P i =A i ·W imay be divided into multiple parallel operations. Each of these operations may include multiple parallel operations (1-bit multiplication, summation, scaling, summation). In an exemplary implementation, these operations include performing a 1-bit digital multiplication, a summation in charge, and a conversion to digital. These operations may be performed as follows: 1. All 1-bit digital numbers are multiplied (digitally) to form a product. For example, S i a i0 ·w i0 sign bit S i determines the sign of the result. These products have values ​​of -1, 0, or 1. 2. (In significance-based columns) All products in a column are summed. This summation is performed analogue via charge summation. The summation result is converted to digital via ADC conversion, resulting in a column output c i0 ~c i4 occurs. 3. The sum of all columns is scaled by its significance digitally. Scaling in this case is a simple bit shift. For example, c i1 =S i a i0 ·w i1 +S i a i0 ·w i1 is 2 1 This involves scaling the digital result c i1 A left shift of the on-bits is required. 4. The sum of all scaled columns is the final product P i are digitally summed to obtain

[0085] Figures 28A-28B collectively illustrate an architecture for implementing the parallel multiplication function shown in Figure 27. Figure 28A illustrates a parallel 4-bit signed integer multiplier. Figure 28B illustrates a MASAR array 150 that implements the parallel 4-bit signed integer multiplier of Figure 28A. This architecture includes a product cell with three rows and five columns, followed by scaling and summation.

[0086] Each cell in Figure 28A contains a weight memory, e.g., SRAM 202, programmed as shown. The sign of each cell output in Figure 28A is determined by a sign bit S i It is controlled by S i is calculated in the "sign bit cell" at the top right of the array. The sign bit is routed from the sign bit cell to the other cells. This example is shown as signed, but similar calculations may be performed for two's complement values.

[0087] Figure 28B shows a MASAR array 150 implementation of parallel multi-bit multipliers 204. In this case, the MASAR cells 104 perform one-bit products and sums (in the charge domain). Each MASAR column 102 is then used as a SAR ADC (2-bit in this example) to convert the sums from charge to digital using techniques similar to those described for single-bit calculations. Scaling may then be performed digitally, for example via bit shifting 160. The final sum 162 in Figure 28B is also done digitally.

[0088] Figure 29A shows a parallel 4-bit signed integer multiplier. This multiplier includes five MASAR columns 102 for each sum of products and three rows of simplified MASAR cells 104. The sign bit S i are distributed to all MASAR cells 104. In the example shown, only one row of cells has a weight memory SRAM 202. In this case, the weight values ​​may be routed to other cells (as indicated by the diagonal arrows). Figure 29B shows a product representation of the diagram in Figure 29A, where the routing is indicated by the diagonal arrows.

[0089] The architectures shown in Figures 27, 28A-28B, and 29A-29B implement a single 4-bit product. Additional MAR cells 104 may be utilized to implement multiple products in parallel.

[0090] Figure 30 shows N M1 shows an example of a single channel / kernel multi-bit MASAR array 150 for computing 4-bit signed integer MACs.

[0091] Figure 31 shows an example of multiple channels / kernels in parallel to accommodate larger parallel computations. K There are N channels, each with 4-bit precision. M MACs can be computed in parallel. Mathematically this is as follows:

number

[0092] While the previous examples have used Np = 4-bit signed integers, this architecture can be scaled to greater or less than 4 bits of precision. This involves scaling the number of product cell rows N as shown in Equations 23 and 24. pr and the number of columns N pc Scaling of the N pr =N p -1 formula 23 N pc =2 N P -3 formula 24

[0093] Total number of rows in MASAR column 102 r and the number of MACs N M and the number of zero-input rows N Z The relationship between can be determined using Equations 25 and 26. N M =floor(Nr / (N p -1)) Equation 25 N z =N r -N M ·(Np-1) Equation 26

[0094] Here, to enable the use of binary weighted capacitor DACs in each MASAR column 102, the number of rows N rmust be a power of 2. For example, N r =2 k = 256 (k = 8) and N p = 4 bits. In this case, from the above formula, N pr =3, N pc =5, N Z = 1, and N M In this example, the 256-row by 5-column MASAR array 150 can compute 85 parallel MACs with 4-bit precision. k Note that zero-input rows have been added to ensure there are k MASAR cells 104. This is necessary because each MASAR column 102 is also a k=8-bit SAR ADC. In another example, a 256 row by 13 column, 8-bit precision MASAR array 150 has N M = 36 MACs can be computed. In the 8-bit case, N pr =7, N pc = 13, and N Z =4.

[0095] Figure 32 shows an example of a single channel 8-bit signed integer parallel MASAR array 150 accelerator. As shown for the 4-bit case in Figure 30, multiple 8-bit precision channels can be placed in parallel. Note that adding additional channels does not change the ADC resolution of each MASAR column 102. However, N M or N r Increasing the number of 0 input rows may increase the required resolution. Also, in the 8-bit case, the number of 0 input rows increases to 4. In this case, the 0 input rows can be used for other purposes such as input bias or calibration. If other capacitor DAC approaches are used to advantage, the additional 0 input rows may not be necessary.

[0096] While parallel multi-bit architectures improve computational speed, serial architectures are more compact. This section describes how multi-bit digital multiplication can be decomposed into serial computations, enabling smaller multi-bit MASAR array 150 accelerators.

[0097] Figure 33A shows an example of a MASAR array 150 implementing serial ordering for a 4-bit signed integer Np = 4 bits. This approach serializes the multiplication of the input activations with the weights. The multiplication is performed by serializing the least significant activation bit a i0 Starting with (N p -1) th It may be performed with a series of operations ending in a bit.

[0098] Figure 34 shows exemplary operations performed for the case of 4-bit signed integers using the MASAR array 150 of Figure 33A. In these steps, the partial products pp i0 ~pp i2 is calculated. At step 0 (s=0), pp i0 (S i a i0 ·w i2 ,S i a i0 ·w i1 ,S i a i0 ·w i0 ) are calculated in parallel, and then 0 In step 1 (s=1), pp i1 All terms in are calculated, and 2 1 is scaled by and added to the result of step 0. In step 2 (s=2), i2 is calculated, and 2 2 and added to the result of step 1. The output of step 2 is the final product P i =A i ·W i Although this example is shown for signed integers, a similar process may be performed for two's complement values.

[0099] Referring back to FIG. 33A, FIG. 33A shows an implementation of performing the operations of FIG. 34 using a simplified model of a MAS cell 104. Each cell has an SRAM 202 for storing weights as shown. st In the column, the sign bit of the multiplication is the most significant bit a i3 and w i3 For the remaining three columns, the partial product terms are calculated in parallel and stored as charges on the columns. is and a scale factor of 2 s changes for each step (s=0,1,2) of the multiplication. The charges on the columns representing the partial products are converted to digital form before scaling. The previous step output P stored in a register i,s-1 is added to the current scaled output. The register shown in Figure 33A is i,s Also stores the current result of P i,s-1 =0 and P i,0 =2 0 ·pp i0 At step 1, P i,s-1 =P i,0 and P i,1 =2 1 ·pp i1 +P i,0 At step 2, P i,s-1 =P i,1 and P i,2 =2 2 ·pp i2 +P i,1 =P i That is,

[0100] Figure 33B shows a diagram of a serial MASAR product cell for Np=4 bits with comparators 114 and digital logic 116. In this case, only a 1-bit SAR ADC is required to convert each partial product. i While only one row is needed to calculate p-1 In general, a series product cell is about Np times smaller than a parallel product cell, so one product or MAC implementation requires about N p Twice as slow.

[0101] The architecture shown in Figure 33A implements only one serial 4-bit product. To implement multiple products in parallel, serial MASAR product cells must be added to the columns (bit lines).

[0102] Figure 35 shows K=N K N in the channel or kernel M 1 shows an exemplary serial 4-bit precision MAC array 150 for computing MACs with products i=1 to N. As shown, each channel has M There are N rows. Each row can compute one product (series). Therefore, each channel has N rows with 4-bit accuracy, as given in Equation 22. M As explained earlier, multiple channels / kernels can be deployed in parallel to accommodate larger computations. This means that N K Channels, if any, are also shown. Activation may be propagated along the rows of channels. Weights may be stored in SRAM 202 located within MASAR cells 14. Sign bits may be calculated and routed along each row of individual channels, similar to that shown in FIG. 33A.

[0103] It should also be noted that the serial MASAR accelerator can be scaled to higher or lower bit precision. For example, N M Compute N 16-bit MACs p = 16-bit precision accelerator is 16 columns × (N M +1) rows of serial MASAR accelerators.

[0104] While many of the above examples are described in terms of signed integer values, it should be noted that MASAR column 102 and MASAR array 150 may also be used for two's complement arithmetic. Similar to unsigned numbers, an N-bit two's complement number can be written as 2 N represents one of the possible values ​​of , but in a different range. Therefore, in two's complement arithmetic, 2 is used for an N-bit output.N However, for signed values, 2 bits are added to the N-bit output to account for the sign. N+1 A line may be required.

[0105] Figures 36-40 collectively illustrate an example of a serial 3-bit precision MASAR array 150 calculation performed using two's complement numbers. As with the signed example described above, activations may be supplied sequentially to the MASAR array 150. In Figure 36, the input bits are supplied from LSB to MSB, but other orders are possible (Figures 38-40). Partial products (PP) are calculated similarly to that shown in Figure 34. An accumulator or integrator can add scaled partial products to the previous result until the final step of the calculation is completed.

[0106] FIG. 36 shows an exemplary two's complement bit-serial architecture using three-bit activations and weights. As shown, activations are fed bit-by-bit to the MASAR column 102. Each activation is multiplied by a corresponding weight, which may be stored, for example, near the multiplier 204, e.g., in the SRAM 202. For each bit, a digital summation may be performed to generate a partial product for that bit position. These partial products are then scaled. For example, for a three-bit two's complement number, the LSB bit is shifted 2^0, the next LSB is shifted 2^1, and the third bit is shifted 2^2. The sum of these three intermediate values ​​becomes the final sum 162.

[0107] Figure 37 shows an example of adding partial products to a serial calculation performed by the architecture of Figure 36. The left side shows the individual values ​​that are summed to generate the partial products, while the right side additionally shows the shifts applied to each value of the partial products. The equations below summarize these calculations.

[0108] Figure 38 shows an example of the first step of the partial product calculation of Figure 37. As shown, the MSB of the 3-bit calculation is performed using the MASAR column 102. When this bit is added to the result, it is shifted left by two positions and also has its sign reversed.

[0109] Figure 39 shows an example of the second step of the partial product calculation of Figure 37. As shown, the middle bit of the 3-bit calculation is performed using MASAR column 102. This bit is shifted left by one position when it is added to the result.

[0110] Figure 40 shows an example of the third step of the partial product calculation of Figure 37. As shown, the LSB of the 3-bit calculation is performed using the MASAR column 102. This bit does not require any shifting when added to the result.

[0111] Although exemplary embodiments have been described above, these embodiments are not intended to describe all possible configurations encompassed by the claims. It should be understood that the terms used herein are terms of description rather than limitation, and that various changes can be made without departing from the spirit and scope of the present disclosure. As noted above, features of various embodiments can be combined to form additional embodiments of the present disclosure that may not be explicitly described or illustrated. While various embodiments have been described as offering advantages over other embodiments or prior art implementations or as being preferred with respect to one or more desired characteristics, those skilled in the art will recognize that, depending on the particular application and implementation, compromises in one or more characteristics or characteristics may be made to achieve overall desired system attributes. These attributes may include, but are not limited to, cost, strength, durability, life cycle cost, marketability, appearance, packaging, size, maintainability, weight, manufacturability, ease of assembly, and the like. Thus, to the extent that any embodiments are described as less desirable than other embodiments or prior art implementations with respect to one or more characteristics, these embodiments do not fall outside the scope of the present disclosure and may be desirable for particular applications.

Claims

1. A multiply-accumulate successive approximation (MASAR) sequence, a plurality of MASAR cells each including a multiplier configured to perform digital multiplication between an input activation received for an input and an operand to calculate a result, and a unit capacitor configured to store the result as an analog charge; and digital logic configured to perform an analog summation of the analog charges on the unit capacitors of the plurality of MASAR cells to determine a digital output of the multiplication by configuring the unit capacitors as a capacitive digital-to-analog converter (CDAC) in a successive approximation register (SAR) analog-to-digital converter (ADC).

2. The MASAR sequence of claim 1 , wherein the operands specify weights or biases of a neural network.

3. The MASAR column of claim 1 , wherein each of the plurality of MASAR cells further comprises a memory configured to maintain the operand.

4. 4. The MASAR column of claim 3, wherein each memory has a memory input and a memory output, and each of the plurality of MASAR cells further includes a column select control line connected to the memory, the memory being configured to utilize the value on the column select control line to switch between (i) storing a value on the memory input as an operand in the memory, and (ii) applying a value in the memory to the multiplier from the memory output.

5. 2. The MASAR column of claim 1, wherein the plurality of MASAR cells are configured to receive the operands from an input of the operand separate from an input of the input activation.

6. each of the plurality of MASAR cells further includes a multiplexer (MUX) having at least first and second MUX inputs and a MUX output, the MUX configured to receive a result on the first MUX input, receive a bit guess input from the digital logic on the second MUX input, and apply the MUX output to the unit capacitor; 2. The MASAR array of claim 1, wherein the MUX is further configured to be controlled by an enable MAC control line to select between (i) storing the result in the unit capacitor and (ii) utilizing the unit capacitor to determine the analog sum of the charges.

7. 7. The MASAR string of claim 6, wherein the digital logic is further configured to utilize a SAR to convert the analog charge to a digital result by controlling the unit capacitance of individual MASAR cells via the bit guess inputs to form the CDAC.

8. 8. The MASAR sequence of claim 7, wherein the SAR comprises extrapolating bits of the digital output of the multiplication from most significant bit to least significant bit.

9. further comprising a comparator having a comparator input and a comparator output, each of the unit capacitors being connected to the comparator input via a common bit line, and the digital logic being configured to receive the comparator output; the common bit line is connected to a switch controllable by a RESET line, such that when the RESET line is set, the common bit line is connected to a reference voltage, and when the RESET line is de-asserted, the common bit line is de-asserted from the reference voltage; 2. The MASAR array of claim 1, wherein the RESET line is set when performing a digital multiplication, and the RESET line is unset when performing the analog summation of the analog charges.

10. 2. The MASAR sequence of claim 1, wherein the digital output utilizes N+1 bits for signed integer arithmetic and N bits for two's complement arithmetic.

11. The digital output is N ＢＧ a bit value, and the plurality of MASAR cells are at least 2 Ｎｒ The MASAR array of claim 1 comprising cells.

12. 12. The MASAR string of claim 11, wherein the digital logic is configured to control the unit capacitance of each of the MASAR cells via bit speculation inputs to form the capacitive digital-to-analog converter (CDAC).

13. The bit guess input is M=0:N ＢＧ -1 to N ＢＧ bit width, and each bit estimation line M is Ｍ 13. The MASAR array of claim 12, wherein the MASAR array is connected to:

14. The bit guess input is M=0:MAX(N ＢＧ )-X to M<MAX(N ＢＧ ) bit width, and each bit inference line M is Ｍ＋Ｘ 13. The MASAR array of claim 12, wherein said analog input is connected to said input of said first analog input, thereby providing a coarse mapping of the analog charge on said unit capacitors to determine a digital output.

15. The MASAR column of claim 12 , wherein bit speculation lines are connected to spatially randomized sets of the plurality of MASAR cells across the MASAR column.

16. 13. The MASAR string of claim 12, wherein a first subset of the plurality of MASAR cells is connected to the bit guess input for ADC conversion, and a second subset of the MASAR cells is connected to a reference voltage to perform conversion range shifting.

17. 17. The MASAR string of claim 16, wherein the first subset of the plurality of MASAR cells comprises least significant bits (LSBs) of the digital output and the second subset of the MASAR cells comprises most significant bits (MSBs) of the digital output, thereby providing a range-shifted full resolution mapping of a subset of the range of values ​​of the MASAR string.

18. 17. The MASAR string of claim 16, wherein the first subset of the plurality of MASAR cells comprises MSBs of the digital output and the second subset of the MASAR cells comprises LSBs of the digital output, thereby providing a coarse resolution mapping of a full range of values ​​for the MASAR string.

19. 17. The MASAR string of claim 16, wherein the first subset of the plurality of MASAR cells comprises MSBs and LSBs of the digital output, and the second subset of the MASAR cells comprises the remaining bits of the digital output, thereby providing a range-shifted coarse resolution mapping of a subset of the range of values ​​of the MASAR string.

20. A MASAR sequence, a plurality of MASAR cells, each of the MASAR cells comprising: a multiplier configured to perform digital multiplication between the input activations received for the input and an operand to calculate a result; a unitary capacitor configured to store the result as an analog charge; a multiplexer (MUX) having at least first and second inputs and an output, said MUX configured to receive a result on a first input, receive a bit guess input from the digital logic on a second input, and apply an output to said unit capacitor; The MASAR string includes digital logic configured to perform analog summation of the analog charges of the unit capacitors of a plurality of MASAR cells by controlling the unit capacitance of each MASAR cell via the bit guess input to form a CDAC, and to utilize the SAR to determine the digital output of a MAC; and the MASAR string includes a comparator having a comparator input and a comparator output, each of the unit capacitors is connected to a comparator input via a common bit line, the digital logic is configured to receive the comparator output, the common bit line is connected to a RESET switch controllable by a RESET line; the MUX is further configured to be controlled by an enable MAC control line to select between (i) storing the result in the unit capacitor and (ii) utilizing the unit capacitor to determine the analog sum of the charges; and the RESET switch is configured to be controlled to select between (i) connecting the common bit line to a reference voltage and (ii) disconnecting the common bit line from the reference voltage.

21. For charge storage in MAC mode operation, the enable MAC control line is set to store a result in the unit capacitor and the RESET switch is set to connect the unit capacitor to the reference voltage; For charge addition in MAC mode operation, the enable MAC control line is set to store a result in the unit capacitor and the RESET switch is unset to disconnect the unit capacitor from the reference voltage; 21. The MASAR array of claim 20, wherein in an ADC conversion mode, the enable MAC control line is set to connect the bit guess input of the digital logic to the unit capacitor, and the RESET switch is unset to disconnect the unit capacitor from the reference voltage.

22. 22. The MASAR column of claim 21 , wherein each of the plurality of MASAR cells further includes a memory having a memory input and a memory output, the memory configured to maintain the operand and a column select control line connected to the memory, the memory configured to utilize the value on the column select control line to switch between (i) storing a value on the memory input as an operand in the memory, and (ii) applying a value in the memory to the multiplier from the memory output.

23. 1. A method for performing multiplication and multiply-accumulate functions using a plurality of MASAR cells and digital logic, comprising: utilizing a multiplier of each of the plurality of MASAR cells to perform a digital multiplication between an input activation received for an input of the respective MASAR cell and an operand to calculate a result; storing the result of the digital multiplication as an analog charge in a unit capacitor of each MASAR cell; and performing an analog summation of the analog charges on the unit capacitors of the plurality of MASAR cells under control of digital logic to determine a digital output of the multiplication by configuring the unit capacitors as a capacitive digital-to-analog converter (CDAC) in a successive approximation register (SAR) analog-to-digital converter (ADC).

24. 24. The method of claim 23, further comprising controlling a MUX via an enable MAC control line to select between (i) storing the result in the unitary capacitor and (ii) utilizing the unitary capacitor to determine the analog sum of the charges.

Images