Mixed-signal circuits for bitwise multiplication with different precisions.
Patent Information
- Application Number
- JP2024513738
- Authority / Receiving Office
- JP · JP
- Patent Type
- Applications
- Current Assignee / Owner
- Priority Date
- 2021-09-14
- Filing Date
- 2022-09-13
- Publication Date
- 2024-11-08
- Estimated Expiration
- 2042-09-13
AI Technical Summary
Analog-to-digital conversion (ADC) energy is a limiting factor in mixed-signal circuit design, particularly in performing bitwise multiplications with different precisions, which affects the efficiency of neural network computations.
A mixed-signal multiply-accumulate (MS-MAC) circuit is implemented, combining digital and analog processing to perform bitwise multiplications with varying precisions by using different architectures for dot product engines based on their noise contribution, ensuring higher precision for critical calculations and lower precision for less critical ones.
This approach reduces computational noise and power consumption in neural network accelerators by optimizing bitwise multiplications, enhancing the accuracy and efficiency of mixed-signal circuits.
Smart Images

Figure 00000000_0000_ABST
Abstract
Description
[Technical field]
[0001] The field relates to electronics, and more specifically to mixed-signal circuits. Summary of the Invention
[0002] Mixed-signal circuits operate in both analog and digital signal domains. Therefore, analog-to-digital conversion (ADC) is usually required to perform at least some calculations. However, ADC conversion energy can be a limiting factor in mixed-signal circuit design. SUMMARY OF THE PRESENT EMBODIMENTS Embodiments of the present invention provide improved techniques for performing different parts of a bitwise multiplication with different precisions in mixed-signal circuits.
[0003] In one embodiment, an apparatus includes at least one processor and at least one memory including instruction code configured to cause the apparatus, by the at least one processor, to at least perform a first portion of a bitwise multiplication of a first and second digital input with a first precision and to perform at least a second portion of the bitwise multiplication of the first and second digital inputs with a second precision different from the first precision.
[0004] In another embodiment, a method of performing a multiply-accumulate operation includes performing a first portion of a bitwise multiplication of first and second digital inputs in a first precision and performing at least a second portion of the bitwise multiplication of the first and second digital inputs in a second precision different from the first precision, the method being performed by a processing circuit configured to execute instruction code.
[0005] In another embodiment, an article of manufacture includes a non-transitory computer readable storage medium having executable instruction code embodied thereon that, when executed by a processor, causes the processor to perform: performing a first portion of a bitwise multiplication of first and second digital inputs in a first precision; and performing at least a second portion of the bitwise multiplication of the first and second digital inputs in a second precision, different from the first precision.
[0006] In another embodiment, a system includes a multiply-accumulate unit configured to perform binary multiplication of a first and second vector. The multiply-accumulate unit includes a set of two or more bitwise dot-product summation units each configured to perform bitwise multiplication of a portion of the first and second vectors to generate a digital output. Each bitwise dot-product summation unit of the set of two or more bitwise dot-product summation units includes at least one instance of a bitwise dot-product summation unit coupled to an instance of a successive approximation register analog-to-digital converter unit, the successive approximation register analog-to-digital converter unit configured to convert an analog output of the dot-product summation unit to at least a portion of the digital output. A first subset of the set of two or more bitwise dot-product summation units having a first precision is used to calculate a first portion of the bitwise multiplication of the first and second vectors, and a second subset of the set of two or more bitwise dot-product summation units having a second precision different from the first precision is used to calculate a second portion of the bitwise multiplication of the first and second vectors.
[0007] In another embodiment, a device includes a multiply-accumulate circuit configured to perform binary multiplication of a first and second vector. The multiply-accumulate circuit includes a set of two or more instances of a bitwise dot-product summation circuit, each configured to perform a bitwise multiplication of a portion of the first and second vectors to generate a digital output. Each instance of the bitwise dot-product summation circuit of the set of two or more instances of the bitwise dot-product summation circuit includes at least one instance of a bitwise dot-product computation circuit coupled to at least one instance of a successive approximation register analog-to-digital converter circuit, the successive approximation register analog-to-digital converter circuit configured to convert an analog output of the bitwise dot-product computation circuit to at least a portion of the digital output. A first subset of the set of two or more instances of the bitwise dot-product summation circuit having a first precision is used to compute a first portion of the bitwise multiplication of the first and second vectors, and a second subset of the set of two or more instances of the bitwise dot-product summation circuit having a second precision different from the first precision is used to compute a second portion of the bitwise multiplication of the first and second vectors.
[0008] These and other features and advantages of the embodiments described herein will become more apparent from the accompanying drawings and the following detailed description. [Brief description of the drawings]
[0009] [Figure 1A] FIG. 1 illustrates a neural network accelerator in which one or more exemplary embodiments may be implemented. [Figure 1B] FIG. 1 illustrates a mathematical representation of the computations performed in a neural network accelerator in which one or more exemplary embodiments may be implemented. [Diagram 2] FIG. 2 illustrates a mixed signal multiply-accumulate circuit in which one or more exemplary embodiments may be implemented. [Figure 3A] FIG. 1 illustrates a dot product engine in which one or more exemplary embodiments may be implemented. [Figure 3B] FIG. 1 illustrates a dot product engine in which one or more exemplary embodiments may be implemented. [Figure 4] FIG. 1 illustrates an algebraic expression associated with a dot product calculation in which one or more exemplary embodiments may be implemented. [Diagram 5] FIG. 13 illustrates a plot of the influence coefficient of bit position on the additive noise contribution of different terms in a dot product calculation in accordance with an example embodiment. [Figure 6] FIG. 1 illustrates a mixed signal multiply-accumulate circuit using different hardware architectures for different dot product engines in accordance with an example embodiment. [Figure 7] FIG. 2 illustrates an architecture of a dot product engine in accordance with an example embodiment. [Figure 8] FIG. 1 illustrates a mixed signal multiply-accumulate circuit using different hardware architectures for different dot product engines in accordance with an example embodiment. [Figure 9] FIG. 2 illustrates a method for performing a low-noise mixed-signal multiply-accumulate operation according to an example embodiment. [Figure 10] FIG. 1 illustrates an example implementation of an artificial intelligence system in accordance with an illustrative embodiment. [Figure 11] FIG. 1 illustrates an exemplary processor system according to an exemplary embodiment. [Figure 12] FIG. 1 illustrates a cloud computing environment in accordance with an exemplary embodiment. [Figure 13] FIG. 1 illustrates an abstraction model layer in accordance with an exemplary embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS
[0010] It should be understood that the various features illustrated in the accompanying drawings are schematic views that are not drawn to scale. Moreover, the same or similar reference numerals are used throughout the drawings to denote the same or similar features, elements, or structures, and thus detailed descriptions of the same or similar features, elements, or structures will not be repeated for each drawing. Moreover, the terms "exemplary" and "illustrative" as used herein mean "serving as an example, instance, or illustration." Any embodiment or design described herein as "exemplary" or "illustrative" should not be construed as preferred or advantageous over other embodiments or designs.
[0011] Furthermore, when the phrase "configured to" is used in connection with a circuit (the term "circuitry" may be used interchangeably herein), structure, element, or component, etc. that performs one or more functions or otherwise provides some functionality, it should be understood that the phrase is intended to encompass embodiments in which the circuit, structure, element, or component, etc. is implemented in hardware, software, and / or combinations thereof, as well as implementations that include hardware, which may include discrete circuit elements (e.g., transistors, inverters, logic gates, etc.), programmable elements, processing devices, one or more integrated circuits or other types of circuitry, or both, and / or combinations thereof. Thus, by way of example only, when a circuit, structure, element, component, etc. is defined as being configured to provide a particular function, it is intended to encompass embodiments that include an element, processing device, integrated circuit, or other type of circuit, or both, that enables the circuit, structure, element, component, etc. to perform the particular function when it is in an operational state (e.g., when connected to or otherwise deployed in a system, when powered on, when receiving input, or when generating output, or a combination thereof), and to encompass embodiments when the circuit, structure, element, component, etc. is in a non-operational state (e.g., when not connected to or otherwise deployed in a system, when not powered on, when not receiving input, or when not generating output, or a combination thereof) or when in a partially operational state.
[0012] Neural network models are one of the most widely used types of machine learning (ML) algorithms in artificial intelligence systems. For example, neural network models can be used to recognize underlying relationships and / or patterns in a set of data through a process that mimics the way the human brain works. A neural network model is first trained with a training data set (training phase), and then the trained neural network model is used to recognize relationships and / or patterns in a target data set (inference phase).
[0013] Many workloads implementing neural network models require a large number of matrix multiplications, which typically involve multiply-accumulate (MAC) operations. It has been recognized that accumulation operations are generally energy inefficient when performed in the digital signal domain, but are significantly more energy efficient in the analog signal domain. Thus, it has been further recognized that more efficient multiply-accumulate operations can be achieved by combining digital signal domain techniques for the multiplications and analog signal domain techniques for the accumulations. Circuits that combine analog and digital processing to perform MAC operations are called mixed signal MAC (MS-MAC) circuits.
[0014] FIG. 1A illustrates a neural network accelerator 100 in which one or more exemplary embodiments may be implemented. In some exemplary embodiments, the neural network accelerator 100 is implemented in one or more graphics processing units (GPUs), one or more field programmable gate arrays (FPGAs), one or more application-specific integrated circuits (ASICs), one or more multi-core central processing units (CPUs), or other types of circuitry, or combinations thereof. By way of example, the neural network accelerator 100, or a portion thereof, may include one or more of the above devices customized for a particular purpose or otherwise configured for a particular purpose, including circuitry (e.g., circuits, processors, memories, etc.) programmed with executable program code (e.g., instruction code, computer program code, etc.). By way of example only, the particular purpose may be the implementation and execution of an artificial intelligence system (e.g., machine learning algorithms). In addition, the neural network accelerator 100 may be part of a system-on-chip (SoC), such as a neuromorphic computer system, but may also be used in a variety of other applications that may benefit from such artificial intelligence based decision making.
[0015] As shown, the neural network accelerator 100 includes a matrix multiplication engine 102 configured to perform calculations corresponding to the convolutional and fully connected layers of a neural network, which may typically be performed using low precision integer arithmetic on data from a memory 104. The results are then transferred via an output first-in-first-out (FIFO) data structure 106 to a special function unit 108, which performs calculations corresponding to one or more activation functions, such as, but not limited to, batch normalization, a sigmoid function, a rectified linear unit (ReLU) function, and a SoftMax function. The calculations in the special function unit 108 typically need to be performed with high precision to preserve the precision of the neural network that is mapped to the accelerator.
[0016] However, it is recognized that matrix multiplication represents a large portion of the computations in a neural network workload, and thus the matrix multiplication engine 102 may consume a significant percentage of the total power consumed by the neural network accelerator 100. It is further recognized that the low precision integer computations performed in the matrix multiplication engine 102 may be performed by operating at least partially in the analog domain, resulting in an accelerator that has lower total power consumption as compared to accelerators that perform all of their computations in the digital domain using digital circuitry.
[0017] 1B illustrates a mathematical representation 110 according to an exemplary embodiment for enabling a vector dot product (also referred to herein as an inner product, as will be further described) to be calculated by one or more multiply-add operations in the neural network accelerator 100 of FIG. 1A. In the example of FIG. 1B, the equation on the left mathematically represents the dot product of two vectors, x and w, each of size 64, where the elements x i and w iEach of the has 4 bits. Each 4-bit by 4-bit product (4b*4b) is divided into 16 1-bit by 1-bit products (1b*1b). The equation on the right represents the 1b*1b multiplication mathematically, where x i,m and w i,n Each of has one bit.
[0018] First, bitwise products are performed and then summation is performed based on the weight of the bits. Essentially, 4-bit by 4-bit terms are split into 16 1-bit products, which are accumulated (summed) and then appropriately scaled by powers of 2 by the outer summation to generate the total sum. Based on power considerations, the 1b*1b multiplications are performed in the digital domain, the inner summation of 1b*1b multiplications is performed in the analog domain, and the outer summation is performed in the digital domain. The inner summation is performed in the analog domain because here analog circuits perform better in terms of power than digital circuits. The 16-way addition may be performed in the digital domain since there is little advantage to doing it in the analog domain. In one example embodiment, the inputs and outputs of the inner summation are digital.
[0019] 1B represents one exemplary embodiment, and other embodiments may use different divisions of analog and digital domain calculations. It should also be appreciated that the dot product calculation shown as mathematical representation 110 is intended as an example for ease of explanation. Thus, the element x i and w i Implementations of the dot product of vectors x and w when x, w, and w have different numbers of bits and one vector has a different number of elements compared to the other vector are supported by alternative embodiments based on the teachings provided herein.
[0020] 2 illustrates a mixed signal multiply-accumulate circuit 200 for performing integer matrix multiplication according to an example embodiment. Of course, some or all of the mixed signal multiply-accumulate circuit 200 may be implemented as part of the matrix multiplication engine 102 of FIG. 1 in some embodiments. As shown, each bit x of a first digital input X is i and the digital weight w of the second digital input W. i and a corresponding bit representing , are input to one of sixteen one-bit (1-b) dot-product engines 210-1, 210-2..., 210-15, 210-16 (collectively referred to herein as dot-product engines 210). As explained further below, the dot-product engines 210 may be referred to as counters because they effectively count the number of ones that occur in a binary multiplication operation. In one example, the first digital input X represents an activation function represented by a four-bit element, while the second digital input W represents a weight represented by a four-bit element. The dot-product engines 210-1, 210-2..., 210-15, 210-16 output digital partial sums 220-1, 220-2..., 220-15, 220-16 (collectively referred to herein as partial sums 220), respectively. The digital partial sums 220 are input to a digital reduction circuit 230 which performs an outer summation to generate the MS-MAC output.
[0021] 3A and 3B show an example of an MS-MAC circuit configured for dot-product summation performed by the dot-product engine 210 according to an exemplary embodiment. In the exemplary embodiment of FIG. 3A and FIG. 3B, it is assumed that each of the input vectors X and W includes 63 elements, with each element including 4 bits. In general, the MS-MAC circuit performs multiplication in the digital signal domain using digital logic and accumulation in the analog domain using charge-sharing capacitors. The MS-MAC circuit effectively multiplies the digital inputs in the digital domain, accumulates the results of the multiplication in the analog domain, and generates a binary weighted digital code based on the accumulated results. The number of bits of the binary weighted digital code is referred to as the analog resolution or quantization level (p) of the analog-to-digital conversion operation performed by the MS-MAC circuit. To generate a unique binary weighted code for a set of 63-bit inputs, p is 6, i.e., D[0:5]. Throughout FIG. 3A and FIG. 3B, circuit elements providing the same or similar functionality use the same or similar reference numbers.
[0022] 3A, the MS-MAC circuit 300 includes an analog dot-product engine 302 and a successive approximate register (SAR) ADC 304. As shown, the analog dot-product engine 302 outputs an output V IN where the output V IN is the sum of weights generated by the analog dot product engine 302, Σx i w i The analog dot product engine 302 receives inputs x[62:0] and w[62:0] and outputs V IN ∝Σx i w iThe output of the comparator 310 is input to the SAR controller 306. The SAR controller 306 outputs a binary weighted code D[0:5], which is provided to a set of capacitors 312 (having capacitance values C, 2C, 4C, 8C, 16C, 32C, respectively). Of course, each bit of the binary weighted code D[0:5] is coupled to a set of capacitors 312 (e.g., from most significant bit (MSB) to least significant bit (LSB)). For 6-bit quantization, the MSB (D5) of the binary weighted output of the SAR controller 306 is coupled to a 32C capacitor, the next bit (D4) is coupled to a 16C capacitor, the next bit (D3) is coupled to an 8C capacitor, the next bit (D2) is coupled to a 4C capacitor, the next bit (D1) is coupled to a 2C capacitor, and the LSB (D0) is coupled to a 1C capacitor. As shown, the binary weighted code D[0:5] output from the SAR controller 306 is provided to the bottom plate of a capacitor 312 through a set of buffers and switches 314. The SAR ADC 304 also includes a reset switch 316, which when opened allows the top plate of the capacitor 312 to electrically float.
[0023] 3B illustrates an MS-MAC circuit 320 that illustrates an example implementation of the analog dot-product engine 302. As shown in FIG. 3B, the analog dot-product engine 302 includes a set of 63 AND gates 322, each of which performs a multiplication of a corresponding set of inputs x[62:0] and w[62:0]. The outputs of the AND gates 322 (i.e., x0*w0, x1*w1, ..., x 62 *w 62 ) is coupled through a reset switch 324 to the bottom plate of a capacitor 326, each of which has the same capacitance value C. The analog dot product engine 302 further includes a reset switch 328. The top plate of the capacitor 326 provides an output V IN ∝Σxi w i to provide.
[0024] In an exemplary embodiment, the dot product in the mixed signal domain
[0025]
number
[0026] is calculated, where X i and W i may be low precision (e.g., a 4-bit integer INT4). As described elsewhere herein, computing the dot product has advantages over fully digital implementations from a power perspective. However, mixed-signal computations are susceptible to noise and bit errors. Exemplary embodiments described herein provide techniques for making such mixed-signal domain computations more robust to noise.
[0027] 4 illustrates an algebraic expression 400 associated with the dot product calculations performed by the dot product engine 210. The dot product engine 210, illustratively implemented as a pop counter in the analog domain, counts the number of ones (non-zeros) out of 64 bitwise multiplications. This count for 64 bitwise multiplications may be expressed as:
[0028]
number
[0029] Here, a binary number (the product) is decomposed into multiple bit positions. After decomposition, 16 terms are generated, each of which is calculated by an analog pop counter (e.g., one of the dot product engines 210). The final output is calculated in the digital backend (e.g., by a shifter and an adder).
[0030]
number
[0031] The calculation of is illustratively performed in the analog domain,
[0032]
number
[0033] The calculation of is performed in the digital domain. As shown in FIG. 4, each "box" may be replaced by an analog pop counter (e.g., one of the dot product engines 210). Thus,
[0034]
number
[0035] may be implemented via a dot product engine 210-1,
[0036]
number
[0037] may be implemented via a dot product engine 210-2,
[0038]
number
[0039] may be implemented via a dot product engine 210-3, ...
[0040]
number
[0041] The term may be implemented using the dot product engine 210-16.
[0042] More specifically, consider that there are 64 X's at input 202 and 64 W's at input 204, and since each X and W has 4 bits of precision, 64 multiplications (e.g., products) are performed and all of those products are added together. Further, consider a 1 out of 64 multiplication, called X by W (where the capital letters X and W mean that each of these are represented by a 4-bit binary number). X may be rewritten as 8a+4b+2c+1d, and W may be rewritten as 8p+4q+2r+1s. Thus, X by W is: X*W=(8a+4b+2c+1d)(8p+4q+2r+1s) X*W=64ap+32bp+16cp+8dp+32aq+16bq+8cq+4dq+16ar+8br+4cr+2dr+8as+4bs+2cs+1ds X*W=64ap+32(bp+aq)+16(cp+bq+ar)+8(dp+cq+br+as)+4(dq+cr+bs)+2(dr+cs)+1ds Thus, we see that different terms are multiplied by coefficients 64, 32, 16, ..., 1, and there are some overlaps. For example, the three terms cp, bq, and ar are multiplied by the same coefficient 16. For binary numbers, 16 (=2 4 ) may be accomplished by a simple 4-bit shift. Thus, multiplication by coefficients 64, 32, 16, 8, 4, 2, and 1 may be represented as <<6, <<5, <<4, <<3, <<2, <<1, and <<0 (where "<<" means a left shift). This is shown in digital partial sums 220 discussed above with respect to Figures 2, 3A, and 3B (e.g., digital partial sums 220-1 is <<6, digital partial sums 220-2 and 220-3 are <<5, digital partial sums 220-4 through 220-6 are <<4, digital partial sums 220-7 through 220-10 are <<3, digital partial sums 220-11 through 220-13 are <<2, digital partial sums 220-14 and 220-15 are <<1, and digital partial sum 220-16 is <<0).
[0043] Each analog pop counter (e.g., dot product engine 210) produces an equal probability of a bit error rate (BER) regardless of bit position. The pop counter with the highest weighted sum of products (e.g., the term with influence coefficient 64)
[0044]
number
[0045] When a bit error occurs in, its impact is the pop counter with the lowest weighted sum of products (i.e., the term with an impact factor of 1).
[0046]
number
[0047] 2, the highest weighted multiply-and-sum is associated with 1-b dot-product engine 210-1 and digital partial sum 220-1, and the lowest weighted multiply-and-sum is associated with 1-b dot-product engine 210-16 and digital partial sum 220-16.
[0048] 5 shows a table 500 that lists the impact coefficients of each term (e.g., each term of algebraic expression 400 of FIG. 4) along with the count of their respective occurrences. As discussed above, due to the characteristics of successive approximation register (SAR) analog-to-digital converters (ADCs) that may be used to implement dot product engine 210, the error from each analog pop counter may only be on the order of + / -1.
[0049] X i Let be the additive noise in the i-th term, then the total noise is determined according to the following formula: η=X1+2X2+2X3+4X4+4X5+4X6+8X7+8X8+8X9+8X 10 +16X11 +16X 12 +16X 13 +32X 14 +32X 15 +64X 16 Where:
[0050]
number
[0051] where the error E is determined according to the following formula: E[X]=0, Var[X i ]=E[X 2 ]-(E[X]) 2 =BER Here, Var[X i ] is X i where E[η]=0 and BER is the bit error rate.
[0052]
number
[0053] The higher weighted terms (e.g., X 16 , X 15 , X 14 ) has a very large effect on the total noise. For example, X 16 , X 15 , and X 14 Consider the case where each of is reduced by 100x. In this case, the formula becomes:
[0054]
number
[0055] Here the last three additive noise terms are reduced by 100x and therefore can be considered to have negligible contribution.
[0056] Thus, to make the MAC engine more robust against noise, some embodiments use different architectures for different dot product engines 210 (e.g., different 1-b multiplication pop counters). These different architectures provide different accuracies to each other. A first architecture having a higher accuracy (and thus less noise) is used for some dot product engines 210 (e.g., those corresponding to higher weighted terms), and a second architecture having a relatively lower accuracy is used for other dot product engines 210 (e.g., those corresponding to relatively lower weighted terms), whereby the overall computational accuracy may increase.
[0057] In the following description, it is assumed that only the following two different types of architectures are used. That is, the pop counters associated with the three highest weighted terms (e.g., in the above example, the dot product engines 210-1, 210-2, and 210-3 corresponding to the higher weighted terms X 16 , X 15 , and X 14 ), and a second type of architecture for the remaining pop counters (e.g., the dot product engines 210-4 to 210-16 corresponding to the relatively lower weighted terms X 13 ~X1), where the first architecture provides a higher accuracy than the second architecture. However, it should be recognized that in other implementations, it is also possible to use more than three different types of architectures (e.g., those having different relative accuracies) for different subsets of pop counters. For example, in some embodiments, the following three different types of architectures are used. That is, the pop counters associated with the three highest weighted terms (e.g., in the above example, the higher weighted terms X 16 , X 15 , and X 14A first type of architecture for the dot product engines 210-1, 210-2, and 210-3 corresponding to the next three highest weighted terms (e.g., the middle weighted term X in the above example) 13 , X 12 , and X 11 , and the remaining pop counters (e.g., the relatively lower weighted terms X 10 .X1), where the first architecture provides greater precision than the second architecture, which in turn provides greater precision than the third architecture. Various other examples are possible with different pop counters using different architectures, using different number type architectures providing different precision.
[0058] Different types of architectures may provide different accuracy (e.g., noise improvement or BER) in various ways, including, but not limited to, improving accuracy by using a lower resolution pop counter to improve the signal to noise ratio (SNR) by effectively increasing the minimum voltage step; using larger capacitors with lower thermal noise (kT / C) and lower capacitor mismatch; using lower noise comparators (e.g., where the comparator noise may be dominated by the thermal noise of the transistors in the comparator); using more precise offset cancellation in the comparator; combinations thereof, etc.
[0059] 6 illustrates a mixed signal multiply-accumulate circuit 600 according to an example embodiment, which is similar to mixed signal multiply-accumulate circuit 200 and can be used to perform integer matrix multiplication. Of course, in some embodiments, some or all of mixed signal multiply-accumulate circuit 600 may be implemented as part of matrix multiplication engine 102 of FIG. 1. As shown, each bit x of a first digital input X is i and the digital weight w of the second digital input W. i and a corresponding bit representing , are input to one of sixteen one-bit (1-b) dot-product engines 610-1, 610-2..., 610-15, 610-16 (collectively referred to herein as dot-product engines 610). The dot-product engines 610, like the dot-product engine 210, may be referred to as counters because they effectively count the number of ones that occur in a binary multiplication operation. In one example, the first digital input X represents an activation function represented by a four-bit element, while the second digital input W represents a weight represented by a four-bit element. The dot-product engines 610-1, 610-2..., 610-15, 610-16 output digital partial sums 620-1, 620-2..., 620-15, 620-16 (collectively referred to herein as partial sums 620), respectively. The digital partial sums 620 are input to a digital reduction circuit 630 which performs an outer summation to generate the MS-MAC output.
[0060] While each dot-product engine 210 in the mixed-signal multiply-accumulate circuit 200 of Figure 2 is assumed to use the same architecture, different dot-product engines 610 in the mixed-signal multiply-accumulate circuit 600 of Figure 6 use different architectures. More specifically, continuing the example above, three dot-product engines 610-1, 610-2, and 610-3 (associated with term weights x64, x32, and x32, respectively) use a first architecture, while the remaining dot-product engines 610-4 through 610-16 (associated with term weights x16 through x1) use a second architecture, where the first architecture provides higher precision than the second architecture. Such different precisions may be implemented using various techniques as described above.
[0061] In some embodiments, different precision for different architectures may be achieved by introducing quantization noise into the architecture with the lower precision. FIG. 7 shows an example implementation of an MS-MAC circuit 700 (e.g., a pop counter) connected to an additional capacitor bank 720. The MS-MAC circuits 700 and 720 together may be referred to as implementing a "split" pop counter. The dot product engine 700 includes a SAR controller 706 and a comparator 710. The SAR controller 706 calculates the sum of x0, w0 to x 62 , w 62 The MS-MAC circuit 700 controls a set of sub-cell controllers 762-1 through 762-63 (together, sub-cell controller 762) that receive a respective set of inputs from the SAR controller 706. The sub-cell controllers 762 may implement digital (logical) AND gates with outputs coupled to a set of switch pairs 764-1 through 764-62 (together, switch pair 764). As will be further explained, the switch pairs 764 enable the MS-MAC circuit 700 to switch between a summation phase of operation and a SAR phase of operation via control inputs generated by the SAR controller 706.
[0062] Switch pair 764 is connected to the bottom plate of each of a set of capacitors 766-1, ..., 766-63, which are together referred to as capacitors 766. Each capacitor 766 has the same capacitance C0 = ... = C 63 A top plate of the capacitor 766 is connected to a first input of the comparator 710 and to a pre-charge switch 768. As will be further explained, the pre-charge switch 768 couples a common-mode voltage V CM The second input of the comparator 710 is V CM Connected to the voltage V CM is set to the natural common mode voltage of the comparator 710, which is the voltage at which the comparator 710 is believed to have the best sensitivity / noise characteristics. The output of the comparator 710 is connected to the SAR controller 706. The SAR controller 706 outputs a binary weighted code D[0:5]. Although not explicitly shown in FIG. 7, the data input and the control input may be buffered through different buffers. To synchronize the timing between the SAR controller 706 and the comparator 710, a clock signal (clk, not explicitly shown) may be used, which may be buffered by a clock buffer (not explicitly shown).
[0063] As shown, the MS-MAC circuit 700 is coupled to an additional capacitor bank 720, which has a capacitance of x 63 , w 63 x 126 , w 126The AND / NAND gates 763 include a set of digital (logical) AND or NAND gates 763-64, ... 763-127 (together AND / NAND gates 763) that receive a respective set of inputs. The AND / NAND gates 763 have outputs coupled to a set of switch pairs 765-64 through 765-127 (together switch pair 765). The switch pair 765 is coupled to the bottom plates of capacitors 767-64, ... 767-127 (together capacitors 767). The top plates of the capacitors 767 are coupled via switch 770 such that they contribute to the voltage at the common node 772 when switch 770 is closed. As will be further explained, a precharge switch 769 couples a common-mode voltage V CM is selectively applied.
[0064] In MS-MAC circuit 700, both capacitors 766 and 767 are used during summation, whereas only capacitor 766 is used during SAR conversion. Capacitors 766 and 767 are controlled individually during the input summation phase, whereas capacitor 766 is controlled in a binary weighted manner during the SAR conversion operation.
[0065] During the first operating phase (sum), precharge switches 768 and 769 are closed, so that the voltage V SUM is V CM At the same time, the sub-cell controller 762 and the AND / NAND gate 763 switch the input x i , w i The product of each pair of capacitors 766 and 767 is calculated, where switches 764 and 765 control whether the bottom plate of each of the capacitors 766 and 767 is charged to a reference voltage (Vdd) or to ground (0V), depending on the logic output of the sub-cell controller 762 and the AND / NAND gate 763. Thus, each capacitor 766 stores a charge Q i =C(x i *w i *VDD -V CM ) to accumulate.
[0066] During the second operation phase (SAR), first switch 770 is opened, and then precharge switches 768 and 769 are opened. This means that the additional capacitor bank 720 is not used during the SAR operation phase. Precharge switch 768 is opened, and common node 772 connecting the top plates of capacitors 766 is allowed to electrically float. Switch pair 764 is then configured to pass the control output of SAR controller 706 through subcell controller 762, such that the output of SAR controller 706 controls the operational configuration of capacitors 766. That is, during this phase, the set of capacitors 766 is configured as an array of binary weighted capacitances, i.e., a subset of capacitors 766 is connected to predefined bit control lines, e.g., LSB to MSB, of SAR controller 706. Each bit control line corresponds to a bit of the binary weighted output (code) generated by SAR controller 706, e.g., D[0:5] for 6 quantization levels. That is, a given capacitor 766 is connected to a given bit control line when the SAR control signal closes the corresponding switch of switch pair 764. For 6-bit quantization, the MSB (D5) of the binary weighted output of the SAR controller 706 is coupled to capacitors 766-32 through 766-63 (32 capacitors), the next bit (D4) is coupled to capacitors 766-16 through 766-31 (16 capacitors), the next bit (D3) is coupled to capacitors 766-8 through 766-15 (8 capacitors), the next bit (D2) is coupled to capacitors 766-4 through 766-7 (4 capacitors), the next bit (D1) is coupled to capacitors 766-2 and 766-3 (2 capacitors), and the LSB (D0) is coupled to capacitor 766-1 (1 capacitor).
[0067] Depending on the result of the multiplication by the sub-cell controller 762 and the resulting charge stored on the set of capacitors 766, a decision is made by the comparator 710 in successive iterations for each of the output bits D[0:5] (1 or 0). Thus, six SAR iterations are performed to generate a 6-bit binary weighted output for the 127 data inputs shown in FIG.
[0068] To summarize the dot product calculation performed by the MS-MAC circuit 700 of Figure 7, in the first phase of operation (first cycle), the sub-cell controller 762 and the AND / NAND gate 763 multiply the data received at their respective inputs and their outputs drive the capacitors 766 and 767. Then in the second phase of operation (second to seventh cycles, corresponding to a 6-bit binary weighted output), normal SAR ADC operation occurs as described above.
[0069] The MS-MAC circuit 700 of FIG. 7 advantageously saves area and power associated with the comparator 710 and the SAR controller 706, and avoids large voltage swings on the capacitor 767 during the SAR ADC phase.
[0070] 8 illustrates a mixed signal multiply-accumulate circuit 800 according to an example embodiment, which is similar to mixed signal multiply-accumulate circuit 600 and can be used to perform integer matrix multiplication. Of course, in some embodiments, some or all of mixed signal multiply-accumulate circuit 800 may be implemented as part of matrix multiplication engine 102 of FIG. 1. As shown, each bit x i and the digital weight w of the second digital input W. iand corresponding bits representing , are input to one of a set of dot-product engines 810-1a, 810-1b, 810-2a, 810-2b, 810-3a, 810-3b, 810-4, 810-5, ... 810-16 (together referred to herein as dot-product engines 810). Dot-product engines 810-1a, 810-1b, 810-2a, 810-2b, 810-3a, and 810-3b provide a higher precision architecture for the three highest weighted terms, and dot-product engines 810-4, 810-5, ... 810-16 provide a lower precision architecture for the remaining lower weighted terms. 7 and described above, while a higher precision architecture is used for dot product engines 810-1a, 810-1b, 810-2a, 810-2b, 810-3a, and 810-3b. Dot product engines 810-1a and 810-1b (together dot product engine 810-1) are used for the highest weighted terms, dot product engines 810-2a and 810-2b (together dot product engine 810-2) are used for the second highest weighted terms, and dot product engines 810-3a and 810-3b (together dot product engine 810-3) are used for the third highest weighted terms.
[0071] Mixed signal multiply-accumulate circuit 800 is configured to perform 128 sums of 4-bit digital inputs 802 (X) and 804 (W). For terms scaled by {1, 2, 4, 8, 16} (e.g., the lowest weighted multiply-accumulate of 13), the "split" pop counter architecture of Figure 7 may be used in dot product engines 810-4 through 810-16. The "split" pop counter architecture used in dot product engines 810-4 through 810-16 may only count the number of ones to the nearest even number, so the output (e.g., digital partial sums 820-4 through 820-16) will be inaccurate (e.g., noisy). However, the split pop counter architecture used in dot product engines 810-4 through 810-16 is lower power than the "normal" or non-split pop counter used in dot product engines 810-1 through 810-3 for terms scaled by {32, 64} (e.g., the three highest weighted sums of products). For each of these terms, the computation is split between two pop counters that produce two results (e.g., 820-1a and 820-1b for the highest weighted sum of products, 820-2a and 820-2b for the second highest weighted sum of products, and 820-3a and 820-3b for the third highest weighted sum of products), which are summed in adder 830. For example, 820-1a is the sum of the first 63 terms for the highest weighted sum of products, and 820-1b is the sum of the last 63 terms.
[0072] In total, the mixed signal multiply-accumulate circuit 800 has 19 pop counters: six 63b pop counters used for the dot-product engines 810-1 through 810-3 associated with the three highest weighted multiply-accumulate, and thirteen 127b split pop counters used for the remaining dot-product engines 810-4 through 810-16 associated with the relatively lower weighted multiply-accumulate. The dot-product engines 810-1 through 810-3 have a 6-bit design based on 63b pop counters, with a target BER of 10. -12Dot product engines 810-4 through 810-16 sum 127 terms in a 6-bit design that uses a 63b pop counter circuit but with an additional capacitor bank as described above with respect to FIG. 7. A lower order pop counter introduces quantization noise, but is expected to save energy in the comparators, digital backend, and SAR controller. In the example of FIG. 8, dot product engines 810-1 through 810-3 use a higher precision architecture and dot product engines 810-4 through 810-16 use a relatively lower precision architecture. Power and area are saved for less critical calculations (e.g., for the relatively lower weighted sums of products of 13) while more critical calculations (e.g., the three highest weighted sums of products) are performed with higher precision.
[0073] It should again be appreciated that the particular number and arrangement of dot product engine architectures shown in Figure 8 are presented as examples only. For example, in other embodiments, a higher precision architecture may be used for terms scaled by {16, 32, 64} instead of just terms scaled by {32, 64}. Furthermore, three or more different types of architectures may be used for different pop counters or dot product engines used for three or more different subsets of terms (e.g., a first precision for terms scaled by {32, 64}, a second precision for terms scaled by {8, 16}, and a third precision for terms scaled by {1, 2, 4}). Additionally, while Figure 8 shows an example having a 127b split pop counter (e.g., for dot product engines 810-4 through 810-16) and a 63b pop counter (e.g., for dot product engines 810-1 through 810-3), this is not a requirement. The fractional pop counter used for the relatively lower weighted sums of products may be a 63b fractional pop counter (e.g., where an additional capacitor bank is added to the x 16 capacitor bank shown in the example of FIG. 7). 63 , w 63 x 126, w 126 Instead of input x 32 , w 32 x 62 , w 62 32 capacitors for 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 43, 44, 45, 46, 47, 48, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 61, 62, 63, 64, 65, 6
[0074] In some embodiments, a mixed-signal multiply-add circuit is designed that distinguishes between "important" and "unimportant" calculations in different dot-product calculation engines by sorting the dot-product calculation engines by their noise contribution to the final result and selecting different precision circuit architectures for different sets of dot-product calculation engines (e.g., using the highest precision architecture for the dot-product calculation engines that contribute the most noise to the final result and using a relatively lower precision architecture for the dot-product calculation engines that contribute less noise to the final result). The different architectures used for different subsets of dot-product calculation engines may be designed in various ways. For example, some embodiments provide higher precision by performing important bitwise dot products "exactly" (e.g., without added quantization noise) while performing less important bitwise dot products "approximately" (e.g., with added quantization noise). Advantageously, the "approximate" calculation of the less important bitwise dot products may be achieved by using a circuit that provides power or area savings or both compared to the circuit used for the "exact" calculation of the important bitwise dot products.
[0075] In other embodiments, in addition to or instead of performing critical bitwise dot products exactly (e.g., without additive quantization noise) while performing less critical bitwise dot products approximately (e.g., with additive quantization noise), the circuitry used in the dot product engine that performs the critical bitwise dot products may be designed with larger area and / or power consumption (e.g., due to larger unit capacitor size, lower noise comparators, etc.) compared to the dot product engine that performs the non-critical bitwise dot products. This ensures that the critical dot products have lower computational errors (e.g., bit errors) than the non-critical dot products (e.g., because bit errors can be introduced by thermal noise, manufacturing variations in the unit capacitors, offsets, etc.).
[0076] 9 illustrates a method 900 for performing a low noise mixed signal multiply-accumulate operation. The method 900 begins at step 902 by determining the additive noise contributions of different terms of a bitwise multiplication of a first and second digital input. For example, FIGS. 4 and 5 illustrate a method for performing a low noise mixed signal multiply-accumulate operation. i and W i9 illustrates determining the effect of additive noise contributions of different terms on 63 bitwise multiplications of a digital input. In step 904, the terms are sorted based on the determined additive noise contributions of the terms on the bitwise multiplications of the first and second digital inputs. In step 906, two or more different dot-product summation architectures are selected for use by two or more different portions of the bitwise multiplications of the first and second digital inputs. For example, a first dot-product summation architecture having a first precision may be selected for use by performing a first portion of the bitwise multiplications of the first and second digital inputs, and at least a second dot-product summation architecture having a second precision different from the first precision may be selected for use by performing a second portion of the bitwise multiplications of the first and second digital inputs. The first portion of the bitwise multiplications may include k highest weighted sums of products of the n total sums of products of the bitwise multiplications of the first and second digital inputs, while the second portion of the bitwise multiplications may include at least a portion of the relatively lower weighted sums of products of the remaining n bitwise multiplications of the first and second digital inputs. In step 908, two or more different portions of the bitwise multiplication of the first and second digital inputs are performed using the two or more different selected dot-product-sum architectures.
[0077] As described above, three or more different dot-product-sum architectures may be used for three or more different portions of the bitwise multiplication of the first and second digital inputs, and thus step 906 may include selecting three or more different dot-product-sum architectures for three or more different portions of the bitwise multiplication of the first and second digital inputs. For example, the first portion may be for the k highest weighted sum of products of the n total sum of products of the bitwise multiplication of the first and second digital inputs, the second portion may be for the next highest weighted sum of products of the l total sum of products of the n, and the third portion may be for the relatively lower weighted sum of products of the remaining n k l of the bitwise multiplication of the first and second digital inputs.
[0078] 10 illustrates an example implementation of an artificial intelligence system 1000 according to an example embodiment. As shown, the system 1000 includes a data set 1010, a neural network model 1020, and a MAC unit with low noise ADC 1030. As described elsewhere herein, the MAC unit with low noise ADC 1030 is implemented using an architecture with different precision for significant and non-significant bitwise dot product calculations to perform calculations for the neural network model 1020 using the data set 1010, which may include training the data in a training mode or inferring the data set in an inference mode.
[0079] In one exemplary embodiment, the artificial intelligence system 1000 is implemented by one or more application specific integrated circuits (ASICs). An ASIC is an integrated circuit (IC) chip or device customized for a specific purpose, including logic (e.g., circuits, processors, memory, etc.) programmed by executable program code (e.g., instruction code, computer program code, etc.) or otherwise configured for a specific purpose. The specific purpose in this exemplary case is the implementation and execution of an artificial intelligence system (e.g., machine learning algorithms). An ASIC may also be considered a system-on-chip (SoC). Some ASIC implementations that may be used by one or more exemplary embodiments use user-selectable cell libraries of basic logic functions (e.g., multiplexers, comparators, etc., including multiple VLSI transistor devices to provide various functions such as switching, comparing, etc.) to allow configuration (and reconfiguration) of the system.
[0080] It will further be appreciated that artificial intelligence system 1000, and portions thereof, may be realized in alternative circuit / processor-based technologies, such as, for example, technologies including one or more multi-core central processing units (CPUs), one or more graphics processing units (GPUs), and one or more field programmable gate arrays (FPGAs). In some embodiments, artificial intelligence system 1000 may be implemented as a combination of two or more circuit / processor-based technologies (e.g., ASICs, CPUs, GPUs, FPGAs, etc.).
[0081] As described herein, the techniques illustrated in Figures 1-10 may also include providing a system, the system including separate software modules, each of the separate software modules being implemented in a tangible computer-readable recordable storage medium. For example, all of the modules (or any subset thereof) may reside on the same medium, or each may reside on a different medium. The modules may include any or all of the components shown in the figures and / or described herein. In embodiments of the invention, the modules may be executed, for example, on a hardware processor. Thus, method steps may be performed using the separate software modules of the above-described system executed on a hardware processor. Additionally, a computer program product may include a tangible computer-readable recordable storage medium having code adapted to be executed to perform at least one method step described herein, including provisioning of the system with the separate software modules.
[0082] 1-10 may be implemented via a computer program product that may include computer usable program code stored on a computer readable storage medium of a data processing system, the computer usable program code being downloaded over a network from a remote data processing system. Additionally, in embodiments of the present invention, the computer program product may include computer usable program code stored on a computer readable storage medium of a server data processing system, the computer usable program code being downloaded over a network to the remote system for use on the computer readable storage medium by the remote data processing system.
[0083] Embodiments of the present invention or elements thereof may be implemented in the form of an apparatus including a memory and at least one processor coupled to the memory and configured to perform the exemplary method steps.
[0084] In some embodiments, an apparatus includes at least one processor and at least one memory including instruction code configured to cause the apparatus, by the at least one processor, to at least perform a first portion of a bitwise multiplication of a first and second digital input with a first precision and to perform at least a second portion of the bitwise multiplication of the first and second digital inputs with a second precision different from the first precision.
[0085] The bitwise multiplication of the first and second digital inputs may include computing a set of weighted sums of bitwise dot products of the first and second digital inputs, where a first portion of the bitwise multiplications of the first and second digital inputs includes a first subset of the set of weighted sums of bitwise dot products of the first and second digital inputs and a second portion of the bitwise multiplications of the first and second digital inputs includes a second subset of the set of weighted sums of bitwise dot products of the first and second digital inputs. The set of weighted sums of bitwise dot products of the first and second digital inputs may include n weighted sums of bitwise dot products of the first and second digital inputs, where the first portion of the bitwise multiplications performed with the first precision includes computing k highest weighted sums of the bitwise dot products of the first and second digital inputs and the second portion of the bitwise multiplications performed with the second precision includes computing at least a portion of the remaining n k weighted sums of the bitwise dot products of the first and second digital inputs. The value of k may be selected based at least in part on a determination of the additive noise contribution of different terms of the bitwise multiplication of the first and second digital inputs.
[0086] The bitwise multiplication of the first and second digital inputs may be computed using one of a set of bitwise dot product summation units, where a first precision is provided using a first type of hardware for a first subset of the set of bitwise dot product summation units that perform a first portion of the bitwise multiplication of the first and second digital inputs, and a second precision is provided using a second type of hardware, different from the first type of hardware, for a second subset of the set of bitwise dot product summation units that perform a second portion of the bitwise multiplication of the first and second digital inputs.
[0087] Each of the sets of bitwise dot product summation units may include a pop counter, the pop counters of a first subset of the sets of bitwise dot product summation units may have a first resolution and the pop counters of a second subset of the sets of bitwise dot product summation units may have a second resolution different from the first resolution.
[0088] Each of the sets of bitwise dot product summation units may include a pop counter implemented using a SAR ADC including a comparator. A first type of comparator used in the SAR ADC of the first subset of the set of bitwise dot product summation units may have a first noise profile, and a second type of comparator used in the SAR ADC of the second subset of the set of bitwise dot product summation units may have a second noise profile different from the first noise profile. The first type of comparator used in the SAR ADC of the first subset of the set of bitwise dot product summation units may additionally or alternatively have a first offset cancellation precision, and the second type of comparator used in the SAR ADC of the second subset of the set of bitwise dot product summation units may have a second offset cancellation precision different from the first offset cancellation precision.
[0089] Each of the set of bitwise dot-product summation units may include a pop counter implemented using a SAR ADC including a plurality of capacitors, and the plurality of capacitors used in the SAR ADC of the first subset of the set of bitwise dot-product summation units may have a respective first size and the plurality of capacitors used in the SAR ADC of the second subset of the set of bitwise dot-product summation units may have a respective second size different from the first size. The first subset of the set of bitwise dot-product summation units may additionally or alternatively include a single instance of a SAR ADC coupled to a plurality of additional capacitors that sum the first number of inputs, and each of the second subset of the set of bitwise dot-product summation units may include two or more instances of a SAR ADC that each sum a second number of inputs that is less than the first number of inputs.
[0090] In some embodiments, a method of performing a multiply-accumulate operation includes performing a first portion of a bitwise multiplication of first and second digital inputs in a first precision and performing at least a second portion of the bitwise multiplication of the first and second digital inputs in a second precision different from the first precision, the method being performed by a processing circuit configured to execute an instruction code.
[0091] The bitwise multiplication of the first and second digital inputs may include computing a set of weighted sums of bitwise dot products of the first and second digital inputs, where a first portion of the bitwise multiplications of the first and second digital inputs includes a first subset of the set of weighted sums of bitwise dot products of the first and second digital inputs and a second portion of the bitwise multiplications of the first and second digital inputs includes a second subset of the set of weighted sums of bitwise dot products of the first and second digital inputs. The set of weighted sums of bitwise dot products of the first and second digital inputs may include n weighted sums of bitwise dot products of the first and second digital inputs, where the first portion of the bitwise multiplications performed with the first precision includes computing k highest weighted sums of the bitwise dot products of the first and second digital inputs and the second portion of the bitwise multiplications performed with the second precision includes computing at least a portion of the remaining n k weighted sums of the bitwise dot products of the first and second digital inputs.
[0092] In some embodiments, an article of manufacture includes a non-transitory computer-readable storage medium having executable instruction code embodied thereon that, when executed by a processor, causes the processor to perform a first portion of a bitwise multiplication of first and second digital inputs in a first precision and to perform at least a second portion of the bitwise multiplication of the first and second digital inputs in a second precision, different from the first precision.
[0093] The bitwise multiplication of the first and second digital inputs may include computing a set of weighted sums of bitwise dot products of the first and second digital inputs, where a first portion of the bitwise multiplications of the first and second digital inputs includes a first subset of the set of weighted sums of bitwise dot products of the first and second digital inputs and a second portion of the bitwise multiplications of the first and second digital inputs includes a second subset of the set of weighted sums of bitwise dot products of the first and second digital inputs. The set of weighted sums of bitwise dot products of the first and second digital inputs may include n weighted sums of bitwise dot products of the first and second digital inputs, where the first portion of the bitwise multiplications performed with the first precision includes computing k highest weighted sums of the bitwise dot products of the first and second digital inputs and the second portion of the bitwise multiplications performed with the second precision includes computing at least a portion of the remaining n k weighted sums of the bitwise dot products of the first and second digital inputs.
[0094] In some embodiments, the system includes a multiply-add unit configured to perform binary multiplication of a first and second vector. The multiply-add unit includes a set of two or more bitwise dot-product sum units each configured to perform a bitwise multiplication of a portion of the first and second vectors to generate a digital output. Each bitwise dot-product sum unit of the set of two or more bitwise dot-product sum units includes at least one instance of a bitwise dot-product sum unit coupled to an instance of a SAR ADC unit, the SAR ADC unit configured to convert an analog output of the dot-product sum unit to at least a portion of the digital output. A first subset of the set of two or more bitwise dot-product sum units having a first precision is used to calculate a first portion of the bitwise multiplication of the first and second vectors, and a second subset of the set of two or more bitwise dot-product sum units having a second precision different from the first precision is used to calculate a second portion of the bitwise multiplication of the first and second vectors.
[0095] The bitwise multiplication of the first and second vectors may include computing a set of weighted sums of bitwise dot products of the first and second vectors, where a first portion of the bitwise multiplications of the first and second vectors includes a first subset of the set of weighted sums of bitwise dot products of the first and second vectors and a second portion of the bitwise multiplications of the first and second vectors includes a second subset of the set of weighted sums of bitwise dot products of the first and second vectors. The set of weighted sums of bitwise dot products of the first and second vectors may include n weighted sums of bitwise dot products of the first and second vectors, where the first subset of the set of weighted sums of bitwise dot products of the first and second vectors includes the k highest weighted sums of bitwise dot products of the first and second vectors and the second subset of the set of weighted sums of bitwise dot products of the first and second vectors includes at least a portion of the remaining n k weighted sums of bitwise dot products of the first and second vectors.
[0096] The system may be implemented as part of an artificial intelligence system, or as part of one or more integrated circuits, or both.
[0097] In some embodiments, a device includes a multiply-accumulate circuit configured to perform a binary multiplication of a first and second vector. The multiply-accumulate circuit includes a set of two or more instances of a bitwise dot-product summation circuit, each configured to perform a bitwise multiplication of a portion of the first and second vectors to generate a digital output. Each instance of the bitwise dot-product summation circuit of the set of two or more instances of the bitwise dot-product summation circuit includes at least one instance of a bitwise dot-product computation circuit coupled to at least one instance of a SAR ADC circuit, the SAR ADC circuit configured to convert an analog output of the bitwise dot-product computation circuit to at least a portion of the digital output. A first subset of the set of two or more instances of the bitwise dot-product summation circuit having a first precision is used to compute a first portion of the bitwise multiplication of the first and second vectors, and a second subset of the set of two or more instances of the bitwise dot-product summation circuit having a second precision different from the first precision is used to compute a second portion of the bitwise multiplication of the first and second vectors.
[0098] The bitwise multiplication of the first and second vectors may include computing a set of weighted sums of bitwise dot products of the first and second vectors, where a first portion of the bitwise multiplications of the first and second vectors includes a first subset of the set of weighted sums of bitwise dot products of the first and second vectors and a second portion of the bitwise multiplications of the first and second vectors includes a second subset of the set of weighted sums of bitwise dot products of the first and second vectors. The set of weighted sums of bitwise dot products of the first and second vectors may include n weighted sums of bitwise dot products of the first and second vectors, where the first subset of the set of weighted sums of bitwise dot products of the first and second vectors includes the k highest weighted sums of bitwise dot products of the first and second vectors and the second subset of the set of weighted sums of bitwise dot products of the first and second vectors includes at least a portion of the remaining n k weighted sums of bitwise dot products of the first and second vectors.
[0099] A first subset of the set of two or more instances of the bitwise dot product summation circuit may have a first resolution, and a second subset of the two or more instances of the bitwise dot product summation circuit may have a second resolution different from the first resolution.
[0100] Each of the set of two or more instances of the bitwise dot product summation circuit may include a plurality of logical AND gates, each logical AND gate receiving as input a first vector element of the first vector and a second vector element of the second vector, an output of each of the plurality of logical AND gates coupled to a first plate of a plurality of capacitors, the plurality of capacitors in each of a first subset of the set of the two or more instances of the bitwise dot product summation circuit may have a respective first size, and the plurality of capacitors in each of a second subset of the set of the two or more instances of the bitwise dot product summation circuit may have a respective second size different from the first size.
[0101] Each SAR ADC circuit of the set of two or more instances of the bitwise dot product summation circuit may include a comparator, and a first type of comparator used in the SAR ADC circuits of a first subset of the set of two or more instances of the bitwise dot product summation circuit may be different from a second type of comparator used in the SAR ADC circuits of a second subset of the set of two or more instances of the bitwise dot product summation circuit. The first type of comparator may have a first noise profile and the second type of comparator may have a second noise profile that is different from the first noise profile. The first type of comparator may additionally or alternatively have a first offset cancellation accuracy and the second type of comparator may have a second offset cancellation accuracy that is different from the first offset cancellation accuracy.
[0102] Each instance of a bitwise dot product summation circuit in a first subset of the set of two or more instances of the bitwise dot product summation circuit may include at least two instances of a bitwise dot product calculation circuit and a SAR ADC circuit, and each instance of a bitwise dot product summation circuit in a second subset of the two or more instances of the bitwise dot product summation circuit may include a single instance of a bitwise dot product calculation circuit and a SAR ADC circuit coupled to an additional capacitor bank. Each of the at least two instances of a bitwise dot product calculation circuit and a SAR ADC circuit of each instance of a bitwise dot product summation circuit in the first subset of the set of two or more instances of the bitwise dot product summation circuit may sum a first number of inputs, and each of the single instance of a bitwise dot product calculation circuit and a SAR ADC circuit coupled to an additional capacitor bank of each instance of a bitwise dot product summation circuit in the second subset of the set of two or more instances of the bitwise dot product summation circuit may sum a second number of inputs that is greater than the first number of inputs.
[0103] Additionally, embodiments of the present invention may employ software executed on a computer or workstation. With reference to FIG. 11, such implementations may employ, for example, a processor 1102, memory 1104, and input / output interfaces formed, for example, by a display 1106 and a keyboard 1108. The term "processor" as used herein is intended to include any processing device, such as a processing device including, for example, a multi-core CPU, a GPU, an FPGA, or other forms of processing circuitry, such as, for example, one or more ASICs, or a combination thereof. Furthermore, the term "processor" may refer to two or more separate processors. The term "memory" is intended to include memory associated with a processor (e.g., CPU, GPU, FPGA, ASIC, etc.), such as RAM (random access memory), ROM (read only memory), fixed memory devices (e.g., hard drives, etc.), removable memory devices (e.g., diskettes, etc.), and flash memory. Additionally, the phrase "input / output interface" as used herein is intended to include, for example, mechanisms for inputting data to a processing unit (e.g., a mouse, etc.) and mechanisms for providing results related to the processing unit (e.g., a printer, etc.). The processor 1102, memory 1104, and input / output interfaces, such as a display 1106 and a keyboard 1108, may be interconnected, for example, via a bus 1110 as part of the data processing unit 1112. Suitable interconnections, for example, via the bus 1110, may also be provided for a network interface 1114, such as a network card, which may be provided for interfacing with a computer network, and a media interface 1116, such as a diskette or CD-ROM drive, which may be provided for interfacing with a media 1118.
[0104] Thus, computer software including instructions or code for carrying out the methods of the present invention described herein may be stored in associated memory devices (e.g., ROM, fixed or removable memory, etc.) and may be loaded partially or entirely (e.g., into RAM, etc.) and implemented by a CPU when ready for use. Such software may include, but is not limited to, firmware, resident software, and microcode, etc.
[0105] A data processing system suitable for storing and / or executing program code will include at least one processor 1102 coupled directly or indirectly to memory elements 1104 through a system bus 1110. The memory elements may include local memory used during the actual implementation of the program code, bulk storage, and cache memory that provides temporary storage of at least some of the program code to reduce the number of times the code must be read from bulk storage during implementation.
[0106] Input / output or I / O devices (including but not limited to keyboard 1108, display 1106, and pointing device) may be coupled to the system directly (e.g., via bus 1110) or through intervening I / O controllers (omitted for clarity).
[0107] Network adapters, such as network interface 1114, may also be coupled to the system to enable the data processing system to become coupled to other data processing systems or remote printers or storage devices through intervening private or public networks. Modems, cable modems, and Ethernet® cards are just a few of the currently available types of network adapters.
[0108] A "server," as used herein, including in the claims, includes a physical data processing system that executes a server program, such as system 1112 shown in FIG. 11. It will be understood that such a physical server may or may not include a display and keyboard.
[0109] The invention may be a system, method, or computer program product, or combination thereof, at any possible level of technical detail integration. The computer program product may include a computer-readable storage medium (or media) having computer-readable program instructions for causing a processor to perform aspects of the invention.
[0110] A computer readable storage medium may be a tangible device capable of holding and storing instructions for use by an instruction execution device. A computer readable storage medium may be, for example, but not limited to, an electronic storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any suitable combination of the foregoing. A non-exhaustive list of more specific examples of computer readable storage media includes: portable computer diskettes, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), SRAM, portable compact disc read-only memory (CD-ROM), digital versatile disk (DVD), memory stick, floppy disk, mechanically encoded devices such as punch cards or raised structures in grooves having recorded instructions, and any suitable combination of the foregoing. As used herein, computer-readable storage media should not be construed as being transitory signals per se, such as, for example, electric waves or other freely propagating electromagnetic waves, electromagnetic waves propagating through a waveguide or other transmission medium (e.g., light pulses passing through a fiber optic cable), or electrical signals transmitted through wires.
[0111] The computer readable program instructions described herein may be downloaded from a computer readable storage medium into each computing / processing device or may be downloaded to an external computer or external storage device over a network, such as the Internet, a local area network, a wide area network, or a wireless network, or a combination thereof. The network may include copper transmission cables, optical transmission fiber, wireless transmission, routers, firewalls, switches, gateway computers, or edge servers, or a combination thereof. A network adapter card or network interface in each computing / processing device receives the computer readable program instructions from the network and forwards the computer readable program instructions to a computer readable storage medium in the respective computing / processing device for storage.
[0112] The computer readable program instructions for carrying out the operations of the present invention may be assembler instructions, instruction-set-architecture (ISA) instructions, machine instructions, machine-dependent instructions, microcode, firmware instructions, state setting data, configuration data for an integrated circuit, or source or object code written in any combination of one or more programming languages, including object-oriented programming languages such as Smalltalk, or C++, and procedural programming languages such as the "C" programming language or similar programming languages. The computer readable program instructions may be executed entirely on the user's computer, partially on the user's computer as a stand-alone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the latter scenario, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or the connection to the external computer may be made (e.g., through the Internet using an Internet Service Provider). In some embodiments, electronic circuitry including, for example, programmable logic circuits, field programmable gate arrays (FPGAs), or programmable logic arrays (PLAs), may execute computer readable program instructions by using state information of the computer readable program instructions to personalize the electronic circuitry to perform aspects of the invention.
[0113] Aspects of the present invention are described herein with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer readable program instructions.
[0114] These computer readable program instructions may be provided to a processor of a computer or other programmable data processing apparatus to produce a machine, such that the instructions executed by the processor of the computer or other programmable data processing apparatus cause means for implementing the functions / operations specified in one or more blocks of the flowcharts and / or block diagrams. These computer readable program instructions may also be stored on a computer readable storage medium capable of instructing a computer, programmable data processing apparatus, or other device, or combination thereof, to function in a particular manner, such that the computer readable storage medium on which the instructions are stored includes an article of manufacture including instructions that implement aspects of the functions / operations specified in one or more blocks of the flowcharts and / or block diagrams.
[0115] The computer readable program instructions may also be loaded into a computer, other programmable data processing apparatus, or other device to cause the computer, other programmable apparatus, or other device to perform a series of operational steps to generate a computer-implemented process, such that the instructions executed on the computer, other programmable apparatus, or other device implement the functions / operations specified in one or more blocks of the flowcharts and / or block diagrams.
[0116] The flowcharts and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of the present invention. In this regard, each block of the flowcharts or block diagrams may represent a module, segment, or portion of instructions, which includes one or more executable instructions for implementing the specified logical function(s). In some alternative implementations, the functions shown in the blocks may occur in a different order than that shown in the figures. For example, two blocks shown in succession may actually be accomplished as a single step, may be executed simultaneously, may be executed substantially simultaneously in a partially or fully overlapping manner in time, or the blocks may sometimes be executed in the reverse order, depending on the functionality involved. In addition, it will be noted that each block of the block diagrams and / or flowchart illustrations, and combinations of blocks in the block diagrams and / or flowchart illustrations, may be implemented by a special-purpose hardware-based system that performs the specified functions or operations or executes a combination of special-purpose hardware and computer instructions.
[0117] It should be noted that any of the methods described herein may include the additional step of providing a system including separate software modules implemented in a computer readable storage medium. The modules may include, for example, any or all of the components detailed herein. The method steps may then be performed using the separate software modules and / or sub-modules of the above-mentioned system executed in the hardware processor 1102. Furthermore, the computer program product may include a computer readable storage medium having code adapted to be implemented to perform at least one method step described herein, including provisioning of the system with the separate software modules.
[0118] In each case, it should be understood that the components illustrated herein may be implemented in various forms of hardware, software, or combinations thereof, such as, for example, an application specific integrated circuit(s) (ASIC), a suitably programmed digital computer having functional circuitry and associated memory. Given the teachings of the invention provided herein, those skilled in the relevant art will be able to envision other implementations of the components of the invention.
[0119] Although this disclosure includes a detailed description of cloud computing, it should be understood that implementation of the teachings described herein is not limited to a cloud computing environment. Rather, embodiments of the invention may be implemented in conjunction with any other type of computing environment now known or later developed.
[0120] Cloud computing is a model of service delivery for enabling convenient, on-demand network access to a shared pool of configurable computing resources (e.g., networks, network bandwidth, servers, processing, memory, storage, applications, virtual machines, and services) that can be rapidly provisioned and released with minimal administrative effort or interaction with a service provider. This cloud model may include at least five characteristics, at least three service models, and at least four deployment models.
[0121] The characteristics are as follows:
[0122] On-demand self-service. Cloud consumers can unilaterally provision computing capacity, such as server time and network storage, automatically as needed, without the need for human interaction with the provider of the service.
[0123] Pervasive Network Access. Functionality is available across the network and accessed through standard mechanisms that facilitate use by heterogeneous thin- and thick-client platforms (e.g., cell phones, laptops, and PDAs).
[0124] Resource Pooling. To serve multiple consumers using a multi-tenant model, the provider's computing resources are pooled, with different physical and virtual resources dynamically allocated and reallocated according to demand. The consumer typically has no control or knowledge over the exact location of the resources provided, but there is a sense of location independence in that they may be able to specify a location at a higher level of abstraction (e.g. country, state, or data center).
[0125] Rapid Elasticity. Capabilities can be rapidly and elastically provisioned, possibly automatically, to quickly scale out, and rapidly released to quickly scale in. To the consumer, the capabilities available for provisioning often appear unlimited, and any amount can be purchased at any time.
[0126] Metered services. Cloud systems automatically control and optimize resource usage by utilizing metering at some level of abstraction appropriate to the type of service (e.g., storage, processing, bandwidth, and active user accounts). Resource usage can be monitored, controlled, and reported to provide transparency to both providers and consumers of the services used.
[0127] The service model is as follows:
[0128] Software as a Service (SaaS). The functionality provided to the consumer is the use of the provider's applications running on a cloud infrastructure. The applications are accessible from a variety of client devices through thin-client interfaces, such as web browsers (e.g., web-based email). The consumer does not manage or control the underlying cloud infrastructure, including networks, servers, operating systems, storage, or even individual application functionality, with the possible exception of limited user-specific application configuration settings.
[0129] Platform as a Service (PaaS). The functionality offered to the consumer is the deployment of consumer-created or acquired applications, written using programming languages and tools supported by the provider, onto a cloud infrastructure. The consumer does not manage or control the underlying cloud infrastructure, including the network, servers, operating systems, or storage, but does have control over the deployed applications and, in some cases, the application hosting environment configuration.
[0130] Infrastructure as a Service (IaaS). The functionality provided to the consumer is the provisioning of processing, storage, network, and other basic computing resources on which the consumer can deploy and run any software, which may include operating systems and applications. The consumer does not manage or control the underlying cloud infrastructure, but has control over the operating systems, storage, deployed applications, and sometimes limited control over the choice of network forming components (e.g., host firewalls).
[0131] The deployment models are as follows:
[0132] Private Cloud: This cloud infrastructure is operated solely for an organization. It may be managed by that organization or a third party and may exist on-premise or off-premise.
[0133] Community Cloud: This cloud infrastructure is shared by multiple organizations to support a specific community with common concerns (such as mission, security requirements, policies, and compliance considerations), and may be managed by those organizations or a third party and may reside on-premise or off-premise.
[0134] Public cloud. This cloud infrastructure is made available to the general public or large industry groups and is owned by organizations that sell cloud services.
[0135] Hybrid cloud: This cloud infrastructure is a composite of two or more clouds (private, community, or public) that remain their own entities but are bound together by standard or proprietary technologies that enable data and application portability (e.g., cloud bursting for load balancing between clouds).
[0136] Cloud computing environments are service-oriented and focus on statelessness, low coupling, modularity, and semantic interoperability. At the heart of cloud computing is an infrastructure that includes a network of interconnected nodes.
[0137] 12, an exemplary cloud computing environment 1250 is shown. As shown, the cloud computing environment 1250 includes one or more cloud computing nodes 1210 with which local computing devices used by cloud consumers, such as a personal digital assistant (PDA) or mobile phone 1254A, a desktop computer 1254B, a laptop computer 1254C, or an automotive computer system 1254N, or combinations thereof, may communicate. The nodes 1210 may communicate with each other. These nodes may be physically or virtually grouped together in one or more networks, such as a private, community, public, or hybrid cloud, or combinations thereof, as described above (not shown). This enables the cloud computing environment 1250 to provide infrastructure, platform, or software, or combinations thereof, as a service for which the cloud consumers do not need to maintain resources at their local computing devices. It will be understood that the types of computing devices 1254A-N shown in FIG. 12 are intended to be exemplary only, and that the computing node 1210 and cloud computing environment 1250 can communicate with any type of computing device through any type of network or network-addressable connection (e.g., using a web browser) or both.
[0138] Referring now to Figure 13, a set of functional abstraction layers provided by cloud computing environment 1250 (Figure 12) is shown. It should be understood in advance that the components, layers, and functions shown in Figure 13 are intended to be merely exemplary, and embodiments of the present invention are not limited thereto. As shown, the following layers and corresponding functions are provided:
[0139] Hardware and software layer 1360 includes hardware and software components. Examples of hardware components include mainframe 1361, RISC (Reduced Instruction Set Computer) architecture based servers 1362, servers 1363, blade servers 1364, storage devices 1365, and networks and network forming components 1366. In some embodiments, software components include network application server software 1367 and database software 1368.
[0140] From the abstraction layer provided by the virtualization layer 1370, the following examples of virtual entities may be provided: virtual servers 1371, virtual storage 1372, virtual networks including virtual private networks 1373, virtual applications and operating systems 1374, and virtual clients 1375.
[0141] In one example, management layer 1380 may provide the functions described below: Resource provisioning 1381 provides dynamic procurement of computing and other resources used to execute tasks within the cloud computing environment. Metering and pricing 1382 provides cost tracking as resources are used within the cloud computing environment and billing or invoicing for the consumption of these resources. In one example, these resources may include application software licenses. Security provides identity verification for cloud consumers and tasks, as well as protection of data and other resources. User portal 1383 provides access to the cloud computing environment for consumers and system administrators. Service level management 1384 provides allocation and management of cloud computing resources so that required service levels are met. Service level agreement (SLA) planning and fulfillment 1385 provides advance arrangement and procurement of cloud computing resources where future demand is forecasted by SLAs.
[0142] The Workload Layer 1390 provides examples of functions for which a cloud computing environment may be used. Examples of workloads and functions that may be provided from this layer include mapping and navigation 1391, software development and lifecycle management 1392, virtual classroom instructional delivery 1393, data analytics processing 1394, transaction processing 1395, and artificial intelligence algorithms (with low noise MS-DPE / MS-MAC computation) processing 1396 according to one or more embodiments of the present invention.
[0143] The terms used herein are merely for the purpose of describing particular embodiments and are not intended to limit the present invention. As used herein, the singular forms "a", "an" and "the" are intended to include the plural unless the context clearly indicates otherwise. It will be further understood that the terms "comprises" and / or "comprising" as used herein specify the presence of the described features, steps, operations, elements or components, or combinations thereof, but do not exclude the presence or addition of other features, steps, operations, elements, components, or groups thereof, or combinations thereof.
[0144] The description of various embodiments of the present invention is provided for illustrative purposes, but is not intended to be exhaustive or limiting to the disclosed embodiments. Many modifications and variations will be apparent to those skilled in the art without departing from the scope and spirit of the described embodiments. The terms used in this specification are selected to best explain the principles of the embodiments, practical applications, or technical improvements to the technology found in the market, or to enable other skilled in the art to understand the embodiments disclosed herein.
Claims
1. At least one processor; at least one memory containing instruction code, The instruction code causes the at least one processor to: performing a first portion of a bitwise multiplication of first and second digital inputs with a first precision; performing at least a second portion of the bitwise multiplication of the first and second digital inputs in a second precision different from the first precision; An apparatus configured to cause the
2. 2. The apparatus of claim 1 , wherein the bitwise multiplication of the first and second digital inputs comprises computing a set of weighted sums of bitwise dot products of the first and second digital inputs, the first portion of the bitwise multiplications of the first and second digital inputs comprising a first subset of the set of weighted sums of the bitwise dot products of the first and second digital inputs, and the second portion of the bitwise multiplications of the first and second digital inputs comprising a second subset of the set of weighted sums of the bitwise dot products of the first and second digital inputs.
3. 3. The apparatus of claim 2, wherein the set of weighted sums of the bitwise dot products of the first and second digital inputs comprises n weighted sums of the bitwise dot products of the first and second digital inputs, the first portion of the bitwise multiplications performed with the first precision comprises calculating a highest weighted sum of k of the bitwise dot products of the first and second digital inputs, and the second portion of the bitwise multiplications performed with the second precision comprises calculating at least a portion of the remaining n-k weighted sums of the bitwise dot products of the first and second digital inputs.
4. 4. The apparatus of claim 3, wherein a value of k is selected based at least in part on a determination of additive noise contributions of different terms of the bitwise multiplication of the first and second digital inputs.
5. 2. The apparatus of claim 1, wherein the bitwise multiplication of the first and second digital inputs is computed using one of a set of bitwise dot-product summation units, the first precision being provided using a first type of hardware for a first subset of the set of bitwise dot-product summation units that perform the first portion of the bitwise multiplication of the first and second digital inputs, and the second precision being provided using a second type of hardware, different from the first type of hardware, for a second subset of the set of bitwise dot-product summation units that perform the second portion of the bitwise multiplication of the first and second digital inputs.
6. 6. The apparatus of claim 5, wherein each of the sets of bitwise dot product summation units includes a pop counter, the pop counters of the first subset of the sets of bitwise dot product summation units having a first resolution and the pop counters of the second subset of the sets of bitwise dot product summation units having a second resolution different from the first resolution.
7. 6. The apparatus of claim 5, wherein each of the set of bitwise dot product summation units includes a pop counter implemented using a successive approximation register analog-to-digital converter including a comparator, and a first type of comparator used in the successive approximation register analog-to-digital converter circuit of the first subset of the set of two or more instances of the bitwise dot product summation circuit is different from a second type of comparator used in the successive approximation register analog-to-digital converter circuit of the second subset of the set of two or more instances of the bitwise dot product summation circuit.
8. the first type comparator having a first noise profile and the second type comparator having a second noise profile different from the first noise profile; and The first type comparator has a first offset cancellation accuracy, and the second type comparator has a second offset cancellation accuracy different from the first offset cancellation accuracy. The apparatus of claim 7, wherein the at least one of
9. 6. The apparatus of claim 5, wherein each of the set of bitwise dot product summation units includes a pop counter implemented using a successive approximation register analog-to-digital converter including a plurality of capacitors, the plurality of capacitors used in the successive approximation register analog-to-digital converters of the first subset of the set of bitwise dot product summation units having respective first sizes, and the plurality of capacitors used in the successive approximation register analog-to-digital converters of the second subset of the set of bitwise dot product summation units having respective second sizes different from the first sizes.
10. 6. The apparatus of claim 5, wherein each of the set of bitwise dot-product summation units includes one or more pop counters implemented using a successive approximation register analog-to-digital converter each including a plurality of capacitors, the first subset of the set of bitwise dot-product summation units includes a single instance of the successive approximation register analog-to-digital converter coupled to a plurality of additional capacitors that sum a first number of inputs, and the second subset of the set of bitwise dot-product summation units each includes two or more instances of the successive approximation register analog-to-digital converter each summing a second number of inputs less than the first number of inputs.
11. performing a first portion of a bitwise multiplication of first and second digital inputs with a first precision; performing at least a second portion of the bitwise multiplication of the first and second digital inputs in a second precision different from the first precision, A method, wherein the method is performed by a processing circuit configured to execute instruction code.
12. 12. The method of claim 11 , wherein the bitwise multiplication of the first and second digital inputs comprises computing a set of weighted sums of bitwise dot products of the first and second digital inputs, the first portion of the bitwise multiplications of the first and second digital inputs comprising a first subset of the set of weighted sums of the bitwise dot products of the first and second digital inputs, and the second portion of the bitwise multiplications of the first and second digital inputs comprising a second subset of the set of weighted sums of the bitwise dot products of the first and second digital inputs.
13. A computer program for causing a computer processor to carry out each of the steps of the method according to claim 11 or 12.
14. 1. A system including a multiply-accumulate unit configured to perform binary multiplication of first and second vectors, the system comprising: the multiply-add unit comprises a set of two or more bitwise dot-product-sum units each configured to perform a bitwise multiplication of a portion of the first and second vectors to generate a digital output, the bitwise multiplication of the first and second vectors comprising computing a set of weighted sums of bitwise dot products of the first and second vectors; each bitwise dot-product summation unit of the set of two or more bitwise dot-product summation units includes at least one instance of a bitwise dot-product summation unit coupled to an instance of a successive approximation register analog-to-digital converter unit, the successive approximation register analog-to-digital converter unit configured to convert an analog output of the dot-product summation unit to at least a portion of the digital output; a first subset of the set of two or more bitwise dot product summation units having a first precision is used to calculate a first portion of the bitwise multiplication of the first and second vectors, and a second subset of the set of two or more bitwise dot product summation units having a second precision different from the first precision is used to calculate a second portion of the bitwise multiplication of the first and second vectors.
15. 15. The system of claim 14, wherein the bitwise multiplication of the first and second vectors comprises computing a set of weighted sums of bitwise dot products of the first and second vectors, the first portion of the bitwise multiplications of the first and second vectors comprises a first subset of the set of weighted sums of the bitwise dot products of the first and second vectors, and the second portion of the bitwise multiplications of the first and second vectors comprises a second subset of the set of weighted sums of the bitwise dot products of the first and second vectors.
16. The system of claim 14 , wherein the system is implemented as part of an artificial intelligence system.
17. The system of claim 14 , wherein the system is implemented as part of one or more integrated circuits.
18. 1. A device comprising: a multiply-accumulate circuit configured to perform binary multiplication of first and second vectors, the multiply-accumulate circuit comprising: the sum-of-products circuitry includes a set of two or more instances of a bitwise dot-product-sum circuit, each configured to perform a bitwise multiplication of a portion of the first and second vectors to generate a digital output; each instance of the bitwise dot product summation circuit of the set of two or more instances of the bitwise dot product summation circuit includes at least one instance of a bitwise dot product computation circuit coupled to at least one instance of a successive approximation register analog-to-digital converter circuit, the successive approximation register analog-to-digital converter circuit configured to convert an analog output of the bitwise dot product computation circuit to at least a portion of the digital output; a first subset of the set of two or more instances of the bitwise dot product summation circuit having a first precision is used to compute a first portion of the bitwise multiplication of the first and second vectors, and a second subset of the set of two or more instances of the bitwise dot product summation circuit having a second precision different from the first precision is used to compute a second portion of the bitwise multiplication of the first and second vectors.
19. 20. The device of claim 18, wherein the bitwise multiplication of the first and second vectors comprises computing a set of weighted sums of bitwise dot products of the first and second vectors, the first portion of the bitwise multiplications of the first and second vectors comprising a first subset of the set of weighted sums of the bitwise dot products of the first and second vectors, and the second portion of the bitwise multiplications of the first and second vectors comprising a second subset of the set of weighted sums of the bitwise dot products of the first and second vectors.
20. 20. The device of claim 18, wherein the first subset of the set of the two or more instances of the bitwise dot product summation circuit has a first resolution and the second subset of the two or more instances of the bitwise dot product summation circuit has a second resolution different from the first resolution.
21. 20. The device of claim 18, wherein each of the set of two or more instances of the bitwise dot product summation circuit includes a plurality of logical AND gates, each of the plurality of logical AND gates receiving as inputs a first vector element of the first vector and a second vector element of the second vector, an output of each of the plurality of logical AND gates coupled to a first plate of a plurality of capacitors, the plurality of capacitors in each of the first subset of the set of two or more instances of the bitwise dot product summation circuit having a respective first size, and the plurality of capacitors in each of the second subset of the set of two or more instances of the bitwise dot product summation circuit having a respective second size different from the first size.
22. 20. The device of claim 18, wherein the successive approximation register analog-to-digital converter circuit of each of the set of two or more instances of the bitwise dot product summation circuit includes a comparator, and a first type of comparator used in the successive approximation register analog-to-digital converter circuit of the first subset of the set of two or more instances of the bitwise dot product summation circuit is different from a second type of comparator used in the successive approximation register analog-to-digital converter circuit of the second subset of the set of two or more instances of the bitwise dot product summation circuit.
23. 20. The device of claim 18, wherein each instance of the bitwise dot product summation circuit in the first subset of the set of the two or more instances of the bitwise dot product summation circuit includes at least two instances of the bitwise dot product calculation circuit and the successive approximation register analog-to-digital converter circuit, and each instance of the bitwise dot product summation circuit in the second subset of the two or more instances of the bitwise dot product summation circuit includes a single instance of the bitwise dot product calculation circuit and the successive approximation register analog-to-digital converter circuit coupled to an additional capacitor bank.
24. 24. The device of claim 23, wherein each of the at least two instances of the bitwise dot product calculation circuit and the successive approximation register analog-to-digital converter circuit of each instance of the bitwise dot product summation circuit in the first subset of the set of the two or more instances of the bitwise dot product summation circuit sums a first number of inputs, and each of the single instances of the bitwise dot product calculation circuit and the successive approximation register analog-to-digital converter circuit coupled to the additional capacitor bank of each instance of the bitwise dot product summation circuit in the second subset of the set of the two or more instances of the bitwise dot product summation circuit sums a second number of inputs that is greater than the first number of inputs.
