Hybrid Matrix Multiplier
The hybrid computing hardware accelerator addresses the inefficiencies of traditional matrix multiplication by using a digital-analog hybrid approach, achieving rapid and energy-efficient matrix computations.
Patent Information
- Application Number
- JP2024526822
- Authority / Receiving Office
- JP · JP
- Patent Type
- Applications
- Current Assignee / Owner
- Priority Date
- 2021-11-10
- Filing Date
- 2021-11-25
- Publication Date
- 2024-11-15
- Estimated Expiration
- 2041-11-25
AI Technical Summary
Existing computing systems face challenges in performing matrix multiplications with large matrices and high data rates, requiring significant time and power consumption, especially in safety-critical and mobile device applications.
A hybrid computing hardware accelerator utilizing a digital binary single-bit multiplier with an analog multiply-accumulate operator, where single-bit multiplication results are stored as charge on capacitors, allowing for almost instantaneous sum of products operations with minimal power consumption.
The hybrid circuit provides fast and low-power matrix multiplication, significantly reducing computational time and energy usage compared to traditional digital methods.
Smart Images

Figure 2024542400000001_ABST
Abstract
Description
[Technical field]
[0001] The present disclosure relates generally to processing architectures, devices, and methods for matrix multiplication, and more particularly to hybrid multiply-accumulate circuits. [Background technology]
[0002] Matrix multiplication is an important operation in many mathematical computations. For example, in linear algebra, matrix multiplication can be used to solve simultaneous linear equations such as differential equations. This type of computation has applications in, for example, pattern matching, artificial intelligence, analytic geometry, engineering, physics, natural sciences, computer science, computer animation, and economics. Summary of the Invention [Problem to be solved by the invention]
[0003] Matrix multiplication is typically performed on digital computers running stored programs. The programs describe the operations to be performed, and hardware within the computer, such as digital multipliers and adders, performs the operations. In some computing systems, specially designed hardware can accelerate the speed of calculations. Some applications, especially safety-critical tasks, require real-time processing to provide useful output in a useful time. Furthermore, applications on portable devices have limited power available. Despite these high-speed computing systems, solving problems with large matrices and high data rates can take time and consume more power than necessary. Therefore, there is a need for computing hardware accelerators that can perform matrix multiplication at higher speeds and with lower power consumption. [Means for solving the problem]
[0004] In particular, embodiments of the present disclosure may provide a hybrid computing hardware accelerator for performing matrix multiplication using multiply-accumulate operations. The computing hardware accelerator of the present disclosure is comprised of a digital binary single-bit multiplier with an analog multiply-accumulate unit. The data values of the single-bit multipliers are each stored in a digital memory, and the single-bit multiplication results are stored as charges on capacitors. The charges on the capacitors are combined to sum (accumulate) the values, providing the multiply-accumulate operation. By combining the charges on the capacitors, the operation is nearly instantaneous, dependent on the speed at which the charges flow in the conductors, and does not require external power. Thus, embodiments of the present disclosure may provide a very fast and low-power multiply-accumulate circuit. Charge is denoted as Q in electronic systems, so each single-bit multiply-accumulate circuit is referred to herein as a qmac, which is a hybrid circuit using digital multiplication and analog accumulation.
[0005] According to an embodiment of the present disclosure, a hybrid multiply-accumulate circuit comprises an array of single-bit multiply-accumulate circuits, each single-bit multiply-accumulate circuit comprising (i) a first storage element for storing a first single-bit value, (ii) a second storage element for storing a second single-bit value, (iii) a bit multiplier circuit for multiplying the first single-bit value by the second single-bit value to compute a product, and (iv) an analog storage circuit, the bit multiplier circuit operable to store a charge representative of the product in the analog storage circuit. The array of single-bit multiply-accumulate circuits is operable to combine the charges stored in each analog storage circuit to provide a cumulative charge representative of a sum of the products. The analog storage circuit may be a capacitor.
[0006] According to some embodiments, the hybrid multiply-accumulate circuit comprises a switch circuit coupled to the bit multiplier circuit and the analog storage circuit operable in a first mode to transfer charge from the bit multiplier circuit to the analog storage circuit and in a second mode to isolate the bit multiplier circuit from the analog storage circuit and connect the analog storage circuits of the array together to provide the stored charge. Some embodiments comprise a clear circuit coupled to the analog storage circuits of the array operable to remove charge from the analog storage circuits of the array. In some embodiments, the bit multiplier circuit is a functional AND gate or performs the function of an AND gate.
[0007] In some embodiments of the present disclosure, the hybrid multiply-accumulate circuit comprises an analog-to-digital converter that converts charges stored in the analog storage circuits in the array to digital accumulation values. Some embodiments comprise a shift circuit or shift electrical connection that multiplies the digital accumulation values by a power of two. Some embodiments comprise a digital adder operable to add the digital accumulation values to generate a digital matrix value. The digital adder can be pipelined.
[0008] In some embodiments, there is no analog-to-digital converter converting the output of the parallel-connected qmacs10 analog circuits 16, and summing of the outputs of the array of hybrid multiply-accumulate circuits is performed by an analog summer operable to sum the stored charges to generate analog matrix values. Some embodiments consist of voltage multipliers connected to analog storage circuits in the array to multiply the stored charges by powers of two. Such summing and multiplication can be performed by an opamp configured as a summer having an opamp input connected to an analog storage circuit operable to provide analog matrix values. The opamp input of the opamp can be configured to multiply or divide the opamp input by a power of two. Some embodiments include an analog-to-digital converter converting the analog matrix values to generate digital matrix values, and the output of the opamp is digitized.
[0009] In some embodiments, the bit multiplication circuit is comprised of a number of switches (e.g., a series switch circuit consisting of pairs of MOS transistors) connected in series, with a first MOS transistor controlled by a positive control signal and a second MOS transistor controlled by an inverted (negative) version of the same control signal, one of the series connected switches being controlled by a weight value and the other being controlled by an input value representing a matrix multiplication of the weight value and the input value.
[0010] According to an embodiment of the present disclosure, a hybrid matrix multiplier comprises digital storage elements, each of the digital storage elements operable to store a digital value, a multiplication circuit for multiplying the stored digital values to generate a product, and an analog storage circuit operable to store the product. The voltage connection can provide power for operating the digital storage elements, the multiplication circuit, and the analog storage circuit. In some embodiments, the power supply connection provides power for operating the digital storage elements, the multiplication circuit, and the analog storage circuit and has a voltage of 1V or less (e.g., 500mV or less, 100mV or less, 50mV or less, or 10mV or less). The multiplication circuit can be comprised of a series-connected switch consisting of a pair of MOS transistors. Effect of the Invention
[0011] DETAILED DESCRIPTION OF THE DRAWINGS Embodiments of the present disclosure provide a high-speed, efficient, low-power, compact hybrid hardware accelerator that performs matrix multiplication using multiply-add operations. [Brief description of the drawings]
[0012] [Figure 1A] 1 illustrates mathematically a matrix multiplication operation that is useful in understanding embodiments of the present disclosure. [Figure 1B] 1 illustrates mathematically a matrix multiplication operation that is useful in understanding embodiments of the present disclosure. [Figure 1C] 1 illustrates a simplified computer program matrix multiplication operation that is useful in understanding embodiments of the present disclosure. [Figure 1D] 1 illustrates a simplified computer program matrix multiplication operation that is useful in understanding embodiments of the present disclosure. [Diagram 2] FIG. 2 is a functional circuit diagram of a single-bit multiply-accumulate circuit according to an exemplary embodiment of the present disclosure. [Diagram 3] 3 is a circuit diagram of a one-dimensional array of single-bit multiply-accumulate circuits shown in FIG. 2 according to an exemplary embodiment of the present disclosure. [Figure 4A]FIG. 2 is a functional circuit diagram of a single-bit multiply-accumulate circuit with a switch circuit and a clear circuit, in accordance with an exemplary embodiment of the present disclosure. [Figure 4B] FIG. 4B is an abstract diagram of the functional schematic diagram of FIG. 4A in accordance with an exemplary embodiment of the present disclosure. [Figure 4C] FIG. 4B is a timing diagram for operating the single-bit multiply-accumulate circuit of FIG. 4A according to an exemplary embodiment of the present disclosure. [Diagram 5] FIG. 4B is a schematic diagram of a one-dimensional array of single-bit multiply-accumulate circuits shown in FIG. 4A according to an exemplary embodiment of the present disclosure. [Figure 6] 1 is a graphical representation of a multiplication operation with a multiplication integral value that is useful in understanding embodiments of the present disclosure. [Figure 7] FIG. 2 is a schematic diagram of a two-dimensional array of single-bit multiply-accumulate circuits with digital summing circuits, according to an exemplary embodiment of the present disclosure. [Figure 8] FIG. 2 is a schematic diagram of a two-dimensional array of single-bit multiply-accumulate circuits with analog summing circuits in accordance with an exemplary embodiment of the present disclosure. [Figure 9] FIG. 2 is a circuit diagram of an analog summing circuit according to an exemplary embodiment of the present disclosure. [Figure 10] FIG. 2 is a circuit diagram of an analog summing circuit according to an exemplary embodiment of the present disclosure. [Figure 11A] FIG. 2 is a circuit diagram of a vector-matrix hybrid multiply-accumulate circuit according to an exemplary embodiment of the present disclosure. [Figure 11B] 11B is a diagram showing matrix values in the vector matrix hybrid multiply-add circuit of FIG. 11A. [Figure 12] FIG. 9 is a circuit diagram of a vector-matrix hybrid multiply-accumulate circuit consisting of a two-dimensional array of single-bit multiply-accumulate circuits with analog summing circuits as shown in FIG. 8 in accordance with an exemplary embodiment of the present disclosure. [Figure 13] FIG. 1 is a schematic diagram of analog voltage controlled cascaded switches illustrating low power single-bit multiplication according to an exemplary embodiment of the present disclosure. [Figure 14]FIG. 2 is a schematic diagram of a low power analog voltage controlled switch according to an exemplary embodiment of the present disclosure. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS
[0013] Certain embodiments of the present disclosure relate to a number of single-bit hybrid multiply-accumulate circuits (each referred to as a qmac) that include two digital single-bit binary storage elements, each storing a single-bit value, a multiplier that multiplies the two single-bit values to calculate a product, and an analog charge storage element, such as a capacitor, to hold the product as a charge (or voltage). A one-dimensional array of qmacs can calculate and sum one-dimensional arrays (vectors) of single-bit products. A two-dimensional array of qmacs can calculate the product of two multi-bit digital multipliers. A multiplicand is a value that is multiplied with another value to calculate the multiplied product. A two-dimensional array of qmacs for calculating multi-bit multiplications can be of size NxM-1, where N is the number of bits in one of the two digital multiplications and M is the number of bits in the other of the two digital multiplications. A vector matrix multiplication and accumulation for two linear vectors (one-dimensional arrays of numbers) with M values can be calculated in M two-dimensional arrays and accumulated as one value.
[0014] As shown in Figure 1A, the calculation C=AxB where A, B, and C are matrices is matrix multiplication. If A is an m×n matrix and B is an n×p matrix, then C is an m×p matrix. When k=1~n, i=1~m, and j=1~p, C i,j =ΣA i,k ·B k,jFor k=1 to N, the sum of the products of A and B is a multiply-accumulate (mac) operation. Thus, matrix multiplication is a series of (i×j) multiply-accumulate operations of size k, with each multiply-accumulate operation providing one value of matrix C. FIG. 1B illustrates the calculation C=A×B for p=1, where C and B are linear (e.g., one-dimensional or vector) matrices. FIG. 1C is a simplified software program illustrating the matrix calculation of FIG. 1A, and FIG. 1D is a simplified software program illustrating the matrix calculation of FIG. 1B. The loop from k=0 to (n-1) is a multiply-accumulate operation requiring n multiplications and n additions.
[0015] According to an embodiment of the present disclosure, as shown in FIG. 2 and FIG. 3, a hybrid multiply-and-accumulate operation can be performed by an array of qmacs 10, each qmacs 10 comprising a first digital single-bit binary storage element 12 for storing a first bit A, a second digital single-bit binary storage element 12 for storing a second bit B, and a bit multiplier 14 (bit multiplication circuit 14) for multiplying the multipliers A and B and generating a product that is stored as a charge on a bit capacitor 16 (analog storage circuit 16). In some embodiments, the storage element 12 is an SRAM cell, a DRAM cell, a flip-flop (e.g., a D flip-flop), or a pair of inverters with their inputs and outputs connected, as shown in the inset of FIG. 2. In some embodiments, the bit multiplier 14 is an AND gate that provides a positive value (e.g., 1) only if both A and B are positive (e.g., 1), thus providing multiplication. The AND gate can be implemented as a transistor with its source connected to the storage element 12 for A and its gate connected to the storage element 12 for B (or vice versa) as shown in FIG. 2, which provides the charge Q stored in the bit capacitor 16 when the product of the multipliers A and B has a value of 1. If the values of A or B are the same for different qmacs 10, the storage element 12 for the constant can be shared by multiple qmacs 10 (e.g., a single storage element 12 can provide input values to multiple qmacs 10, as shown in FIG. 7, described below). As will be appreciated by those with knowledge of analog and digital circuit design, FIGS. 2 and 3 are simplified designs, and much more complex designs, such as those shown in FIGS. 13 and 14, described below, that can operate at very low voltages and power, are included as embodiments of the present disclosure. For example, the amount of charge stored in the bit capacitor 16 can be made very small to reduce the power used by the qmac 10 and increase the circuit speed. Making the bit capacitor 16 very small can reduce the area of the bit capacitor 16 in an integrated circuit.Thus, in some embodiments, bit multiplier 14 very precisely controls the current over time to store charge on bit capacitors 16 in order to maintain accuracy and precision of the multiply-accumulate operations. Bit multiplier 14 can therefore be designed to very precisely control the amount of charge stored on bit capacitors 16, e.g., in response to carefully calibrated timing signals and voltages.
[0016] FIG. 3 shows four qmacs 10 with bit capacitors 16 (analog storage circuits 16) connected in parallel to sum four products in a hybrid sum-of-products circuit 20. The four parallel qmacs 10 provide a sum-of-products operation in which four single-bit A values are each multiplied by a single-bit B value. The single-bit B values may be the same or different. Thus, FIG. 3 shows a circuit for performing a sum-of-products operation on four single-bit binary values (e.g., k=4 in the mathematical illustrations of FIGS. 1A-1D). In this manner, the array of single-bit sum-of-products circuits 10 is operable together to combine the charge stored in each analog storage circuit 16 to provide a cumulative charge representative of the sum of the products of the qmacs 10.
[0017] The sum of the charges on the parallel connected bit capacitors 16 provides an analog accumulated value output O that is converted to a digital value in an analog-to-digital converter (ADC) 30 or used as an analog value for further calculations. Since a parallel capacitor has a capacitance equal to the sum of the capacitances of the parallel connected capacitors, the absolute value of the voltage or charge (output O) must be scaled by the number of capacitors n. Since the charge of a capacitor is equal to voltage times capacitance (Q=CV), if the capacitance increases for a constant charge, the voltage decreases accordingly. For example, if all capacitors store a charge Q equivalent to a value of one, the sum of the values would be four (example of Figure 3), but since the four capacitors are electrically connected in parallel, the voltage remains one. Therefore, the voltage output must be scaled by the number of capacitors (e.g., by a factor of four in the diagram of Figure 3).
[0018] The hybrid multiply-accumulate circuit can operate with less power than a digital equivalent circuit using, for example, a digital adder. The net current or charge leakage from the small bit capacitors 16 can be very small, and the analog storage circuit 16 and other analog operations can operate at very low voltages, for example, 1 volt or less (e.g., 500 mV or less, 100 mV or less, 50 mV or less, or 10 mV or less), lower than the voltages used in conventional digital logic circuits (e.g., 5 V, 3.6 V, 3.3 V, or 1.65 V). Some embodiments of the present disclosure can operate at substantially 10 mV.
[0019] The circuits of Figures 2 and 3 are simplified representations of qmacs 10 and their implementation in an array of multiple multiply-accumulate circuits. As mentioned above, precise control of the charge stored in the bit capacitors 16 helps maintain the precision and accuracy of the multiply-accumulate operations. As shown in Figure 4A, a more complex circuit of qmacs 10 controls the electrical connections between qmacs 10 in an array of qmacs 10 using a switch circuit 18 (also designated S in the figure) connected to the output of the bit multiplier 14 and the bit capacitors 16. When the switch circuit 18 is on, a charge Q representing the product of bits A and B is stored in the bit capacitors 16 via the left transistor of the switch circuit 18. When the switch circuit 18 is off, the left transistor is off and an inverter consisting of the center transistor of the switch circuit 18 applies a positive signal to a connecting switch consisting of the right transistor of the switch circuit 18, connecting the bit capacitors 16 in parallel.
[0020] The switch circuit 18 shown in FIG. 4A is a simplified circuit, and more complex circuits can be implemented to provide the switch function, and such complex circuits are also included in the present disclosure. Here, in a first mode, the switch circuit 18 is on, and the products of the multiplications by the bit multipliers 14 are applied individually to transfer charge to the bit capacitors 16 in each qmac 10. In a second mode, the switch circuit 18 is off, the bit capacitors 16 are connected in parallel, and the charge Q on the bit capacitors 16 of each qmac 10 is isolated from the bit multipliers 14 and summed to provide an accumulation value output O. FIG. 4B is an abstraction of the single-bit multiply-accumulate circuit 10 of FIG. 4A, where A and B are single-bit digital storage elements 12, M is the bit multiplier 14, S is the switch circuit 18, and C is the clear circuit 19.
[0021] FIG. 4C illustrates a multiply-accumulate cycle of qmac 10. Load signal A and load signal B are set to store corresponding values in storage elements 12, provided by, for example, a computer or other state machine controller, and are multiplied by bit multipliers 14. At the same time, the clear signal goes high and the switch signal goes low to isolate and clear bit capacitors 16. Once bit capacitors 16 are cleared, the clear signal is set low and the switch signal is set high to allow a charge Q, representing the product of A and B, to be deposited onto bit capacitors 16. Once charge Q has been loaded onto bit capacitors 16, the switch signal is set low to isolate bit multipliers 14 from bit capacitors 16 and connect all bit capacitors 16 in parallel, thereby summing the charges Q on bit capacitors 16 to provide Output Valid O. The summed charge Q, which is equal to output value O, is appropriately scaled and converted to a digital value in analog-to-digital converter 30 or used as an analog value for further calculations. When the switch circuit 18 switches between the first and second modes, the entire operation takes place in two cycles.
[0022] Figure 5 shows an array of qmacs 10 forming a hybrid multiply-accumulate circuit 20 using the abstract representation of Figure 4B. In some embodiments, a single clear circuit 19 can be used to clear charge from all of the connected bit capacitors 16 when the switch circuits 18 are off, but switch circuits 18 connected between the bit capacitors 16 can prevent charge removal for all of the bit capacitors 16. In some embodiments, a clear circuit 19 is provided for each qmac 10, and the clear circuits 19, like the switch circuits 18, are commonly controlled in the hybrid multiply-accumulate circuit 20.
[0023] FIG. 6 illustrates a full multiplication for 2 binary, multi-digit, multi-bit values. Although FIG. 6 illustrates the case of values having 4 bits, any number of bits can be used for the hybrid multiply-add circuits 20 with the number of qmacs 10 corresponding to the number of bits being multiplied. The number of qmacs 10 in each hybrid multiply-add circuit 20 corresponds to the number of bits in A, and the number of hybrid multiply-add circuits 20 corresponds to the number of simultaneous multiply-add operations. If the number of qmacs 10 is less than the number of bits in A, or the number of simultaneous multiply-add operations is less than the number of bits in B, partial calculations can be performed and the products stored and combined under the control of an external computer or controller, such as a state machine.
[0024] As shown in the 4-bit example of FIG. 6, each column of products shown is a single bit of value B multiplied by a bit of value A. In FIG. 6, the columns are spatially shifted relative to each other to represent the relative magnitude (location) of the products in each column, as is conventional for multiplication written by hand on paper. The products (multiplication values) of each column 21 are summed in each hybrid multiply-accumulate circuit 20 to form an accumulated result (sum output value O), as shown in FIG. 5. Each column 21 of products may be calculated and summed in a different hybrid multiply-accumulate circuit 20. The accumulated results (accumulated output value O) of the hybrid multiply-accumulate circuits 20 are then summed (added) to obtain the final value of the multi-bit multiplication.
[0025] The multiplication and accumulation of each column 21 of products can be performed by a one-dimensional array of qmacs 10. As shown in FIG. 7, each column of qmacs 10 forms a hybrid multiply-accumulate circuit 20 that shares a common B storage element 12. The array of qmacs 10 of each hybrid multiply-accumulate circuit 20 (which in this example corresponds to the multiplication illustrated in FIG. 6) calculates and sums the column 21 of products as an output value O. The output values O of each hybrid multiply-accumulate circuit 20 can be added. Since each column 21 of products has a different scale value (relative magnitude), the values of each column 21 of products must be scaled to multiply by their scale value, e.g., 1 to 6, before being added. Multiple multiplication operations can be performed without reloading the bit value (B storage element 12) whose bit does not change, for example, if the bit value represents a weight common to the multiplication of multiple input values.
[0026] The array of hybrid multiply-accumulate circuits 20 forming hybrid multi-bit multiplier 22 provides extremely fast operation having many fewer cycles than conventional digital circuits. Furthermore, the addition step to sum the output value O (if performed digitally) can be split into stages (e.g., adding a pair of values at a time) and pipelined, making the operation even faster, and multiply-accumulate operations on different values can be overlapped in time, e.g., under the control of a computer or state machine controller.
[0027] In some embodiments of the present disclosure, the sum of the output values O from the hybrid multiply-accumulate circuit 20 is calculated digitally. In some embodiments, the sum of the output values O from the hybrid multiply-accumulate circuit 20 is calculated using analog circuitry. As shown in Figure 7, the output values are converted with an analog-to-digital converter 30 to provide digital bit values that are stored, for example, in a register or other memory, the digital bit values are scaled, for example, by shifting them relative to each other (each shift corresponds to a power of two), and the scaled bit values are summed using a digital adder.
[0028] In the example shown in FIG. 8, the analog integration result of each hybrid multiply-accumulate operation (columns of qmacs10) is a voltage (or charge), which is multiplied (e.g., by a voltage multiplier VM) by an amount corresponding to the location of the analog integration, the multiplied analog integrations are added, e.g., using an analog adder, and the final integration is converted to a digital value in an analog-to-digital converter 30. In such an embodiment, the entire calculation can be done in two switch cycles (excluding any clear or load cycles), resulting in a very fast operation compared to conventional embodiments. FIG. 8 shows an embodiment with a separate storage element 12 for each qmac10.
[0029] Multiplication and summation of analog voltages can be implemented in some embodiments using an operational amplifier (opamp) 40 configured in summing mode. FIG. 9 shows an inverting summing (summing) opamp 40. The output Vo of the opamp 40 is a sum of each of the voltages V1-V N The sum of R' / R n where n is the particular column and N is the number of columns 21 of products to be summed (e.g., 7 in the example of FIG. 7). Each voltage corresponds to the output O of a column of qmacs 10. For example, R1 may correspond to the lowest value to be summed, such that R' / R1=1 / 64, R' / R2=1 / 32, R' / R3=1 / 16, R' / R4=1 / 8, R' / R5=1 / 4, R' / R6=1 / 1, and R' / R7=1. The inverting output of opamp 40 may be converted to a digital value using analog-to-digital converter 30 and scaled appropriately.
[0030] FIG. 10 shows a non-inverting sum (addition) operational amplifier 40. The output Vo of the operational amplifier 40 is N The sum of R1 to R Nwhere V is equal to V' / R multiplied by the ratio of R' / R, which is equal to V1 / VN. The voltage values V1 through VN can be scaled with resistor implemented voltage dividers. For example, the resistor connected to V1 can have a ratio of 63:1, the resistor connected to V2 can have a ratio of 31:1, the resistor connected to V3 can have a ratio of 15:1, and the voltages can be top-scaled to correspond to the location of the added value. The output of the op-amp 40 can be scaled by a ratio of (R+R') / R (e.g. 64) and converted to a digital value using the analog-to-digital converter 30.
[0031] The embodiments of Figures 7 and 8 with analog summing can provide faster operation, and the embodiment of Figure 6 with digital summing can provide higher accuracy. The embodiments of the present disclosure are not limited by the number of bits shown. For example, the hybrid multiply-accumulate circuit 20 can have 64, 128, 256, 512, 1024, 2048, 4096, 8192, or 16384 or more qmacs10s, and the same number of hybrid multiply-accumulate circuits 20 can be employed in an array to provide fast multiplication by many bits. The embodiments of the present disclosure can be provided as a hardware accelerator for a conventional computer or graphics processor. Data can be fed to the hardware accelerator in a pipelined manner with two or more shift registers at the input and output. The hardware implementation of the array of hybrid multiply-accumulate circuits 20 must be sized to efficiently correspond to the size of the input vector. If the array of hybrid multiply-accumulate circuits 20 is too large for the task, much of the circuitry will not be used (e.g., the number of qmacs10s is too large). If the array of hybrid multiply-accumulate circuit 20 is too small, the vector multiplication must be decomposed into smaller vectors.
[0032] As shown in FIG. 6, a two-dimensional multiplication array of single-bit multiply-add circuit 10 can perform multi-bit multiplication (e.g., as shown in FIGS. 7 and 8). A hybrid multi-bit multiplier 22 consisting of multiple arrays such as those in FIGS. 8 and 9 forming a hybrid matrix multiply-add circuit 24 can compute the entire vector multiplication. Each multi-bit multiplication of a vector multiply-add operation (e.g., as shown in FIG. 1B) can generate a digital product (as shown in FIG. 7 or after analog-to-digital conversion of the analog sum output value O), which can be digitally added using a digital adder. In some embodiments, each multi-bit multiplication of a vector multiply-add operation (e.g., as shown in FIG. 1B) can generate an analog product (output value O as shown in FIG. 8), which can be added using similar circuitry as shown in FIGS. 1-6. A circuit similar to bit multiplier 14 can be used to store the analog product P (shown in FIG. 8) in a capacitor (e.g., similar to bit capacitor 16 but with a larger storage capacity for a larger charge). As shown in Figure 12, switch and clear circuits 18, 19 similar to those in Figure 5 can store charge Q on capacitors, sum the charges by connecting the capacitors in parallel, and convert the summed charges with an analog-to-digital converter 30 to provide the entire vector-matrix multiplication in one cycle. Figure 11A shows a hybrid matrix multiply-accumulate circuit 24, and Figure 11B associates the hybrid multi-bit multiplier 22 with the multiplicands in a vector multiply-accumulate calculation.
[0033] The embodiments of the present disclosure can provide a very low voltage multiply-accumulate circuit 10 using, for example, a voltage of 10 mV to 1 V. A bit multiplier 14 using a conventional AND gate may require, for example, six relatively large transistors operating at a relatively high voltage to implement a bit multiplication circuit that can adequately control the charge Q stored in the analog storage circuit 16 (for example, 1.65 to 5 V). In contrast, as shown in FIG. 13, the bit multiplier 14 of the present disclosure can operate at a relatively low voltage (for example, not exceeding 1 V, and as low as 10 mV) and low power, and can configure a series-connected series switch circuit 15 that can adequately control the charge Q stored in the analog storage circuit 16 with, for example, only four relatively small transistors.
[0034] As shown in Figure 13, a series of three serial switch circuits 15 and an analog storage circuit 16 can provide a qmac 10 functionally similar to that shown in Figures 4A and 4B. Each serial switch circuit 15 has two differential voltage inputs V and Vbar (where Vbar is the inverse of V), two voltage inputs In and Inbar (where Inbar is the inverse of In), and an output O. Thus, each of the signals A, B, and switches (described in more detail below) in Figures 13 and 14 are differential signals. The first serial switch circuit 15 has a reference voltage V REFP (For example, V REF , a high or positive value such as 10 mV) and its inverse value V REFN (e.g., a low or negative value such as 0 mV) and has two inputs, a value A (e.g., a weight value) and its inverse, Abar. As shown in the inset of FIG. 13 for serial switch circuit 15A, when A is high (e.g., positive or 10 mV) and Abar is consequently low (e.g., 0 mV), the output O is V, as shown by the undashed connection. REF The series switch circuit 15B is configured such that when A is low (e.g., negative or 0 mV) and Abar is consequently high (e.g., 10 mV), the output O is switched to V, as shown by the undashed line in the inset of FIG. REFNIt is.
[0035] Thus, if A is positive, O is positive, and if A is negative, O is negative. The second series switch 15 in series has an input value B and its inverse value Bbar, and transmits the value O from the first series switch 15 to V REFP Let it be a positive value, V REFN is an inverted voltage value (e.g., 0 volts). Thus, if O is low (negative), then the output P from the second serial switch circuit 15 will be low (negative) regardless of what value B is. If O is high (positive) and B is high (positive), then the output O from the second serial switch circuit 15 will be high (positive), and if B is low, then the output P from the second serial switch circuit 15 will be low (negative). In this way, the first two serial switch circuits 15 perform an AND function with less circuitry and power.
[0036] The third serial switch circuit 15 can be used to implement the switch circuit 18, and includes an input switch value and its inverse (corresponding to the switch values in FIGS. 4A and 4B), V REF The value O from the second serial switch 15 is taken as the inversion voltage value, and the common V SUM Thus, when the switch is high, the output O charges the analog storage circuit 16. When the switch is low, the charge Q on the analog storage circuit 16 is commonly connected to other analog storage circuits 16 in the array of qmacs10 (e.g., as shown as analog qmac10 array output in FIG. 3) to provide a summation operation.
[0037] FIG. 14 shows some embodiments of a low-voltage qmac 10 consisting of three series-connected series switch circuits 15. Each switch circuit 15 consists of a pair of simple MOS (metal oxide semiconductor) transistors with separate differential inputs and a common output. One of the pair of simple MOS transistors is controlled by a positive control signal, and the other is controlled by an inverted (negative) version of the same control signal, e.g., the positive and negative outputs of any single-bit storage element 12 (e.g., a pair of D flip-flops or inverters as shown and described with respect to FIG. 2). The function of the circuit is as described above with respect to FIG. 13. A series of such series switch circuits 15 can require fewer, simpler transistors that operate at much lower voltages (e.g., 10 mV instead of 1 percent or less, e.g., 0.624 percent, or 1.65 volts) and therefore require much less power. The combined (added) voltage of the analog storage circuit 16 can be as follows: V SUM =((n*V REFP )(Nn)*V REFN )) / N. Here, V REFN =0 Volts: V SUM =(n*V REFP ) / N, Here, n is the number of capacitors, and N is the number of qmacs10s connected in a row. SUM is scaled or transformed as described above (not shown in FIG. 14 with explicit circuit 19).
[0038] Thus, in accordance with some embodiments of the present disclosure, a hybrid matrix multiplier includes digital storage elements 12, each of the digital storage elements 12 operable to store a digital value, a multiplication circuit 14 for multiplying the stored digital values to generate a product, an analog storage circuit 16 operable to store the product, and a power connection (e.g., V REFP and V REFN). The power supply connections can have a voltage of 1V or less, 500mV or less, 100mV or less, 50mV or less, or 10mV or less. The bit multiplication circuit 14 can be comprised of switches 15 connected in series.
[0039] In some embodiments, the hardware implementation of the hybrid matrix multiply-accumulate circuit 24, the hybrid multi-bit multiplier 22, or the hybrid multiply-accumulate circuit 20 does not exactly match the computation desired for a particular application. In such applications, the computation can be divided into subproblems that better fit the available hardware, and the results can be combined to provide the desired computation. The subproblems can be executed sequentially in time, such that the hardware is time-shared or time-multiplexed. A portion of the value (e.g., bits of the multiplier B) can be stored in the storage element 12 for multiple hardware operations, thereby reducing the power and time used by the hardware.
[0040] The embodiments of the present disclosure enable vector multiply-accumulate calculations that use very little energy at a very high rate. Instead of requiring n loops of a program (e.g., as shown in Figures 1C and 1D, each of which requires multiple machine code cycles to run), the entire calculation is performed in one cycle. Many large matrix operations, for example in machine learning applications, have many zero values in the matrix and require relatively low bit precision to iterate the solution of the matching problem. Thus, the embodiments of the present disclosure provide an efficient circuit for such applications.
[0041] The embodiments of the present disclosure are not limited to the specific examples shown and described herein, and a skilled designer will readily appreciate that various implementations of analog and digital circuits can be employed to perform the described operations and that such implementations are included in the embodiments of the present disclosure.
[0042] Embodiments of the present disclosure may be used in neural networks, pattern matching computers, or machine learning computers to provide efficient and timely processing with reduced power and hardware requirements. Such embodiments may constitute computing accelerators, such as neural network accelerators, pattern matching accelerators, machine learning accelerators, or artificial intelligence computing accelerators designed for static or dynamic processing workloads.
[0043] Having described specific implementations of the embodiments, it will now be apparent to those of ordinary skill in the art that other implementations incorporating the concepts of the present disclosure may be used. Accordingly, the present disclosure should not be limited to specific implementations, but rather should be limited only by the spirit and scope of the following claims.
[0044] Throughout this specification, where devices and systems are described as having, including, or consisting of certain elements, or processes and methods are described as having, including, or consisting of certain steps, it is further contemplated that there are devices and systems of the disclosed technology that consist essentially of, or consist of, the disclosed elements, and that there are processes and methods according to the disclosed technology that consist essentially of, or consist of, the disclosed processing steps.
[0045] It should be understood that the order of steps or order for performing certain actions is immaterial so long as the disclosed technology remains operable. Moreover, in some circumstances, two or more steps or actions may be performed simultaneously. Although the disclosure has been described in detail with particular reference to certain embodiments thereof, it will be understood that variations and modifications can be effected within the spirit and scope of the following claims. [Explanation of symbols]
[0046] C Clear Circuit M multiplier circuit / multiplier O Output Value S Switch / Switch circuit VM Voltage Multiplier 10 qmac / Single-bit multiply-accumulate circuit 12 Single-bit storage element 14-bit multiplier / bit multiplier circuit 15, 15A, 15B Serial switch circuit 16 Capacitor / Analog Storage Circuits 18 Switch Circuit 19 Rear Circuit 20 Hybrid multiply-accumulate circuit 21 Product Sequence 22 Hybrid Multi-bit Multiplier 24 Hybrid matrix multiply-accumulate circuit 30 Analog to Digital Converter 40 Operational Amplifiers
Claims
1. An array of single-bit multiply-accumulate circuits, each of which comprises: (i) a first storage element for storing a first single-bit value; (ii) a second storage element for storing a second single-bit value; and (iii) a bit multiplier circuit for multiplying the first single-bit value by the second single-bit value to calculate a product; (iv) an analog storage circuit; and Equipped with the bit multiplication circuit is operable to store a charge representing the product in an analog storage circuit; A hybrid multiply-accumulate circuit, wherein the array of single-bit multiply-accumulate circuits is operable to combine charges stored in the analog storage circuits to provide an integrated charge representing a sum of products.
2. 2. The hybrid multiply-accumulate circuit of claim 1, wherein the analog storage circuit is a capacitor.
3. 2. The hybrid multiply-accumulate circuit of claim 1, further comprising: a switch circuit coupled to the bit multiply circuit and the analog storage circuit, the switch circuit operable in a first mode to transfer charge from the bit multiply circuit to the analog storage circuit and in a second mode to isolate the bit multiply circuit from the analog storage circuit and connect a plurality of the analog storage circuits to each other in the array to provide the accumulated charge.
4. 2. The hybrid multiply-accumulate circuit of claim 1, further comprising a clear circuit coupled to a plurality of said analog storage circuits of said array and operable to remove charge from a plurality of said analog storage circuits of said array.
5. 5. The hybrid multiply-accumulate circuit of claim 4, wherein each of the single-bit multiply-accumulate circuits comprises a clear circuit coupled to the analog storage circuit and operable to remove charge from the analog storage circuit.
6. 2. The hybrid multiply-accumulate circuit of claim 1, wherein said bit multiplying circuit is a functional AND gate.
7. 2. The hybrid multiply-accumulate circuit of claim 1, further comprising an analog-to-digital converter that converts the charges stored in said analog storage circuits in said array into digital accumulation values.
8. 8. The hybrid multiply-accumulate circuit of claim 7, further comprising a shift circuit or shift electrical connection for multiplying said digital storage value by a power of two.
9. 2. The hybrid multiply-accumulate circuit according to claim 1, further comprising a voltage multiplier for multiplying the charge stored in said analog storage circuit in said array by a power of two.
10. a first storage element for storing a first value; a second storage element for storing a second value; a multiplication circuit for multiplying a first value by a second value to calculate a product; A hybrid multiply-accumulate circuit having an analog storage circuit.
11. the first value and the second value are binary single-bit digital values; A hybrid multiply-accumulate circuit operable to store a charge in the analog storage circuit representing a product.
12. An array of hybrid multiply-accumulate circuits according to claim 7; a digital adder operable to add the digital product values to generate a digital matrix value; A hybrid matrix multiplier with
13. The hybrid matrix multiplier of claim 12 , wherein the digital adder is pipelined.
14. An array of hybrid multiply-accumulate circuits according to claim 1; an analog summer operable to sum the accumulated charges to generate an analog matrix value; A hybrid matrix multiplier with
15. 15. The hybrid matrix multiplier of claim 14, further comprising an op-amp coupled to the analog storage circuit and configured as a summer of op-amp inputs and operable to provide the analog matrix values.
16. 16. The hybrid matrix multiplier of claim 15, wherein the op-amp is configured to multiply or divide the op-amp input by a power of two.
17. 15. The hybrid matrix multiplier of claim 14, further comprising an analog-to-digital converter for converting the analog matrix values to produce digital matrix values.
18. 2. The hybrid multiply-accumulate circuit according to claim 1, wherein said bit multiplying circuit is composed of switches connected in series.
19. a plurality of digital storage elements operable to store digital values; a multiplication circuit for multiplying the stored digital values to produce a product; an analog storage circuit for storing said product; a power supply connection for providing power to operate the digital storage element, the multiplication circuit, and the analog storage circuit, the power supply connection having a voltage of 1V or less (e.g., 500mV or less, 100mV or less, 50mV or less, or 10mV or less); 2. The hybrid multiply-accumulate circuit according to claim 1, comprising:
20. 20. The hybrid multiply-accumulate circuit according to claim 19, wherein the multiplier circuit is composed of switches connected in series. Hybrid Matrix Multiplier The hybrid multiply-accumulate circuit includes an array of single-bit multiply-accumulate circuits, each single-bit multiply-accumulate circuit having a first storage element for storing a first single-bit value, a second storage element for storing a second single-bit value, a multiplication circuit for multiplying the first single-bit value by the second single-bit value to compute a product, and an analog storage circuit. The multiplication circuit is operable to store a charge representative of the product in the analog storage circuit. The analog storage circuit is operable to combine the charges stored in each analog storage circuit to provide an accumulated charge representative of the sum of the products. The hybrid matrix multiplier includes the array of hybrid multiply-accumulate circuits and an adder operable to add the accumulated values to generate a matrix value. The matrix value and the adder may be digital or analog.
Citation Information
Patent Citations
Inner product arithmetic circuit
JP1993324695A
Semiconductor device
JP2019168783A
Switched capacitor vector-matrix multiplier
US20210216280A1
Product-sum calculator
WO2021171880A1
