Composable AI-assisted compute fabric

Abstract

The compute fabric includes, in part, a number of compute tiles arranged within the memory blocks, a networking circuit coupled to the compute tiles and adapted to enable communication between the compute tiles and to allow the compute tiles to communicate with systems external to the compute fabric, and a controller configured to control the compute tiles. Each compute tile includes, in part, a number of multiplier bit cells (MBCs) arranged along M rows and N columns, where M and N are integers greater than 1. Each MBC is configured to multiply a first bit by a second bit to generate a multiplication value, convert the multiplication value into a charge, and store the charge in a capacitor arranged within the MBC.

Description

[Technical Field]

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS This application claims the benefit under 35 U.S.C. 119(e) of U.S. Patent Application No. 63 / 449,032, filed February 28, 2023, the contents of which are incorporated herein by reference in their entirety.

[0002] The present disclosure relates to compute fabrics, and more particularly to compute fabrics controlled by machine learning / artificial intelligence systems. [Background technology]

[0003] An important part of artificial intelligence and machine learning is the computationally intensive task of matrix multiplication. Matrix multiplication, or matrix product, is an arithmetic operation that generates one matrix from two matrices using entries in a field, or more generally, a ring or semiring. Matrix multiplication is designed to represent the composition of linear maps represented by matrices. Matrix multiplication is therefore a fundamental tool of linear algebra, and as such has numerous applications in many areas of mathematics, as well as in applied mathematics, statistics, physics, economics, and engineering. More specifically, if A is an n-by-m matrix and B is an m-by-p matrix, their matrix product AB is an n-by-p matrix, where the m entries across the rows of A are multiplied with the m entries down the columns of B and added to produce the entries of AB. When two linear maps are represented by matrices, the matrix product represents the composition of the two maps.

[0004] Computing matrix multiplication is a central operation in all computational applications of linear algebra. Its computational complexity is O(n 3 ) (for an n×n matrix) (this complexity is O(n 2.373)). This nonlinear complexity means that matrix multiplication is often an important part of many algorithms. This is reinforced by the fact that many operations on matrices have the same complexity, such as matrix inversion, determinants, and solving systems of linear equations. Therefore, various algorithms have been devised to compute large matrix products, taking into account the architecture of the computer.

[0005] Matrix multiplication is at the heart of all machine learning algorithms and is the most computationally expensive task in these applications. Most machine learning implementations use general-purpose CPUs and perform matrix multiplication in a serial fashion. Serial computation in the digital domain, together with limited memory bandwidth, limits the maximum throughput and power efficiency of a computing system. Summary of the Invention

[0006] A compute fabric according to one embodiment of the present disclosure includes, in part, a number of compute tiles arranged in memory blocks, a networking circuit coupled to the compute tiles and adapted to enable communication between the compute tiles and further enable the compute tiles to communicate with systems external to the compute fabric, and a controller configured to control the compute tiles. Each compute tile includes, in part, a number of multiplier bit cells (MBCs) arranged along M rows and N columns, where M and N are integers greater than 1. Each MBC is configured to multiply a first bit by a second bit to generate a multiplication value, convert the multiplication value to a charge, and store the charge in a capacitor arranged in the MBC.

[0007] In one embodiment, the number of multiplication bitcells is configured to multiply a first binary number by a second binary number, the first bit being a bit located in the first binary number and the second bit being a bit located in the second binary number. In one embodiment, the controller is configured to control power usage associated with the number of multiplication bitcells. In one embodiment, the controller is configured to control latency associated with the number of multiplication bitcells.

[0008] In one embodiment, the controller is configured to control throughput associated with the number of multiplier bitcells. In one embodiment, the controller is configured to control parallelization of the number of compute tiles. In one embodiment, the controller is configured to control the flow of data between the number of compute tiles and the networking circuitry. In one embodiment, each MBC includes, in part, circuitry configured to perform multiply-and-accumulate (MAC) operations and static random access memory cells. In one embodiment, a first binary number is input to the compute fabric, and a second binary number is stored in a memory block.

[0009] In one embodiment, the controller is configured to control the resolution of the compute tile by dynamically programming a number of clock cycles, where a first binary number is provided to at least one of the plurality of compute tiles corresponding to the number of clock cycles. In one embodiment, the controller is configured to control the resolution of the compute tile by selecting a number of memory cells used for a MAC operation. In one embodiment, the controller is configured to control the resolution of the compute tile by programming a number of steps to be performed in a binary search associated with a successive approximation register disposed within the compute tile.

[0010] In one embodiment, the compute fabric receives a first set of input bits associated with a first matrix, receives a second set of input bits associated with a second matrix, distributes a first subset of the first input bits to a first group of compute tiles, distributes a second subset of the first input bits to a second group of compute tiles, distributes a first subset of the second input bits to a third group of compute tiles, distributes a second subset of the second input bits to a fourth group of compute tiles, and The processor is further configured to: instruct the first group of compute tiles and the third group of compute tiles to generate a matrix multiplication of a first subset of the one input bits with a first subset of the second input bits to generate a first partial sum; instruct the second group of compute tiles and the fourth group of compute tiles to generate a matrix multiplication of a second subset of the first input bits with a second subset of the second input bits to generate a second partial sum; and combine the first and second partial sums to generate a result of the multiplication of the first matrix with the second matrix.

[0011] In one embodiment, the compute tiles are arranged along one or more rows. In one embodiment, the compute tiles are arranged along one or more columns. In one embodiment, the compute tiles are arranged along an array of one or more rows and one or more columns. In one embodiment, the controller is configured to control the resolution of a successive approximation register (SAR) analog-to-digital converter (ADC) arranged within the compute tile. In one embodiment, the controller is configured to vary a reference voltage used by the ADC. In one embodiment, the controller is configured to vary the number of calculations performed by the compute tile.

[0012] In one embodiment, the compute fabric further includes, in part, a performance monitor, and the controller is trained to vary the configuration of the compute fabric via reinforcement learning, in part, including setting a configuration state of the compute fabric to a first state, measuring a compute fabric performance characteristic with the performance monitor, receiving a reward signal in response to the measured performance characteristic, and repeating the setting, measuring, and receiving until the received reward reaches a maximum value.

[0013] In one embodiment, the performance characteristics include one or more of power usage, throughput, latency, and resolution. In one embodiment, the configuration state of the compute fabric is defined by one or more of the data path width between compute tiles, the number of bits of input data into which a first bit is placed, the resolution of a successive approximation register (SAR) analog-to-digital converter (ADC) associated with the compute tile, a reference voltage used by the ADC, and the number of calculations performed by the compute tile.

[0014] A method of computation according to one embodiment of the present disclosure includes, in part, forming a number of compute tiles within a memory block, enabling communication among the compute tiles and between the compute tiles and an external system, and controlling the compute tiles. Each compute tile includes, in part, a number of multiplier bit cells (MBCs) arranged along M rows and N columns, where M and N are integers greater than 1. Each MBC is configured to multiply a first bit by a second bit to generate a multiplication value, convert the multiplication value to a charge, and store the charge in a capacitor disposed within the MBC.

[0015] In one embodiment, the multiplier bitcells are configured to multiply a first binary number by a second binary number, the first bit being a bit located in the first binary number and the second bit being a bit located in the second binary number. The method, according to one embodiment, includes, in part, varying power usage associated with the plurality of multiplier bitcells. The method, according to one embodiment, includes, in part, varying latency associated with the multiplier bitcells.

[0016] The method includes, in part, varying a throughput associated with a multiplier bitcell, in accordance with one embodiment. The method includes, in part, varying a parallelization of a compute tile, in accordance with one embodiment. The method includes, in part, varying a flow of data between compute tiles, in accordance with one embodiment.

[0017] In one embodiment, each MBC includes, in part, circuitry configured to perform multiply-accumulate (MAC) operations and static random access memory cells. In one embodiment, a first binary number is input to the compute fabric and a second binary number is stored in the memory block.

[0018] The method, according to one embodiment, includes, in part, varying the resolution of the compute tile by dynamically programming a number of clock cycles, where a first binary number is provided to at least one of the plurality of compute tiles corresponding to the number of clock cycles. The method, according to one embodiment, includes, in part, varying the resolution of the compute tile by selecting a number of memory cells used for the MAC operation. The method, according to one embodiment, includes, in part, controlling the resolution of the compute tile by programming a number of steps performed in a binary search associated with a successive approximation register located within the compute tile.

[0019] According to one embodiment, the method includes receiving a first set of input bits associated with a first matrix, receiving a second set of input bits associated with a second matrix, distributing a first subset of the first input bits to a first group of compute tiles, distributing a second subset of the first input bits to a second group of compute tiles, distributing a first subset of the second input bits to a third group of compute tiles, distributing a second subset of the second input bits to a fourth group of compute tiles, and The method includes, in part, instructing the first group of compute tiles and the third group of compute tiles to generate a matrix multiplication of a first subset of the one input bits and a first subset of the second input bits to generate a first partial sum; instructing the second group of compute tiles and the fourth group of compute tiles to generate a matrix multiplication of a second subset of the first input bits and a second subset of the second input bits to generate a second partial sum; and combining the first and second partial sums to generate a result of the multiplication of the first matrix and the second matrix.

[0020] In one embodiment of the method, the compute tiles are arranged along one or more rows. In one embodiment of the method, the compute tiles are arranged along one or more columns. In one embodiment of the method, the compute tiles are arranged along an array of one or more rows and one or more columns.

[0021] In one embodiment of the method, the controller is configured to control the resolution of a successive approximation register (SAR) analog-to-digital converter (ADC) located within the compute tile. In one embodiment, the method includes, in part, varying a reference voltage used by the ADC. In one embodiment, the method further includes, in part, varying the number of calculations performed by the compute tile.

[0022] In one embodiment, the method further includes, in part, setting a configuration state of the compute fabric to a first state, measuring a compute fabric performance characteristic, receiving a reward signal in response to the measured performance characteristic, and repeating the setting, measuring, and receiving until the received reward reaches a maximum value.

[0023] In one embodiment of the method, the performance characteristics include, in part, one or more of power usage, throughput, latency, and resolution. In one embodiment of the method, the configuration state of the compute fabric is defined by one or more of a data path width between compute tiles, a number of bits of input data into which a first bit is placed, a resolution of a successive approximation register (SAR) analog-to-digital converter (ADC) associated with the compute tile, a reference voltage used by the ADC, and a number of calculations performed by the compute tile.

[0024] The following detailed description, figures, and appended claims illustrate properties and advantages of innovations, embodiments, and / or examples of the claimed invention. All figures depict innovations, embodiments, and / or examples of the claimed invention for illustrative purposes only and do not limit the scope of the claimed invention. Such figures are not necessarily drawn to scale and are part of this disclosure. [Brief explanation of the drawings]

[0025] [Figure 1] FIG. 10 is a top-level diagram of a multiplier according to one embodiment of the present disclosure. [Figure 2A] FIG. 1 illustrates an exemplary switched-capacitor matrix multiplier using successive approximation registers (SARs) for use with embodiments of the present disclosure. [Figure 2B] FIG. 1 illustrates an exemplary switched-capacitor matrix multiplier using successive approximation registers (SARs) for use with embodiments of the present disclosure. [Figure 2C]FIG. 1 illustrates an exemplary switched-capacitor matrix multiplier using successive approximation registers (SARs) for use with embodiments of the present disclosure. [Figure 3A] FIG. 2 illustrates an exemplary 2-bit switched-capacitor matrix multiplier, in accordance with one embodiment of the present disclosure. [Figure 3B] FIG. 2 illustrates an exemplary 2-bit switched-capacitor matrix multiplier, in accordance with one embodiment of the present disclosure. [Figure 4A] FIG. 2 illustrates an exemplary 3-bit switched-capacitor matrix multiplier, in accordance with one embodiment of the present disclosure. [Figure 4B] FIG. 2 illustrates an exemplary 3-bit switched-capacitor matrix multiplier, in accordance with one embodiment of the present disclosure. [Figure 4C] FIG. 2 illustrates an exemplary 3-bit switched-capacitor matrix multiplier, in accordance with one embodiment of the present disclosure. [Figure 4D] FIG. 2 illustrates an exemplary 3-bit switched-capacitor matrix multiplier, in accordance with one embodiment of the present disclosure. [Figure 4E] FIG. 2 illustrates an exemplary 3-bit switched-capacitor matrix multiplier, in accordance with one embodiment of the present disclosure. [Figure 4F] FIG. 2 illustrates an exemplary 3-bit switched-capacitor matrix multiplier, in accordance with one embodiment of the present disclosure. [Figure 4G] FIG. 2 illustrates an exemplary 3-bit switched-capacitor matrix multiplier, in accordance with one embodiment of the present disclosure. [Figure 4H] FIG. 2 illustrates an exemplary 3-bit switched-capacitor matrix multiplier, in accordance with one embodiment of the present disclosure. [Figure 5] FIG. 10 illustrates how the worst-case error in a MAC result can be calculated for an imperfect MAC operation. [Figure 6] FIG. 2 is a high-level simplified schematic diagram of a multiplication bitcell according to one embodiment of the present disclosure. [Figure 7] FIG. 2 illustrates an array of multiplication bit cells according to one embodiment of the present disclosure. [Figure 8] FIG. 2 illustrates connections between an array of multiplication bit cells according to one exemplary embodiment of the present disclosure. [Figure 9] FIG. 1 illustrates a simplified high-level block diagram of a hardware accelerator compute fabric, according to one embodiment of the present disclosure. [Figure 10] FIG. 10 illustrates the hardware accelerator compute fabric of FIG. 9 being controlled by a machine learning agent in accordance with another embodiment of the present disclosure. [Figure 11] FIG. 1 illustrates a configurable compute fabric according to one embodiment of the present disclosure. [Figure 12] 12 illustrates example source and destination addresses in an instruction cache of the compute fabric of FIG. 11 that instruct a controller of the compute fabric how to configure the data path of the compute fabric, according to one embodiment of the present disclosure. [Figure 13] FIG. 12 illustrates a controller and some other blocks of the compute fabric of FIG. 11 during a training phase, in accordance with one embodiment of the present disclosure. DETAILED DESCRIPTION OF THE INVENTION

[0026] One aspect of the present disclosure relates to a general-purpose, low-power, switched-capacitor vector-matrix multiplier (VMM). Significant power efficiency is achieved by performing multiply-accumulate operations in the analog domain and storing weight values ​​locally, eliminating power-intensive data communication between memory and a computing device. The vector-matrix multiplier computes N dot products of n-dimensional inputs and m-dimensional weights in parallel, as shown in equation (1) below.

[0027]

number

[0028] The dot product multiplication described by equation (1) can be expanded bitwise as follows:

number

[0029]

number

number

[0030] In general, N, n, and m set an upper limit for k (the output resolution) in equation (1). k≦n+m+log2(N) (3)

[0031] For n-bit inputs and m-bit weights, nm cycles will be required to compute the result, where "." represents a multiplication operation. However, if k is set lower than its upper limit, not all cycles are required. For example, for a 256-input dot-product multiplier with 8-bit inputs, weights, and outputs, and a successive approximation register (SAR) described further below with 8-bit resolution, only 49 cycles (as opposed to 64) are required to ensure that the approximate multiply-accumulate (MAC) result is within 1 least significant bit (LSB) of its true value in the worst case where all inputs and weights are 255 (with random inputs and weights, this further reduces to only 36 cycles).

[0032] Output resolutions higher than that of the SAR can be achieved by performing SAR quantization on the partial MAC results one or more times throughout the MAC operation. For example, a 16-bit output can be achieved by performing SAR quantization once every eight MAC cycles, and then scaling and summing the results in the digital domain. In this manner, any output resolution from 1 to n+m+log2(N) can be achieved with this architecture.

[0033] Analog implementation provides a natural medium for implementing fully parallel computational arrays with high integration density and energy efficiency. By summing the charge on each capacitor in a large capacitor bank, a switched-capacitor vector-matrix multiplier can achieve massively parallel multiply-accumulate with low latency.

[0034] The switched-capacitor vector-matrix multiplier includes a successive approximation register (SAR) analog-to-digital converter (ADC) (located in FIG. 2A , described below) per MAC, or per neuron, as further described below with reference to FIG. 8 . The capacitor bank of each SAR is not binary-weighted but is expanded to provide one connection per multiplication bit cell, as further described below. The inputs and weights are implemented digitally in a bit-serial manner. An SAR architecture according to one embodiment provides a low-power solution that essentially includes both a DAC and an ADC. While the DAC converts the digital code back to an analog charge that is stored in the MAC operation, the ADC quantizes the result of each MAC operation, which can be used as an input to the next stage. This process makes the architecture scalable and can be cascaded many times to implement very large neural networks. Performing quantization with a SAR ADC has the added benefit of dynamically reducing the resolution of the result for applications requiring faster, more power-efficient, but less accurate, computation. An NxN matrix multiplier can be constructed by an array of N switched-capacitor vector-vector internal multipliers, each with N inputs and a log2(N)-bit resolution SAR. The distributed nature of the local storage of the results eliminates the need for high-bandwidth memory, significantly increasing the power efficiency of the system. Furthermore, digital storage of the results allows for reconfigurable digital post-processing, which can be used to apply nonlinearities.

[0035] Multiplication of matrices larger than the physical structure of the switched capacitor matrix multiplier can be achieved by performing partial matrix multiplications of the size of the available switched capacitor matrix multiplier, then storing and recombining the partial results locally.

[0036] The digital interface of the SAR's capacitive DAC input and state machine output can be modified to incorporate dot-product calculations into the SAR. By multiplying the digital output of the SAR's state machine by the bitwise product of the input and weight, the SAR can operate in two distinct phases: an accumulation phase, in which the input and weight are multiplied bitwise using a simple AND gate and the result is stored at the shared node of the capacitive DAC, and a conversion phase, in which a regular SAR operation results in digital quantization of the accumulated result. More resolution can be incorporated into the final MAC output by scaling down the previous MAC result by a factor of two and adding it to the MAC result of the next consecutive bit of the input or weight before SAR quantization begins. In this manner, the resolution of the input and weight can be arbitrarily high and set on the fly, at the expense of energy and speed.

[0037] The interface between the DAC inputs of the SAR and the state machine outputs can be embedded in memory. By storing weights locally using cross-coupled inverters, memory accesses and calculations can be performed locally, while avoiding the need for energy-intensive data movement in and out of memory. Such a distributed memory system can be thought of as a static random access memory (SRAM) with embedded bitwise multipliers (AND gates), whose memory cells are capacitively coupled to the bit lines (the shared nodes of the capacitive DAC) through the SAR's unit capacitors. In this manner, all bits stored in the SRAM can be read simultaneously, as long as the SAR has sufficient precision to resolve the amount of charge injected by a single memory cell. Due to this in-memory calculation, significant area and power savings can be achieved.

[0038] FIG. 1 illustrates a top-level view of an exemplary switched-capacitor vector-vector internal multiplier 100. In FIG. 1, multiplier 100 receives an n-bit input (X0...X N-1 )102 and m-bit weights (W0...W N-1) 103. The multiplier 100 produces one k-bit output (Y) 104.

[0039] FIG. 2A illustrates an exemplary successive approximation register (SAR) 200 ADC implementation for use with some embodiments of the present disclosure. The SAR 200 is shown as comprising multiple (e.g., six in FIG. 2A) connected MAC circuits 201A-201F. Each of the MAC circuits 201A-201F (shown in FIG. 2B as an expanded view of 201) comprises a multiplexer 206, a capacitor 205, a successive approximation register (SAR) local memory 228 for storing weight values, an AND gate 209, and an inverter 208, with the AND gate and inverter connected in series. A 2-bit input 207 to the multiplexer 206 selects between the output of the AND gate 209 for a value of '0', the output of the inverter 208 for a value of '1', ground 204 for a value of '2', and an input 210 from the SAR state machine 218 for a value of '3' to drive the SAR capacitor 205.

[0040] In some embodiments, SAR 200 further comprises reset switches S1-S7 211-217. Switches S4 214 and S5 215 connect the shared output of MAC circuits 201A and 201B 222 to ground and V mid Connect to V mid is set to half the SAR supply voltage. Switches S2 212 and S3 213 connect the shared output of MAC circuits 201C and 201D 223 to ground and V mid , respectively. Switch S1 211 connects the shared output of MAC circuits 201E and 201F 221 to ground. Switch S6 216 connects the shared output of MAC circuits 201E and 201F 221 to the shared output of MAC circuits 201C and 201D 223. Switch S7 217 connects the shared output of MAC circuits 201A and 201B 222 to the shared output of MAC circuits 201C and 201D 223. Timing diagram 227 illustrates the orientation of switches S1 through S7 within the analog MAC operation and quantization stage of the matrix multiplier. Signal

[0041]

number

number

[0042]

number

[0043] In an embodiment, SAR 200 further comprises a comparator 219 and a state machine 218. Comparator 219 supplies a reference voltage V to provide an input for state machine SM218. ref 220 with output voltages 221 of MAC circuits 201E-201F. State machine SM218 provides output b0 224 that is fed back to MAC circuit 201E, output b1 225 that is fed back to MAC circuits 201C and 201D, and a two-bit output Range_sel, one bit of which is fed back to MAC circuit 201A and the other bit of which is fed back to MAC circuit 201B.

[0044] In some embodiments, the SAR 200 receives a two-bit signal

number

number

[0045]

number

number

[0046]

number

number

[0047]

number

number

number

[0048] 3A and 3B illustrate a 2-bit implementation of an exemplary switched-capacitor matrix multiplier 300 according to one embodiment of the present disclosure. In some embodiments, the switched-capacitor matrix multiplier 300 comprises a plurality of (e.g., four in FIGS. 3A and 3B) SARs 200A-200D, each having a configuration as described and depicted with respect to FIG. 2A. In some embodiments, the switched-capacitor matrix multiplier 300 further comprises a plurality of (e.g., four in FIGS. 3A and 3B) local memories 301A-301D for storing input values.

[0049] Figures 4A, 4B, 4C, 4D, 4E, 4F, 4G, and 4H illustrate a 3-bit implementation of an exemplary switched-capacitor matrix multiplier 400 according to one embodiment of the present disclosure. As shown, the switched-capacitor matrix multiplier 400 comprises multiple SARs (e.g., eight as shown), each having a configuration similar to that discussed above with reference to Figures 2A and 2B. However, as shown in Figures 4A, 4B, 4C, 4D, 4E, 4F, 4G, and 4H, each SAR may comprise 12 MAC circuits, each having functionality similar to that discussed above. Such a configuration provides a matrix multiplier with a different resolution and number of inputs than those shown in Figures 2A and 2B. Specifically, the configurations of Figures 4A-4H provide a 3-bit matrix multiplier configured to utilize eight inputs. It should be understood that the 2-bit matrix multiplier configurations of Figures 2A and 2B and the 3-bit matrix multipliers of Figures 4A-4H are merely examples of various matrix multiplier configurations in which various embodiments may be implemented.

[0050] Figure 5 illustrates how the worst-case error in a MAC result can be calculated. The worst-case error can be calculated by summing the numbers in the figure starting from the top right corner and moving down and to the left. Thus, the worst-case error after 49 cycles is:

number

[0051] It should be understood that "weight" and "per-bit" weight are used interchangeably herein.

[0052] Example Multiply-Accumulate (MAC) and Quantization Operations The MAC stage or operation of an exemplary switched-capacitor vector-matrix multiplier multiplies the least significant bits (LSBs) (k=j=0) of the inputs and weights,

number

[0053]

number

number

[0054] Switch S7 then closes, shorting nodes 222 and 223 in FIG. 2A, and the top plates of the SAR capacitors in MAC circuits 201C and 201D are

number

[0055]

number

[0056]

number

[0057] S7 closes again,

number

number

[0058] In the quantization stage / operation / process of an exemplary switched-capacitor vector-matrix multiplier,

number

[0059]

number

number

[0060] As described above, in accordance with some embodiments of the present disclosure, matrix multiplication is performed entirely within a memory, such as a static random access memory (SRAM). FIG. 6 is a high-level simplified schematic diagram of a multiplication bit cell (MBC) 600 according to one embodiment of the present disclosure. The MBC 600 is shown as including, in part, an SRAM cell 610 configured to store weights and a MAC unit 601, as described in detail above, e.g., MAC circuits (units) 201A-201F, as described with reference to FIGS. 2A and 2B. The MBC 600 is shown as receiving, in part, input signals I1 and I3 and generating an output signal Vout representing the product of signals I1 and I3. Signal I1 represents the input signal applied to the MBC 600, and signal I3 represents the weight with which signal I1 is multiplied. The weight represented by signal I3 may be stored within the SRAM cell 610. The integration of the MBC does not affect the operation of the SRAM cells storing the weights. MBC 600 can be activated or deactivated for multiplication with logic input signal I2. The SRAM cells remain operable for read / write access as memory cells independent of the logic state of signal I2. As a result, MBC 600 has dual functionality, operating as a standard memory element, as depicted by back-to-back inverters synonymous with SRAM, and as a multiplication bit cell, accessible to higher-level systems, as further explained below.

[0061] 7 shows an M×N array 700 of MBCs according to one embodiment of the present disclosure. For example, row 1 contains MBCs 600 11 , 600 12 , and 600 1N and row M is shown as containing MBC600 M1 , 600 M2 , and 600 MN Each MBC600 ij corresponds to the MBC 600 shown in Figure 6, where i and j refer to the row and column numbers, respectively, in which the cell is arranged. According to one embodiment, the array 700 of MBCs is arranged entirely within an SRAM.

[0062] 8 illustrates connections between an exemplary M×N array 800 of MBCs, where M represents the number of rows in the array 800 and N represents the number of columns in the array 800. Each column of the array 800 is referred to herein as a neuron. For example, MBC cells 800 11 , 800 21 ...800 M1 Column 1 of array 800 containing MBC cell 800 is shown as being associated with neuron 8051. Similarly, for example, 1N , 800 2N ...800 MN Column N of array 800 containing N is shown as being associated with neuron 805N. Array 800 therefore contains N neurons.

[0063] Array 800 is disposed within a memory block (e.g., a block of SRAM) configured to perform matrix multiplication. In one embodiment, the matrix multiplication may be performed in the analog domain, as described above, using a digital-to-analog (DAC) converter, a multiply-and-accumulate (MAC) circuit, and an analog-to-digital (ADC) converter. It should be understood that in one embodiment, the DAC, MAC, and ADC operations may be performed entirely within the memory block, e.g., an SRAM block, in which array 800 is disposed. Such a memory block, denoted as array 800, is alternatively referred to herein as a Compute-and-Quantize-In-Memory (CQIM) block, or alternatively referred to herein as a CQIM array. In one embodiment, the matrix multiplication may be performed in the digital domain using digital multiplication circuitry.

[0064] As explained above, each column of the CQIM array 800 forms a neuron adapted to multiply one or more inputs by one or more weights and accumulate the result, also referred to herein as a vector-dot product. For example, the MBC cell 800 ijCQIM array 800, shown as including an M×N array of MBC cells 800 (each of which corresponds to MBC 600 of FIG. 6), is arranged in rows 8001 and 8002. 11 , 800 12 , 800 21 , and 800 22 The MBC cell 800 may be configured to perform a vector-dot product of input activation signals IA1 and IA2 applied to the MBC cell 800, where i is a row index ranging from 1 to M and j is a column index ranging from 1 to N. 11 , 800 12 , 800 21 , and 800 22 The weights stored in 11 , W 12 , W 21 , and W22 22 , the vector-dot product can be expressed as shown below:

[0065]

number

[0066] In one embodiment, each MBC forms a CQIM tile configured to perform matrix multiplication. In another embodiment, two or more MBCs form a CQIM tile. Such two or more MBCs may be located in the same row, or in the same column, or in different rows and columns. For example, in one embodiment, MBC 800 11 , 800 12 MBCs such as the MBC 800 may be configured to form CQIM tiles. 11 , 800 21 MBCs such as 800 may be configured to form tiles. 11 , 800 12 , 800 21 , and 800 22MBCs arranged in different rows and columns, such as 800, 801, 802, 803, 804, 805, 806, 807, 808, 809, 810, 811, 812, 813, 814, 815, 816, 817, 818, 819, 820, 821, 822, 823, 824, 825, 826, 827, 828, 829, 830, 831, 832, 833, 834, 835, 836, 837, 838, 839, 840, 841, 842, 843, 844, 845, 846, 847, 84 11 and 800 MN may be configured to form tiles.

[0067] Each row of the array is shown as receiving an Input Activate (IA) signal. For example, IA1 is the MBC800 11 , 800 12 , and 800 1N is shown as being applied to the IA M is MBC800 M1 , 800 M2 , and 800 MN is shown as being applied to the IA k is MBC600 k1 , 600 k2 , and 600 kN where k is a row index ranging from 1 to M in this example. Each input activate signal corresponds to a different signal I1 shown in FIG.

[0068] IA with a value represented by one or more bits received by each MBC i The signal is multiplied by a weight, e.g., stored in its MBC, as described in detail above. The result of each such multiplication is then converted to a charge by a capacitor located in its MBC, such as capacitor 605 shown in FIG. 6. For example, IA1 is multiplied by a weight stored in its MBC 800. 11 The result is converted to a charge and multiplied by the weight stored in the MBC800 21 IA2 is supplied to MBC800 21 The result is converted to a charge and multiplied by the weight stored in the MBC800 11 Such charges are then accumulated sequentially within each neuron to produce a MAC result for that neuron.

[0069] 6, a logic block 620 located within MBC cell 600, in part in response to signal I2, sends the multiplication result to capacitor 625 for storage before the result is converted to an electrical charge. In one embodiment, the logic block may be configured to provide signals that control the serial and parallel operations of MBC cell 600.

[0070] Each neuron 805 j is logical block 825 j , and a comparator 815 receiving a reference voltage VREF. j As described in detail above with reference to FIGS. 2A-2C, 3A-3B, and 4A-4H, comparator 815 j is part of the integrated SAR that performs the binary search. j further drives the signal applied to MAC circuit 201A shown in FIG. 2A, for example. Application of a voltage derived from the same VREF that the comparator is comparing allows the binary search algorithm to iterate until the desired number of SAR output bits is achieved. The MAC results are then converted, processed by the SAR operation described above, and quantized on a column-by-column basis as shown in array 800 above.

[0071] According to some embodiments of the present disclosure, a user may select the IAs, weights, and neurons 8051, 8052...805 N Output activate (OA) signals OA1, OA2...OA are respectively provided by N It allows you to independently program (i.e., configure) the resolution of calculations for all data types, including IA data types. j The values ​​are represented by digital bits and applied sequentially, see for example FIG. 2, so that the user can j By programming the number of clock cycles required for bits to be sequentially fed to their associated CQIM tiles, the IA jThe resolution of the value bits can be dynamically set, where j is the row index ranging from 1 to M in the example shown in FIG. 8. The number of clock cycles is j The OA bit resolution determines how many of the value bits are processed by the CQIM tile. As shown in FIGS. 2A-2C, 3A-3B, 4A-4H, 6, and 8, when the desired number of clock cycles has been processed by the circuit, logic block 601 of the MBC block as shown in FIG. 6 is instructed to start the SAR operation and complete the quantization. The resolution of the weight parameters can be set by selecting the number of memory bit cells used for the MAC operation. As shown in FIGS. 2A-2C, 3A-3B, and 4A-4H, for a 600 (see FIG. 6) n-cell MBC, the resolution of the weight values ​​can be set using only a subset of blocks 201A-201F shown in FIG. 2A. The resolution of the OA bits can be set by programming the number of steps in the SAR binary search as described in the previous paragraph. For example, if the number of SAR conversion steps is set to 4, the SAR result will be a 4-bit digital output.

[0072] FIG. 9 is a simplified high-level block diagram of a hardware accelerator compute fabric (HACF) 900 according to one embodiment of the present disclosure. The HACF 900 is formed using multiple CQIM tiles that communicate with each other and external devices via a network-on-chip (NoC) 950 to perform computations. Only six CQIM tiles 952, 954, 956, 958, 960, and 962 are shown in FIG. 9 . However, the HACF 900 may have more or fewer CQIM tiles than six. In one embodiment, the CQIM tiles may implement one or more layers of a neural network system. Although not shown, the NoC 950 is configured to route the output of one layer of the neural network system to the input of a subsequent layer of the neural network system. Higher-dimensional matrix sizes may be distributed along multiple CQIM tiles, and the partial sums generated by each tile are summed together to generate the result of the matrix multiplication.

[0073] 10 is a block diagram of an HACF 1000 controlled by a machine learning agent 1010 according to another embodiment of the present disclosure. The HACF 1000 is shown as including, in part, multiple CQIM tiles 1052, 1054, 1056, 1058, 1060, and 1062, an NoC 1050, and a machine learning agent 1010. The HACF 1000 can be configured and controlled by the machine learning agent 1010 to, among other benefits, increase energy efficiency, reduce latency, increase throughput, enable parallelization of CQIM tiles (i.e., the number of CQIM tiles that can be connected in parallel by the machine learning agent 1010 of FIG. 10 to enable parallelization of operations within the compute fabric), optimize data flow on the NoC, and optimally map computations onto the compute fabric.

[0074] The switched-capacitor-based configurable matrix multiplier, SAR integrated CQIM tiles, and associated arrays described above can be optimized to achieve desired performance metrics, different modes of operation, such as power consumption, latency, throughput, and the like. Performance metrics can be measured using many different techniques. On-chip counters can count system or reference clock cycles to measure latency and throughput, which can be timed to program execution, program counters, or other timing and system management signals within the architecture. To measure power, for example, sense resistors can be placed around the chip to measure the current consumed by the design. Voltage can be measured near the point of load using sense amplifiers, current references, and ADCs. Using each of these measurements alone or in combination can provide a measure of power and energy from one section of a chip or multiple sections of a chip, allowing for optimization of consumed power and energy. For example, in one embodiment, described above with reference to the array 800 shown in FIG. 8, multiple CQIM tiles can be integrated together on the same die, across multiple dies, or across multiple systems.

[0075] 11 illustrates a configurable compute fabric 1100 according to one embodiment of the present disclosure. The compute fabric 1110 includes a dynamic random access memory (DRAM) 1102, a compute / memory datapath logic block (alternatively referred to herein as datapath logic) 1104, a compiled algorithm / workload / program 1106, an instruction SRAM / controller / cache 1108, a dynamic memory / compute allocation controller (alternatively referred to herein as a DMCA controller) 1112, and N compute modules 11101, 11102, ... 1110. N(collectively referred to herein as compute modules 1110) and a system performance monitor 1120. Each compute module 1110 k may be, for example, a CQIM tile as described above with reference to array 800 of Figure 8, or may be a digital compute block, where k is an integer that varies from 1 to N in this example. DMCA controller 1112 corresponds to machine learning agent 1110 shown in Figure 10.

[0076] The CTRL signals applied to the compute module 1110 are generated by a controller 1112. The DMCA controller generates control signals CTRL1, CTRL2...CTRL3 that control, for example, in-memory addressing, ADC resolution, calculation order, configurability for the calculation accuracy or precision of the compute fabric 1100, and the like. N , W2...W3 are provided by the datapath logic 1104. N and X1, X2...X N(X and W represent the data and weight to be multiplied together) control, among other things, the width, i.e., bit depth, of the data used by the compute module 1110. The compute module 1110 can be unified into one memory addressing space of the DRAM 1102, thus enabling a logical mapping of data onto the compute fabric 1100. Such mapping allows the compute order to be optimized so that the compute fabric is fully utilized by computing in parallel and sequentially across the fabric, with partial results computed and available to the next stage or layer of computation. As shown in FIG. 11, each compute module 1110 can be formed using one or more CQIM tiles, as shown in FIG. 8, or one or more digital compute blocks (we shall refer to the components of a digital compute block), thereby allowing flexibility in computation order and realizing more advanced arithmetic or bitwise operations beyond multiply-accumulate.

[0077] In one embodiment, the data path logic 1104 connects the compute modules 11101...1110 N 11. The datapath logic 1104a is a configurable network-on-chip (network-on-chip 950 shown in FIG. 9) interface adapted to connect together. The datapath logic 1104a can be a chip that conforms to any number of communications or networking protocols. External memory 1102, which can be DRAM, is also connected to the datapath logic 1104 to provide access to data needed by the compiled algorithm / workload / program 1106.

[0078] The DMCA controller 1112 is also shown connected to a system performance monitor 1120 that provides feedback to the DMCA controller regarding the response of the compute fabric 1100 to ongoing computational operations. The system performance monitor 1120 measures performance metrics such as throughput, latency, energy consumption, or any other performance metrics. The system performance monitor 1120 may be formed on the same die that includes the compute module 1110. Alternatively, the system performance monitor 1120 may be off-chip, or both on-chip and off-chip, and may be adapted to provide detailed metrics of performance at the SoC or system level. The DMCA controller 1112 is further configured to control the DRAM 1102, or any other memory, internal or external, to optimally load data used by the compiled algorithm / workload / program into the compute module 1110 or other components of the compute fabric 1100.

[0079] The DMCA controller 1112 is further configured to control the flow of data between the compute modules 1100 by configuring the data path width between the compute modules and the memory 1102. The DMCA controller is further configured to decode instructions received from the instruction cache 1108 and provide commands to the compute modules and other components of the compute fabric to execute programs. Figure 12 is an example of source and destination addresses in the instruction cache 1108 that instruct the DMCA controller 1112 how to command and configure the data path between the compute modules and the memory 1102.

[0080] Control signals provided by the DMCA controller 1112 provide flexibility and selectivity within the compute fabric 1100. These control signals configure state and control registers within each compute module 1110. In an embodiment in which the compute modules 1110 are CQIM tiles, such as those shown in array 800 of FIG. 8, the control signals configure the bit resolution of the weights and inputs, the number of inputs included in the multiply-accumulate function, the output resolution of the SAR ADC, the reference voltage for the ADC, row configuration and merging, the number of calculations performed, and increase performance. Control signals with more bits (i.e., wider) can be used to control system-level parameters of the compute fabric, such as supply voltage, ambient temperature of the system, and other parameters.

[0081] The compiled algorithm / workload / program 1106 is loaded onto the compute fabric 110 and stored in an instruction cache. The instruction buffer is connected to and provides instructions to the DMCA 1112 as shown in Figure 11. The DMCA controller 1112 issues instructions and configuration signals to the compute modules 1110 and memory 1102 for optimal computation.

[0082] In one embodiment, DMCA controller 1112 may be a machine learning agent (system), such as machine learning agent 1010 shown in FIG. 10, that may be trained on algorithms and measurable metrics of compute fabric 1100, such as throughput, latency, energy dissipation, and the like. As the machine learning system is trained through reinforcement learning techniques, it identifies an optimal configuration for compute fabric 1100 to meet performance metrics identified as important during training. The training also determines the optimal allocation of compute resources, memory allocation, and data path traffic to meet the performance metrics.

[0083] As an embodiment of training, reinforcement learning techniques may be utilized to optimally determine the configuration of compute fabric 1100 to meet performance requirements. In one embodiment, such reinforcement learning may use the flow shown in FIG. 13. FIG. 13 shows DMCA controller 1112 coupled to compute / memory block 1300. Compute / memory block 1300 includes, in part, compute module 1110, datapath logic 1104, memory 1102, and instruction cache 1108 of compute fabric 1100. A state variable, S, provided by compute / memory block 1300 T , S T+1 ... include, for example, instructions and control signals provided by the compute module 1110, the datapath logic 1104, the memory 1102, and the instruction cache 1108 of the compute fabric 1100. The state variables also represent modified values ​​of the instructions shown in Figure 12 that define the state of the system. The reward variable R shown in Figure 13 T is generated by the system performance monitor 1120 shown in Figure 12. As the DMCA 1112 configures the system during training, metrics such as power, throughput, and latency are monitored by the DMCA controller 1112 using the reward variable R T Generate a reward signal to learn how to configure the state variables to maximize the action variable A. T is a signal that configures the compute / memory block 1300 to maximize the desired performance metric of the compute fabric 1100. At each iteration of the training step, the variable S T and R T is determined according to a training algorithm that drives the optimal action variable AT, as shown in FIG. 13, e.g., S T+1 and R T+1 will be updated to.

[0084] The size of the trained model depends on the size and number of compute modules, memory, data path width, and the like. Machine learning algorithms running on the compute fabric 1100 can be hierarchical to reduce overall complexity and model size. Such hierarchical learning algorithms form nested algorithms that can be run in concert to increase the overall control and efficiency of the compute fabric and its data paths.

[0085] The trained model can then be deployed by the DMCA controller 1112, allowing the DMCA controller 1112 to reason about the optimal performance of the compute fabric 1100 and the algorithm / workload / program 1106 being executed by the user. As instructions are read from the algorithm / workload / program 1106 into the instruction cache 1108, the DMCA controller infers the optimal configuration based on the calculations encoded into the instructions. Once the DMCA controller has global visibility of the compute fabric 1100, resource allocation scheduling, memory allocation, and data path control to avoid congestion, for example, are managed according to the trained model.

[0086] Figure 12 depicts an example of a compiled algorithm / workload / program 1106 of Figure 11. The instruction shown includes four fields: an opcode, a source memory address, a destination memory address, and a configuration field. The opcode is a binary field, represented in hexadecimal, that indicates which function to perform. The source and destination fields provide addresses to the memory allocation controller for moving data into or out of the compute module 1110. The configuration field provides the configuration of the compute module 1110 that is designated to perform the function identified in the opcode field.

[0087] In the compiled program shown in FIG. 12, the compute modules 1110 are assumed to be arranged along an array having 16 rows and 89 columns, thus forming an 8-bit wide configuration. The input data is also 8 bits wide. In the first instruction, identified as an opcode of 0x0 and shown in the first row, the instruction causes data to be read into a memory address storing weights for the compute modules, resulting in a configuration for the weight data that is 8 bits wide. The second instruction, with opcode 0x02, is similar to the first instruction but causes input data to be read for the vector-dot product operation. The third instruction, with opcode 0xA0, initiates a vector-dot product operation for the compute module 1110 and is addressed at 0xA000_1000. The instruction further causes the result to be stored in address 0x9000_0000. The configuration for this instruction commands that the output be 8 bits in resolution. The fourth instruction, with opcode 0xB0, is an example of a digital compute function, such as scalar multiplication, on data stored in memory stored in the source address field, with the result of the operation stored in the destination address field. A configuration command is provided to the digital compute block to select the function to be performed. The fifth instruction, 0xFF, indicates, for example, program completion.

[0088] As can be seen from the above example, the algorithm can be further optimized within the configuration of the program. For example, the instructions shown in Figure 12 can be optimized by configuration to achieve, for example, higher bit resolution or improved latency as the program is executed, and the like, without degrading the results of the algorithm.

[0089] For example, assume that a model is being trained to optimize latency. The trained model output modifies the compiled program shown in Figures 11 and 12 in the following ways: First, it may optimize source and destination addresses to minimize the physical distance traveled. Since latency is being optimized, the model has been trained to bring the compute fabric to an optimum point by selecting memory that is addressed physically close to each other to minimize latency.

[0090] Models can similarly be trained to adjust configuration parameters to optimize the operation of compute fabric 1100. In the example shown in FIG. 12 and described above, the compute fabric is configured to provide 8-bit resolution. For multiply-accumulate operations, the output resolution required for full output resolution is y=n+m+log2(N) where y is the output resolution, n and m are the weight resolution and input resolution, respectively, and N is the number of weight and input pairs. For example, when N=1, n=8, and m=8, the total output resolution is 16, but configured to 8 bits. Under such conditions, the trained model adjusts the first two instructions to read only the 4 MSBs of weight and input data for the multiply-accumulate operation, since that is all that is required to satisfy the 8-bit output configuration. The result is an optimization of the latency performance parameter in conjunction with optimizing the addressing.

[0091] The foregoing detailed description presents each individual feature, structure, function, or characteristic described herein, and any combination of two or more such features, structures, functions, or characteristics, regardless of whether such feature, structure, function, or characteristic, or combination thereof, solves any problem disclosed herein, and without limitation on the scope of the claims, to the extent that such feature, structure, function, or characteristic, or combination thereof, is based on the specification as a whole in view of the knowledge of one of ordinary skill in the art. When an embodiment of a claimed invention includes a particular feature, structure, function, or characteristic, it is within the knowledge of one of ordinary skill in the art to use such feature, structure, function, or characteristic in connection with other embodiments, for example, as a substitute for another feature, structure, function, or characteristic, whether or not explicitly described.

[0092] In view of the foregoing detailed description, it will be apparent to those skilled in the art that many changes may be made within the scope of the innovations, embodiments, and / or examples described herein, such as the function and arrangement of elements, without departing from the principles described herein. One or more elements of one embodiment may be substituted for one or more elements in another embodiment, as would be apparent to one skilled in the art. The embodiments described herein are selected to illustrate the principles of the invention and its useful application, thereby enabling those skilled in the art to understand how various embodiments and variations may be adapted to the particular uses presented.

[0093] The foregoing detailed description of innovations, embodiments, and / or examples of the claimed inventions has been provided for purposes of illustration and description. It is not intended to be exhaustive or to limit the claimed inventions to the exact forms described, but rather to be accorded the broadest scope consistent with the principles and features disclosed herein. Many variations will be recognized by those skilled in the art. Without limitation, any and all equivalents described, shown, or incorporated by reference in this patent application are specifically incorporated by reference into the description herein of the innovations, embodiments, and / or examples. In addition, any and all variations described, shown, or incorporated by reference herein with respect to any one embodiment should also be considered taught with respect to all other embodiments. Any such variations include both currently known variations and future variations; for example, any element used herein includes future equivalent elements that provide the same function, regardless of the structure of the future equivalents.

Claims

1. A compute fabric, comprising: a plurality of compute tiles arranged within the memory block; networking circuitry coupled to the plurality of compute tiles, the networking circuitry adapted to enable communication between the plurality of compute tiles and to enable the plurality of compute tiles to communicate with systems external to the compute fabric; a controller configured to control the plurality of compute tiles; each of the plurality of compute tiles a plurality of multiplication bit cells (MBCs) arranged along M rows and N columns, where M and N are integers greater than 1, each MBC comprising: multiplying the first bit by the second bit to generate a multiplied value; converting the multiplied value into an electric charge; storing the charge in a capacitor disposed within the MBC; The compute fabric is configured to:

2. 2. The compute fabric of claim 1, wherein the plurality of multiplier bit cells are configured to multiply a first binary number by a second binary number, the first bits being bits located in the first binary number and the second bits being bits located in the second binary number.

3. The compute fabric of claim 1 , wherein the controller is configured to control power usage associated with the plurality of multiplier bitcells.

4. The compute fabric of claim 1 , wherein the controller is configured to control a latency associated with the plurality of multiplier bitcells.

5. The compute fabric of claim 1 , wherein the controller is configured to control a throughput associated with the plurality of multiplier bitcells.

6. The compute fabric of claim 1 , wherein the controller is configured to control parallelization of the plurality of compute tiles.

7. The compute fabric of claim 1 , wherein the controller is configured to control a flow of data between the plurality of compute tiles and the networking circuitry.

8. 3. The compute fabric of claim 2, wherein each MBC comprises circuitry configured to perform multiply-and-accumulate (MAC) operations and static random access memory cells.

9. The compute fabric of claim 8 , wherein the first binary number is an input to the compute fabric and the second binary number is stored within the memory block.

10. 10. The compute fabric of claim 9, wherein the controller is configured to control a resolution of the plurality of compute tiles by dynamically programming a number of clock cycles, the first binary number being provided to at least one of the plurality of compute tiles corresponding to the number of clock cycles.

11. The compute fabric of claim 9 , wherein the controller agent is configured to control a resolution of the plurality of compute tiles by selecting a number of memory cells used for a MAC operation.

12. 10. The compute fabric of claim 9, wherein the controller is configured to control a resolution of the plurality of compute tiles by programming a number of steps performed in a binary search associated with a successive approximation register located within the compute tile.

13. The compute fabric includes: receiving a first set of input bits associated with a first matrix; receiving a second set of input bits associated with a second matrix; distributing a first subset of the first input bits to a first group of the plurality of compute tiles; distributing a second subset of the first input bits to a second group of the plurality of compute tiles; distributing a first subset of the second input bits to a third group of the plurality of compute tiles; distributing a second subset of the second input bits to a fourth group of the plurality of compute tiles; instructing the first group of the plurality of compute tiles and the third group of the plurality of compute tiles to generate a matrix multiplication of the first subset of the first input bits and the first subset of the second input bits to generate a first partial sum; instructing the second group of the plurality of compute tiles and the fourth group of the plurality of compute tiles to generate a matrix multiplication of the second subset of the first input bits and the second subset of the second input bits to generate a second partial sum; combining the first and second partial sums to generate a result of multiplication of the first matrix and the second matrix; The compute fabric of claim 1 , further configured to:

14. The compute fabric of claim 1 , wherein the plurality of compute tiles are arranged along one or more rows.

15. The compute fabric of claim 1 , wherein the plurality of compute tiles are arranged along one or more columns.

16. The compute fabric of claim 1 , wherein the plurality of compute tiles are arranged along an array of one or more rows and one or more columns.

17. The compute fabric of claim 1 , wherein the controller is configured to control a resolution of a successive approximation register (SAR) analog-to-digital converter (ADC) located within a compute tile.

18. The compute fabric of claim 17 , wherein the controller is further configured to vary a reference voltage used by the ADC.

19. The compute fabric of claim 1 , wherein the controller is further configured to vary a number of calculations performed by a compute tile.

20. and a performance monitor, wherein the controller is trained to vary the configuration of the compute fabric via reinforcement learning, the reinforcement learning comprising: setting a configuration state of the compute fabric to a first state; measuring performance characteristics of the compute fabric with the performance monitor; receiving a reward signal in response to the measured performance characteristic; repeating said setting, said measuring, and said receiving until said received reward reaches a maximum value; The compute fabric of claim 1 , comprising:

21. 21. The compute fabric of claim 20, wherein the performance characteristics include one or more of power usage, throughput, latency, and resolution.

22. 21. The compute fabric of claim 20, wherein the configuration state of the compute fabric is defined by one or more of a data path width between the compute tiles, a number of bits of input data into which the first bit is placed, a resolution of a successive approximation register (SAR) analog-to-digital converter (ADC) associated with a compute tile, a reference voltage used by the ADC, and a number of calculations performed by a compute tile.

23. forming a plurality of compute tiles within a memory block; enabling communications among the plurality of compute tiles and between the compute tiles and an external system; controlling the plurality of compute tiles; each of the plurality of compute tiles comprising: a plurality of multiplication bit cells (MBCs) arranged along M rows and N columns, where M and N are integers greater than 1, each MBC comprising: multiplying the first bit by the second bit to generate a multiplied value; converting the multiplied value into an electric charge; storing the charge in a capacitor disposed within the MBC; The method is configured to:

24. 24. The method of claim 23, wherein the plurality of multiplication bit cells are configured to multiply a first binary number by a second binary number, the first bits being bits located in the first binary number and the second bits being bits located in the second binary number.

25. 24. The method of claim 23, further comprising varying power usage associated with the plurality of multiplier bitcells.

26. 24. The method of claim 23, further comprising varying a latency associated with the plurality of multiplication bitcells.

27. 24. The method of claim 23, further comprising varying a throughput associated with the plurality of multiplication bitcells.

28. The method of claim 23 , further comprising varying a parallelization of the plurality of compute tiles.

29. The method of claim 23 , further comprising: varying a flow of data between the plurality of compute tiles.

30. 25. The method of claim 24, wherein each MBC comprises circuitry configured to perform multiply-accumulate (MAC) operations and static random access memory cells.

31. 31. The method of claim 30, wherein the first binary number is an input to the compute fabric and the second binary number is stored within the memory block.

32. 32. The method of claim 31 , further comprising controlling a resolution of the plurality of compute tiles by dynamically programming a number of clock cycles, wherein the first binary number is provided to at least one of the plurality of compute tiles corresponding to the number of clock cycles.

33. 32. The method of claim 31, further comprising varying the resolution of the plurality of compute tiles by selecting a number of memory cells used for a MAC operation.

34. 32. The method of claim 31, further comprising controlling a resolution of the plurality of compute tiles by programming a number of steps performed in a binary search associated with successive approximation registers located within the compute tiles.

35. receiving a first set of input bits associated with a first matrix; receiving a second set of input bits associated with a second matrix; distributing a first subset of the first input bits to a first group of the plurality of compute tiles; distributing a second subset of the first input bits to a second group of the plurality of compute tiles; distributing a first subset of the second input bits to a third group of the plurality of compute tiles; distributing a second subset of the second input bits to a fourth group of the plurality of compute tiles; instructing the first group of the plurality of compute tiles and the third group of the plurality of compute tiles to generate a matrix multiplication of the first subset of the first input bits and the first subset of the second input bits to generate a first partial sum; instructing the second group of the plurality of compute tiles and the fourth group of the plurality of compute tiles to generate a matrix multiplication of the second subset of the first input bits and the second subset of the second input bits to generate a second partial sum; combining the first and second partial sums to generate a result of multiplication of the first matrix by the second matrix; 23. The method of claim 22, further comprising:

36. The method of claim 22 , wherein the plurality of compute tiles are arranged along one or more rows.

37. The method of claim 22 , wherein the plurality of compute tiles are arranged along one or more columns.

38. The method of claim 22 , wherein the plurality of compute tiles are arranged along an array of one or more rows and one or more columns.

39. 23. The method of claim 22, wherein the controller is configured to control the resolution of a successive approximation register (SAR) analog-to-digital converter (ADC) located within a compute tile.

40. 40. The method of claim 39, further comprising varying a reference voltage used by the ADC.

41. The method of claim 22 , further comprising varying the number of calculations performed by the compute tiles.

42. setting a configuration state of the compute fabric to a first state; measuring performance characteristics of the compute fabric; receiving a reward signal in response to said measured performance; repeating said setting, said measuring, and said receiving until said received reward reaches a maximum value; 23. The method of claim 22, further comprising:

43. 43. The method of claim 42, wherein the performance characteristics include one or more of power usage, throughput, latency, and resolution.

44. 23. The method of claim 22, wherein the configuration state is defined by one or more of a data path width between the compute tiles, a number of bits of input data into which the first bit is placed, a resolution of a successive approximation register (SAR) analog-to-digital converter (ADC) associated with the compute tile, a reference voltage used by the ADC, and a number of calculations performed by the compute tile.

Images