Transition current management

Abstract

In some examples, the device (100) includes control logic (129) configured to detect an idle cycle, an operand generator (140) configured to provide a composite operand in response to detecting the idle cycle, and a computer operation circuit (120). The computer operation circuit is configured to perform a first computer operation on the composite operand during the idle cycle. The computer operation circuit is configured to perform a second computer operation on the architecture operand during the active cycle.

Description

[Background technology]

[0001] A hardware accelerator may be implemented to perform certain operations more efficiently than would be possible on a general-purpose processor, such as a central processing unit (CPU). For example, a matrix multiplication accelerator (MMA) may be implemented to perform matrix mathematical operations more efficiently than would be possible on a general-purpose processor. Machine learning algorithms may be expressed as matrix operations, with matrix multiplication tending to be more performance-critical. Thus, machine learning is an example of an application domain in which an MMA may be implemented to perform matrix mathematical operations, such as matrix multiplication.

[0002] In hardware implementations of matrix multiplication, such as with an MMA, the calculations may be performed in parallel pipelined computer operations that may involve near-simultaneous evaluation of multiplications, dot-product sums, and multiplications. Such computer operations generally involve a significant amount of hardware components operating at relatively high signal transition frequencies. For example, some computer systems that include an MMA may perform approximately 4096 to approximately 8192 matrix multiplications per clock cycle at gigahertz rates. The amount of hardware components and / or signal transition frequencies involved in hardware-implemented matrix multiplication may contribute to a relatively high current demand while the computer operations involving the MMA are active (e.g., during an active cycle).

[0003] During some phases of program execution (e.g., during idle cycles), a computing system including an MMA may not need to perform matrix mathematical operations, so that the computing operations involving the MMA are inactive. For example, the computing system may not need to perform matrix mathematical operations due to program structure or transient resource dependencies (e.g., cache misses). While the computing operations involving the MMA are inactive (e.g., during idle cycles), the current demand may be low (e.g., on the order of leakage current in the MMA) compared to the current demand while the computing operations involving the MMA are active (e.g., during active cycles). Therefore, relatively large transient currents (di / dt) may occur when the computing operations involving the MMA start (e.g., when the MMA transitions from an idle cycle to an active cycle) and stop (e.g., when the MMA transitions from an active cycle to an idle cycle). Summary of the Invention

[0004] In some examples, a device includes control logic configured to detect an idle cycle, an operand generator configured to provide a composite operand in response to detecting the idle cycle, and a computer operation circuit configured to perform a first computer operation on the composite operand during the idle cycle, and a computer operation circuit configured to perform a second computer operation on the architecture operand during the active cycle. [Brief explanation of the drawings]

[0005] [Figure 1] FIG. 2 is a block diagram of an exemplary device for processing data.

[0006] [Figure 2] FIG. 1 is a block diagram of an example implementation of a device for processing data with a tightly coupled matrix multiplication accelerator (MMA).

[0007] [Figure 3] FIG. 1 is a block diagram of an example implementation of a device for processing data with a loosely coupled MMA.

[0008] [Figure 4] FIG. 1 illustrates an example implementation of a matrix multiplication operation in a device for processing data.

[0009] [Figure 5] FIG. 1 illustrates an example implementation of activity leveling during matrix multiplication operations in a device for processing data.

[0010] [Figure 6] FIG. 2 is a block diagram of an example implementation of an operand generator.

[0011] [Figure 7] FIG. 2 is a block diagram of an example implementation of an operand generator.

[0012] [Figure 8] FIG. 1 is a block diagram of an example implementation of a pseudorandom number generator.

[0013] [Figure 9] FIG. 2 illustrates exemplary waveforms over time in a device for processing data.

[0014] [Figure 10] FIG. 2 illustrates exemplary waveforms over time in a device for processing data.

[0015] [Figure 11] FIG. 2 illustrates exemplary waveforms over time in a device for processing data.

[0016] [Figure 12] FIG. 1 is a diagram showing exemplary waveforms versus frequency in a device for processing data.

[0017] The use of the same reference numbers or other reference designators in the drawings indicates the same or similar (functional and / or structural) features. DETAILED DESCRIPTION OF THE INVENTION

[0018] As described above, a relatively large transient current (di / dt) may occur when a computer operation involving a computer operation circuit, such as a matrix multiplication accelerator (MMA), begins and the computer operation circuit transitions from an idle cycle to an active cycle. A relatively large transient current may also occur when a computer operation involving the computer operation circuit stops and the computer operation circuit transitions from an active cycle to an idle cycle. The large transient current that occurs when the computer operation circuit transitions between an active cycle and an idle cycle may increase the inductance sensitivity of a package (or board) design. For example, a direct relationship may exist between inductance and transient current, such that an increase in the magnitude of the transient current (|di / dt|) may increase the impedance of the inductance, and a decrease in the magnitude of the transient current may decrease the impedance of the inductance.

[0019] Increasing the transient current drawn by the MMA or other computing circuitry during transitions between active and idle cycles can also increase package design complexity and manufacturing costs. For example, flattening the response of the power distribution network supplying the current drawn by the MMA to avoid resonances that may be excited by narrow current demand pulse widths associated with such increases in transient current can increase package design complexity. In another example, some of the power distribution network components involved in supplying current to the MMA may be strengthened to accommodate such increases in transient current, which can increase manufacturing costs.

[0020] Some aspects of the present description relate to transient current management for managing transient current in a device during parallel matrix computer operations using activity leveling. In at least one example, the device includes an operand generator configured to provide composite operands. Generally, an operand may be the subject of a mathematical operation or a computer instruction. The operand may include architecture operands and composite operands. Architecture operands may represent operands that are processed, manipulated, transformed, or created during some phase of program execution by a general-purpose processor, such as a central processing unit (CPU) or application control logic (ACL). Composite operands, in various examples, may represent operands generated or created by an operand generator external to any phase of program execution by the general-purpose processor, the results of which may be discarded without being used by any program.

[0021] A computer operation involving a given computer operation circuit may consume power by being performed on composite operands provided by the operand generator during an otherwise idle cycle. The power consumed by performing the computer operation on composite operands provided by the operand generator during an idle cycle may reduce the magnitude of transient currents drawn by the circuit during transitions between active and idle cycles (referred to herein as activity leveling). By reducing the transient currents drawn by the circuit during transitions between active and idle cycles, the increased inductance sensitivity, complexity, and manufacturing costs of package designs associated with such increased transient currents may be avoided.

[0022] FIG. 1 is a block diagram of an example device 100 for processing data. At least some implementations of device 100 represent an application environment for managing transients during parallel matrix computing using activity leveling. Device 100 includes processor 110, which represents a general-purpose processor such as a CPU or ACL. Device 100 also includes MMA 120, which represents a hardware accelerator coupled to processor 110 via interface 130. MMA 120 is merely one example of various computing circuitry, including input data formatter 121, output data formatter 123, buffer controller 125, matrix multiplier array 127, and control logic 129. In some examples, control logic 129 is external to the computing circuitry (e.g., MMA 120) but within device 100. The interface 130 includes a first source data bus (SRC1) 131, a second source data bus (SRC2) 133, a result data bus (DST RESULTS) 135, a command interface (COMMAND) 137, and a status interface (STATUS) 139.

[0023] In operation, processor 110 is configured to provide control signals at command interface 137 that cause MMA 120 to control the operation of input data formatter 121, output data formatter 123, buffer controller 125, and matrix multiplier array 127. In some examples, MMA 120 may store data structures (not explicitly shown) that determine how input data formatter 121, output data formatter 123, buffer controller 125, and matrix multiplier array 127 operate, and processor 110 may control the contents of the data structures via command interface 137. The control signals that processor 110 provides at command interface 137 may include opcode instructions, stall signals, formatting instructions, and other signals that modify the operation of MMA 120. The opcode instructions may include opcode instructions that define matrix math operations, such as matrix multiplication operations, direct vector-matrix multiplication (which may be useful for performing matrix-matrix multiplication), convolution, and other parallel matrix computer operations. The opcode instructions may also include opcode instructions that define non-matrix math operations, such as matrix transpose operations, matrix initialization operations, and other matrix-related operations that do not involve matrix math operations. The formatting instructions may include formatting instructions that define how MMA 120 interprets input data provided on first source data bus 131 or second source data bus 133. The formatting instructions may also include formatting instructions that define how MMA 120 presents results to processor 110 as output data provided on result data bus 135.

[0024] Input data formatter 121 is configured to convert data provided on first source data bus 131 and data provided on second source data bus 133 into architecture operands for internal use within MMA 120 using formatting instructions provided in command interface 137. Output data formatter 123 is configured to convert result data generated by computer operations involving MMA 120 into output data provided on result data bus 135 using formatting instructions provided in command interface 137. Buffer controller 125 is configured to provide and / or manage memory for storing the architecture operands provided by input data formatter 121 and for storing result data provided by matrix multiplier array 127. Matrix multiplier array 127 is configured to perform parallel matrix computer operations using the operands provided by input data formatter 121. Matrix multiplier array 127 is also configured to provide result data generated by the parallel matrix computer operations to buffer controller 125 for storage. Control logic 129 is configured to alter the operation of input data formatter 121, output data formatter 123, buffer controller 125, and matrix multiplier array 127 in response to receiving control signals provided by processor 110 at command interface 137. Control logic 129 is also configured to provide signals at status interface 139, for query by processor 110, indicating the status of MMA 120 or the status of operations performed by MMA 120.

[0025] In some examples, input data formatter 121, output data formatter 123, buffer controller 125, and control logic 129 are implemented using hardware circuitry. For example, any suitable hardware circuitry configured to manipulate data bits to facilitate the specific operations attributed to input data formatter 121, output data formatter 123, buffer controller 125, and control logic 129 herein may be useful. Taking output data formatter 123 as an example, an exemplary 8-bit by 8-bit vector multiplication produces a 16-bit result. Multiple such 16-bit results may be summed, and overflow (e.g., summing two 16-bit numbers results in a 17-bit result) is possible. Thus, multiplication may be performed with 32-bit precision. However, in the exemplary implementation in which the output has 8 bits, output data formatter 123 may be hardware configured to select which 8 bits of the 32-bit sum to provide as output. Output data formatter 123 may also be a hardware configuration for performing other operations on the output data, such as scaling or saturation operations.

[0026] Device 100 also includes operand generator 140 configured to provide composite operands when activity leveling is enabled. Computer operations involving MMA 120 may be performed on composite operands provided by operand generator 140 during otherwise idle cycles, consuming power. The power consumed by performing computer operations on composite operands provided by operand generator 140 during idle cycles may reduce the magnitude of transient current drawn by MMA 120 during transitions between active and idle cycles. In at least one example, the magnitude of transient current drawn by MMA 120 during transitions between active and idle cycles may be further reduced when operand generator 140 provides composite operands that have statistical similarity to architectural operands provided by processor 110. Computational operations performed on composite operands provided by operand generator 140 during idle cycles may be architecturally transparent (e.g., without any discernible impact on device architecture, such as memory) by discarding any result data generated by such computational operations without modifying memory provided by buffer controller 125 for storing the result data. As shown in FIG. 1, operand generator 140 may be implemented by device 100 within MMA 120, within processor 110, or external to both processor 110 and MMA 120. Composite data bus 142 (e.g., a bus for providing composite data to components) may provide data from operand generator 140 to other components, such as to buffer controller 125, as shown.

[0027] In some examples, operand generator 140 includes any suitable hardware circuit logic configured to perform the acts attributed herein to operand generator 140. Figure 6, described in detail below, provides an example hardware configuration for operand generator 140.

[0028] The term “statistical similarity” refers to a similarity between composite and architectural operands that facilitates a relatively consistent amount of current draw from MMA 120. More specifically, the current demand of a multiplier may depend on how the inputs to the multiplier are changing. For example, if the same data is provided to the multiplier's inputs every clock cycle, the multiplier may consume nearly zero power per clock cycle. This is because in static complementary metal-oxide semiconductor (CMOS) technology, a circuit consumes a significant amount of power only when the inputs to the circuit change (ignoring leakage power). However, a multiplier whose inputs change during each clock cycle will consume the maximum amount of power per clock cycle. Maintaining a consistent current draw from MMA 120 is desirable. However, because the current draw of MMA 120 over time depends on the sequence of input operands, the sequences of composite and architectural operands may appear similar. So, for example, if on average 3 of the 8 bits of the architecture operand change with each clock cycle, then the composite operand may have 3 of the 8 bits changing with each clock cycle.

[0029] 2 and 3 illustrate sample implementations of examples described herein in a broader context (e.g., FIGS. 2 and 3 illustrate system-level implementations of examples described herein). For example, FIG. 2 illustrates a block diagram of a tightly coupled application context 200 in accordance with various examples. In at least one example, tightly coupled generally refers to an application context in which processor 110 or another general-purpose processor may directly access MMA 120 without interacting with an intervening controller. Tightly coupled application context 200 is an example implementation of device 100 in which fabric 210 couples dynamic random access memory (DRAM) 220 with system-on-chip (SOC) 230. Fabric 210 provides an interconnect architecture for communicating data and / or control signals between components coupled to fabric 210, such as DRAM 220, local memory 240 of processor 110, and one or more peripheral interfaces 250 of SOC 230. In tightly coupled application context 200, MMA 120 may be tightly coupled to processor 110 and to local memory 240 of processor 110. Thus, MMA 120 may be directly accessed by processor 110 in tightly coupled application context 200, thereby supporting the processing of data from any number of peripherals 260 coupled to one or more peripheral interfaces 250. Peripherals 260 generally represent hardware devices that provide data such as image data, audio data, sensor data, radar data, encrypted data, and other data that can be evaluated using matrix mathematical operations.

[0030] 3 is a block diagram of a loosely coupled application context 300 in various examples. In at least one example, loosely coupled generally refers to an application context in which processor 110 or another general-purpose processor interacts with an intervening controller to indirectly access MMA 120. Loosely coupled application context 300 is an example implementation of device 100 in which fabric 210 couples DRAM 220 with SOC 310. In loosely coupled application context 300, MMA 120 is loosely coupled to processor 110 via intermediate controller 320. Thus, processor 110 may indirectly access MMA 120 via intermediate controller 320 in loosely coupled application context 300. The local memory 330 of the intermediate controller 320 may be coupled to the fabric 210 to communicate data and / or control signals with other components coupled to the fabric 210, such as the DRAM 220, the local memory 240 of the processor 110, and one or more peripheral interfaces 250.

[0031] FIG. 4 illustrates an example implementation of matrix multiplication in device 100 for processing data. More specifically, FIG. 4 shows some, but not all, contents of buffer controller 125 (FIG. 1), including buffers useful for storing operands, as described below. FIG. 4 represents various example operands and results of a matrix multiplication operation using matrix notation of the form X[n], where each pair of box brackets (e.g., []) represents a dimension of the matrix, and n is the number of elements that make up that dimension of the matrix. For example, FIG. 4 uses "A

[64] " to represent a row of a matrix multiplier, which in this example is a one-dimensional matrix containing 64 elements, and these 64 elements make up its first dimension. In another example, FIG. 4 uses "B

[64]

[64] " to represent a two-dimensional multiplicand matrix, where the first dimension contains 64 elements and the second dimension contains 64 elements.

[0032] 1 and 4 simultaneously, control logic 129 receives opcode instructions provided by processor 110 at command interface 137 while a computer operation involving MMA 120 is active. The opcode instructions define a matrix multiplication operation for which a computer operation involving MMA 120 is active. Thus, this example implementation does not involve MMA 120 transitioning between active and idle cycles.

[0033] Buffer controller 125 may be configured to include and / or manage memory having a two-stage pipeline structure, including buffers for storing architecture operands provided by input data formatter 121 and for storing result data provided by matrix multiplier array 127. Buffer controller 125 may also include additional circuitry, such as circuitry for managing buffers shown in FIG. 4, although such circuitry is not explicitly shown in FIG. 4. The two-stage pipeline structure may include a foreground and a background, as shown in FIG. 4. The foreground and background are constructs. As described below and shown in FIG. 4, mathematical operations occur in the foreground, and preparation for the foreground operations occurs in the background. In other words, matrix multiplier array 127 may perform operations on data stored in the foreground of the two-stage pipeline structure. Buffer controller 125 may use the background of the two-stage pipeline structure for data transfer operations.

[0034] In response to control logic 129 receiving an opcode instruction, MMA 120 loads data corresponding to a row of a multiplier matrix from first source data bus 131. Input data formatter 121 converts the data that MMA 120 loads from first source data bus 131 into architectural multiplier operands. Input data formatter 121 provides the architectural multiplier operands to buffer controller 125 for storage in foreground multiplier buffer 411. Multiple dot product calculator operations are calculated in parallel within matrix multiplier array 127 using elements of the architectural multiplier operands stored in foreground multiplier buffer 411 and columns of multiplicand operands stored in foreground multiplicand buffer 412 (the contents of which are provided by background multiplicand buffer 412, which accumulates as described below). Matrix multiplier array 127 provides results of these multiple dot product calculator operations to buffer controller 125. During an active cycle, buffer controller 125 stores the result provided by matrix multiplier array 127 in row 414 of foreground product buffer 413 (eg, as a result of an add-assign operation indicated by the symbol "+=").

[0035] While the calculator operations are occurring within matrix multiplier array 127, a first background data transfer occurs between buffer controller 125 and input data formatter 121 while calculator operations are occurring within matrix multiplier array 127 using foreground multiplier buffer 411 and foreground multiplicand buffer 412. The first background data transfer involves input data formatter 121 providing formatted data to buffer controller 125 for storage in background multiplicand buffer 422 using data that MMA 120 loads from second source data bus 133. While these calculator operations are occurring within matrix multiplier array 127, a second background data transfer also occurs between buffer controller 125 and output data formatter 123. The second background data transfer involves buffer controller 125 providing output data formatter 123 with data to be stored in background product buffer 423 (which receives its contents from foreground product buffer 413, as shown in FIG. 4) for conversion into result data that MMA 120 provides to processor 110 via result data bus 135.

[0036] FIG. 5 is a diagram illustrating an example implementation of activity leveling in device 100 for processing data. FIG. 5 represents various example operands and results of a matrix multiplication operation using matrix notation of the form X[n], where each pair of box brackets (e.g., []) represents a dimension of the matrix, and n is the number of elements that make up that dimension of the matrix. For example, FIG. 5 uses "A

[64] " to represent a row of a matrix multiplier, which in this example is a one-dimensional matrix containing 64 elements, and these 64 elements make up its single dimension. In another example, FIG. 5 uses "B

[64]

[64] " to represent a two-dimensional multiplicand matrix, whose first dimension contains 64 elements and whose second dimension contains 64 elements.

[0037] 1 and 5, device 100 includes a multiplexer (MUX) 502 (MUX 502 is not explicitly shown in FIG. 1) having a first multiplexer input, a second multiplexer input, a multiplexer output, and a control terminal. The first multiplexer input of MUX 502 is coupled to input data formatter 121. The second multiplexer input of MUX 502 is coupled to composite data bus 142. The multiplexer output of MUX 502 is coupled to buffer controller 125. Control logic 129 provides a leveling signal (IDLE) to the control terminal of MUX 502 and to operand generator 140. FIG. 1 does not explicitly show control logic 129 coupled to operand generator 140 to provide IDLE.

[0038] In this example implementation, control logic 129 receives control signals provided by processor 110 at command interface 137 while a computer operation involving MMA 120 is active. The control signals received by control logic 129 cause the computer operation involving MMA 120 to halt. Thus, Figure 5 illustrates an example operation of MMA 120 upon transitioning from an active cycle to an idle cycle. In at least one example, the control signal is a stall signal asserted by processor 110 in response to encountering a stall condition prior to the idle cycle. In at least one example, the control signal is an opcode instruction defining a non-matrix mathematical operation.

[0039] Control logic 129 detects an idle cycle in response to receiving a control signal provided by processor 110 at command interface 137. In response to detecting the idle cycle, control logic 129 enables activity leveling in MMA 120 by asserting leveling signal IDLE. In response to control logic 129 enabling activity leveling, operand generator 140 provides a composite operand on composite data bus 142 before the idle cycle for storage in foreground multiplier buffer 411. In at least one example, providing the composite operand involves operand generator 140 selecting the composite operand from a sample buffer that stores a set of sampled architectural operands (e.g., architectural multiplier operands) using a circular index or a pseudo-random index. In at least one example, operand generator 140 constructs a set of sampled architecture operands by sampling the architecture multiplier operands provided by input data formatter 121 to buffer controller 125 over a number of active cycles preceding an idle cycle detected by control logic 129, so that a pattern or trend in the architecture operands is determined. In at least one example, the composite operands provided by operand generator 140 have a statistical similarity to the architecture operands provided by processor 110, such as the composite operands provided by any example implementation of operand generator 140 described with reference to either FIG. 6 or FIG. 7.

[0040] MUX 502 couples composite data bus 142 to buffer controller 125 in response to control logic 129 enabling activity leveling signal IDLE. Buffer controller 125 stores composite operands in foreground multiplier buffer 411 in response to MUX 502 coupling composite data bus 142 to buffer controller 125. During idle cycles when activity leveling is enabled, multiple dot product calculator operations are calculated in parallel in matrix multiplier array 127 using elements of the composite operands stored in foreground multiplier buffer 411 and columns of multiplicand operands stored in foreground multiplicand buffer 412. Matrix multiplier array 127 provides the results of these multiple dot product calculator operations to buffer controller 125. During idle cycles when activity leveling is enabled, buffer controller 125 discards the results provided by matrix multiplier array 127 without modifying foreground product buffer 413. As described in more detail below, performing computational operations involving MMA 120 using composite operands provided by operand generator 140 with activity leveling enabled may reduce the magnitude of transient current drawn by MMA 120 during transitions between active and idle cycles.

[0041] FIG. 6 is a block diagram of an example implementation of operand generator 140. In FIG. 6, operand generator 140 provides a composite operand that has a statistical similarity to an architecture operand provided by processor 110. As shown in FIG. 6, operand generator 140 includes a distance circuit 602, a logic gate 604, an accumulation register 606, a shift circuit 608, an averaging register 610, a thermometer encoder 612, a shuffle circuit 614, and a pseudorandom number generator 616. Distance circuit 602 is configured to compute a Hamming distance or population count of a particular element of an architecture operand provided by processor 110 (an “architecture operand element”) for an active cycle. In at least one example, Hamming distance is a metric for comparing two binary data strings that measures the number of bit positions where the two binary data strings differ. Logic gate 604 is configured to update the Hamming distance value stored in accumulation register 606 for the architecture operand element during the active cycle. Updating the Hamming distance value stored in accumulation register 606 involves logic gate 604 performing a bitwise AND logic operation on the stored Hamming distance value and the Hamming distance computed by distance circuit 602.

[0042] Shifting circuit 608 is configured to update the average Hamming distance value stored in averaging register 610 for the architecture operand element once every 2n active cycles, where n is a natural number. Updating the average Hamming distance value stored in averaging register 610 for the architecture operand element involves shifting circuit 608 performing a bitwise right shift operation on the Hamming distance value stored in accumulating register 606 for the architecture operand element. Shifting circuit 608 is also configured to reset or clear the Hamming distance value stored in accumulating register 606 in response to updating the average Hamming distance value stored in averaging register 610. In at least one example, logic gate 604, accumulating register 606, shifting circuit 608, and / or averaging register 610 can be replicated to increase the sampling rate of the Hamming distances or population counts of the architecture operand elements provided by processor 110 for active cycles.

[0043] Thermometer encoder 612 is configured to convert the average Hamming distance value stored in averaging register 610 from binary to an 8-bit thermometer-encoded value having the average Hamming distance. A pseudorandom number provided by pseudorandom number generator 616 can control shuffle circuit 614 to generate composite operand elements having statistical similarity to the architecture operand elements using the thermometer code provided by thermometer encoder 612. Generating the composite operand elements can involve shuffle circuit 614 randomly shuffling the 8-bit thermometer-encoded values using a shuffling algorithm (e.g., the Fisher-Yates algorithm or the Knuth algorithm) controlled using the pseudorandom number provided by pseudorandom number generator 616. Operand generator 140 can use the composite operand elements generated by shuffle circuit 614 to generate composite operands for processing by matrix multiplier array 127 during idle cycles.

[0044] FIG. 7 is a block diagram of an example implementation of operand generator 140. In FIG. 7, operand generator 140 provides a composite operand that has a statistical similarity to the architecture operand provided by processor 110. As shown in FIG. 7, operand generator 140 includes an averaging circuit 710, an averaging register 720, a mask generator 730, a logic gate 740, and a pseudo-random number generator 616. Averaging circuit 710 is configured to update an average value for a particular element of the architecture operand (an “architecture operand element”) stored in averaging register 720 based on a comparison of the stored average value with the current value of the architecture operand element provided by processor 110 for the active cycle. Updating the average value of the architecture operand element stored in averaging register 720 involves incrementing the average value by one when the result of the comparison indicates that the average value exceeds the current value of the architecture operand element. Updating the average value of the architecture operand element stored in the averaging register 720 also involves decrementing the average value by one when the result of the comparison indicates that the current value of the architecture operand element exceeds the average value. In at least one example, the averaging circuit 710 generates the average value of the architecture operand element using a least mean squares ("LMS") algorithm. In at least one example, the LMS algorithm is a fixed-step LMS algorithm.

[0045] Mask generator 730 is configured to compute a binary mask from the average values for the architecture operand elements stored in averaging register 720. Computing the binary mask involves mask generator 730 identifying the most significant set bit in the average values stored in averaging register 720. Computing the binary mask also involves mask generator 730 setting each bit between the most significant set bit and the least significant bit of the average values stored in averaging register 720. Logic gate 740 is configured to generate a composite operand element having a statistical similarity to the architecture operand element. Generating the composite operand element involves logic gate 740 performing a bitwise AND logical operation on the binary mask provided by mask generator 730 and a pseudorandom number provided by pseudorandom number generator 616. Operand generator 140 may use the composite operand element generated by logic gate 740 to generate a composite operand for processing by matrix multiplier array 127 during idle cycles.

[0046] 8 is a block diagram of an example implementation of pseudo-random number generator 616. In FIG. 8, pseudo-random number generator 616 includes a first linear feedback shift register (LFSR) 811, a second LFSR 812, a third LFSR 813, and a fourth LFSR 814. As shown in FIG. 8, each LFSR of pseudo-random number generator 616 is configured to store a different 32-bit seed provided at the input of that LFSR. For example, first LFSR 811 is configured to store a first seed (Seed[n]), second LFSR 812 is configured to store a second seed (Seed[n+1]), third LFSR 813 is configured to store a third seed (Seed[n+2]), and fourth LFSR 814 is configured to store a fourth seed (Seed[n+3]). The output of each LFSR in the pseudorandom number generator 616 may provide a sequence of pseudorandom values beginning with an initial value set by a seed stored in that LFSR. The LFSR may scan through the possible sequence of values that can be represented by N non-zero bits. The starting seed reflects where each 8-bit quantity begins in the sequence. Any non-zero starting value may be useful as a seed.

[0047] The output of each LFSR of pseudo-random number generator 616 is coupled to the input of a different bit-reversal register. For example, the output of the first LFSR 811 is coupled to the input of the first bit-reversal register 821, the output of the second LFSR 812 is coupled to the input of the second bit-reversal register 822, the output of the third LFSR 813 is coupled to the input of the third bit-reversal register 823, and the output of the fourth LFSR 814 is coupled to the input of the fourth bit-reversal register 824. Each bit-reversal register of pseudo-random number generator 616 may perform a bit-reversal operation on a pseudo-random value provided at the input of the bit-reversal register, and may provide a pseudo-random value at the output of the bit-reversal register.

[0048] The pseudo-random number generator 616 also includes a logic circuit 830 comprising a plurality of logic gates. In Figure 8, the plurality of logic gates of the logic circuit include a first exclusive-OR (XOR) gate 831, a second XOR gate 832, a third XOR gate 833, and a fourth XOR gate 834. The output of each XOR gate is configured to provide a different pseudo-random number to the operand generator 140 for generating a composite operand. Each XOR gate is configured to provide a pseudo-random number at the output of the XOR gate in response to a bitwise XOR logic operation performed on data provided at the output of one LFSR and data provided at the output of one bit-reversal register driven by data provided at the output of another LFSR.

[0049] For example, the first XOR gate 831 is configured to provide a first pseudo-random number (prng[n][31:0]) in response to a bitwise XOR logic operation performed on the data provided at the output of the first LFSR 811 and the data provided at the output of the first bit-reversal register 821 driven by the data provided at the output of the second LFSR 812. In another example, the second XOR gate 832 is configured to provide a second pseudo-random number (prng[n+1][31:0]) in response to a bitwise XOR logic operation performed on the data provided at the output of the second LFSR 812 and the data provided at the output of the second bit-reversal register 822 driven by the data provided at the output of the third LFSR 813.

[0050] In another example, the third XOR gate 833 is configured to provide a third pseudo-random number (prng[n+2][31:0]) in response to a bitwise XOR logic operation performed on the data provided at the output of the third LFSR 813 and the data provided at the output of the third bit-reversal register 823 driven by the data provided at the output of the fourth LFSR 814. In another example, the fourth XOR gate 834 is configured to provide a fourth pseudo-random number (prng[n+3][31:0]) in response to a bitwise XOR logic operation performed on the data provided at the output of the fourth LFSR 814 and the data provided at the output of the fourth bit-reversal register 824 driven by the data provided at the output of the first LFSR 811.

[0051] An LFSR having an output that provides data to a bitwise XOR logic operation of one XOR gate may form a pair of counter-rotating LFSRs with another LFSR that provides data to drive a bit-reversal register that provides data to a bitwise XOR logic operation of an XOR gate. For example, the first LFSR 811 and the second LFSR 812 may form a pair of counter-rotating LFSRs for the first XOR gate 831. In another example, the second LFSR 812 and the third LFSR 813 may form a pair of counter-rotating LFSRs for the second XOR gate 832. In another example, the third LFSR 813 and the fourth LFSR 814 may form a pair of counter-rotating LFSRs for the third XOR gate 833. In another example, the fourth LFSR 814 and the first LFSR 811 may form a pair of counter-rotating LFSRs for the fourth XOR gate 834.

[0052] In at least one example, using a counter-rotating LFSR to provide pseudo-random numbers to operand generator 140 for generating composite operands may reduce cycle-to-cycle correlation within a sequence of pseudo-random numbers. Reducing such cycle-to-cycle correlation may reduce electromagnetic interference (EMI) associated with performing matrix mathematical operations. In at least one example, using a counter-rotating LFSR to provide pseudo-random numbers to operand generator 140 for generating composite operands may reduce die size by reducing the footprint of pseudo-random number generator 616.

[0053] FIG. 9 is a diagram 900 of example waveforms illustrating simulated operation of an example implementation of MMA 120 on the same data set. Diagram 900 includes an x-axis corresponding to time in picoseconds (pS). Diagram 900 also includes a y-axis corresponding to power in microwatts (μW), with power expressed as a percentage of the maximum value (100%) shown on the y-axis. Diagram 900 also includes a waveform 902 representing power consumption as a function of time by MMA 120 with activity leveling disabled. Diagram 900 also includes a waveform 904 representing power consumption as a function of time by MMA 120 with activity leveling disabled. At time 906, an active cycle 908 begins when a computer operation (e.g., a matrix multiplication) involving MMA 120 begins. For example, a computer operation involving MMA 120 may begin in response to MMA 120 receiving an opcode instruction from processor 110 that defines a matrix mathematical operation. During the active cycle 908, waveforms 902 and 904 each approach a first power level 910 that approximates the full-rate power of the MMA 120. Comparing waveforms 902 and 904, it can be seen that during the active cycle 908, the power consumption by the MMA 120 with activity leveling enabled is comparable to the power consumption by the MMA 120 with activity leveling disabled.

[0054] At time 912, an idle cycle 914 begins when the computer operation involving the MMA 120 stops. For example, the computer operation involving the MMA 120 may stop in response to the MMA 120 receiving an opcode instruction from the processor 110 that defines a non-matrix mathematical operation. Between the active cycle 908 and the idle cycle 914, the waveform 902 decreases from a first power level 910 to a second power level 916. The second power level 916 approximates the static leakage power of the MMA 120. Between the active cycle 908 and the idle cycle 914, the waveform 904 decreases from the first power level 910 to a third power level 918. The third power level 918 is lower than the first power level 910 but higher than the second power level 916. Therefore, the variation in power consumption by an MMA 120 with activity leveling enabled when transitioning between an active cycle 908 and an idle cycle 914 is less than the variation in power consumption by an MMA 120 with activity leveling disabled.

[0055] At time 920, an active cycle 922 begins when a computer operation (e.g., a matrix multiplication) involving MMA 120 begins. For example, the computer operation involving MMA 120 may begin in response to MMA 120 receiving an opcode instruction from processor 110 that defines a matrix mathematical operation. During idle cycle 914 and active cycle 922, waveform 902 and waveform 904 each approach a first power level 910 that approximates the full-rate power of MMA 120. During idle cycle 914 and active cycle 922, waveform 902 increases from a second power level 916 to the first power level 910. During idle cycle 914 and active cycle 922, waveform 904 increases from a third power level 918 to the first power level 910. The difference between the third power level 918 and the first power level 910 is less than the difference between the second power level 916 and the first power level 910. Thus, the variation in power consumption by the MMA 120 with activity leveling enabled is less than the variation in power consumption by the MMA 120 with activity leveling disabled when transitioning between the idle cycle 914 and the active cycle 922. Diagram 900 illustrates that by enabling activity leveling, the variation in power consumption by the MMA 120 when transitioning between the active cycle and the idle cycle can be reduced.

[0056] FIGS. 10 and 11 are example waveform diagrams, respectively, illustrating simulated operation of an example implementation of the MMA 120 for the same data set. In particular, diagram 1000 of FIG. 10 and diagram 1100 of FIG. 11 illustrate power consumption and transient current magnitude (|di / dt|), respectively, from this simulated operation. Diagram 1000 includes an x-axis corresponding to time in picoseconds (pS). Diagram 1000 also includes a y-axis corresponding to power in microwatts (μW), with power expressed as a percentage of the maximum value (100%) shown on the y-axis. Diagram 1000 also includes a waveform 1002 representing power consumption as a function of time by the MMA 120 with activity leveling disabled. Diagram 1000 also includes a waveform 1004 representing power consumption as a function of time by the MMA 120 with activity leveling enabled. Diagram 1100 includes an x-axis corresponding to time in picoseconds (pS). Diagram 1100 also includes a y-axis corresponding to the magnitude of the transient current in amperes per second (A / S), expressed as positive and negative multiples of the basic unit 1 U. Diagram 1100 includes a waveform 1102 that represents the magnitude of the transient current drawn by MMA 120 with activity leveling disabled as a function of time. Diagram 1100 also includes a waveform 1104 that represents the magnitude of the transient current drawn by MMA 120 with activity leveling enabled as a function of time.

[0057] At time 1006, each implementation of MMA 120 transitions from an active cycle 1008 to an idle cycle 1010 when a computer operation (e.g., a matrix multiplication) involving MMA 120 stalls. The transition computer operation involving MMA 120 may stall when, for example, processor 110 asserts a stall signal provided to MMA 120 in response to processor 110 encountering a stall condition, such as a stall condition associated with program structure or a transient resource dependency (e.g., a cache miss). Between active cycle 1008 and idle cycle 1010, waveform 1002 decreases from a first power level 1012 to a second power level 1014. First power level 1012 approximates the full-rate power of MMA 120. Second power level 1014 approximates the static leakage power of MMA 120. During the active cycle 1008 and the idle cycle 1010, the waveform 1004 decreases from a first power level 1012 to a third power level 1016. The difference between the first power level 1012 and the third power level 1016 is less than the difference between the first power level 1012 and the second power level 1014. Thus, when transitioning between the active cycle 1008 and the idle cycle 1010, the fluctuation in power consumption by the MMA 120 with activity leveling enabled is less than the fluctuation in power consumption by the MMA 120 with activity leveling disabled.

[0058] 11 , waveforms 1102 and 1104 each include a spike near time 1006, which corresponds to an increase in the transient current drawn by each exemplary implementation of MMA 120 when transitioning between active cycle 1008 and idle cycle 1010. Comparing waveforms 1102 and 1104, it can be seen that the magnitude of the transient current drawn by an MMA 120 with activity leveling enabled is less than the magnitude of the transient current drawn by an MMA 120 with activity leveling disabled when transitioning between active cycle 1008 and idle cycle 1010. Thus, the magnitude of the transient current drawn by an MMA 120 with activity leveling enabled is less than the magnitude of the transient current drawn by an MMA 120 with activity leveling disabled when transitioning between active cycle 1008 and idle cycle 1010.

[0059] 10 , an active cycle 1018 begins at time 1020 when a computer operation (e.g., a matrix multiplication) involving MMA 120 begins. For example, the computer operation involving MMA 120 may begin in response to MMA 120 receiving an opcode instruction from processor 110 that defines a matrix mathematical operation. During idle cycle 1010 and active cycle 1018, waveform 1002 and waveform 1004 each approach a first power level 1012 that approximates the full-rate power of MMA 120. During idle cycle 1010 and active cycle 1018, waveform 1002 increases from a second power level 1014 to the first power level 1012. During idle cycle 1010 and active cycle 1018, waveform 1004 increases from a third power level 1016 to the first power level 1012. The difference between the third power level 1016 and the first power level 1012 is less than the difference between the second power level 1014 and the first power level 1012. Thus, the variation in power consumption by the MMA 120 with activity leveling enabled when transitioning between the idle cycle 1010 and the active cycle 1018 is less than the variation in power consumption by the MMA 120 with activity leveling disabled. Diagram 1000 illustrates that by enabling activity leveling, the variation in power consumption by the MMA 120 when transitioning between the active cycle and the idle cycle can be reduced.

[0060] 11 , waveforms 1102 and 1104 each include a spike near time 1020, which corresponds to the increase in transient current drawn by each exemplary implementation of MMA 120 when transitioning between idle cycle 1010 and active cycle 1018. Comparing waveforms 1102 and 1104, it can be seen that the magnitude of the transient current drawn by MMA 120 with activity leveling enabled when transitioning between idle cycle 1010 and active cycle 1018 is smaller than the magnitude of the transient current drawn by MMA 120 with activity leveling disabled. Thus, the fluctuation in power demand by MMA 120 with activity leveling enabled when transitioning between idle cycle 1010 and active cycle 1018 is smaller than the fluctuation in power demand by MMA 120 with activity leveling disabled. Diagram 1100 illustrates that by enabling activity leveling, the fluctuations in power demand by MMA 120 when transitioning between active and idle cycles can be reduced.

[0061] 12 shows example waveform diagrams 1200, each illustrating simulated operation of an example implementation of MMA 120 for the same data set. Diagram 1200 includes waveform 1202, which represents power consumption as a function of frequency by MMA 120 with activity leveling disabled. Diagram 1200 also includes waveform 1204, which represents power consumption (expressed as a percentage of the maximum value (100%) shown on the y-axis) as a function of frequency by MMA 120 with activity leveling enabled. Comparing waveforms 1202 and 1204 reveals an overall reduction in power consumption by MMA 120 with activity leveling enabled relative to power consumption by MMA 120 with activity leveling disabled.

[0062] Although an example is provided in which MMA 120 performs operations on compositing operands, the principles of performing statistical analysis on a set of architecture operands to determine a corresponding set of compositing operations to use during idle cycles apply equally to any suitable computing circuitry, such as a CPU, a graphics processing unit (GPU), a fast Fourier transform (FFT) accelerator, a digital signal processor (DSP), or other signal processing circuitry.

[0063] The term "coupled" is used throughout this specification. This term may encompass connections, communications, or signal paths that enable a functional relationship consistent with this description. For example, in a first example, device A is coupled to device B if device A generates signals to control device B to perform certain operations; or, in a second example, device A is coupled to device B via an intervening component C such that device B is controlled by device A via control signals generated by device A, where the intervening component C does not substantially change the functional relationship between device A and device B.

[0064] A device that is "configured" to perform a task or function may be configured (e.g., programmed and / or hardwired) to perform that function by a manufacturer at the time of manufacture, and / or may be configurable (or reconfigurable) by a user after manufacture to perform that function and / or other additional or alternative functions. Such configuring may be achieved by firmware and / or software programming of the device, by the construction and / or layout of the hardware components and interconnections of the device, or by a combination of these.

[0065] Unless specifically stated otherwise, the use of "about," "approximately," or "substantially" in front of a value means ±10 percent of the stated value. Variations in the described examples are possible, and other examples are possible, within the scope of the claims.

Claims

1. A device, an interface adapted to be coupled to a processor; a matrix multiplication accelerator (MMA) coupled to the interface; Including, the MMA includes a memory having a multiplier buffer, a multiplicand buffer, and a product buffer; The MMA detecting an idle cycle using a control signal provided at the interface by the processor; Responsive to detecting the idle cycle, loading a composite operand into the multiplier buffer; during said idle cycles, performing a matrix mathematical operation using said composite operand and a multiplicand operand stored in said multiplicand buffer to generate a result stored in said product buffer; It is configured as follows: device.

2. 10. The device of claim 1, wherein the MMA is configured to discard the result without updating the product buffer.

3. 2. The device of claim 1, wherein the MMA is configured to detect the idle cycle when the control signal is an opcode instruction, and the opcode instruction defines a non-matrix mathematical operation.

4. 2. The device of claim 1, wherein the control signal is a stall signal asserted by the processor prior to the idle cycle.

5. 2. The device of claim 1, further comprising an operand generator coupled between the MMA and the interface, the MMA further configured to receive the composite operand from the operand generator.

6. The device of claim 1 , wherein the MMA is further configured to receive the composite operand from the interface.

7. 2. The device of claim 1, further comprising: a multiplexer having a multiplexer output, a first multiplexer input, and a second multiplexer input, the multiplexer output coupled to the multiplier buffer, the first multiplexer input coupled to the interface, and the second multiplexer input coupled to an operand generator of the device.

8. A device, an interface adapted to be coupled between a processor and a matrix multiplication accelerator (MMA), the MMA including a multiplier buffer and an operand generator coupled to the interface, the operand generator comprising: receiving a leveling signal having an asserted value in response to detecting an idle cycle; generating a composite operand in response to receiving the leveling signal having the asserted value; The device is configured to provide the combined operand at the interface for storage in the multiplier buffer before the idle cycle.

9. 9. The device of claim 8, wherein the operand generator is configured to select the composite operand from a sample buffer that stores sampled architectural operands provided to the multiplier buffer during an active cycle preceding the idle cycle.

10. 9. The device of claim 8, wherein the composite operand has a statistical similarity to an architectural operand provided by the processor during an active cycle preceding the idle cycle.

11. 9. The device of claim 8, wherein the operand generator comprises a pseudorandom number generator, the operand generator further comprising: configured to generate composite operand elements for the composite operand using pseudorandom numbers provided by the pseudorandom number generator, the pseudorandom number generator configured to provide the pseudorandom numbers using a pair of counter-rotating linear feedback shift registers with different seeds; device.

12. 9. The device of claim 8, wherein the operand generator is configured to control a Fisher-Yates algorithm or a Knuth algorithm using pseudorandom numbers provided by a pseudorandom number generator.

13. 9. The device of claim 8, wherein the operand generator is configured to generate composite operand elements of the composite operand using average values of architecture operand elements.

14. 14. The device of claim 13, wherein the operand generator is configured to generate the mean values of the architecture operand elements using a least mean squares algorithm.

15. 9. The device of claim 8, wherein the operand generator comprises: computing a binary mask using average values of the architecture operand elements; generating a composite operand element for the composite operand using the binary mask and a pseudorandom number provided by a pseudorandom number generator; The device is configured to:

16. A device, control logic configured to detect idle cycles; an operand generator configured to provide a composite operand in response to detecting the idle cycle; a computer arithmetic circuit; Includes The computing circuitry comprises: performing a first computer operation on the composite operand during the idle cycle; configured to perform a second computer operation on the architectural operand during the active cycle; device.

17. 17. The device of claim 16, wherein the computer operation circuitry is configured to discard a result of the first computer operation.

18. 17. The device of claim 16, wherein the computer operation circuitry is configured to store the composite operand in a multiplier buffer before the idle cycle.

19. 20. The device of claim 18, wherein the operand generator is configured to select the composite operand from a sample buffer that stores sampled architectural operands provided to the multiplier buffer during an active cycle preceding the idle cycle.

20. 17. The device of claim 16, wherein the composite operand has a statistical similarity to another architectural operand provided by the processor during an active cycle preceding the idle cycle.

Images