Transient current management

Abstract

In an example, an apparatus (100) includes control logic (129) configured to detect an idle period; an operand generator (140) configured to provide a composite operand in response to the detection of the idle period; and a calculation circuit (120). The calculation circuit is configured to perform a first calculation on the composite operand during the idle period. The computing circuitry is configured to perform a second computation on the architectural operand during the activity cycle.

Description

Background Art

[0001] Hardware accelerators may be implemented to perform certain operations more efficiently than such operations would be performed on a general purpose processor such as a central processing unit (CPU). For example, a matrix multiplication accelerator (MMA) may be implemented to perform matrix math operations more efficiently than they would be performed on a general purpose processor. Machine learning algorithms may be expressed as matrix operations, which tend to be dominated in performance by matrix multiplications. Thus, machine learning is an example of an application area in which an MMA may be implemented to perform matrix math operations such as matrix multiplications.

[0002] In hardware implementations of matrix multiplication, such as by MMA, the computations may be performed with parallel pipelined computations that may involve nearly simultaneous evaluation of multiplications, dot product sums, and accumulations. Such computations typically involve a large number of hardware components operating at relatively high signal transition frequencies. For example, some computing systems including MMAs may perform about 4096 to about 8192 matrix multiplications per clock cycle at gigahertz rates. The amount of hardware components and / or signal transition frequencies involved in hardware-implemented matrix multiplications may contribute to relatively high current requirements while computations involving MMAs are active (e.g., during active cycles).

[0003] During a certain stage of program execution (e.g., during an idle cycle), a computing system including an MMA may not need to perform matrix math operations, such that computations involving the MMA are inactive. For example, the computing system may not need to perform matrix math operations due to program structure or transient resource dependencies (e.g., cache misses). When computations involving the MMA are inactive (e.g., during an idle cycle), current demand may be lower (e.g., with respect to leakage level currents in the MMA) relative to current demand when computations involving the MMA are active (e.g., during an active cycle). Therefore, relatively high transient currents (di / dt) may occur when computations involving the MMA start (e.g., when the MMA transitions from an idle cycle to an active cycle) and stop (e.g., when the MMA transitions from an active cycle to an idle cycle). Summary of the invention

[0004] In an example, an apparatus includes: control logic configured to detect an idle cycle; an operand generator configured to provide a synthesized operand in response to detection of an idle cycle; and computation circuitry. The computation circuitry is configured to perform a first computation on the synthesized operand during the idle cycle. The computation circuitry is configured to perform a second computation on the architectural operand during an active cycle. BRIEF DESCRIPTION OF THE DRAWINGS

[0005] Figure 1 is a block diagram of an example apparatus for processing data.

[0006] Figure 2 is a block diagram of an example implementation of a device for processing data utilizing a tightly coupled matrix multiplication accelerator (MMA).

[0007] Figure 3 is a block diagram of an example implementation of a device for processing data utilizing a loosely coupled MMA.

[0008] Figure 4 is a diagram illustrating an example implementation of a matrix multiplication operation in a device for processing data.

[0009] Figure 5 A diagram illustrating an example implementation of activity equalization during a matrix multiplication operation in a device for processing data.

[0010] Figure 6 is a block diagram of an example implementation of an operand generator.

[0011] Figure 7 is a block diagram of an example implementation of an operand generator.

[0012] Figure 8 is a block diagram of an example implementation of a pseudorandom number generator.

[0013] Fig. 9 is a graph of an example waveform versus time in an apparatus for processing data.

[0014] Fig.10 is a graph of an example waveform versus time in an apparatus for processing data.

[0015] Fig.11 is a graph of an example waveform versus time in an apparatus for processing data.

[0016] Fig.12 is a graph of an example waveform versus frequency in an apparatus for processing data.

[0017] The same reference numbers or other reference designators are used in the drawings to indicate the same or similar features (functionally and / or structurally). DETAILED DESCRIPTION

[0018] As described above, a relatively high transient current (di / dt) may occur when a computation involving a computational circuit, such as a matrix multiplication accelerator (MMA), begins and the circuit transitions from an idle period to an active period. A relatively high transient current may also occur when a computation involving a computational circuit stops and the circuit transitions from an active period to an idle period. The high transient current that occurs when the computational circuit transitions between an active period and an idle period may increase the inductive sensitivity of a package (or board) design. For example, there may be a direct relationship between inductance and transient current, such that the impedance of the inductor may increase when the magnitude of the transient current (|di / dt|) increases, and decrease when the magnitude of the transient current decreases.

[0019] Increased transient currents drawn by the MMA or other computing circuits when transitioning between active and idle periods may also increase package design complexity and production costs. For example, flattening the response of the power distribution network that supplies the current drawn by the MMA to avoid resonances that may be excited by the narrow current demand pulse widths associated with such increases in transient current may increase package design complexity. In another example, components involved in the power distribution network components that supply current to the MMA are hardened to accommodate such increases in transient current, which may increase production costs.

[0020] Aspects of the present specification relate to transient current management that manages transient current in a device during parallel matrix computation using activity balancing. In at least one instance, the device includes an operand generator configured to provide a synthetic operand. Typically, an operand can be an object of a mathematical operation or a computer instruction. An operand can include an architectural operand and a synthetic operand. An architectural operand can represent an operand that is processed, manipulated, transformed, or created by a general purpose processor (e.g., a central processing unit (CPU) or an application control logic (ACL)) during a certain stage of program execution. In various instances, a synthetic operand can represent an operand generated or created by an operand generator external to any stage of program execution by a general purpose processor, and the result can be discarded without being used by any program.

[0021] Calculations involving a given computational circuit may be performed on synthesized operands provided by an operand generator during otherwise idle cycles to consume power. The power consumed by performing calculations on synthesized operands provided by an operand generator during idle cycles may reduce the magnitude of transient current drawn by the circuit when transitioning between active and idle cycles (referred to herein as activity balancing). Reducing the transient current drawn by the circuit when transitioning between active and idle cycles may avoid increases in package design inductance sensitivity, complexity, and production costs associated with increases in such transient current.

[0022] Figure 11 is a block diagram of an example device 100 for processing data. At least some embodiments of the device 100 represent an application environment for managing transient currents during parallel matrix calculations using active equalization. The device 100 includes a processor 110 representing a general purpose processor (e.g., a CPU or an ACL). The device 100 also includes an MMA 120 representing a hardware accelerator coupled to the processor 110 through an interface 130. The MMA 120 is only one example of a large number of computational circuits and includes an input data formatter 121, an output data formatter 123, a buffer controller 125, a matrix multiplier array 127, and control logic 129. In the example, the control logic 129 is external to the computational circuit (e.g., MMA 120), but still within the device 100. The interface 130 includes a first source data bus (SRC1) 131, a second source data bus (SRC2) 133, a result data bus (DST RESULTS) 135, a command interface (COMMAND) 137, and a status interface (STATUS) 139.

[0023] In operation, the processor 110 is configured to provide control signals at the command interface 137 that cause the MMA 120 to control the operation of the input data formatter 121, the output data formatter 123, the buffer controller 125, and the matrix multiplier array 127. In some examples, the MMA 120 may store data structures (not explicitly shown) that determine the manner in which the input data formatter 121, the output data formatter 123, the buffer controller 125, and the matrix multiplier array 127 are to operate, and the processor 110 may control the contents of the data structures via the command interface 137. The control signals provided by the processor 110 at the command interface 137 may include opcode instructions, stall signals, formatting instructions, and other signals that modify the operation of the MMA 120. The opcode instructions may include opcode instructions that define matrix math operations, such as matrix multiplication operations, direct vector by matrix multiplications (which may be used to perform matrix by matrix multiplications), convolutions, and other parallel matrix calculations. The opcode instructions may also include opcode instructions that define non-matrix math operations, such as matrix transposition operations, matrix initialization operations, and other matrix-related operations that do not involve matrix math operations. The formatting instructions may include formatting instructions that define how the MMA 120 is to interpret input data provided at the first source data bus 131 or at the second source data bus 133. The formatting instructions may also include formatting instructions that define how the MMA 120 is to present results to the processor 110 as output data provided at the result data bus 135.

[0024] The input data formatter 121 is configured to use the formatting instructions provided at the command interface 137 to transform the data provided at the first source data bus 131 and the data provided at the second source data bus 133 into architectural operands for internal use within the MMA 120. The output data formatter 123 is configured to use the formatting instructions provided at the command interface 137 to transform the result data generated by the calculation involving the MMA 120 into output data provided at the result data bus 135. The buffer controller 125 is configured to provide and / or manage memory for storing the architectural operands provided by the input data formatter 121 and for storing the result data provided by the matrix multiplier array 127. The matrix multiplier array 127 is configured to perform parallel matrix calculations using the operands provided by the input data formatter 121. The matrix multiplier array 127 is also configured to provide the result data generated by the parallel matrix calculations to the buffer controller 125 for storage. The control logic 129 is configured to modify the operation of the input data formatter 121, the output data formatter 123, the buffer controller 125, and the matrix multiplier array 127 in response to receiving control signals provided by the processor 110 at the command interface 137. The control logic 129 is also configured to provide signals indicating the status of the MMA 120 or the status of operations performed by the MMA 120 at the status interface 139 for interrogation by the processor 110.

[0025] In some instances, input data formatter 121, output data formatter 123, buffer controller 125 and control logic 129 are implemented using hardware circuit logic. For example, any suitable hardware circuit logic configured to manipulate data bits to promote the specific operations attributed to input data formatter 121, output data formatter 123, buffer controller 125 and control logic 129 herein may be useful. Taking output data formatter 123 as an example, example 8-bit by 8-bit vector multiplication produces 16-bit results. There may be multiple such 16-bit results to be summed together, and overflow (e.g., two 16-bit numbers are summed to produce 17-bit results) can be considered. Therefore, accumulation can be performed with 32-bit precision. However, in the example implementation scheme where the output has 8 bits, output data formatter 123 can be configured by hardware to select which eight bits in the 32-bit summation will be provided as output. Output data formatter 123 can also be configured by hardware to perform other operations on the data to be output, such as scaling and saturation operations.

[0026] The device 100 also includes an operand generator 140 configured to provide synthesized operands when activity equalization is enabled. Computations involving the MMA 120 may be performed on synthesized operands provided by the operand generator 140 during otherwise idle cycles to consume power. The power consumed by performing calculations on synthesized operands provided by the operand generator 140 during idle cycles may reduce the magnitude of transient current drawn by the MMA 120 when transitioning between active cycles and idle cycles. In at least one example, the magnitude of transient current drawn by the MMA 120 when transitioning between active cycles and idle cycles may be further reduced when the operand generator 140 provides synthesized operands that have statistical similarity to architectural operands provided by the processor 110. Computations performed on synthesized operands provided by the operand generator 140 during idle cycles may be architecturally transparent (e.g., have no discernible impact on the device architecture, such as memory) by discarding any result data generated by such calculations without modifying the memory provided by the buffer controller 125 for storing the result data. As Figure 1 As shown, operand generator 140 may be implemented by device 100 within MMA 120, within processor 110, or external to both processor 110 and MMA 120. A synthesized data bus 142 (e.g., a bus for providing synthesized data to components) may provide data from operand generator 140 to other components, such as to buffer controller 125, as shown.

[0027] In some examples, operand generator 140 includes any suitable hardware circuit logic configured to perform the actions attributed herein to operand generator 140. Figure 6 An example hardware configuration for operand generator 140 is provided.

[0028] The term "statistical similarity" refers to the similarity between the synthesized operands and the architectural operands, which facilitates drawing a relatively consistent amount of current from the MMA 120. More specifically, the current demand of a multiplier may depend on how the inputs of the multiplier change. For example, if the same data is provided to the inputs of the multiplier every clock cycle, the multiplier may consume almost zero power per clock cycle because in static complementary metal oxide semiconductor (CMOS) technology, a circuit consumes a large amount of power only when the inputs of the circuit change (ignoring leakage power). However, a multiplier with each input changing during each clock cycle will consume the maximum amount of power per clock cycle. It is desirable to maintain a consistent current draw from the MMA 120. However, because the current draw of the MMA 120 over time depends on the sequence of the input operands, the sequences of the synthesized operands and the architectural operands can be made to look similar. Thus, for example, if the architectural operands change 3 out of 8 bits on average per clock cycle, the synthesized operands may change 3 out of 8 bits per clock cycle.

[0029] Figure 2 and 3 Sample implementations of the examples described herein are presented in a broader context (e.g., Figure 2 and 3 shows a system-level implementation of the examples described herein). For example, Figure 2 1 is a block diagram of a tightly coupled application context 200 in various examples. In at least one example, tightly coupled generally refers to an application context in which a processor 110 or another general purpose processor can directly access an MMA 120 without interacting with an intermediate controller. The tightly coupled application context 200 is an example implementation of the device 100 in which a mesh fabric 210 couples a dynamic random access memory (DRAM) 220 with a system on a chip (SOC) 230. The mesh fabric 210 provides an interconnect architecture for transmitting data signals and / or control signals between each component coupled to the mesh fabric 210, such as the DRAM 220, the local memory 240 of the processor 110, and one or more peripheral interfaces 250 of the SOC 230. In the tightly coupled application context 200, the MMA 120 can be tightly coupled to the processor 110 and the local memory 240 of the processor 110. Thus, the MMA 120 may be directly accessed by the processor 110 in the tightly coupled application context 200 to support processing of data from any number of peripherals 260 coupled to one or more peripheral interfaces 250. The peripherals 260 generally represent hardware devices that provide data, such as image data, audio data, sensor data, radar data, cryptographic data, and other data that may be evaluated using matrix math operations.

[0030] Figure 3 1 is a block diagram of a loosely coupled application context 300 in various examples. In at least one example, loose coupling generally refers to an application context in which a processor 110 or another general purpose processor interacts with an intermediate controller to indirectly access an MMA 120. The loosely coupled application context 300 is an example implementation of the device 100 in which a mesh fabric 210 couples a DRAM 220 with a SOC 310. In the loosely coupled application context 300, the MMA 120 is loosely coupled to the processor 110 through the intermediate controller 320. Therefore, the processor 110 can indirectly access the MMA 120 through the intermediate controller 320 in the loosely coupled application context 300. The local memory 330 of the intermediate controller 320 can be coupled to the mesh fabric 210 to transmit data signals and / or control signals to other components coupled to the mesh fabric 210 (e.g., the DRAM 220, the local memory 240 of the processor 110, and one or more peripheral interfaces 250).

[0031] Figure 4 1 is a diagram illustrating an example implementation of matrix multiplication in a device 100 for processing data. More particularly, Figure 4 A buffer controller 125 ( Figure 1 ) in some, but not all, of the contexts described below. Figure 4 Various example operands and results of matrix multiplication operations using matrix notation of the form X[n] are represented, where each pair of square brackets (e.g., []) represents a dimension of the matrix, and n is the number of elements comprising that dimension of the matrix. For example, Figure 4 "A

[64] " is used to represent a row of matrix multipliers, which in this example is a single dimension matrix with 64 elements including that single dimension. In another example, Figure 4 “B

[64]

[64] ” is used to represent a multiplicand matrix having two dimensions: a first dimension including 64 elements; and a second dimension including 64 elements.

[0032] In this example implementation, and with reference to Figure 1 and 4 , control logic 129 receives an opcode instruction provided by processor 110 at command interface 137 when a computation involving MMA 120 is active. The opcode instruction defines a matrix multiplication operation, where a computation involving MMA 120 is active. Therefore, this example implementation does not involve MMA 120 transitioning between active cycles and idle cycles.

[0033] The buffer controller 125 may be configured to include and / or manage a memory having a two-stage pipeline structure including buffers for storing architectural operands provided by the input data formatter 121 and for storing result data provided by the matrix multiplier array 127. The buffer controller 125 may also include additional circuitry, such as for managing Figure 4 The buffer circuit system shown in Figure 4 This circuit system is not explicitly shown. The two-stage pipeline structure can include a foreground and a background, such as Figure 4 The front end and back end are constructs. As described below, and as Figure 4 As shown in FIG. 1 , mathematical operations occur in the foreground, and preparation for the foreground operations occurs in the background. In other words, the matrix multiplier array 127 can perform operations on data stored in the foreground of the two-stage pipeline structure. The buffer controller 125 can use the background of the two-stage pipeline structure for data transfer operations.

[0034] The MMA 120 loads data corresponding to a row of the multiplier matrix from the first source data bus 131 in response to the control logic 129 receiving the opcode instruction. The input data formatter 121 transforms the data loaded by the MMA 120 from the first source data bus 131 into architectural multiplier operands. The input data formatter 121 provides the architectural multiplier operands to the buffer controller 125 for storage in the foreground multiplier buffer 411. Multiple dot product calculations are calculated in parallel within the matrix multiplier array 127 using the elements of the architectural multiplier operands stored in the foreground multiplier buffer 411 and the columns of the multiplicand operands stored in the foreground multiplicand buffer 412 (whose contents are provided by the background multiplicand buffer 412, which is filled as described below). The matrix multiplier array 127 provides the results of those multiple dot product calculations to the buffer controller 125. During active cycles, buffer controller 125 stores the results provided by matrix multiplier array 127 in row 414 of foreground product buffer 413 (eg, as a result of an addition assignment operation, represented by the symbol "+=").

[0035] When computations are occurring within the matrix multiplier array 127, a first background data transfer occurs between the buffer controller 125 and the input data formatter 121 while computations are occurring within the matrix multiplier array 127 using the foreground multiplier buffer 411 and the foreground multiplicand buffer 412. The first background data transfer involves the input data formatter 121 providing formatted data to the buffer controller 125 for storage in the background multiplicand buffer 422 using data loaded from the second source data bus 133 by the MMA 120. A second background data transfer also occurs between the buffer controller 125 and the output data formatter 123 while those computations are occurring within the matrix multiplier array 127. The second background data transfer involves the buffer controller 125 providing the output data formatter 123 with data stored in the background product buffer 423 (which receives its contents from the foreground product buffer 413, as shown in FIG. 1 ). Figure 4 ) to transform the data in the MMA 120 into result data that is provided to the processor 110 via the result data bus 135.

[0036] Figure 5 is a diagram illustrating an example implementation of activity equalization in a device 100 for processing data. Figure 5 Various example operands and results of matrix multiplication operations using matrix notation of the form X[n] are represented, where each pair of square brackets (e.g., []) represents a dimension of the matrix, and n is the number of elements comprising that dimension of the matrix. For example, Figure 5 "A

[64] " is used to represent a row of matrix multipliers, which in this example is a single dimension matrix with 64 elements including that single dimension. In another example, Figure 5 “B

[64]

[64] ” is used to represent a multiplicand matrix having two dimensions: a first dimension including 64 elements; and a second dimension including 64 elements.

[0037] refer to Figure 1 and 5 , the device 100 includes a multiplexer (MUX) 502 having a first multiplexer input, a second multiplexer input, a multiplexer output and a control terminal (although Figure 1 MUX 502 is not explicitly shown. A first multiplexer input of MUX 502 is coupled to input data formatter 121. A second multiplexer input of MUX 502 is coupled to composite data bus 142. A multiplexer output of MUX 502 is coupled to buffer controller 125. Control logic 129 provides an equalization signal (IDLE) to a control terminal of MUX 502 and operand generator 140. Figure 1 The control logic 129 coupled to the operand generator 140 to provide IDLE is not explicitly shown.

[0038] In this example implementation, control logic 129 receives a control signal provided by processor 110 at command interface 137 when a computation involving MMA 120 is active. The control signal received by control logic 129 causes the computation involving MMA 120 to cease. Thus, Figure 5 An example operation of the MMA 120 when transitioning from an active cycle to an idle cycle is described. In at least one example, the control signal is a stall signal asserted by the processor 110 in response to encountering a stall condition prior to the idle cycle. In at least one example, the control signal is an opcode instruction defining a non-matrix math operation.

[0039] The control logic 129 detects an idle cycle in response to receiving a control signal provided by the processor 110 at the command interface 137. In response to detecting the idle cycle, the control logic 129 enables active equalization in the MMA 120 by asserting the equalization signal IDLE. The operand generator 140 provides synthesized operands on the synthesized data bus 142 before the idle cycle in response to the control logic 129 enabling active equalization for storage in the foreground multiplier buffer 411. In at least one example, providing the synthesized operands involves the operand generator 140 selecting the synthesized operands from a sampling buffer storing a set of sampled architectural operands (e.g., architectural multiplier operands) using a loop index or a pseudo-random index. In at least one example, the operand generator 140 constructs the set of sampled architectural operands by sampling the architectural multiplier operands provided to the buffer controller 125 by the input data formatter 121 during a plurality of active cycles prior to the idle cycle detected by the control logic 129 to determine a pattern or trend in the architectural operands. In at least one example, the synthesized operands provided by operand generator 140 have statistical similarities to the architected operands provided by processor 110, such as those provided by the processor 110. Figure 6 or Figure 7 The synthesized operands provided by any of the example implementations of operand generator 140 described.

[0040] The MUX 502 couples the composite data bus 142 and the buffer controller 125 in response to the control logic 129 enabling the active equalization signal IDLE. The buffer controller 125 couples the composite data bus 142 and the buffer controller 125 in response to the MUX 502 to store the composite operand in the foreground multiplier buffer 411. During idle cycles when utilizing active equalization is enabled, multiple dot product calculations are calculated in parallel within the matrix multiplier array 127 using the elements of the composite operands stored in the foreground multiplier buffer 411 and the columns of the multiplicand operands stored in the foreground multiplicand buffer 412. The matrix multiplier array 127 provides the results of those multiple dot product calculations to the buffer controller 125. During idle cycles when utilizing active equalization is enabled, the buffer controller 125 discards the results provided by the matrix multiplier array 127 without modifying the foreground product buffer 413. As described in more detail below, performing calculations involving MMA 120 using synthesized operands provided by operand generator 140 with activity equalization enabled may reduce the magnitude of transient current drawn by MMA 120 when transitioning between active and idle cycles.

[0041] Figure 6 is a block diagram of an example implementation of operand generator 140. Figure 6 In the embodiment, operand generator 140 provides synthetic operands that have statistical similarity to the architectural operands provided by processor 110. Figure 6 As shown, the operator generator 140 includes a distance circuit 602, a logic gate 604, an accumulation register 606, a shift circuit 608, an averaging register 610, a thermometer encoder 612, a shuffle circuit 614, and a pseudo-random number generator 616. The distance circuit 602 is configured to calculate a Hamming distance or population count of a particular element ("architectural operand element") of an architectural operand provided by the processor 110 in an active cycle. In at least one example, the Hamming distance is a metric used to compare two binary data strings that measures the number of bit positions at which the two binary data strings differ. The logic gate 604 is configured to update the Hamming distance value of the architectural operand element stored in the accumulation register 606 during the active cycle. Updating the Hamming distance value stored in the accumulation register 606 involves the logic gate 604 performing a bitwise AND logic operation on the stored Hamming distance value and the Hamming distance calculated by the distance circuit 602.

[0042] The shift circuit 608 is configured to nThe average Hamming distance value of the architectural operand element stored in the averaging register 610 is updated once per active cycle, where n is a natural number. Updating the average Hamming distance value of the architectural operand element stored in the averaging register 610 involves the shift circuit 608 performing a bitwise right shift operation on the Hamming distance value of the architectural operand element stored in the accumulator register 606. The shift circuit 608 is also configured to reset or clear the Hamming distance value stored in the accumulator register 606 in response to updating the average Hamming distance value stored in the averaging register 610. In at least one example, the logic gates 604, the accumulator register 606, the shift circuit 608, and / or the averaging register 610 can be replicated to increase the sampling rate of the Hamming distances or population counts of the architectural operand elements provided by the processor 110 in the active cycle.

[0043] Thermometer encoder 612 is configured to convert the average Hamming distance value stored in averaging register 610 from binary to an 8-bit thermometer coded value having the average Hamming distance. The pseudo-random number provided by pseudo-random number generator 616 can control shuffle circuit 614 to generate synthetic operand elements having statistical similarity to architectural operand elements using the thermometer code provided by thermometer encoder 612. Generating the synthetic operand elements can involve shuffle circuit 614 randomly shuffling the 8-bit thermometer coded value using a shuffle algorithm (Fisher-Yates algorithm or Knuth algorithm) controlled using the pseudo-random number provided by pseudo-random number generator 616. Operand generator 140 can use the synthetic operand elements generated by shuffle circuit 614 to generate synthetic operands for matrix multiplier array 127 for processing during idle cycles.

[0044] Figure 7 is a block diagram of an example implementation of operand generator 140. Figure 7 In the embodiment, operand generator 140 provides synthetic operands that have statistical similarity to the architectural operands provided by processor 110. Figure 7As shown, the operator generator 140 includes an averaging circuit 710, an averaging register 720, a mask generator 730, a logic gate 740, and a pseudo-random number generator 616. The averaging circuit 710 is configured to update the average value of a particular element ("architectural operand element") of an architectural operand stored in the averaging register 720 based on a comparison between the stored average value and the current value of the architectural operand element provided by the processor 110 in an active cycle. Updating the average value of the architectural operand element stored in the averaging register 720 involves incrementing the average value by one when the result of the comparison indicates that the average value exceeds the current value of the architectural operand element. Updating the average value of the architectural operand element stored in the averaging register 720 also involves decrementing the average value by one when the result of the comparison indicates that the current value of the architectural operand element exceeds the average value. In at least one example, the averaging circuit 710 uses a least mean square ("LMS") algorithm to generate the average value of the architectural operand element. In at least one example, the LMS algorithm is a fixed step size LMS algorithm.

[0045] The mask generator 730 is configured to calculate a binary mask based on the average value of the architecture operand elements stored in the averaging register 720. Calculating the binary mask involves the mask generator 730 identifying the most significant set bit in the average value stored in the averaging register 720. Calculating the binary mask also involves the mask generator 730 setting each bit between the most significant set bit and the least significant bit of the average value stored in the averaging register 720. The logic gate 740 is configured to generate a synthetic operand element with statistical similarity to the architecture operand element. Generating the synthetic operand element involves the logic gate 740 performing a bitwise AND logic operation on the binary mask provided by the mask generator 730 and the pseudo-random number provided by the pseudo-random number generator 616. The operand generator 140 can use the synthetic operand element generated by the logic gate 740 to generate a synthetic operand for the matrix multiplier array 127 to be processed during an idle cycle.

[0046] Figure 8 616 is a block diagram of an example implementation of a pseudo-random number generator. Figure 8 In FIG. 6 , the pseudo random number generator 616 includes a first linear feedback shift register (LFSR) 811, a second LFSR 812, a third LFSR 813, and a fourth LFSR 814. Figure 8As shown, each LFSR of the pseudo-random number generator 616 is configured to store a different 32-bit seed provided at the input of the LFSR. For example, the first LFSR 811 is configured to store a first seed (seed[n]), the second LFSR 812 is configured to store a second seed (seed[n+1]), the third LFSR 813 is configured to store a third seed (seed[n+2]), and the fourth LFSR 814 is configured to store a fourth seed (seed[n+3]). The output of each LFSR of the pseudo-random number generator 616 can provide a sequence of pseudo-random values ​​that starts with an initial value set by the seed stored in the LFSR. The LFSR will traverse possible sequences of numbers that can be represented by N bits other than zero. The starting seed will reflect where each of the 8-bit quantities starts in the sequence. Any non-zero starting value can be used as a seed.

[0047] The output of each LFSR of the pseudo-random number generator 616 is coupled to the input of a different bit-reversal register. For example, the output of the first LFSR 811 is coupled to the input of the first bit-reversal register 821, the output of the second LFSR 812 is coupled to the input of the second bit-reversal register 822, the output of the third LFSR 813 is coupled to the input of the third bit-reversal register 823, and the output of the fourth LFSR 814 is coupled to the input of the fourth bit-reversal register 824. Each bit-reversal register of the pseudo-random number generator 616 can perform a bit-reversal operation on a pseudo-random value provided at the input of the bit-reversal register to provide a pseudo-random value at the output of the bit-reversal register.

[0048] The pseudo-random number generator 616 also includes a logic circuit 830 having a plurality of logic gates. Figure 8 , the plurality of logic gates of the logic circuit include a first exclusive OR (XOR) gate 831, a second XOR gate 832, a third XOR gate 833, and a fourth XOR gate 834. The output of each XOR gate is configured to provide a different pseudo-random number to the operand generator 140 for generating a composite operand. Each XOR gate is configured to provide a pseudo-random number at the output of the XOR gate in response to a bit-by-bit XOR logic operation performed on data provided at the output of one LFSR and data provided at the output of a bit reversal register driven by data provided at the output of another LFSR.

[0049] For example, the first XOR gate 831 is configured to provide a first pseudo-random number (prng[n][31:0]) in response to a bitwise XOR logic operation performed on data provided at the output of the first LFSR 811 and data provided at the output of the first bit inversion register 821 driven by data provided at the output of the second LFSR 812. In another example, the second XOR gate 832 is configured to provide a second pseudo-random number (prng[n+1][31:0]) in response to a bitwise XOR logic operation performed on data provided at the output of the second LFSR 812 and data provided at the output of the second bit inversion register 822 driven by data provided at the output of the third LFSR 813.

[0050] In another example, the third XOR gate 833 is configured to provide a third pseudo-random number (prng[n+2][31:0]) in response to a bitwise XOR logic operation performed on data provided at the output of the third LFSR 813 and data provided at the output of the third bit-reversal register 823 driven by data provided at the output of the fourth LFSR 814. In another example, the fourth XOR gate 834 is configured to provide a fourth pseudo-random number (prng[n+2][31:0]) in response to a bitwise XOR logic operation performed on data provided at the output of the fourth LFSR 814 and data provided at the output of the fourth bit-reversal register 824 driven by data provided at the output of the first LFSR 811.

[0051] The LFSR having an output that provides data to the bitwise XOR logic operation of the XOR gate may form a pair of counter-rotated LFSRs with another LFSR that provides data for driving a bit reversal register that provides data to the bitwise XOR logic operation of the XOR gate. For example, the first LFSR 811 and the second LFSR 812 may form a pair of counter-rotated LFSRs relative to the first XOR gate 831. In another example, the second LFSR 812 and the third LFSR 813 may form a pair of counter-rotated LFSRs relative to the second XOR gate 832. In another example, the third LFSR 813 and the fourth LFSR 814 may form a pair of counter-rotated LFSRs relative to the third XOR gate 833. In another example, the fourth LFSR 814 and the first LFSR 811 may form a pair of counter-rotated LFSRs relative to the fourth XOR gate 834.

[0052] In at least one example, using a counter-rotating LFSR to provide pseudo-random numbers to the operand generator 140 for generating the synthesized operands can reduce cycle-to-cycle correlation within the pseudo-random number sequence. Reducing this cycle-to-cycle correlation can mitigate electromagnetic interference (EMI) associated with performing matrix math operations. In at least one example, using a counter-rotating LFSR to provide pseudo-random numbers to the operand generator 140 for generating the synthesized operands can reduce die size by reducing the footprint of the pseudo-random number generator 616.

[0053] Fig. 9 900 is a graph of example waveforms, each of which shows simulated operation of an example implementation of MMA 120 on the same data set. Graph 900 includes an x-axis corresponding to time in picoseconds (pS). Graph 900 also includes a y-axis corresponding to power in microwatts (μW), expressed as a percentage of the maximum value (100%) shown on the y-axis. Graph 900 also includes a waveform 902, which represents the power consumption of MMA 120 over time with activity equalization disabled. Graph 900 also includes a waveform 904, which represents the power consumption of MMA 120 over time with activity equalization enabled. At time 906, an activity period 908 begins as a calculation (e.g., a matrix multiplication) involving MMA 120 begins. For example, the calculation involving MMA 120 may begin in response to MMA 120 receiving an opcode instruction defining a matrix math operation from processor 110. During the active period 908, waveforms 902 and 904 each approach a first power level 910 that approximates the full rate power of the MMA 120. A comparison between waveforms 902 and 904 shows that during the active period 908, the power consumption of the MMA 120 with active equalization enabled is comparable to the power consumption of the MMA 120 with active equalization disabled.

[0054] At time 912, an idle period 914 begins as computations involving the MMA 120 cease. For example, computations involving the MMA 120 may cease in response to the MMA 120 receiving an opcode instruction defining a non-matrix math operation from the processor 110. Between the active period 908 and the idle period 914, the waveform 902 decreases from the first power level 910 to the second power level 916. The second power level 916 approximates the static leakage power of the MMA 120. Between the active period 908 and the idle period 914, the waveform 904 decreases from the first power level 910 to the third power level 918. Although less than the first power level 910, the third power level 918 is higher than the second power level 916. Therefore, when transitioning between the active period 908 and the idle period 914, the change in power consumption of the MMA 120 with active balancing enabled is less than the change in power consumption of the MMA 120 with active balancing disabled.

[0055] At time 920, an active period 922 begins as a computation (e.g., a matrix multiplication) involving the MMA 120 begins. For example, the computation involving the MMA 120 may begin in response to the MMA 120 receiving an opcode instruction defining a matrix math operation from the processor 110. Between the idle period 914 and the active period 922, the waveforms 902 and 904 each approach a first power level 910, which is approximately the full rate power of the MMA 120. Between the idle period 914 and the active period 922, the waveform 902 increases from the second power level 916 to the first power level 910. Between the idle period 914 and the active period 922, the waveform 904 increases from the third power level 918 to the first power level 910. The difference between the third power level 918 and the first power level 910 is less than the difference between the second power level 916 and the first power level 910. Thus, when transitioning between idle periods 914 and active periods 922, the change in power consumption of MMA 120 with activity balancing enabled is less than the change in power consumption of MMA 120 with activity balancing disabled. Graph 900 shows that the change in power consumption of MMA 120 when transitioning between active and idle periods can be reduced by enabling activity balancing.

[0056] Fig.10 and Fig.11 are diagrams of example waveforms, each showing simulated operation of an example implementation of MMA 120 on the same data set. In particular, Fig.10 Figure 1000 and Fig.11Graph 1100 of EMBODIMENTS shows power consumption and transient current magnitudes (|di / dt|) from the simulated operation, respectively. Graph 1000 includes an x-axis corresponding to time in picoseconds (pS). Graph 1000 also includes a y-axis corresponding to power in microwatts (μW), expressed as a percentage of the maximum value (100%) shown on the y-axis. Graph 1000 also includes a waveform 1002 representing the power consumption of MMA 120 over time with active equalization disabled. Graph 1000 also includes a waveform 1004 representing the power consumption of MMA 120 over time with active equalization enabled. Graph 1100 includes an x-axis corresponding to time in picoseconds (pS). Graph 1100 also includes a y-axis corresponding to transient current magnitudes in amperes per second (A / S), expressed as positive and negative multiples of the base unit 1U. Graph 1100 includes a waveform 1102 representing the magnitude of transient current drawn by MMA 120 over time with active equalization disabled. Graph 1100 also includes a waveform 1104 representing the magnitude of transient current drawn by MMA 120 over time with active equalization enabled.

[0057] At time 1006, when computations (e.g., matrix multiplications) involving MMA 120 are stopped, each implementation of MMA 120 transitions from an active cycle 1008 to an idle cycle 1010. For example, computations involving MMA 120 may be stopped when processor 110 asserts a stall signal provided to MMA 120 in response to processor 110 encountering a stall condition (e.g., a stall condition associated with a program structure or a transient resource dependency (e.g., a cache miss). Between the active cycle 1008 and the idle cycle 1010, waveform 1002 decreases from a first power level 1012 to a second power level 1014. The first power level 1012 approximates the full rate power of MMA 120. The second power level 1014 approximates the static leakage power of MMA 120. Between the active cycle 1008 and the idle cycle 1010, waveform 1004 decreases from the first power level 1012 to a third power level 1016. The difference between the first power level 1012 and the third power level 1016 is less than the difference between the first power level 1012 and the second power level 1014. Therefore, when transitioning between the active period 1008 and the idle period 1010, the change in power consumption of the MMA 120 with active balancing enabled is less than the change in power consumption of the MMA 120 with active balancing disabled.

[0058] refer to Fig.11, waveforms 1102 and 1104 each include a spike near time 1006 that corresponds to an increase in transient current drawn by each example implementation of MMA 120 when transitioning between active periods 1008 and idle periods 1010. A comparison between waveforms 1102 and 1104 shows that when transitioning between active periods 1008 and idle periods 1010, the magnitude of the transient current drawn by MMA 120 with active balancing enabled is less than the magnitude of the transient current drawn by MMA 120 with active balancing disabled. Thus, when transitioning between active periods 1008 and idle periods 1010, the magnitude of the transient current drawn by MMA 120 with active balancing enabled is less than the magnitude of the transient current drawn by MMA 120 with active balancing disabled.

[0059] refer to Fig.10 , at time 1020, an active period 1018 begins as a computation (e.g., a matrix multiplication) involving the MMA 120 begins. For example, the computation involving the MMA 120 may begin in response to the MMA 120 receiving an opcode instruction defining a matrix math operation from the processor 110. Between the idle period 1010 and the active period 1018, the waveforms 1002 and 1004 each approach a first power level 1012, which approximates the full rate power of the MMA 120. Between the idle period 1010 and the active period 1018, the waveform 1002 increases from the second power level 1014 to the first power level 1012. Between the idle period 1010 and the active period 1018, the waveform 1004 increases from the third power level 1016 to the first power level 1012. The difference between the third power level 1016 and the first power level 1012 is less than the difference between the second power level 1014 and the first power level 1012. Thus, when transitioning between idle periods 1010 and active periods 1018, the change in power consumption of MMA 120 with activity balancing enabled is less than the change in power consumption of MMA 120 with activity balancing disabled. Diagram 1000 shows that the change in power consumption of MMA 120 when transitioning between active and idle periods can be reduced by enabling activity balancing.

[0060] refer to Fig.11, waveforms 1102 and 1104 each include a spike near time 1020 that corresponds to an increase in transient current drawn by each example implementation of MMA 120 when transitioning between idle period 1010 and active period 1018. A comparison between waveforms 1102 and 1104 shows that when transitioning between idle period 1010 and active period 1018, the magnitude of the transient current drawn by MMA 120 with active balancing enabled is less than the magnitude of the transient current drawn by MMA 120 with active balancing disabled. Thus, when transitioning between idle period 1010 and active period 1018, the change in current demand of MMA 120 with active balancing enabled is less than the change in current demand of MMA 120 with active balancing disabled. Diagram 1100 shows that the change in current demand of MMA 120 when transitioning between active and idle periods can be reduced by enabling active balancing.

[0061] Fig.12 1200 is a graph of example waveforms that each show simulated operation of an example implementation of the MMA 120 on the same data set. Graph 1200 includes a waveform 1202 that represents power consumption of the MMA 120 as a function of frequency with active equalization disabled. Graph 1200 also includes a waveform 1204 that represents power consumption of the MMA 120 as a function of frequency with active equalization enabled (expressed as a percentage of the maximum value (100%) shown on the y-axis). A comparison between waveforms 1202 and 1204 shows a global reduction in power consumption of the MMA 120 with active equalization enabled relative to power consumption of the MMA 120 with active equalization disabled.

[0062] Although an example of an MMA 120 that performs operations on synthetic operands is provided, the principles of performing statistical analysis on a set of architectural operands to determine a corresponding set of synthetic operations used during idle cycles are equally applicable to any suitable computing circuitry, such as a CPU, a graphics processing unit (GPU), a fast Fourier transform (FFT) accelerator, a digital signal processor (DSP), or other signal processing circuitry.

[0063] The term "coupled" is used throughout this specification. The term may encompass connections, communications, or signal paths that achieve a functional relationship consistent with this specification. For example, if device A generates a signal to control device B to perform an action, then: in a first instance, device A is coupled to device B; or in a second instance, device A is coupled to device B through an intermediate component C, provided that the intermediate component C does not substantially change the functional relationship between device A and device B, so that device B is controlled by device A via the control signal generated by device A.

[0064] A device "configured to" perform a task or function may be configured (e.g., programmed and / or hardwired) by the manufacturer at the time of manufacture to perform the described function, and / or may be configured (or reconfigurable) by a user after manufacture to perform the described function and / or other additional or alternative functions. Configuration may be performed through firmware and / or software programming of the device, through the construction and / or layout of the device's hardware components and interconnections, or a combination thereof.

[0065] Unless otherwise stated, "about," "approximately," or "substantially" preceding a value means + / - 10% of the stated value. Modifications may be made in the described examples, and other examples may be made within the scope of the claims.

Claims

1. A device comprising: an interface adapted to be coupled to a processor; and a matrix multiplication accelerator (MMA) coupled to the interface, wherein the MMA includes a memory having a multiplier buffer, a multiplicand buffer, and a product buffer, and the MMA is configured to: detecting an idle period using a control signal provided by the processor at the interface; In response to detecting the idle cycle, loading the multiplier buffer with a synthesized operand; and During the idle period, a matrix math operation is performed using the resultant operand and the multiplicand operand stored in the multiplicand buffer to produce a result to be stored in the product buffer.

2. The apparatus of claim 1, wherein the MMA is configured to discard the result without updating the product buffer.

3. The apparatus of claim 1, wherein the control signal is an opcode instruction, and the MMA is configured to detect the idle cycle when the opcode instruction defines a non-matrix math operation. 4 . The apparatus of claim 1 , wherein the control signal is a stall signal asserted by the processor prior to the idle cycle.

5. The device according to claim 1, wherein: The device further includes an operand generator coupled between the MMA and the interface, and the MMA is further configured to receive the synthesized operand from the operand generator.

6. The apparatus of claim 1, wherein the MMA is further configured to receive the synthesized operand from the interface.

7. The device of claim 1 , further comprising a multiplexer having a multiplexer output, a first multiplexer input, and a second multiplexer input, wherein the multiplexer output is coupled to the multiplier buffer; the first multiplexer input is coupled to the interface, and the second multiplexer input is coupled to an operand generator of the device.

8. A device comprising: an interface adapted to be coupled between a processor and a matrix multiplication accelerator (MMA), wherein the MMA includes a multiplier buffer; and an operand generator coupled to the interface, wherein the operand generator is configured to: Responsive to detecting an idle period, receiving an equalization signal having an asserted value; generating a synthesized operand in response to receiving the equalized signal having the asserted value; and Prior to the idle cycle, the synthesized operand is provided at the interface for storage in the multiplier buffer.

9. The apparatus of claim 8, wherein the operand generator is configured to select the synthesized operand from a sampling buffer that stores sampled architectural operands provided to the multiplier buffer during an active cycle preceding the idle cycle.

10. The apparatus of claim 8, wherein the synthesized operands have statistical similarity to architectural operands provided by the processor during an active cycle preceding the idle cycle.

11. The apparatus of claim 8, wherein the operand generator comprises a pseudo-random number generator, and the operand generator is further configured to: A composite operand element for the composite operand is generated using a pseudo-random number provided by the pseudo-random number generator, wherein the pseudo-random number generator is configured to provide the pseudo-random number using a pair of counter-rotating linear feedback shift registers having different seeds.

12. The apparatus of claim 8, wherein the operand generator is configured to: The Fisher-Yates algorithm or the Knuth algorithm is controlled using pseudo-random numbers provided by a pseudo-random number generator.

13. The apparatus of claim 8, wherein the operand generator is configured to: An average value of the architectural operand elements is used to generate a composite operand element for the composite operand.

14. The apparatus of claim 13, wherein the operand generator is configured to: The mean value of the architectural operand elements is generated using a least mean square algorithm.

15. The apparatus of claim 8, wherein the operand generator is configured to: Compute a binary mask using the average of the architectural operand elements; and Composite operand elements for the composite operand are generated using the binary mask and a pseudo-random number provided by a pseudo-random number generator.

16. An apparatus comprising: control logic configured to detect an idle period; an operand generator configured to provide a synthesized operand in response to said detection of said idle cycle; and A computing circuit configured to: During the idle period, performing a first calculation on the composite operand; and During the active cycle, a second computation is performed on the architectural operand.

17. The apparatus of claim 16, wherein the computation circuit is configured to discard a result of the first computation.

18. The apparatus of claim 16, wherein the computation circuit is configured to store the synthesized operand in a multiplier buffer prior to the idle cycle.

19. The apparatus of claim 18, wherein the operand generator is configured to select the synthesized operand from a sampling buffer that stores sampled architectural operands provided to the multiplier buffer during an active cycle preceding the idle cycle.

20. The apparatus of claim 16, wherein the synthesized operand has a statistical similarity to another architectural operand provided by a processor during an active cycle preceding the idle cycle.

Images