Systolic array cell with output post-processing

Abstract

To provide a method for implementing a matrix multiplication unit that includes multiple cells arranged in a systolic array, a matrix multiplication unit, and a data processing cell.SOLUTION: In an architecture 300, a matrix multiplication unit includes multiple cells arranged in a systolic array. Each cell includes a multiplication circuit 306 for calculating a product of elements of an input matrix. Each cell includes an accumulator 310 that obtains an accumulated value by accumulating the sum of the products output by the multiplication circuit. Each cell also includes a post-processing component 312 that obtains a post-processed value by performing one or more post-processing operations on the accumulated value.SELECTED DRAWING: Figure 3

Description

[Technical field]

[0001] REFERENCE TO RELATED APPLICATIONS This application claims priority to U.S. Provisional Application No. 63 / 116,034, filed November 19, 2020, entitled “SYSTOLIC ARRAY CELLS WITH OUTPUT POST-PROCESSING,” the disclosure of which is incorporated herein by reference.

[0002] Technical Field This specification relates to a systolic array of hardware processing units. [Background technology]

[0003] background A systolic array is a network of processing units that computes data and passes it through a network. Data in a systolic array flows in a pipelined manner between the processing units, and each processing unit may independently compute a partial result based on data received from its upstream neighboring processing unit. The processing units, which may also be referred to as cells, may be wired together to pass data from upstream processing units to downstream processing units. Systolic arrays are used in machine learning applications, for example, to perform matrix multiplication. Summary of the Invention [Means for solving the problem]

[0004] overview In general, one innovative aspect of the subject matter described herein may be embodied in a matrix multiplication unit that includes a plurality of cells arranged in a systolic array. Each cell includes a multiplier circuit configured to determine a product of elements of an input matrix. Each cell includes an accumulator configured to determine an accumulated value by accumulating sums of products output by the multiplier circuit. Each cell also includes a post-processing component configured to determine a post-processed value by performing one or more post-processing operations on the accumulated value.

[0005] These and other implementations may each optionally include one or more of the following features: In some aspects, each cell further includes an output register configured to receive the post-processed value and to shift the post-processed value out of the cell.

[0006] In some aspects, the post-processing component includes a rounding circuit configured to round the accumulated value from a higher precision numeric format to a lower precision numeric format. Each cell may include a number of output wires equal to the number of bits of the lower precision numeric format. This rounding within the cell may reduce the output bandwidth. Reducing the output bandwidth may reduce the number of wires required to extract output data from the cells. The reduction in the number of wires may allow for a smaller die size for the systolic array or a larger number of cells per die without increasing the die size.

[0007] In some aspects, the post-processing component comprises a truncation circuit configured to truncate the accumulated values ​​from a higher precision numeric format to a lower precision numeric format. In some aspects, the post-processing component includes a rectified linear unit (ReLU) circuit configured to output the accumulated values ​​when the accumulated values ​​are positive and to output a value of zero when the accumulated values ​​are negative or zero. In some aspects, the post-processing component is programmable and configured to perform one of a plurality of post-processing operations based on the control signal.

[0008] In general, another innovative aspect of the subject matter described herein may be embodied in a data processing cell that may include a multiplication circuit configured to determine products of elements of an input matrix, an accumulator configured to determine an accumulated value by accumulating sums of the products output by the multiplication circuit, and a post-processing component configured to determine a post-processed value by performing one or more post-processing operations on the accumulated value.

[0009] These and other implementations may each optionally include one or more of the following features: In some aspects, the cell may include an output register configured to receive the post-processed value and to shift the post-processed value out of the data processing cell.

[0010] In some aspects, the post-processing component includes a rounding circuit configured to round the accumulated value from a higher precision numeric format to a lower precision numeric format. The cell may include a number of output wires equal to the number of bits of the lower precision numeric format. This rounding within the cell may reduce the output bandwidth. Reducing the output bandwidth may reduce the number of wires required to extract output data from the cell. The reduction in the number of wires may allow for a smaller die size for the systolic array or a larger number of cells per die without increasing the die size.

[0011] In some aspects, the post-processing component includes a truncation circuit configured to truncate the accumulated value from a higher precision numeric format to a lower precision numeric format. The post-processing component may include a ReLU circuit configured to output the accumulated value when the accumulated value is positive and to output a value of zero when the accumulated value is negative or zero. In some aspects, the post-processing component is programmable and configured to perform one of a plurality of post-processing operations based on the control signal.

[0012] In general, another innovative aspect of the subject matter described herein may be embodied in a method for multiplying matrices, the method including: a first input register of a cell receiving a first input matrix, a second input register of the cell receiving a second input matrix, a multiplication circuit of the cell generating products of elements of the first input matrix and elements of the second input matrix, an accumulator of the cell generating an accumulation value that accumulates the products, and a post-processing component of the cell performing one or more post-processing operations on the accumulation value.

[0013] These and other implementations may each optionally include one or more of the following features: In some aspects, performing the one or more post-processing operations may include rounding the accumulated values ​​from a higher precision numeric format to a lower precision numeric format.

[0014] In some aspects, performing the one or more post-processing operations may include truncating the accumulated value from a higher precision numeric format to a lower precision numeric format. Performing the one or more post-processing operations may include outputting the accumulated value when the accumulated value is positive and outputting a value of zero when the accumulated value is negative or zero.

[0015] In some aspects, performing the one or more post-processing operations may include receiving a control signal and performing a given post-processing operation of the plurality of post-processing operations based on the control signal.

[0016] Some aspects may include an output register receiving the post-processed accumulated values ​​from the post-processing component, and an output register shifting out the post-processed accumulated values ​​from the cells. Effect of the Invention

[0017] The subject matter described herein may be implemented in certain embodiments to achieve one or more of the following advantages: The systolic array cells described herein may include a post-processing component that performs post-processing of the output of the cell before shifting the output of the cell out of the cell. This post-processing within the cell may reduce the output bandwidth, which may reduce the number of wires required to extract the output data from the cell. For example, the post-processing may include reducing the precision of a floating-point number, e.g., from 32 bits to 16 bits, which may then reduce the number of output wires from 32 to 16 if the cell includes one output wire per output bit. The reduction in the number of wires may allow for a smaller die size for the systolic array, or a larger number of cells per die without increasing the die size. The post-processing component may be a programmable element that allows for greater flexibility in the type of post-processing operations that may be performed by each cell.

[0018] Various features and advantages of the foregoing subject matter are described below with reference to the drawings. Further features and advantages will be apparent from the subject matter described herein and from the claims. [Brief description of the drawings]

[0019] [Figure 1] 1 illustrates an exemplary processing system including a matrix calculation unit. [Diagram 2] 1 illustrates an exemplary architecture including a matrix computation unit. [Diagram 3] 1 illustrates an exemplary architecture of a cell in a systolic array. [Figure 4] 1 illustrates an exemplary architecture of a cell in a systolic array. [Diagram 5] FIG. 1 is a flow diagram of an example process for performing matrix multiplication and performing one or more post-processing operations. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0020] Like reference numbers and designations in the various drawings indicate like elements. Detailed Description Generally, this document describes a systolic array of cells that includes post-processing components. The cells may include computation units, such as multiplication and / or addition circuits, for performing computations. For example, the systolic array may perform matrix-matrix multiplication on input matrices, with each cell performing a partial matrix product of a portion of each input matrix. The systolic array of cells may be part of a matrix computation unit of a processing system, such as a dedicated machine learning processor used to train machine learning models and / or perform machine learning computations, a graphics processing unit (GPU), or another suitable processing system that performs matrix multiplication.

[0021] A systolic array may implement an output stationary matrix multiplication technique in which each cell computes a partial sum of products of some of the elements of the input matrix. The elements may be shifted in opposite or orthogonal directions across the rows or across the columns of the systolic array. Each time a cell receives a pair of matrix elements, it multiplies the two elements and accumulates all the partial sums of the products computed by that cell for its portion of the elements of the two input matrices. The elements of the input matrices may be individual elements or submatrices.

[0022] A post-processing component of a cell may perform post-processing operations on partial results calculated by the computational units of the cell. For example, if the computational units accumulate 32-bit floating-point numbers, the post-processing component may round or truncate those floating-point numbers to a lower precision floating-point format, such as a 16-bit floating-point format. The post-processing may be performed outside the systolic array rather than by each cell. However, performing post-processing within each cell may reduce the output bandwidth of each cell and may reduce the number of input and / or output wiring of each cell. For example, each cell may require 16-bit floating-point wiring to receive 32-bit floating-point numbers. and 32 output wires for outputting a 32-bit floating-point number. By rounding or truncating the floating-point numbers within each cell, the number of input and / or output wires for each cell may be reduced by 50%, which may reduce the size of the multiplication unit and / or allow more cells per multiplication unit without increasing the size of the multiplication unit.

[0023] 1 illustrates an exemplary processing system 100 that includes a matrix computation unit 112. System 100 is one example of a system that may implement matrix computation unit 112 with a systolic array of cells having post-processing components.

[0024] System 100 includes a processor 102 that may include one or more computational cores 103. Each computational core 103 may include a matrix computation unit 112 that may be used to perform matrix-matrix multiplication using a systolic array of cells with post-processing components. System 100 may be in the form of a dedicated hardware chip.

[0025] 2 illustrates an exemplary architecture including a matrix computation unit 112. The matrix computation unit is a two-dimensional systolic array 206. The two-dimensional systolic array 206 may be a square array. The array 206 includes a plurality of cells 204. In some implementations, a first dimension 220 of the systolic array 206 corresponds to a column of cells, and a second dimension 222 of the systolic array 206 corresponds to a row of cells. The systolic array 206 may have more rows than columns, more columns than rows, or an equal number of columns and rows. Thus, the systolic array 206 may have a shape other than a square.

[0026] In this example, the systolic array 206 is used for neural network calculations. For example, the matrix calculation unit 112 of FIG. 1 may be implemented as the systolic array 206. In other examples, the systolic array 206 may be used for matrix multiplication or other calculations, such as convolution, correlation, or data sorting, in other applications.

[0027] In the illustrated example, value loader 202 sends activation inputs to the rows of array 206, and weight fetcher interface 208 sends weight inputs to the columns of array 206. However, in some other implementations, activation and weight inputs are transferred to either side of the columns of systolic array 206. If other types of inputs are used rather than activation and weight inputs, weight fetcher interface 208 may be replaced with another value such that the value loader may send inputs in the opposite or orthogonal direction across systolic array 206.

[0028] In another example, the value loader 202 may send activation inputs across the rows of the systolic array 206 and the weight fetcher interface 208 may send weight inputs across the columns of the systolic array 206, or vice versa. In a neural network example, the value loader 202 may send activation inputs to the rows (or columns) of the array 206 and the weight fetcher interface 208 may send weight inputs to the rows (or columns) of the array 206 from the opposite (or orthogonal) side from the side of the value loader 202. In yet another example, the value loader 202 may send activation inputs diagonally across the array 206 and the weight fetcher interface 208 may send weight inputs diagonally across the array, for example in the opposite direction to that of the value loader 202 or in a direction orthogonal to the direction of the value loader 202.

[0029] The value loaders 202 may receive activation inputs from a unified buffer or other suitable source. Each value loader 202 may send a corresponding activation input to a separate left-most cell of the array 206. The left-most cell may be a cell along the left-most column of the array 206. For example, value loader 212 may send an activation input to cell 214. A value loader may send its activation input to a cell 214. A force may also be sent to an adjacent value loader, whose activation input may be used in another left-most cell of array 206. This allows the activation input to be shifted for use in another particular cell of array 206.

[0030] The weight fetcher interface 208 may receive weight inputs from the memory unit. The weight fetcher interface 208 may send corresponding weight inputs to separate topmost cells of the array 206. The topmost cells may be cells along the top row of the array 206. For example, the weight fetcher interface 208 may send weight inputs to cells 214-217.

[0031] In some implementations, the host interface shifts the activation inputs across array 206 along one dimension, e.g., to the right, while shifting the weight inputs across array 206 along the orthogonal dimension, e.g., down. For example, over one clock cycle, the activation input in cell 214 may shift into an activation register in cell 215 to the right of cell 214. Similarly, the weight input in cell 214 may shift into a weight register in cell 218 below cell 214. In other examples, the weight inputs may be shifted in the opposite direction to the activation inputs (e.g., from right to left).

[0032] For example, to multiply two matrices, one representing activation inputs and one representing weights, using the output stationary technique, each cell accumulates the sum of the products of the matrix elements shifted into the cell. At each clock cycle, each cell may process a given weight input and a given activation input to multiply the two inputs. The cell may add each product to an accumulated value maintained by the cell's accumulator. For example, cell 215 may perform a first product of two matrix elements, e.g., a first activation input and a first weight input, and store the product in the accumulator. Cell 215 may shift an activation input to cell 216 and shift a weight input to cell 214. Similarly, cell 215 may receive a second activation input from cell 214 and a second weight input from cell 216. Cell 215 may multiply the second activation input with the second weight input. Cell 215 may add this to the previous accumulated value to generate an updated accumulated value.

[0033] After all of the matrix elements have passed through the rows and columns of the systolic array, each cell may shift out its accumulated value as a partial result of the matrix multiplication. Before shifting out the accumulated value, each cell may post-process the accumulated value and pass the post-processed output to an appropriate accumulator unit 210, e.g., the accumulator unit 210 in the same column as the cell. For example, each cell may round or truncate the number being output to a lower precision number and pass it to the accumulator unit 210. Exemplary individual cells are further described below with reference to Figures 3 and 4.

[0034] The cells may pass, e.g., shift, the post-processed outputs along their columns, e.g., toward the bottom of the columns in the array 206. In some implementations, at the bottom of each column, the array 206 may include an accumulator unit 210 that stores and accumulates each post-processed output from each column. The accumulator unit 210 may accumulate each post-processed output of its column to generate a final accumulated value. The final accumulated value may be forwarded to a vector computation unit or other suitable component.

[0035] A cell 204 of the systolic array 206 may be wired to adjacent cells. For example, cell 215 may be wired to cell 214 and cell 216 using a set of wiring. In some implementations, when shifting output data out of a cell to the accumulator unit 210, the cell may output a numeric value in a single clock cycle. To do so, the cell may provide an output wiring for each bit of the computer numeric format used to represent the output value. For example, if the output value is represented using a 32-bit floating-point format, e.g., float32 or FP32, the cell may have 32 output wires to shift out the entire output value in a single clock cycle.

[0036] In some cases, the input to the accumulator of the computation unit and / or cell has a lower precision than the internal precision of the computation unit and / or accumulator. For example, the floating-point values ​​of the input matrix may be 16 bits, for example, in bfloat16 or BF16 format. However, the multiplier circuit, the adder circuit, and / or the accumulator may operate with higher precision numbers, for example, FP32 numbers. In this example, the output of the accumulator of the upstream cell may be an FP32 number. Thus, to output an FP32 number in one clock cycle, the upstream cell may have 32 output wires to the downstream cell. By using a post processor in each cell as shown in FIG. 3, the number of output wires may be reduced to 16, for example, if the post processor rounds or truncates the FP32 number to a BF16 number. FP32 and BF16 are used only as examples. The cell 204 may operate with other numeric formats having other levels of precision.

[0037] By reducing the number of output wiring in this manner, the overall size of the systolic array may be reduced, i.e., the integrated circuit die on which the systolic array is implemented may be reduced, and / or the number of cells in the systolic array may be increased without increasing the size of the die.

[0038] 3 illustrates an example architecture 300 of a cell in a systolic array. For example, cell 204 of systolic array 206 of FIG. 2 may be implemented using architecture 300. The cell may be used to perform matrix-matrix multiplication of two input matrices. Although the cell is described with respect to performing matrix-matrix multiplication, the cell may be used to perform other calculations, such as convolution, correlation, or data sorting.

[0039] A cell may include input registers, including input register 302 and input register 304. Input register 302 may receive an input matrix via bus 322. For example, depending on the location of the cell in the systolic array, input register 302 may receive elements of the input matrix from a right neighboring cell (e.g., an adjacent cell located to the right of a given cell) or from another component (e.g., a weight fetcher interface, if used in systolic array 206 of FIG. 2). Thus, each element of the input matrix received by input register 302 may be a weight input.

[0040] The input register 304 may also receive elements of the input matrix via bus 324. For example, depending on the cell's location in the systolic array, the input register 304 may receive the input matrix from a left adjacent cell (e.g., an adjacent cell located to the left of a given cell) or from another component (e.g., a value loader or integration buffer, if used in the systolic array 206 of FIG. 2). Thus, each element of the input matrix received by the input register 304 may be an activation input.

[0041] The cell includes a multiplication circuit 306 and an addition circuit 308. The multiplication circuit 306 may determine the product of the matrix elements stored in the input registers 302 and 304. For example, the multiplication circuit 306 may multiply the elements of the input matrix stored in the input register 302 with the elements of the input matrix stored in the input register 304 to determine the product. If the elements of the input matrix received by the input register 302 are weight inputs and the elements of the input matrix received by the input register 304 are activation inputs, the multiplication circuit 306 may multiply the weight inputs by the activation inputs. The multiplication circuit 306 may output the product to the addition circuit 308.

[0042] The summing circuit 308 may obtain a new accumulated value by summing the product with an accumulated value stored in an accumulator 310. The summing circuit 308 may then send the new accumulated value to the accumulator 310. The accumulator 310 may store the current accumulated value.

[0043] After multiplication is completed for all elements of the input matrix, the accumulator 310 may output the accumulated data to a post-processing component 312 of the cell. The post-processing component 312 may be implemented using a circuit and may perform post-processing operations on the accumulated data received from the accumulator 310.

[0044] In some implementations, the post-processing component 312 includes a rounding circuit configured to round the accumulated value from a higher precision numeric format to a lower precision numeric format. For example, the post-processing component 312 may round an FP32 number to a BF16 number.

[0045] The post-processing component 312 may include a truncation circuit to truncate the accumulated values ​​from a higher precision numeric format to a lower precision numeric format. For example, the post-processing component 312 may truncate an FP32 number to a BF16 number.

[0046] The post-processing component 312 may include a rectified linear unit (ReLU) circuit configured to perform a rectified linear activation function on the accumulated data. The ReLU may directly output the accumulated value if the accumulated value is positive. If the accumulated value is negative, the ReLU may output a value of 0. The post-processing component 312 may include a ReLU in combination with a rounding or truncation circuit. In this manner, the post-processing component 312 can reduce the precision of positive values ​​while outputting a value of 0 for negative values.

[0047] The post-processing component 312 may include circuitry for performing other operations on the accumulated data, for example, the post-processing component 312 may include circuitry for performing other activation functions, such as a binary step function, a linear activation function, and / or a non-linear activation function, such as a sigmoid function.

[0048] In some implementations, the post-processing component 312 is a programmable component that may perform multiple post-processing operations. In this way, the host interface (or another component of the core 103) may adjust the post-processing operations for different input matrices, different machine learning calculations, or other purposes. For example, some machine learning calculations may better require or perform when higher precision values ​​are output by the cells. In this example, the post-processing component 312 may be controlled to round the accumulated values, for example, to one of multiple possible lower precision formats, or to pass the higher precision accumulated values ​​directly. The control signal may be used to modify the post-processing operations performed by the programmable post-processing component 312. Continuing with the previous example, the post-processing component 312 may round the accumulated values ​​to a first lower precision format in response to receiving a first control signal, may round the accumulated values ​​to a second lower precision format in response to receiving a second control signal, or may not round at all in response to receiving a third control signal. In another example, the post-processing component 312 may execute a given activation function from a set of possible activation functions of the post-processing component 312 based on the control signal.

[0049] After post-processing is complete, post-processing component 312 may send the post-processed data to output register 314. Output register 314 may use output bus 336 to shift the post-processed data to a neighboring cell, e.g., to the bottom-most neighboring cell, or to an accumulator, depending on the location of the cell in the systolic array.

[0050] In some implementations, the post-processing component 312 is part of the accumulator 310. Since accumulator 310 may include its own register, output register 314 may be omitted in this example.

[0051] If the post-processing operation is idempotent, e.g., a ReLU operation, the post-processing may be performed at every step. In this example, the post-processing component may be placed between the accumulators, which may be used to shift the post-processed data out of the cells.

[0052] The cell also includes buses for shifting matrix elements in and out from other cells. For example, the cell includes bus 324 for receiving matrix elements from a neighboring cell to the left, and bus 332 for shifting matrix elements to a neighboring cell to the right. Similarly, the cell includes bus 322 for receiving matrix elements from a top neighboring cell, and bus 328 for shifting matrix elements to a bottom neighboring cell. The cell also includes bus 330 for receiving accumulated values, e.g., post-processed values, from a top neighboring cell, and bus 334 for shifting accumulated values ​​received from the top neighboring cell to the bottom neighboring cell. Each bus may be implemented as a set of wires.

[0053] 4 shows an example architecture 400 of cells within a systolic array, such as systolic array 206 of FIG. 2. In this example, the cells of the systolic array are used to perform neural network computations. This provides an example of how post-processing circuitry 414 may be used in the systolic array cells of a neural network processing unit.

[0054] The cell may include an activation register 406 that stores an activation input. The activation register may receive an activation input from a left neighboring cell, i.e., a neighboring cell located to the left of a given cell, or from a value loader or buffer, depending on the location of the cell in the systolic array. The cell may include a weight register 402 that stores a weight input. The weight input may be transferred from a top neighboring cell or from a weight fetcher interface, depending on the location of the cell in the systolic array. A multiplication circuit 408 may be used to multiply the weight input from the weight register 402 with the activation input from the activation register 406. The multiplication circuit 408 may output the product to a summation circuit 410.

[0055] The adder circuit 410 may add the product and the accumulated value from the sum in register 404 to generate a new accumulated value. The adder circuit 410 may then send the new accumulated value to the accumulator 411. Once all of the matrix elements of the input matrix have been processed, the accumulator 411 may send the final accumulated value to the post-processing circuit 414. The post-processing circuit 414 may perform one or more post-processing operations on the accumulated value before outputting it to the accumulator unit. As mentioned above, the post-processing may include, for example, rounding the accumulated value, truncating the accumulated value, and / or applying a ReLU to the accumulated value.

[0056] A cell may also shift its weight input and activation input to adjacent cells for processing. For example, weight register 402 may send its weight input to another weight register in the bottom-most adjacent cell. Activation register 406 may send its activation input to another activation register in the next cell to the right. Thus, both the weight input and the activation input may be reused by other cells in the array in subsequent clock cycles.

[0057] In some implementations, the cell also includes a control register. The control register may store a control signal that determines whether the cell should shift either its weight inputs or its activation inputs to an adjacent cell. In some implementations, shifting a weight input or activation input takes one or more clock cycles. The control signal may also determine whether the activation input or weight input is The control signal may determine whether the activation input and weight input are forwarded to the multiplication circuit 408, or whether the multiplication circuit 408 performs an operation on the activation input and weight input. The control signal may also be passed to one or more adjacent cells, for example, using wiring.

[0058] 5 is a flow diagram of an example process 500 for performing matrix multiplication and executing one or more post-processing operations. The process 500 may be performed by each of one or more cells of a systolic array of a multiplication unit.

[0059] A first input register of the cell receives a first input matrix 502. For example, the first input matrix may represent activation inputs.

[0060] A second input register of the cell receives a second input matrix 504. For example, the second input matrix may represent weight inputs.

[0061] Multiplication circuitry of the cell multiplies elements of the input matrices 506. For example, the multiplication circuitry may perform matrix-matrix multiplication by multiplying corresponding elements of a first input matrix by corresponding elements of a second input matrix, one or more at a time.

[0062] The accumulator of the cell accumulates the sum of the products 508. For example, the summing element of the cell may sum the most recent product with the current accumulated value stored in the accumulator and store the updated accumulator value in the accumulator.

[0063] A post-processing component of the cell performs one or more post-processing operations on the accumulated values ​​(510). After all products have been determined for the input matrix, the accumulator may output the final accumulated values ​​to the post-processing component. The post-processing component may then perform rounding, truncation, a ReLU operation, or another suitable operation on the accumulated values. The post-processing component may then output the post-processed values ​​from the cell, for example, via an output register.

[0064] The subject matter and functional operations described herein may be implemented in digital electronic circuitry, tangibly embodied computer software or firmware, computer hardware, or one or more combinations thereof, including the structures disclosed herein and their structural equivalents. The subject matter described herein may be implemented as one or more computer programs, i.e., as one or more modules of computer program instructions encoded on a tangible, non-transitory program carrier for execution by or to control the operation of a data processing device. Alternatively, or in addition, the program instructions may be encoded on an artificially generated propagated signal, such as a machine-generated electrical, optical, or electromagnetic signal, that is generated to encode information for transmission to a receiving device suitable for execution by a data processing device. The computer storage medium may be a machine-readable storage device, a machine-readable storage substrate, a random or serial access memory device, or one or more combinations thereof.

[0065] The processes and logic flows described herein may be performed by one or more programmable processors executing one or more computer programs to perform functions by operating on input data and generating output. The processes and logic flows may also be performed by, and devices may be realized by, special purpose logic circuitry, such as, for example, an FPGA (field programmable gate array), an ASIC (application specific integrated circuit), or a GPGPU (general purpose graphics processing unit).

[0066] Processors suitable for the execution of a computer program include, by way of example, general purpose microprocessors. A computer may be based on a processor or a special purpose microprocessor or both, or any kind of central processing unit. In general, the central processing unit receives instructions and data from a read-only memory or a random access memory or both. The essential elements of a computer are a central processing unit for executing instructions and one or more memory devices for storing instructions and data. In general, a computer will also include one or more mass storage devices for storing data, such as, for example, a magnetic disk, a magneto-optical disk, or an optical disk, and / or be operatively coupled to receive data from or transfer data to the one or more mass storage devices. However, a computer need not have such devices. Furthermore, a computer may be embedded in another device, such as, for example, a mobile phone, a personal digital assistant (PDA), a mobile audio or video player, a game console, a global positioning system (GPS) receiver, or a portable storage device (e.g., a universal serial bus (USB) flash drive).

[0067] Computer-readable media suitable for storing computer program instructions and data include, by way of example, all forms of non-volatile memory, media and memory devices, including semiconductor memory devices, such as EPROM, EEPROM and flash memory devices; magnetic disks, such as internal hard disks or removable disks; magneto-optical disks; and CD-ROM and DVD-ROM disks. The processor and the memory may be supplemented by, or incorporated in, special purpose logic circuitry.

[0068] Although the present specification contains many specific implementation details, these should not be construed as limitations on the scope of any invention or what may be claimed, but rather as descriptions of features that may be specific to particular embodiments of a particular invention. Certain features described in the context of separate embodiments herein may also be implemented in combination in a single embodiment. Conversely, various features described in the context of a single embodiment may also be implemented in multiple embodiments separately or in any suitable subcombination. Furthermore, although features may be described above as acting in a combination, and may even be initially claimed as such, one or more features from a claimed combination may in some cases be deleted from the combination, and the claimed combination may be directed to a subcombination or a variation of the subcombination.

[0069] Similarly, although operations are shown in a particular order in the figures, it should not be understood that such operations need to be performed in the particular order shown, or sequential order, to achieve desirable results, or that all of the shown operations need to be performed. In certain situations, multitasking and parallel processing may be advantageous. Furthermore, it should be understood that the separation of various system modules and components in the above-described embodiments does not require such separation in all embodiments, and that the program components and systems described may generally be integrated into a single software product or packaged into multiple software products.

[0070] Specific embodiments of the subject matter have been described. Other embodiments are within the scope of the following claims. For example, the actions recited in the claims can be performed in a different order and still achieve desirable results. As an example, the processes depicted in the accompanying figures do not necessarily require the particular order shown, or sequential order, to achieve desirable results. In certain implementations, multitasking and parallel processing may be advantageous.

Claims

1. 1. A data processing cell comprising: a multiplication circuit configured to multiply elements of an input matrix; an accumulator configured to obtain an accumulated value by accumulating the sums of the products output by the multiplication circuit; a post-processing component configured to perform one or more post-processing operations on the accumulated values ​​to obtain post-processed values.

2. The data processing cell of claim 1 , further comprising an output register configured to receive the post-processed value and to shift the post-processed value out of the data processing cell.

3. The data processing cell of claim 1 , wherein the post-processing component comprises a rounding circuit configured to round the accumulated value from a higher precision numeric format to a lower precision numeric format.

4. 4. The data processing cell of claim 3, further comprising a number of output wiring equal to the number of bits of said lower precision numeric format.

5. 2. The data processing cell of claim 1, wherein the post-processing component comprises a truncation circuit configured to truncate the accumulated value from a higher precision numeric format to a lower precision numeric format.

6. The post-processing components include: outputting the accumulated value when the accumulated value is positive; 2. The data processing cell of claim 1, comprising a normalized linear unit (ReLU) circuit configured to output a value of zero when the accumulated value is negative or zero.

7. The data processing cell of claim 1 , wherein the post-processing component is programmable and configured to perform one of a plurality of post-processing operations based on a control signal.

8. A matrix multiplication unit comprising a plurality of data processing cells according to claim 1.

9. 1. A method for matrix multiplication comprising the steps of: a first input register of the cell receiving a first input matrix; a second input register of the cell receiving a second input matrix; a multiplication circuit of the cell generating products of elements of the first input matrix and elements of the second input matrix; an accumulator of the cell generating an accumulation value that accumulates the products; a post-processing component of the cell performing one or more post-processing operations on the accumulated values.

10. 10. The method of claim 9, wherein performing the one or more post-processing operations comprises rounding the accumulated value from a higher precision numeric format to a lower precision numeric format.

11. 10. The method of claim 9, wherein performing the one or more post-processing operations comprises truncating the accumulated value from a higher precision numeric format to a lower precision numeric format.

12. performing the one or more post-processing operations outputting the accumulated value if the accumulated value is positive; and outputting a value of 0 when the accumulated value is negative or zero.

13. performing the one or more post-processing operations Receiving a control signal; and performing a given post-processing operation of a plurality of post-processing operations based on the control signal.

14. moreover, an output register receiving the post-processed accumulated value from the post-processing component; 10. The method of claim 9, wherein the post-processing component includes shifting out the post-processed accumulated value from the cell.

15. A matrix multiplication unit, comprising: A plurality of cells arranged in a systolic array, each cell comprising: a multiplication circuit configured to multiply elements of an input matrix; an accumulator configured to obtain an accumulated value by accumulating the sums of the products output by the multiplication circuit; a post-processing component configured to perform one or more post-processing operations on the accumulated values ​​to obtain post-processed values.

16. 16. The matrix multiplication unit of claim 15, wherein each cell further comprises an output register configured to receive the post-processed value and to shift out the post-processed value from the cell.

17. 16. The matrix multiplication unit of claim 15, wherein the post-processing component comprises a rounding circuit configured to round the accumulated values ​​from a higher precision numeric format to a lower precision numeric format.

18. 20. The matrix multiplication unit of claim 17, wherein each cell includes a number of output wiring equal to a number of bits of the lower precision numeric format.

19. 16. The matrix multiplication unit of claim 15, wherein the post-processing component comprises a truncation circuit configured to truncate the accumulated values ​​from a higher precision numeric format to a lower precision numeric format.

20. The post-processing components include: outputting the accumulated value when the accumulated value is positive; 16. The matrix multiplication unit of claim 15, comprising a normalized linear unit (ReLU) circuit configured to output a value of zero when the accumulated value is negative or zero.

21. The matrix multiplication unit of claim 15 , wherein the post-processing component is programmable and configured to perform one of a plurality of post-processing operations based on a control signal.

Images