Matrix multiplication performed using convolution engine comprising array of processing elements

Abstract

An example matrix processor includes processing elements arranged in a grid, where the matrix processor is configured to receive a first matrix and a second matrix, where the first matrix is to be multiplied by the second matrix; transposing the second matrix; organizing the second matrix into a plurality of columns, wherein each column is a row of the second matrix; and sequentially providing the columns of the second matrix and the rows of the first matrix to a processing element over one or more cycles, where the processing element is configured as a multiply-accumulate unit (MAC unit), and where the processing results are stored in the processing element.

Description

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS

[0002] This application claims priority to U.S. Provisional Application No. 63 / 336,586, filed on April 29, 2022, and entitled “MATRIX MULTIPLICATION PERFORMED USING CONVOLUTION ENGINE WHICH INCLUDES ARRAY OF PROCESSING ELEMENTS,” the disclosure of which is hereby incorporated by reference in its entirety. Technical Field

[0003] The present invention relates to matrix multiplication using hardware elements, and more particularly, to matrix multiplication using a convolution engine. Background Art

[0004] Neural networks are relied upon for very different purposes and increasingly form the basis of technology. For example, a neural network may be utilized to perform object classification on an image acquired via a user device (e.g., a smartphone). In this example, the neural network may represent a convolutional neural network that applies convolutional layers, pooling layers, and one or more fully connected layers to classify objects depicted in an image. As another example, a neural network may be used for text translation between languages. For this example, the neural network may represent a recursive neural network.

[0005] Certain processors are typically used for applications of neural networks, e.g., to increase the speed at which they can be trained or used at inference. Example processors may include a graphics processing unit (GPU), which allows for fast computation of operations involving matrices, tensors, and the like. While GPUs may substantially accelerate the processing of neural networks, e.g., compared to a central processing unit (CPU), they are typically designed to perform ordinary linear algebra operations. Thus, they are not designed to specifically accelerate the processing of neural networks. BRIEF DESCRIPTION OF THE DRAWINGS

[0006] Figure 1A is a block diagram illustrating an example matrix processor (convolution engine) included in an example matrix processor system.

[0007] Figure 1B is a block diagram illustrating the conditioning of certain matrices via an example matrix processor to implement matrix multiplication.

[0008] Figure 2A Two matrices are shown to be multiplied via an example matrix processor.

[0009] Figure 2B Formatting two matrices to perform matrix multiplication is shown.

[0010] Figure 2C The processing of two matrices to enable matrix multiplication is shown.

[0011] Figure 3 is a flowchart of an example process of matrix multiplication using the convolution engine.

[0012] Figure 4 is a block diagram illustrating an example vehicle including a matrix processor system.

[0013] Embodiments of the present disclosure and their advantages may be best understood by reference to the following detailed description. It should be understood that the same reference numerals are used to identify the same elements shown in one or more of the accompanying drawings, wherein the display is for the purpose of illustrating embodiments of the present disclosure, rather than for limiting embodiments of the present disclosure. DETAILED DESCRIPTION

[0014] The present application describes techniques that can use a convolution engine to multiply matrices. As will be described, matrices can be multiplied using a matrix processor that includes a grid or array of processing elements. Example processing elements may include multiply-accumulate units (MAC units) arranged in a grid or array. In some embodiments, the grid or array may be a non-systolic array such that each MAC unit locally computes information without data movement within the grid or array.

[0015] The convolution engine can rely on an array or grid of MAC units to efficiently calculate the convolution between input data and weight data. Example input data may include input images, input feature maps, etc. As known to those skilled in the art, the input data may have one or more input channels. For example, an input image may have three channels, each associated with a specific color (e.g., red, green, blue). As another example, the input feature map may have 32, 64, 512 channels, which are formed based on the application of filters or kernels with 32, 64, 512 output channels. Similarly, weight data may be associated with the number of output channels. Therefore, when the input data is convolved with the weight data, the output may be associated with the number of output channels.

[0016] As a simplified case, convolution can convolve a two-dimensional filter with a two-dimensional input. For example, the filter can be a 3×3 filter or kernel, and the input can be a 12×12 input. In this example, the filter can be convolved with a 3×3 window in the input (e.g., the upper left portion of the input). For example, the values ​​included in the 3×3 filter are multiplied by the corresponding values ​​in the 3×3 window. These multiplications are summed and the resulting value is determined in the output (e.g., a 12×12 output, assuming a stride of 1). The resulting value can be positioned in the upper left corner of the output. The filter then slides, such as moves, over subsequent 3×3 windows in the input, and the resulting values ​​are determined for these windows.

[0017] In the above examples, multiplication and addition are used to convolve the filter or kernel with the input. Certain processors such as graphics processing units (GPUs) can be designed for fast matrix operations. For example, these GPUs can be designed to quickly calculate multiplications between matrices. However, neural networks (e.g., convolutional neural networks) require additional hardware or software complexity associated with fast calculation of convolutions. For example, additional hardware or software complexity may be due to the use of sliding filters or kernels across input data. Typically, matrix multiplication is performed between corresponding rows and columns of two matrices. However, in convolution, the filter is multiplied element by element with a corresponding subset of the input matrix. As another example, additional hardware or software complexity may be due to the use of a large number of input and output channels, which complicates the techniques for organizing and subsequently processing input and weight data.

[0018] Therefore, a matrix processor can be used, which is at least partially designed to efficiently and quickly process data for a convolutional neural network. A matrix processor such as a convolution engine can use input data associated with a first number of input channels and output data associated with a second number of output channels to calculate convolution. For example, a matrix processor can use input and weight data that have been organized or formatted to facilitate larger convolution operations. For example, the input data can be in the form of a three-dimensional matrix (e.g., two-dimensional data across multiple input channels). In this example, the output data can span multiple output channels. Therefore, the matrix processor can process larger input data by merging or flattening each two-dimensional output channel into a vector, so that the matrix processor can process the entire channel or a major part thereof. As another example, data can be reused effectively so that weight data can be shared across convolutions.

[0019] In some embodiments, the techniques described herein may be applied to the example matrix processors described in U.S. Pat. No. 11,157,287, U.S. Pat. Pub. No. 2019 / 0026250, and U.S. Pat. No. 11,157,441, which are incorporated herein by reference in their entirety and form a part of the present disclosure as if set forth herein.

[0020] Although convolution engines such as the matrix processors described above are high performance for processing convolutional neural networks, their specialized designs may complicate other processing operations. As described above, GPUs may be designed for certain operations (e.g., matrix multiplication). In order to allow efficient computation of neural networks, GPUs may require added hardware and / or software complexity. Similarly, matrix processors may be designed for efficient computation of neural networks and require additional hardware and / or software complexity to perform other operations (e.g., matrix multiplication).

[0021] Described here is an example of adjusting the operation of a convolution engine (such as a matrix processor) to perform matrix multiplication on an arbitrary matrix. For example, the matrix processor can calculate the matrix multiplication of a first matrix and a second matrix. In this example, the first matrix can be adjusted so that it is formatted in a manner similar to weight data. The matrix processor can then normally use certain convolution parameters to perform so that it quickly generates a processing result (e.g., an output matrix). An example of a convolution parameter can include filter size, the number of input or output channels, etc.

[0022] block diagram

[0023] Figure 1A is a block diagram illustrating an example matrix processor 110 (eg, an example convolution engine) included in the example matrix processor system 100 .

[0024] For example, the matrix processor system 100 can be used to compute a forward pass through the layers of a neural network. In some embodiments, the neural network can represent a convolutional neural network in which image or video information is processed. In the example shown, input data 130 is provided to the matrix processor 100, which includes an array or grid of processing elements 132. The processing elements can be, for example, multiply-accumulate units (MACs). In some embodiments, the array can be non-systolic so that each processing element multiplies and accumulates information over one or more cycles (e.g., clock cycles or processing cycles).

[0025] The input data 130 may represent an input image, one or more feature maps, or a portion thereof. For example, the portion may represent a specific window of the input data (e.g., one or more kernels or filters) to be multiplied by weight data. The input data 130 may also represent information input into a layer of a neural network (such as a convolution, transformer, or fully connected network) that will be multiplied with weight information (e.g., a weight matrix). The weight data 120 may represent one or more filters or kernels for one or more input channels. The weight data 120 may be loaded during one cycle, and optionally different weight data 120 for different output channels may be loaded in subsequent cycles.

[0026] The matrix processor 110 may determine a processing result 112 associated with the input data 130 and the weight data 120. The processing result 122 may represent an output associated with one or more convolutions between the weight data 120 and the input data 130, for example.

[0027] Additional descriptions related to an exemplary matrix processor are included in U.S. Pat. No. 11,157,287, U.S. Pat. Pub. No. 2019 / 0026250, and U.S. Pat. No. 11,157,441, which are hereby incorporated by reference in their entirety and form a part of the present disclosure as if set forth herein.

[0028] Figure 1B is a block diagram illustrating the adjustment of a particular matrix via an example matrix processor 110 to implement matrix multiplication. As described above, the matrix processor 110 can be configured as a convolution engine so that it efficiently computes convolutions associated with input data and weight data. Thus, the matrix processor 110 can be configured to operate based on convolution parameters specified or determined based on the data being processed, based on the particular convolution layer being processed, based on instructions, etc.

[0029] Example convolution parameters may include information about the size associated with the filter and the number of input channels, output channels, and the like. For example, a convolution layer included in a convolutional neural network may apply a filter of a certain volume (e.g., a filter of 3×3×512 volume). In this example, the convolution layer may therefore apply a 3×3 filter to each input channel and produce an output of 512 channels. These filters may be formatted or organized as weight data 120 so that columns of weight data may be sequentially provided to processing elements (e.g., on sequential clock cycles or processing cycles). In some embodiments, each column may represent a filter associated with an output channel. The input data 130 may be similarly formatted or organized as rows, wherein in some embodiments, each row corresponds to a pixel or element associated with an output channel.

[0030] Due to the formatting or organization of information for weight data 120 and input data 130 described above, multiplying two matrices (e.g., A matrix 140A and B matrix 140B) may not be easily adaptable to matrix processor 110. For example, due to the order or arrangement of weight data 120, B matrix 140B may not be aligned with input data 130 for multiplication.

[0031] In some embodiments, matrix processor 110 may be configured to multiply A matrix 140A by B matrix 140B based on adjusting B matrix 140B and optionally setting certain convolution parameters. For example, the convolution parameters may be set so that the size associated with the filter is 1×1. Additionally, B matrix 140B may be transposed 150 and optionally padded 152 (e.g., padded with zeros, which is also referred to herein as applied weights).

[0032] If it will Figure 2A-2C As described in , the B matrix 140B is transposed so that portions of the transposed B matrix 140B can be arranged as weight data 120. For example, the rows of the transposed B matrix 140B can be arranged as corresponding columns, and the corresponding columns will be sequentially applied as weight data 120. In some embodiments, it may be necessary to transpose 150 so that the rows of the B matrix 140B can be used as weight data. In some embodiments, the matrix processor system 100 can provide row data as opposed to column data, and these columns of the B matrix 140B cannot be easily provided as weight data.

[0033] For example, weight data 120 and input data 130 (e.g., [Y, X] matrix) can be flattened in memory (e.g., SRAM, DRAM, etc.) so that the strips of X are continuous. Thus, for hardware, reading strips of X can be easy to read continuously, but reading strips of Y can be harder and may require complex stride loads. For example, the first read from input data 120 in the first cycle can be a natural read (e.g., Figure 2B [B00, B01, B02, ...] in the memory, which can be consecutive bytes in the memory. The same is true for the second cycle reading of [B10, B11, B12, ...]. However, from the weight data (e.g., Figure 2B The first read of the weight data ([A00, A10, A20, ...] in ) may not be contiguous in memory. Thus, for a large matrix, such as 1000×1000, as an example, each element may be a stride of 1000 from the previous element in memory. Therefore, in some embodiments, the weight data is transposed so that it is easier to read (e.g., similar to the input data). In some embodiments, the weight data is not transposed, and memory extraction may be efficient under arbitrarily large stride loads and / or may be able to extract columns of data.

[0034] Additionally, the B matrix 140B may be optionally filled or weighted based on the size of the information that the matrix processor 110 is configured to receive as the weight data 120. For example, in some embodiments, the B matrix 140B may have a specific number of elements or a specific number of bits. In this example, the B matrix 140B may be filled so that the total elements of the bits reach a specific value (e.g., 32 bits, 32 elements). In some embodiments, if the total elements or bits are less than the first threshold number of elements or bits (e.g., 32, 48, etc.), the B matrix 140B may be filled so that the total elements or bits reach the first threshold. In some embodiments, if the total elements or bits are less than the second threshold number of elements or bits (e.g., 64, 96, etc.), but greater than the first threshold, the B matrix 140B may be filled so that the total elements or bits reach the second threshold.

[0035] In some embodiments, if B matrix 140B is greater than the elements or positions of the above-mentioned second threshold number, then B matrix 140B can be clipped or otherwise separated so that two sub-matrices are used. Each sub-matrix can include a portion of B matrix 140B separated along a specific dimension (for example, a sub-matrix can include multiple columns of data). In some embodiments, each sub-matrix can have a size according to the second threshold. Any remaining portion can be filled as described above.

[0036] Blocks 150 and 152 may be implemented as hardware elements of the matrix processor system 100. For example, the hardware element may be used to transpose the B matrix 140B (e.g., in a stream context). In some embodiments, the matrix processor system 100 may receive the transposed 150 and optionally padded 152 matrix B 140B for processing.

[0037] For example, in an initial cycle, the processing element may thus receive information from the weight data 120 and the input data 130. For example, a first portion of the A matrix 140A may be provided to the processing element 132. In this example, the first portion may represent the first row of the A matrix 140A. For example, if the first row has 4 elements, then the processing elements in the first column 134A will receive the first value in the row, the processing elements in the second column 134B will receive the second value in the row, the processing elements in the third column 134C will receive the third value in the row, and the processing elements in the fourth column 134D will receive the fourth value in the row.

[0038] Similarly, in an initial cycle, the processing elements may receive a first portion of the transposed B matrix 140B. For example, a first column of the transposed B matrix 140B may be provided to the processing elements. In this example, the processing elements in the first row 136A may receive a first value in the column, the processing elements in the second row 136B may receive a second value in the column, the processing elements in the third row 136C may receive a third value in the column, and the processing elements in the fourth row 136B may receive a fourth value in the column.

[0039] Processing element 132 may then multiply the value received from A matrix 140A with the value received from transposed B matrix 140B. Subsequently, the remaining portion of A matrix 140A and the remaining portion of transposed B matrix 140B may then be sequentially provided to the processing element for multiplication and accumulation.

[0040] Figure 2A Two matrices are shown to be multiplied via an example matrix processor. In the illustrated example, the A matrix 202A is shown as a 4×4 matrix and the B matrix 202B is shown as another 4×4 matrix. It will be appreciated that the A matrix and the B matrix can be different dimensions without adjusting the operation of the techniques described herein.

[0041] The A matrix 202A is shown transposed. Thus, the elements of the A matrix are flipped on the diagonal.

[0042] As shown, the result of the multiplication is B matrix 202B*A matrix 202A (eg, in the order of matrix multiplication).

[0043] Figure 2B Two matrices are shown formatted to implement matrix multiplication. In this example a matrix processor 110 is shown where the processing elements are arranged as a grid array. For example, processing element 250 is shown as the upper left corner of the array or grid.

[0044] The elements of the A matrix 202A are shown formatted as weight data. For example, the first column 252 of weight data to be provided to the processing element represents Figure 2A The first row (e.g., upper row) of the transposed A matrix 202A shown. As shown, the columns of weight data represent the corresponding rows of the transposed A matrix 202A. These columns can be queued or taken out in sequence. In addition, the B matrix 202B is formatted so that the first row (e.g., upper row) of the B matrix 202A is the first row 254 of the data to be provided to the processing element. As shown, the rows represent the rows of the B matrix 202A that are queued or taken out in sequence.

[0045] With respect to processing element 250, element 250 may thus receive the values ​​A(00) and B(00). These values ​​may be multiplied, for example, by a multiplication-accumulation unit. In the next cycle, as Figure 2C As shown, the result of the multiplication can be added to the subsequent multiplication between A(01) and B(10).

[0046] Figure 2C The processing of two matrices to implement matrix multiplication is shown. In the example shown, the next row of the transposed A matrix 202A is provided to the matrix processor 110 as column 262. Additionally, the next row of the B matrix 202B is provided to the matrix processor 110 as row 264.

[0047] Thus, processing element 250 calculates the product of A(01) and B(10). The products are accumulated so that processing element 250 stores the result of A(00)*B(00)+A(01)*B(10). Although not shown, it is understood that processing can continue so that the processing element of matrix processor 110 receives all columns 270 and all rows 280 (e.g., the values ​​in column 27 and row 280 are loaded into the processing element).

[0048] Thus, each processing element may store an output value associated with the multiplication of the A matrix 202A and the B matrix 202B. For example, the processing unit 250 will store the result of A(00)*B(00)+A(01)*B(10)+A(02)*B(20)+A(03)*B(30). The output value stored in the processing element may then be read out as the processing result associated with the multiplication. For example, the value stored in each row of the processing element may be read out sequentially. As another example, the values ​​stored in a threshold number of rows (e.g., 8 rows, 16 rows, etc.) may be read out sequentially.

[0049] flow chart

[0050] Figure 3 is a flow chart of an example process 300 for matrix multiplication using a convolution engine. For convenience, the process 300 will be described as being performed by the matrix processing system 100.

[0051] At block 302, the system obtains a first matrix and a second matrix to be multiplied. The system may obtain the first matrix and the second matrix based on instructions executed. For example, software executed by the system may cause the first matrix to be retrieved (e.g., from memory).

[0052] At block 304, the system transposes and optionally fills the second matrix. Figure 1B-2A As described, the system transposes the second matrix. Specifically, the result of the multiplication between the first matrix and the second matrix will represent the first matrix*the second matrix. Therefore, the second matrix can be designated as the latter of the matrices in the order of multiplication. In addition, based on the size associated with the second matrix, the system can fill the second matrix as described above.

[0053] At block 306, the system configures the convolution parameters associated with the matrix processor. As described above, the system can set the parameters to indicate a filter size of 1×1. In some embodiments, the system can perform multiplication using a matrix with multiple channels. In these embodiments, the system can perform multiplication as described above on individual channels. Therefore, the channels can be processed separately.

[0054] In block 308, the system obtains the processing results. Figure 2A-2C As described, the system formats or otherwise organizes the values ​​of the first matrix and the second matrix so that they can be provided to the matrix processor as weight data and input data. The multiplication can be performed over one or more cycles, and the processing results can be obtained.

[0055] Vehicle Block Diagram

[0056] Figure 4 A block diagram of a vehicle 400 (e.g., vehicle 102) is shown. Vehicle 400 may include one or more electric motors 402 that cause vehicle 400 to move. Electric motor 402 may include, for example, an induction motor, a permanent magnet motor, etc. As known to those skilled in the art, battery 404 (e.g., one or more battery packs each including a plurality of cells) may be used to power electric motor 402.

[0057] Vehicle 400 also includes a propulsion system 406 that can be used to set a gear (eg, a direction of propulsion) for the vehicle. With respect to electric vehicles, propulsion system 406 can adjust the operation of electric motor 402 to change the direction of propulsion.

[0058] In addition, the vehicle includes a matrix processor system 100 configured to perform matrix multiplication using a convolution engine (e.g., matrix processor 110). The matrix processor system 100 can process data, such as images received from image sensors (e.g., cameras) positioned around the vehicle 400. The matrix processor system 100 can also output information to a display 408 included in the vehicle 400 and receive information (e.g., user input) from a display 408 included in the vehicle 400.

[0059] Other embodiments

[0060] All processes described herein can be implemented and fully automated via software code modules executed by a computing system including one or more computers or processors. The code modules can be stored in any type of non-transitory computer-readable medium or other computer storage device. Some or all of the methods can be implemented in dedicated computer hardware.

[0061] According to the disclosure, many other changes except those described herein will be apparent.For example, according to an embodiment, some actions, events or functions of any algorithm described herein can be performed in different orders, or can be added, merged or completely omitted (for example, not all described actions or events are necessary for the practice of the algorithm). In addition, in certain embodiments, actions or events can be performed concurrently rather than sequentially, for example, by multithreading, interrupt processing or multiple processors or processor cores or on other parallel architectures. In addition, different tasks or processes can be performed by different machines and / or computing systems that can act together.

[0062] The various illustrative logic blocks, modules, and engines described in conjunction with the embodiments disclosed herein may be implemented or executed by a machine designed to perform the functions described herein, such as a processing unit or processor, a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic device, discrete gate or transistor logic, discrete hardware components, or any combination thereof. The processor may be a microprocessor, but in an alternative, the processor may be a controller, a microcontroller, or a state machine, or a combination thereof, etc. The processor may include a circuit system configured to process computer executable instructions. In another embodiment, the processor includes an FPGA or other programmable device that performs logic operations without processing computer executable instructions. The processor may also be implemented as a combination of computing devices, such as a combination of a DSP and a microprocessor, a combination of multiple microprocessors, a combination of one or more microprocessors and a DSP core, or any other such configuration. Although primarily described herein for digital technology, the processor may also primarily include analog components. For example, some or all of the signal processing algorithms described herein may be implemented in an analog circuit system or a mixed analog and digital circuit system. The computing environment may include any type of computer system, including but not limited to a microprocessor-based computer system, a mainframe computer, a digital signal processor, a portable computing device, a device controller, or a computing engine within an appliance, etc.

[0063] Unless otherwise specifically stated, conditional language such as "may," "can," "might," or "could" is understood in the context to be generally used to express that some embodiments include and other embodiments do not include certain features, elements, and / or steps. Thus, such conditional language is generally not intended to imply that one or more embodiments require features, elements, and / or steps in any way, or that one or more embodiments must include logic for deciding, with or without user input or prompting, whether such features, elements, and / or steps are included or to be performed in any particular embodiment.

[0064] Unless expressly stated otherwise, disjunctive language, such as the phrase "at least one of X, Y, or Z," is understood along with the context to be generally used to indicate that an item, term, etc. may be X, Y, or Z, or any combination thereof (e.g., X, Y, and / or Z). Thus, such disjunctive language is generally not intended to, and should not, imply that certain embodiments require that at least one of X, at least one of Y, or at least one of Z each be present.

[0065] Any process description, element or block in the flowcharts described herein and / or depicted in the accompanying drawings should be understood to potentially represent a module, segment or portion of code including one or more executable instructions for implementing a specific logical function or element in the process. Alternative implementations are included within the scope of the embodiments described herein, where elements or functions may be deleted, executed from the order shown or discussed, including substantially simultaneously or in reverse order, depending on the functions involved as will be understood by those skilled in the art.

[0066] Unless expressly stated otherwise, articles such as "a" or "an" should generally be interpreted as including one or more of the described items. Thus, a phrase such as "a device configured to..." is intended to include one or more of the described devices. Such one or more recited devices may also be configured together to perform the recited statements. For example, "a processor configured to perform statements A, B, and C" may include a first processor configured to perform statement A, working in conjunction with a second processor configured to perform statements B and C.

[0067] It should be emphasized that many changes and modifications may be made to the above-described embodiments, and the elements of these embodiments should be understood to be among other acceptable examples. All such modifications and changes are intended to be included within the scope of the present disclosure.

Claims

1. A matrix processor comprising a plurality of processing elements arranged as a grid, wherein the matrix processor is configured to: receiving a first matrix and a second matrix, wherein the first matrix is ​​to be multiplied by the second matrix; transposing the second matrix; organizing the second matrix into a plurality of columns, wherein each column is a row of the second matrix; as well as The columns of the second matrix and the rows of the first matrix are sequentially provided to the processing element in one or more cycles, wherein processing results are stored in the processing element. 2 . The matrix processor according to claim 1 , wherein the order of the multiplication is the first matrix*the second matrix. 3 . The matrix processor of claim 1 , wherein the second matrix is ​​populated based on a size associated with the second matrix.

4. The matrix processor of claim 1, wherein the matrix processor is a convolution engine, and wherein convolution parameters are set for the matrix processor.

5. The matrix processor of claim 4, wherein the convolution parameters indicate a filter of size 1×1.

6. A matrix processor according to claim 5, wherein the convolution parameters indicate a single input channel and / or output channel.

7. The matrix processor of claim 1, wherein the array of processing elements is non-systolic.

8. The matrix processor of claim 1, wherein each processing element stores an element of the multiplication.

9. The matrix processor of claim 1, wherein the processing element is configured as a multiply-accumulate unit (MAC unit).

10. The matrix processor of claim 1, wherein the second matrix is ​​a subset of a larger matrix, and wherein the larger matrix is ​​separated into a plurality of subsets based on a size of the second matrix exceeding a size associated with the processing element.

11. A method implemented by a matrix processor comprising an array of processing elements, wherein the method comprises: Obtaining a first matrix and a second matrix, wherein the first matrix is ​​to be multiplied by the second matrix; transposing the second matrix; organizing the second matrix into a plurality of columns, wherein each column is a row of the second matrix; as well as The columns of the second matrix and the rows of the first matrix are sequentially provided to the processing element in one or more cycles, wherein processing results are stored in the processing element. The method according to claim 11 , wherein the order of the multiplication is the first matrix*the second matrix.

13. The method of claim 11, wherein the second matrix is ​​populated based on a size associated with the second matrix.

14. The method of claim 11, wherein the matrix processor is a convolution engine, and wherein convolution parameters are set for the matrix processor. The method of claim 14 , wherein the convolution parameters indicate a filter of size 1×1.

16. The method of claim 15, wherein the convolution parameters indicate a single input channel and / or output channel.

17. The method of claim 11, wherein the array of processing elements is non-shrinking.

18. The method of claim 11, wherein each processing element stores an element of the multiplication.

19. The method of claim 11, wherein the processing element is configured as a multiply-accumulate unit (MAC unit).

20. The method of claim 11, wherein the second matrix is ​​a subset of a larger matrix, and wherein the larger matrix is ​​separated into a plurality of subsets based on a size of the second matrix exceeding a size associated with the processing element.

Images