An FPGA-based arbitrary-dimensional matrix multiplication operator

Abstract

The application relates to an FPGA-based arbitrary-dimension matrix multiplication operator, which comprises a control module, a reconstruction module, an operation module and an output module; the control module is used for calculating submatrix parameters according to a state signal and an original matrix size, sending a start working signal according to the submatrix parameters and sending a result moving signal after submatrix operation is completed; the reconstruction module is used for dividing an original feature matrix into a plurality of feature submatrices according to the start working signal and dividing an original weight matrix into a plurality of weight submatrices; the operation module is used for simultaneously performing multiplication and accumulation operation on each row and each column element of the plurality of feature submatrices and the plurality of weight submatrices in the form of waves to obtain a plurality of result submatrices, and sending the result moving signal after operation of each result submatrix is completed; and the output module is used for moving and storing the result submatrices in sequence according to the result moving signal. The operator greatly improves the calculation efficiency.

Description

Technical Field

[0001] This invention belongs to the field of computer technology, specifically relating to an arbitrary-dimensional matrix multiplication operator based on FPGA. Background Technology

[0002] Convolution and fully connected operators are common fundamental operators in neural network models, and their operations can be converted into matrix multiplication. The essence of a fully connected operator is the multiplication of the feature matrix and the weight matrix. Convolution operators can be converted into matrix multiplication using `img2col` (which calculates convolution using matrix multiplication). Large-scale matrix multiplication operations can be offloaded from the CPU to dedicated hardware accelerators, thus reducing the CPU's workload. Since the sizes of the original feature and weight matrices in a neural network are often variable, this is one of the reasons why matrix multiplication is difficult to implement.

[0003] In a current technology, a reconfigurable matrix multiplication acceleration system for matrix multiplication-intensive algorithms requires pre-calculating the matrix size and computational complexity of all matrices involved in matrix multiplication. Matrix segments are categorized into six types based on differences in the number of rows and columns, allowing for different matrix partitioning methods before matrix operations. Furthermore, the system's matrix multiplication unit (PE array) has multiple operating modes, each tailored to one of the six matrix types. This system has certain requirements regarding the size of the input matrices used for matrix multiplication and exhibits relatively low computational efficiency.

[0004] Existing scheduling methods based on reconfigurable computational structures for arbitrary-dimensional matrix multiplication employ flexible general-purpose processors (GPPs) and application-specific integrated circuits (ASICs). The number of matrix multiplication operation modules is not fixed and requires dynamic scheduling based on the number of currently idle physical processors (PEs), resulting in slow response times. Furthermore, these methods do not involve matrix partitioning or arranging computational modules in an array. Instead, they determine whether to divide the computation of a single element in the result matrix into multiple parts and send them to multiple PEs, or to use a single PE to compute a single element in the result matrix, based on the number of idle PE arrays. This results in low parallelism and low computational efficiency.

[0005] Therefore, existing techniques for multiplying the original feature matrix and weight matrix in neural networks have drawbacks such as requiring a certain size of the input matrix, slow response time, and low computational efficiency. Summary of the Invention

[0006] To address the aforementioned problems in the existing technology, this invention provides an arbitrary-dimensional matrix multiplication operator based on FPGA. The technical problem to be solved by this invention is achieved through the following technical solution:

[0007] This invention provides an arbitrary-dimensional matrix multiplication operator based on FPGA, comprising: a control module, a reconstruction module, an arithmetic module, and an output module, wherein...

[0008] The control module is connected to the reconstruction module, the calculation module, and the output module. It is used to calculate the submatrix parameters based on the status signal and the original matrix size, send a start working signal based on the submatrix parameters, and send a result transfer signal based on the current submatrix calculation end signal after the submatrix calculation is completed.

[0009] The reconstruction module is used to perform submatrix segmentation on the original feature matrix to obtain several feature submatrixes according to the start working signal, and to perform submatrix segmentation on the original weight matrix to obtain several weight submatrixes.

[0010] The calculation module is used to perform multiplication and accumulation operations on the elements of each row and column of the several feature sub-matrices and the elements of each row and column of the several weight sub-matrices in the form of waves, according to the matrix calculation order of first horizontal and then vertical, to obtain several result sub-matrices, and to send the current sub-matrix calculation end signal after the calculation of each result sub-matrix is ​​completed.

[0011] The output module is used to sequentially move and store the several result sub-matrices according to the result moving signal to form an output matrix.

[0012] In one embodiment of the present invention, the reconstruction module includes an original feature matrix storage module, an original weight matrix storage module, a feature matrix reconstruction module, a weight matrix reconstruction module, a new feature matrix storage module, and a new weight matrix storage module, wherein,

[0013] The original feature matrix storage module is used to store the original feature matrix; the original weight matrix storage module is used to store the original weight matrix.

[0014] The feature matrix reconstruction module is connected to the original feature matrix storage module and the control module, and is used to convert the original feature matrix into a data format that matches the operation module according to the start signal, so as to obtain the plurality of feature sub-matrices;

[0015] The new feature matrix storage module is connected to the feature matrix reconstruction module and is used to store the plurality of feature sub-matrices;

[0016] The weight matrix reconstruction module is connected to the original weight matrix storage module and the control module, and is used to convert the original weight matrix into a data format that matches the operation module according to the start signal, so as to obtain the plurality of weight sub-matrices;

[0017] The new weight matrix storage module is connected to the weight matrix reconstruction module and is used to store the several weight sub-matrices.

[0018] In one embodiment of the present invention, the original feature matrix storage module is a 32-bit wide BRAM; the original weight matrix storage module is a 32-bit wide BRAM;

[0019] The original feature matrix is ​​an 8-bit wide integer of size M×N; the original weight matrix is ​​an 8-bit wide integer of size N×P, where M represents the number of rows in the original feature matrix, N represents the number of columns in the original feature matrix or the number of rows in the original weight matrix, and P represents the number of columns in the original weight matrix.

[0020] The new feature matrix storage module is a 64-bit wide BRAM; the new weight matrix storage module is a 128-bit wide BRAM.

[0021] The number of the plurality of feature sub-matrices is [(M-1) / / 8+1], and the offset of the starting address of adjacent feature sub-matrices is 1; in the new feature matrix storage module, the offset of adjacent data addresses within each feature sub-matrice is equal to the number of the plurality of feature sub-matrices;

[0022] The number of the weighted submatrices is [(P-1) / / 16+1], and the offset of the starting address of adjacent weighted submatrices is 1. In the new weighted matrix storage module, the offset of adjacent data addresses within each weighted submatric is equal to the number of weighted submatrices.

[0023] In one embodiment of the present invention, the arithmetic module includes: a matrix operation control module, an input alignment module, a matrix operation module, and an output buffer module, wherein,

[0024] The matrix operation control module connects the control module and the reconstruction module. It is used to obtain and output the feature submatrix and the weight submatrix based on the submatrix parameters and the storage address of the submatrix. After the submatrix operation is completed, it sends the current submatrix operation end signal to the control module.

[0025] The input alignment module is connected to the matrix operation control module and is used to perform input alignment on the plurality of feature sub-matrices and the plurality of weight sub-matrices according to the matrix element misalignment requirements of the matrix operation module, so as to obtain a plurality of aligned feature sub-matrices and a plurality of aligned weight sub-matrices.

[0026] The matrix operation module is connected to the input alignment module and is used to receive several aligned feature submatrices and several aligned weight submatrices in the order of first horizontal and then vertical matrix calculation. During each submatric operation, the elements of each row and column of the aligned feature submatrices and the elements of each row and column of the aligned weight submatrices are multiplied and accumulated in the form of waves to obtain the several result submatrices.

[0027] The output buffer module is connected to the matrix operation module and the matrix operation control module, and is used to store the several result sub-matrices according to the size of the result sub-matrices sent by the matrix operation control module.

[0028] In one embodiment of the present invention, in the matrix operation module, the propagation clock period from data input to data output during a single submatrix operation is:

[0029] T=N′+P′+M′-2+M′+1=N′+P′+2×M′-1

[0030] Where M′ represents the number of rows in the eigenmatrix, N′ represents the number of columns in the eigenmatrix or the number of rows in the weight submatrix, and P′ represents the number of columns in the weight submatrix.

[0031] In one embodiment of the present invention, the matrix operation module includes a first multiply-accumulate systolic array and a second multiply-accumulate systolic array, wherein,

[0032] The first multiply-accumulate systolic array has several first left-side input channels for corresponding columns of the input-aligned feature submatrix and several first upper-side input channels for corresponding rows of the input-aligned weight submatrix. The first multiply-accumulate systolic array is used to propagate each row of matrix elements of the several first left-side input channels to the right in a horizontal direction and to propagate each column of matrix elements of the several first upper-side input channels to the bottom direction. In each multiply-accumulate module, a multiplier accumulation operation is performed on the column elements of the aligned feature submatrix and the row elements of the aligned weight submatrix to obtain a first result submatrix.

[0033] The second multiply-accumulate pulse array has several second left-side input channels for corresponding columns of the input-aligned feature submatrix, and several second upper-side input channels for corresponding the remaining rows of the input-aligned weight submatrix. The second multiply-accumulate pulse array is used to propagate each row of matrix elements from the several second left-side input channels sequentially to the right, and to propagate each column of matrix elements from the several second upper-side input channels sequentially downwards. In each multiply-accumulate module, a multiplier accumulation operation is performed on the column elements of the aligned feature submatrix and the remaining row elements of the aligned weight submatrix to obtain a second result submatrix. The first result submatrix and the second result submatrix together form the result submatrix.

[0034] The expression for the multiplier accumulation operation is:

[0035] S i+1 ←S i +a i ×b i

[0036] Where S represents the cumulative multiplier value, S i S represents the current accumulated value of the multiplier. i+1 a represents the sum of the multipliers obtained by summing. i and b i These represent the multipliers of the new input.

[0037] In one embodiment of the present invention, the output buffer module includes a first output buffer submodule and a second output buffer submodule, wherein,

[0038] The first output buffer submodule is connected to the first multiply-accumulate systolic array and is used to store the first result submatrix according to the size of the submatrix operation result;

[0039] The second output buffer submodule is connected to the second multiply-accumulate pulsation array and is used to store the second result submatrix according to the size of the submatrix operation result.

[0040] In one embodiment of the present invention, both the first multiply-accumulate systolic array and the second multiply-accumulate systolic array include an 8×8 MAC systolic array;

[0041] Both the first output buffer submodule and the second output buffer submodule include 8 FIFOs.

[0042] In one embodiment of the present invention, the output module includes an output control module and a result storage module, wherein,

[0043] The output control module is connected to the control module, the output terminal of the first output buffer submodule, and the output terminal of the second output buffer submodule. It is used to obtain the first data coordinates of the submatrix, the size of the output matrix, and the size of the result submatrix from the control module according to the result shift signal, and to poll each FIFO in turn according to the first data coordinates of the submatrix, the size of the output matrix, and the size of the result submatrix to shift the output matrix.

[0044] The result storage module is connected to the output control module and is used to store the output matrix.

[0045] In one embodiment of the present invention, the output matrix is ​​a 32-bit wide integer data of size M×P, wherein the result submatrix is ​​less than or equal to 8×16; the row adjacent data offset in the result submatrix is ​​1, and the column adjacent data offset is P; the data address offset of the corresponding position in the row direction of the result submatrix differs by 16; the data address offset of the corresponding position in the column direction of the result submatrix differs by 8*P.

[0046] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0047] The arithmetic unit of this invention, due to the inclusion of a reconstruction module, has no requirements regarding the size of the input matrix or the difference in the number of rows and columns. It performs data reconstruction based on the parameters of the original matrix within the arithmetic unit to achieve submatrix segmentation, thus enabling the processing of matrices of arbitrary dimensions. The arithmetic module, following a matrix calculation order of first horizontal and then vertical, feeds the elements of each row and column of several feature submatrices and several weight submatrices in parallel in a wave-like manner. Multiplication and accumulation operations are performed simultaneously during data propagation, resulting in a fast response time and the ability to produce the output matrix in a short time with minimal resources, greatly improving computational efficiency. Attached Figure Description

[0048] Figure 1 A schematic diagram of an arbitrary-dimensional matrix multiplication operator based on FPGA provided in an embodiment of the present invention;

[0049] Figure 2 A schematic diagram of another FPGA-based arbitrary-dimensional matrix multiplication operator provided in an embodiment of the present invention;

[0050] Figure 3 This is a state transition diagram of the overall control module provided in an embodiment of the present invention;

[0051] Figure 4 This is a schematic diagram of the data format of the original matrix provided in an embodiment of the present invention;

[0052] Figure 5 A schematic diagram of the storage structure of the reconstructed weight matrix in the new weight matrix storage module provided in an embodiment of the present invention;

[0053] Figure 6 A schematic diagram of the storage structure of the reconstructed feature matrix in the new feature matrix storage module, provided in an embodiment of the present invention;

[0054] Figure 7 This is a data flow structure diagram of the 8×8 MAC array in the matrix operation module provided in this embodiment of the invention;

[0055] Figure 8 This is a diagram of the internal structure of the MAC unit provided in an embodiment of the present invention;

[0056] Figure 9 This is a diagram of the output matrix data structure in the output buffer module provided in an embodiment of the present invention;

[0057] Figure 10 This invention provides a 2×2 MAC array clk0 data propagation method according to an embodiment of the invention.

[0058] Figure 11 This is the data propagation method of the 2×2 MAC array clk1 provided in the embodiments of the present invention;

[0059] Figure 12 This is the 2×2 MAC array clk2 data propagation method provided in the embodiments of the present invention;

[0060] Figure 13 This is the 2×2 MAC array clk3 data propagation method provided in the embodiments of the present invention;

[0061] Figure 14 This is the 2×2 MAC array clk4 data propagation method provided in the embodiments of the present invention;

[0062] Figure 15 This is the data propagation method of the 2×2 MAC array clk5-clk8 provided in the embodiment of the present invention. Detailed Implementation

[0063] The present invention will be further described in detail below with reference to specific embodiments, but the implementation of the present invention is not limited thereto.

[0064] Example 1

[0065] Please see Figure 1 and Figure 2 , Figure 1 This is a schematic diagram of an arbitrary-dimensional matrix multiplication operator based on FPGA provided in an embodiment of the present invention. Figure 2 This is a schematic diagram of another arbitrary-dimensional matrix multiplication operator based on FPGA provided in an embodiment of the present invention.

[0066] This embodiment of an FPGA-based arbitrary-dimensional matrix multiplication operator includes a control module, a reconstruction module, an operation module, and an output module. The control module is connected to the reconstruction module, operation module, and output module. It calculates submatrix parameters based on status signals and the original matrix size, sends a start signal based on the submatrix parameters, and sends a result transfer signal based on the end signal of the current submatrix operation after the submatrix operation is completed. The reconstruction module performs submatrix segmentation on the original feature matrix and the original weight matrix based on the start signal, resulting in several feature submatrixes and several weight submatrixes. The operation module, following a horizontal-first, then vertical matrix calculation order, sends several aligned feature submatrixes and several aligned weight submatrixes to the matrix operation module. During each matrix operation, it simultaneously performs multiplication and accumulation operations on the elements of each row and column of the feature submatrix and the elements of each row and column of the weight submatrix in a wave-like manner to obtain several result submatrixes. After the operation of each result submatrix is ​​completed, it sends a current submatrix operation end signal. The output module sequentially transfers and stores several result submatrixes based on the result transfer signal to form an output matrix.

[0067] In one specific embodiment, the control module includes a control information storage module BRAM CTRL32b and a main control module CTRL. The control information storage module BRAM CTRL32b is used to store status signals. The main control module CTRL is connected to the control information storage module BRAM CTRL32b and is used to calculate submatrix parameters based on the status signals and the original matrix size. Then, it sends a start-working signal based on the submatrix parameters. After data reconstruction is completed, it calculates the storage address of the next submatrix based on the current submatrix's position and the storage address of the corresponding result submatrix. After the submatrix calculation is complete, it sends a result transfer signal.

[0068] Specifically, the main control module CTRL controls all state transitions from the original matrix input to the result matrix output through its internal state machine. This includes: the main control module reads the control information storage module BRAM CTRL32b; if the state signal is set, it obtains the original matrix parameters and calculates the sub-matrix parameters based on them. Then, based on the sub-matrix parameters, it sends a start signal to the reconstruction module, notifying it to perform the original matrix data structure conversion. After data reconstruction, it calculates the storage address of the next sub-matrix in the new matrix storage module of the reconstruction module based on the current sub-matrix position, and calculates the storage address of the result sub-matrix calculated from the current sub-matrix in the result storage module of the output module. After the matrix operation module finishes its operation on the current sub-matrix, it notifies the output control module to move the data according to the storage address of the result sub-matrix calculated from the current sub-matrix. When the last sub-matrix finishes its operation, the corresponding state signal is reset, marking the end of one matrix multiplication operation.

[0069] Please see Figure 3 , Figure 3 The state transition diagram of the overall control module provided in the embodiment of the present invention. Figure 3 The task corresponding to each state is as follows:

[0070] IDLE: Initial state, continuously reads the FLAG signal in the control BRAM_CTRL until the signal is 1, then jumps to the GET_COMMAND state.

[0071] GET_COMMAND: Reads the instructions from BRAM_CTRL, obtains the instruction parameters M, N, and P, and jumps to the INFO_RESHAPE state. If an instruction anomaly is found (M, N, or P is any 0), it jumps to the FINISH state. This state is split into GET_COMMAND1 (obtaining M and P) and GET_COMMAND2 (obtaining N) for execution. Here, M is the number of rows in the original feature matrix FM, N is the number of columns in the original feature matrix FM or the original weight matrix WM, and P is the number of columns in the original weight matrix WM.

[0072] INFO_RESHAPE: Notifies FM_reshape and WM_reshape to start working according to the instruction parameters M, N, and P, and jumps to the WAIT_RESHAPE state.

[0073] WAIT_RESHAPE: Waits for FM_reshape and WM_reshape to complete, then jumps to the COM_SUBADDR state.

[0074] COM_SUBADDR: Calculates the storage address of the next submatrix in BRAM_FM and BRAM_WM based on the instruction parameters M, N, P and the current submatrix position, and also calculates the storage address of that submatrix in BRAM_OUT. Jumps to the INFO_MULTIPLY state.

[0075] INFO_MULTIPLY: Notifies the Multiply_ctrl module to start working according to the storage addresses of the submatrix in BRAM_FM and BRAM_WM, and jumps to the WAIT_MULTIPLY state.

[0076] WAIT_MULTIPLY: Waits for the Multiply_ctrl module to finish its work. Then it jumps to the INFO_OUT state.

[0077] INFO_OUT: Notifies the Out_ctrl module to start working according to the storage address of the submatrix in BRAM_OUT, and jumps to WAIT_OUT.

[0078] WAIT_OUT: Waits for the Out_ctrl module to finish its work. Then jumps to the JUDGE_FINISH state.

[0079] JUDGE_FINISH: Determines whether the current submatrix is ​​the last submatrix. If yes, it jumps to the FINISH state; otherwise, it jumps to the COM_SUBADDR state.

[0080] FINISH: Modify the FLAG flag and jump to the IDLE state.

[0081] In one specific embodiment, the reconstruction module includes an original feature matrix storage module BRAM FM 32b, an original weight matrix storage module BRAM WM 32b, a feature matrix reconstruction module FM_reshape, a weight matrix reconstruction module WM_reshape, a new feature matrix storage module BRAM FM 64b, and a new weight matrix storage module BRAM WM 128b. The original feature matrix storage module BRAM FM 32b stores the original feature matrix; the original weight matrix storage module BRAM WM 32b stores the original weight matrix. The feature matrix reconstruction module FM_reshape is connected to the original feature matrix storage module BRAM FM 32b and the main control module CTRL in the control module. It is used to convert the original feature matrix into a data format matching the operation module according to the start signal, obtaining several feature sub-matrices. The new feature matrix storage module BRAM FM 64b is connected to the feature matrix reconstruction module FM_reshape and is used to store several feature sub-matrices. The weight matrix reconstruction module WM_reshape is connected to the original weight matrix storage module BRAM WM 32b and the main control module CTRL in the control module. It is used to convert the original weight matrix into a data format that matches the operation module according to the start signal, and obtain several weight sub-matrices. The new weight matrix storage module BRAM WM 128b is connected to the weight matrix reconstruction module WM_reshape and is used to store several weight sub-matrices.

[0082] Specifically, the data reconstruction modules FM_reshape and WM_reshape read data from the original matrix storage modules BRAM FM32b and BRAM WM 32b, convert the data bit width, and store it into the new matrix storage modules BRAM FM64b and BRAM WM 128b.

[0083] Please see Figure 4 , Figure 4This is a schematic diagram of the data format of the original matrix provided in an embodiment of the present invention. The sizes of the original feature matrix FM (Feature Memory) and the original weight matrix WM (Weight Memory) are not fixed. Therefore, the original feature matrix FM and the original weight matrix WM are required to satisfy the following: the original feature matrix FM is an 8-bit wide integer data of size M×N; the original weight matrix WM is an 8-bit wide integer data of size N×P; and the output matrix Output is a 32-bit wide integer data of size M×P. The original feature matrix storage module BRAM FM 32b is a 32-bit wide BRAM; the original weight matrix storage module BRAM WM 32b is a 32-bit wide BRAM.

[0084] Please see Figure 5 and Figure 6 , Figure 5 This is a schematic diagram of the storage structure of the reconstructed weight matrix in the new weight matrix storage module provided in an embodiment of the present invention. Figure 6 A schematic diagram of the storage structure of the reconstructed feature matrix in the new feature matrix storage module provided in the embodiments of the present invention.

[0085] Since the matrix operation module in the arithmetic module consists of an 8×16 sub-matrix operation unit composed of two Multiply_8x8 modules, each clock cycle requires 8 and 16 numbers to be input through the two input interfaces, directly retrieving 32-bit input data from BRAM FM 32b and BRAM WM 32b without bit width conversion would only output 32 bits per clock cycle, which is insufficient for the 8×16 sub-matrix operation unit and would reduce computational efficiency. Therefore, after the original matrices FM and WM are stored in the 32-bit BRAM, bit width conversion is performed on FM and WM respectively to achieve matrix segmentation. The converted matrices are then placed in 64-bit and 128-bit BRAMs, namely the new feature matrix storage module BRAM FM64b and the new weight matrix storage module BRAM WM 128b. Finally, based on the storage address of the sub-matrix, they are sent to the arithmetic module for multiplication and accumulation operations.

[0086] When FM_reshape and WM_reshape perform matrix partitioning on the original matrices FM and WM, for an M×N original feature matrix FM, there are a total of [(M-1) / / 8+1] feature submatrices (the symbol " / / " is defined as a / / b representing the floor function of the quotient of a divided by b), and the offset of the starting address of adjacent feature submatrices is 1. In the new feature matrix storage module BRAM FM 64b, for a single feature submatrix, the offset of its adjacent data address is [(M-1) / / 8+1], that is, the number of several feature submatrices is consistent with the offset of adjacent data address within a single feature submatrix. For the original weight matrix WM of N×P, there are a total of [(P-1) / / 16+1] weight submatrices, and the offset of the starting address of adjacent weight submatrices is 1. In the new weight matrix storage module BRAM WM128b, for a single weight submatrix, the offset of its adjacent data address is [(P-1) / / 16+1], that is, the number of weight submatrices is consistent with the offset of adjacent data address within a single weight submatrix.

[0087] Because the arithmetic unit in this embodiment has a reconstruction module, it does not have requirements on the size of the input matrix or the difference in the number of rows and columns. The arithmetic unit performs data reconstruction based on the parameters of the original matrix to achieve submatrix segmentation, so it can process matrices of arbitrary dimensions.

[0088] In one specific embodiment, the operation module includes: a matrix operation control module (Multiply_ctrl), an input alignment module, a matrix operation module, and an output buffer module. The Multiply matrix operation control module connects the main control module within the control module and the 64b and BRAM WM components within the reconstruction module. 128b is used to obtain and output feature submatrices and weight submatrices based on submatric parameters and storage addresses. After submatric operations are completed, it sends a result shift signal to the main control module CTRL in the control module and calculates the propagation clock period based on the submatric parameters. The input alignment module is connected to the matrix operation control module Multiply_ctrl and is used to align several feature submatrices and several weight submatrices according to the matrix element misalignment requirements of the matrix operation module, resulting in several aligned feature submatrices and several aligned weight submatrices. The matrix operation module is connected to the input alignment module and is used to receive several aligned feature submatrices and several aligned weight submatrices in the order of first horizontal and then vertical matrix calculation. During each submatric operation, the elements of each row and column of the aligned feature submatrices and the elements of each row and column of the aligned weight submatrices are multiplied and accumulated simultaneously in wave form to obtain several result submatrices. The output buffer module is connected to the matrix operation module and the matrix operation control module and is used to store several result submatrices according to the size of the result submatrices sent by the matrix operation control module.

[0089] In one specific embodiment, the input alignment module includes two identical alignment submodules, input_align. The matrix operation module requires each row of the feature submatrix to be shifted by one position. One alignment submodule, input_align, performs input alignment on the feature submatrix according to this requirement, resulting in an aligned feature submatrix. Similarly, the matrix operation module requires each column of the weight submatrix to be shifted by one position. The other alignment submodule, input_align, performs input alignment on the weight submatrix according to this requirement, resulting in an aligned weight submatrix.

[0090] In one specific embodiment, the matrix operation module includes a first multiply-accumulate systolic array and a second multiply-accumulate systolic array. The first multiply-accumulate systolic array includes a plurality of first left-side input channels and a plurality of first upper-side input channels, and the second multiply-accumulate systolic array includes a plurality of second left-side input channels and a plurality of second upper-side input channels. During each submatrix operation, the data in the first left-side input channels and the second left-side input channels are identical.

[0091] The first left-hand input channels of the first multiply-accumulate systolic array correspond to several columns of the input-aligned feature submatrix, and the first top-hand input channels correspond to a portion of the input-aligned weight submatrix. The first multiply-accumulate systolic array propagates each row of matrix elements from the first left-hand input channels horizontally to the right and each column of matrix elements from the first top-hand input channels downwards. In each multiply-accumulate module, the column elements of the aligned feature submatrix and a portion of the row elements of the aligned weight submatrix are multiplied and accumulated to obtain the first result submatrix. It can be understood that each column of the aligned feature submatrix corresponds one-to-one with each of the first left-hand input channels of the first multiply-accumulate systolic array. For example, the first column of the aligned feature submatrix is ​​input to the first first left-hand input channel of the first multiply-accumulate systolic array, the second column of the aligned feature submatrix is ​​input to the second first left-hand input channel of the first multiply-accumulate systolic array, and so on. The upper rows of the aligned weight submatrix correspond one-to-one with each of the first upper input channels of the first multiply-accumulate systolic array. For example, the first row of the aligned weight submatrix is ​​input to the first upper input channel of the first multiply-accumulate systolic array, the second row of the aligned weight submatrix is ​​input to the second upper input channel of the first multiply-accumulate systolic array, and so on.

[0092] The second multiply-accumulate pulse array has several second left-side input channels corresponding to several columns of the input-aligned feature submatrix, and several second upper-side input channels corresponding to the remaining rows of the input-aligned weight submatrix. The second multiply-accumulate pulse array propagates each row of matrix elements from the several second left-side input channels sequentially to the right horizontally, and each column of matrix elements from the several second upper-side input channels sequentially downwards. In each multiply-accumulate module, the column elements of the aligned feature submatrix and the remaining rows of the aligned weight submatrix are multiplied and accumulated to obtain the second result submatrix. It can be understood that each column of the aligned feature submatrix corresponds one-to-one with each second left-side input channel of the second multiply-accumulate pulse array. For example, the first column of the aligned feature submatrix is ​​input to the first second left-side input channel of the second multiply-accumulate pulse array, the second column of the aligned feature submatrix is ​​input to the second second left-side input channel of the second multiply-accumulate pulse array, and so on. The lower rows of the aligned weight submatrix correspond one-to-one with each of the second upper input channels of the second multiply-accumulated systolic array. For example, the first row of the lower part of the aligned weight submatrix is ​​input to the first second upper input channel of the second multiply-accumulated systolic array, the second row of the lower part of the aligned weight submatrix is ​​input to the second second upper input channel of the second multiply-accumulated systolic array, and so on.

[0093] The first result submatrix and the second result submatrix together form the result submatrix.

[0094] Specifically, both the first and second multiply-accumulate systolic arrays include 8×8 multiply-accumulate (MAC) systolic arrays. That is, the matrix operation module consists of two 8×8 multiply-accumulate systolic arrays (Multiply_8x8), forming an 8×16 submatrix operation unit. Furthermore, the essential process of submatrix multiply-accumulate operation is simply moving the input to the 8×16 submatrix operation unit and then moving the output result away.

[0095] Specifically, the MAC unit is mainly used to build the matrix multiplication module and is a basic component of the matrix multiplication module. The function of the MAC unit is to perform multiplication-accumulation operation, that is, to add the product of the multiplication to the value of the accumulator and then store it into the accumulator, thus completing the operation on one element of the resulting submatrix of the matrix multiplication. Multiplying an M×N matrix A and an N×P matrix B will result in an M×P matrix C, where any element C... i，j satisfy:

[0096]

[0097] Among them, C i，j Let A represent the element in the i-th row and j-th column of matrix C. i，k Let A represent the element in the i-th row and k-th column of matrix A, and B represent the element in the k-th column of matrix B.k，j Let n represent the element in the k-th row and j-th column of matrix B, where n represents the number of columns in matrix A or the number of rows in matrix B.

[0098] For example,

[0099]

[0100] Where a = 1 × 7 + 2 × 9 + 3 × 11 = 58.

[0101] For any element in the resulting submatrix, the result is actually the vector inner product of the corresponding row of the first input matrix and the corresponding column of the second input matrix. A single MAC module can compute one element in the resulting submatrix; multiple MAC modules can be used to compute the values ​​of all elements in a resulting submatrix of a corresponding size in parallel.

[0102] The function of a MAC unit is "multiplier accumulation," that is, adding the product of multiplication to the value of the accumulator and then storing it in the accumulator.

[0103] S i+1 ←S i +a i ×b i

[0104] Where S represents the cumulative multiplier value, S i S represents the current accumulated value of the multiplier. i+1 a represents the sum of the multipliers obtained by summing. i and b i These represent the multipliers of the new input.

[0105] For vectors a and b, their inner product is defined as follows:

[0106]

[0107] In the MAC unit, register S is used to store the current accumulated value S. i , using a i and b i Let a represent two new multipliers input in any given clock cycle. Then, at the rising edge of any given clock cycle, the MAC module can sample the input a. i and b i and the current accumulated value S i Then, you will input 'a'. i and b i The result is fed into the multiplier to obtain the multiplication result and the accumulated value S. i Add them together, and the sum S obtained will be... i+1 Return it to register S. At the next clock cycle, S... i+1This value will be sampled as the new accumulated value. Accordingly, as long as register S is set to 0 at the very beginning of the calculation, the "multiplier accumulation" will continue in each clock cycle until the inner product of the decision vector is calculated, and the calculation result will be output.

[0108] Each 8×8 MAC systolic array has 64 MAC modules used to calculate the product of a matrix A with no more than 8 rows and a matrix B with no more than 8 columns. The design employs a systolic array concept, arranging the MAC modules in an 8x8 square matrix. Each MAC module interacts with only its adjacent MAC modules. When a matrix needs input, it is input simultaneously from the left and top sides of the array. Matrix elements from the left side propagate horizontally to the right, while matrix elements from the top side propagate vertically downwards. Since the array has 8 rows, there are 8 input channels on the left side; similarly, there are 8 input channels on the top side. Each row of the left-side input matrix occupies one input channel; each column of the top-side input matrix occupies one input channel. Within each multiply-accumulate systolic array, the calculation order for the submatrices is defined as horizontal first, then vertical. For example, if FM (M×N) has submatrices FM1 and FM2, and WM (N×P) has submatrices WM1 and WM2, then the order of calculating the submatrices is: FM1×WM1, FM1×WM2, FM2×WM1, FM2×WM2.

[0109] Please see Figure 7 , Figure 7 This is a data flow structure diagram of the 8×8 MAC array in the matrix operation module provided in an embodiment of the present invention. In the diagram, the weight matrix and feature matrix are input into the MAC array from the left and the previous parallel input, and propagate to the right and down at a rate of one clock cycle, performing multiplication and accumulation operations in the current MAC unit.

[0110] Please see Figure 8 , Figure 8 This is a diagram illustrating the internal structure of the MAC unit provided in this embodiment of the invention. In the diagram, f_fata is the horizontal input data, f_data_r is the horizontal propagation data; w_data is the vertical input data, w_data_r is the vertical propagation data; data_l is the output result of the next vertical level, whose value is sent to this level. The selector at this level performs three-level arbitration between data_l and the result of the operation at this level, selecting a value to output as the output result of this level through data_o. It can be seen that f_fata and w_data are multiplied by a multiplier, added to the accumulated value stored in data_reg, and the sum is returned to data_reg. data_reg is then cleared by a selector. When the final operation is performed, the sum is sent to the data_o register and output through data_o. The result is then output upwards after the calculation is complete.

[0111] Furthermore, for both weighted and feature data, a valid signal synchronized with the data transmission occurs during the multiply-accumulate systolic array. The valid signal changes according to logic, and data is considered valid only when the valid signal goes high. Therefore, the three-level arbitration means: when both valid and last are high, multiply-accumulate operations are performed; when the valid_l signal is high, data_l is output; when valid_l is not high, data_o is output. The last signal goes low after the input submatrix data flows through each row or column multiply-accumulate module.

[0112] In this embodiment, each row and column of the segmented submatrix is ​​fed into the matrix operation module in parallel. Multiplication and accumulation operations are performed simultaneously during data propagation, resulting in a matrix with less resources in a shorter time. Furthermore, the first and second multiply-accumulate systolic arrays have only one fixed calculation module and do not transfer partial sums. The multiply-accumulate operations of each MAC run independently, and only the input data and result data are transferred between MACs, which has the advantages of parallelism and high efficiency.

[0113] Furthermore, the matrix operation control module Multiply_ctrl moves data from BRAM_FM_64b and BRAM_WM_128b according to the submatrix data address, and sends it to the two Multiply_8x8 arrays for submatrix operations through the input alignment module. Specifically, data from BRAM_FM_64b is sent to both Multiply_8x8 arrays simultaneously, while data from BRAM_WM_128b is sent to the two Multiply_8x8 arrays separately, high and low 64 bits. For example, for the FM1×WM1 calculation, FM1 has 8 elements as input and WM1 has 16 elements as input, so WM1 needs to be split into two 8-element inputs and input into the two 8×8 MAC pulse arrays. Simultaneously, the matrix operation control module Multiply_ctrl calculates the propagation clock cycle occupied by the submatrix from input to output based on the submatrix parameters to control the transition of the internal state machine, promptly forwarding the state signal of the matrix operation module to the output module, notifying the output module to read the submatrix and allocate the next submatrix to the operation module. The propagation clock cycle is:

[0114] T=N′+P′+M′-2+M′+1=N′+P′+2×M′-1

[0115] Where M′ represents the number of rows in the eigenmatrix, N′ represents the number of columns in the eigenmatrix or the number of rows in the weight submatrix, and P′ represents the number of columns in the weight submatrix.

[0116] In one specific embodiment, the output buffer module includes a first output buffer submodule and a second output buffer submodule, wherein the first output buffer submodule is connected to a first multiply-accumulate systolic array and is used to store a first result submatrix according to the size of the submatrix operation result; the second output buffer submodule is connected to a second multiply-accumulate systolic array and is used to store a second result submatrix according to the size of the submatrix operation result.

[0117] Both the first output buffer submodule and the second output buffer submodule include 8 FIFOs, meaning that both the first output buffer submodule and the second output buffer submodule are Align_fifo modules.

[0118] Specifically, the matrix operation control module Multiply_ctrl can determine the size of the resulting submatrix after multiplying the feature submatrix and the weight submatrix based on their size. The first output buffer submodule and the second output buffer submodule store the output data in 8 FIFOs respectively according to the size of the resulting submatrix provided by Multiply_ctrl.

[0119] Furthermore, when receiving data, the first and second output buffer submodules need to determine whether the data is the required data, because the matrix operation module always outputs an 8×8 submatrix, which contains some unnecessary zero data. Since the required submatrix result is always located in the upper left corner of the 8×8 submatrix, and the unnecessary zero-filling data is located on the right and bottom sides, the size of the current result submatrix and several counters can be used to determine which data is actually needed. After collecting enough data, the first and second output buffer submodules inform the matrix operation control module Multiply_ctrl and wait for the output module to perform data transfer.

[0120] In one specific embodiment, the output module includes an output control module Out_ctrl and a result storage module BRAM_OUT_32b. The output control module Out_ctrl is connected to the output terminals of the control module, the first output buffer submodule, and the second output buffer submodule. It is used to obtain the first data coordinates (Ma, Pa), the size (P), and the result size (sub_P, sub_M) of the submatrix from the control module according to the result shift signal. It then polls each FIFO sequentially according to the first data coordinates (Ma, Pa), the size (P), and the result size (sub_P, sub_M) of the submatrix to shift the output matrix. The result storage module BRAM_OUT_32b is connected to the output control module Out_ctrl and is used to store each result submatrix.

[0121] Specifically, the output control module polls each FIFO sequentially based on the submatrix first data coordinates (Ma, Pa), output matrix size (M×P), and result submatrix size (sub_P, sub_M) obtained from the main control module, but the polling time cannot exceed the size of the current result submatrix. This moves the result submatrix from the output buffer module to BRAM_OUT_32b, and multiple result submatrixes form the entire output matrix in BRAM_OUT_32b.

[0122] Specifically, since matrix partitioning may introduce some "residual blocks" that will be filled to the same size, the size of the valid data portion needs to be known when moving the output results. Furthermore, when performing operations on a submatrix, it's necessary to know which part of the original matrix the calculated submatrix is ​​located in, i.e., the coordinates of the first data element of the submatrix (Ma, Pa). It's also necessary to know when all submatrices have been processed, i.e., the size of the output matrix (M×P) and the size of the resulting submatrix (sub_P, sub_M).

[0123] The output matrix is ​​a 32-bit wide integer data of size M×P, and each result submatrix is ​​less than or equal to 8×16; the row adjacent data in each result submatrix is ​​offset by 1, and the column adjacent data is offset by P; the data address offset of the corresponding position in the result submatrix in the row direction differs by 16; the data address offset of the corresponding position in the result submatrix in the column direction differs by 8*P.

[0124] Please see Figure 9 , Figure 9 This is a diagram of the output matrix data structure in the output buffer module provided in an embodiment of the present invention. The four colored blocks in the diagram correspond to the operation results of the four sub-matrices. Figure 9 As can be seen, not all submatrices output results in a size of 8×16.

[0125] Combination Figure 5 , Figure 6 and Figure 9 ,exist Figure 6 Here we can see that the starting address of the first data in submatrix 1 is 0 (corresponding to data 0a0), and the starting address of the first data in submatrix 2 is 8 (corresponding to data 8a0); Figure 5 Similarly, for the feature matrix, the starting address of the first data in submatrix 1 is 0 (corresponding to data 0b0), and the starting address of the first data in submatrix 2 is 16 (corresponding to data 0b16). Therefore, the starting address of the first data in the resulting submatrix after multiplying feature submatrix 1 and weight submatrix 2 should be 16 (Ma*P+Pa, where P is 20 in this example). Figure 6The initial data address of the yellow block. The first data address of the submatrix resulting from multiplying feature submatrix 2 and weight submatrix 2 should be 176, corresponding to the initial data address of the gray block.

[0126] Taking a 2×2 array as an example, and assuming the input matrix sizes are 2×3 (B) and 3×2 (A) respectively, please refer to the documentation for the data propagation method. Figures 10-15 Figure 10 illustrates the data propagation method of the 2×2 MAC array clk0 provided in an embodiment of the present invention. Figure 11 This is the 2×2 MAC array clk1 data propagation method provided in the embodiments of the present invention. Figure 12 This is the 2×2 MAC array clk2 data propagation method provided in the embodiments of the present invention. Figure 13 This is the 2×2 MAC array clk3 data propagation method provided in the embodiments of the present invention. Figure 14 This is the 2×2 MAC array clk4 data propagation method provided in the embodiments of the present invention. Figure 15 This is the 2×2 MAC array clk5-clk8 data propagation method provided in the embodiments of the present invention. Figure 15 This describes the propagation method for the resulting submatrix. From Figures 10-15 It can be seen that the time from data input to complete output of result is as follows: T = N + P + M - 2 + M + 1 = N + P + 2 × M - 1. Therefore, the arithmetic unit can complete the calculation and reading of the result in a relatively short time.

[0127] The arithmetic unit implemented here is a computationally efficient arbitrary-dimensional matrix multiplication arithmetic unit suitable for implementation on FPGA. It can offload the operations of large-scale convolution operators and fully connected operators in neural network models from the CPU to a dedicated hardware accelerator. Utilizing the parallel data propagation method of a systolic array and a pipelined structure of one-time propagation and internal calculation of results, it completes matrix operations that would otherwise take a long time on the CPU in a short time. Furthermore, the position of valid data in the result matrix can be calculated through the parameters of the submatrix, eliminating the need to store invalid data. Therefore, this arithmetic unit effectively reduces the computation time of matrix multiplication and the resource consumption during matrix multiplication.

[0128] The above description, in conjunction with specific preferred embodiments, provides a further detailed explanation of the present invention. It should not be construed that the specific implementation of the present invention is limited to these descriptions. For those skilled in the art, various simple deductions or substitutions can be made without departing from the concept of the present invention, and all such modifications and substitutions should be considered within the scope of protection of the present invention.

Claims

1. An arbitrary-dimensional matrix multiplication operator based on FPGA, characterized in that, include: The module consists of a control module, a reconfiguration module, a computation module, and an output module. The control module is connected to the reconstruction module, the calculation module, and the output module. It is used to calculate the submatrix parameters based on the status signal and the original matrix size, send a start working signal based on the submatrix parameters, and send a result transfer signal based on the current submatrix calculation end signal after the submatrix calculation is completed. The reconstruction module is used to perform submatrix segmentation on the original feature matrix to obtain several feature submatrixes according to the start working signal, and to perform submatrix segmentation on the original weight matrix to obtain several weight submatrixes. The calculation module is used to perform multiplication and accumulation operations on the elements of each row and column of the several feature sub-matrices and the elements of each row and column of the several weight sub-matrices in the form of waves, according to the matrix calculation order of first horizontal and then vertical, to obtain several result sub-matrices, and to send the current sub-matrix calculation end signal after the calculation of each result sub-matrix is ​​completed. The output module is used to sequentially move and store the several result sub-matrices according to the result moving signal to form an output matrix.

2. The FPGA-based arbitrary-dimensional matrix multiplication operator according to claim 1, characterized in that, The reconstruction module includes an original feature matrix storage module, an original weight matrix storage module, a feature matrix reconstruction module, a weight matrix reconstruction module, a new feature matrix storage module, and a new weight matrix storage module, wherein... The original feature matrix storage module is used to store the original feature matrix; the original weight matrix storage module is used to store the original weight matrix. The feature matrix reconstruction module is connected to the original feature matrix storage module and the control module, and is used to convert the original feature matrix into a data format that matches the operation module according to the start signal, so as to obtain the plurality of feature sub-matrices; The new feature matrix storage module is connected to the feature matrix reconstruction module and is used to store the plurality of feature sub-matrices; The weight matrix reconstruction module is connected to the original weight matrix storage module and the control module, and is used to convert the original weight matrix into a data format that matches the operation module according to the start signal, so as to obtain the plurality of weight sub-matrices; The new weight matrix storage module is connected to the weight matrix reconstruction module and is used to store the several weight sub-matrices.

3. The FPGA-based arbitrary-dimensional matrix multiplication operator according to claim 2, characterized in that, The original feature matrix storage module is a 32-bit wide BRAM; the original weight matrix storage module is a 32-bit wide BRAM; The original feature matrix is ​​an 8-bit wide integer of size M×N; the original weight matrix is ​​an 8-bit wide integer of size N×P, where M represents the number of rows in the original feature matrix, N represents the number of columns in the original feature matrix or the number of rows in the original weight matrix, and P represents the number of columns in the original weight matrix. The new feature matrix storage module is a 64-bit wide BRAM; the new weight matrix storage module is a 128-bit wide BRAM. The number of the plurality of feature sub-matrices is [(M-1) / / 8+1], and the offset of the starting address of adjacent feature sub-matrices is 1; in the new feature matrix storage module, the offset of adjacent data addresses within each feature sub-matrice is equal to the number of the plurality of feature sub-matrices; The number of the weighted submatrices is [(P-1) / / 16+1], and the offset of the starting address of adjacent weighted submatrices is 1. In the new weighted matrix storage module, the offset of adjacent data addresses within each weighted submatric is equal to the number of weighted submatrices.

4. The FPGA-based arbitrary-dimensional matrix multiplication operator according to claim 1, characterized in that, The computation module includes: a matrix operation control module, an input alignment module, a matrix operation module, and an output buffer module, wherein, The matrix operation control module connects the control module and the reconstruction module. It is used to obtain and output the feature submatrix and the weight submatrix based on the submatrix parameters and the storage address of the submatrix. After the submatrix operation is completed, it sends the current submatrix operation end signal to the control module. The input alignment module is connected to the matrix operation control module and is used to perform input alignment on the plurality of feature sub-matrices and the plurality of weight sub-matrices according to the matrix element misalignment requirements of the matrix operation module, so as to obtain a plurality of aligned feature sub-matrices and a plurality of aligned weight sub-matrices. The matrix operation module is connected to the input alignment module and is used to receive several aligned feature submatrices and several aligned weight submatrices in the order of horizontal calculation first and then vertical calculation. During each submatrix operation, the elements of each row and column of the aligned feature submatrix and the elements of each row and column of the aligned weight submatrix are multiplied and accumulated in the form of waves to obtain the several result submatrices. The output buffer module is connected to the matrix operation module and the matrix operation control module, and is used to store the several result sub-matrices according to the size of the result sub-matrices sent by the matrix operation control module.

5. The FPGA-based arbitrary-dimensional matrix multiplication operator according to claim 4, characterized in that, In the matrix operation module, the propagation clock period from data input to result submatrix output during a single submatrix operation is: T=N′+P′+M′-2+M′+1=N′+P′+2×M′-1 Where M′ represents the number of rows in the eigenmatrix, N′ represents the number of columns in the eigenmatrix or the number of rows in the weight submatrix, and P′ represents the number of columns in the weight submatrix.

6. The FPGA-based arbitrary-dimensional matrix multiplication operator according to claim 4, characterized in that, The matrix operation module includes a first multiply-accumulate systolic array and a second multiply-accumulate systolic array, wherein... The first multiply-accumulate systolic array has several first left-side input channels for corresponding columns of the input-aligned feature submatrix and several first upper-side input channels for corresponding rows of the input-aligned weight submatrix. The first multiply-accumulate systolic array is used to propagate each row of matrix elements of the several first left-side input channels to the right in a horizontal direction and to propagate each column of matrix elements of the several first upper-side input channels to the bottom direction. In each multiply-accumulate module, a multiplier accumulation operation is performed on the column elements of the aligned feature submatrix and the row elements of the aligned weight submatrix to obtain a first result submatrix. The second multiply-accumulate pulse array has several second left-side input channels for corresponding columns of the input-aligned feature submatrix, and several second upper-side input channels for corresponding the remaining rows of the input-aligned weight submatrix. The second multiply-accumulate pulse array is used to propagate each row of matrix elements from the several second left-side input channels sequentially to the right, and to propagate each column of matrix elements from the several second upper-side input channels sequentially downwards. In each multiply-accumulate module, a multiplier accumulation operation is performed on the column elements of the aligned feature submatrix and the remaining row elements of the aligned weight submatrix to obtain a second result submatrix. The first result submatrix and the second result submatrix together form the result submatrix. The expression for the multiplier accumulation operation is: S i+1 ←S i +a i ×b i Where S represents the cumulative multiplier value, S i S represents the current accumulated value of the multiplier. i+1 a represents the sum of the multipliers obtained by summing. i and b i These represent the multipliers of the new input.

7. The FPGA-based arbitrary-dimensional matrix multiplication operator according to claim 6, characterized in that, The output buffer module includes a first output buffer submodule and a second output buffer submodule, wherein, The first output buffer submodule is connected to the first multiply-accumulate systolic array and is used to store the first result submatrix according to the size of the submatrix operation result; The second output buffer submodule is connected to the second multiply-accumulate pulsation array and is used to store the second result submatrix according to the size of the submatrix operation result.

8. The FPGA-based arbitrary-dimensional matrix multiplication operator according to claim 7, characterized in that, Both the first multiply-accumulate systolic array and the second multiply-accumulate systolic array include an 8×8 MAC systolic array; Both the first output buffer submodule and the second output buffer submodule include 8 FIFOs.

9. The FPGA-based arbitrary-dimensional matrix multiplication operator according to claim 8, characterized in that, The output module includes an output control module and a result storage module, wherein, The output control module is connected to the control module, the output terminal of the first output buffer submodule, and the output terminal of the second output buffer submodule. It is used to obtain the first data coordinates of the submatrix, the size of the output matrix, and the size of the result submatrix from the control module according to the result shift signal, and to poll each FIFO in turn according to the first data coordinates of the submatrix, the size of the output matrix, and the size of the result submatrix to shift the output matrix. The result storage module is connected to the output control module and is used to store the output matrix.

10. The FPGA-based arbitrary-dimensional matrix multiplication operator according to claim 1, characterized in that, The output matrix is ​​a 32-bit wide integer data of size M×P, wherein the result submatrix is ​​less than or equal to 8×16; the row-adjacent data offset in the result submatrix is ​​1, and the column-adjacent data offset is P; the data address offset of the corresponding position in the result submatrix in the row direction differs by 16; the data address offset of the corresponding position in the result submatrix in the column direction differs by 8*P.

Images