A matrix multiply-add computation acceleration system optimizing bandwidth reduction shared cache overhead

Abstract

The application discloses a kind of matrix multiplication calculation acceleration systems for optimizing bandwidth to reduce shared cache overhead, including matrix multiplication calculation accelerator, global cache, low-delay access link, data bus, off-chip storage and processor core CPU, matrix multiplication calculation accelerator contains pulsating array SA, A loading, B loading, C loading, Y save and PB buffer, and matrix multiplication calculation accelerator connects global cache by low-delay access link.The matrix operation O=IW T Split into mn times Y=AB submatrix operation and make full use of the multiplexing of submatrix data in different row and column directions, reduce global cache data access times and global cache data access bandwidth without affecting performance, and the increased memory level capacity is low in overhead, can be compatible with the original matrix operation mode, while according to bandwidth requirement CPU can dynamically adjust bandwidth allocation between CPU core and matrix multiplication calculation accelerator by adjusting the value of m and n register.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of integrated circuit design scheme for accelerating matrix multiplication and addition calculation, and particularly relates to a matrix multiplication and addition calculation acceleration system for optimizing bandwidth and reducing shared cache overhead. BACKGROUND

[0002] At present, the mainstream matrix multiplication and addition calculation accelerator adopts the structure of systolic array. For a matrix multiplication and addition calculation Y=AB+C, the above structure fixes the weight data B in the calculation unit PE (Processing Engine) and then inputs the multiplied matrix A as input data to the PE, and outputs the data in a pulse and accumulates on C. The calculation unit PE forms a two-dimensional array, the input matrix data is input from the left side in a clock pulse, and the output calculation result is output from the bottom of the array in a clock pulse. However, the above accelerator needs to input a data vector of the multiplied matrix A as a pulse input in each clock cycle, and output a data vector of the AB calculation result as a partial sum for accumulation. The above data vector needs to access the global cache to obtain, which causes the on-chip storage read-write access congestion, consumes additional energy for on-chip memory access, and the data reuse of matrix operation is insufficient.

[0003] As Figure 1As shown, the existing matrix multiplication and addition calculation accelerator preloads the coefficient matrix B from the SA (Systolic Array) top, wherein the SA is a two-dimensional calculation array composed of r rows and c columns of PEs, and the preloaded B matrix remains in the PE array internally during the entire process of pulsating input of the A matrix. In order to mask the delay caused by loading the B matrix, two registers are used inside the PE to save the B matrix, when one of the registers works in the preloading mode, it receives the coefficient value of the corresponding position of the B matrix from the pulsating input of the SA top, and the other register saves the coefficient value of the corresponding position of the B matrix corresponding to the last preloading of the current preloading operation and outputs the value to the multiplication and addition unit MAC inside the PE. The multiplication matrix A is pulsated from the left side of the SA and r input data are required per clock pulse. The output side of the SA is completed by an accumulator to add the C matrix and output the Y matrix, wherein the C matrix and the Y matrix are stored in the same physical address space of the memory, and c calculated partial sums are output by the SA per clock pulse, which are added with c values read from the C buffer and saved to the Y buffer, and the above saving process generates c write operations. The existing matrix multiplication and addition calculation accelerator structure needs a global cache to save the Y matrix, the C matrix and the A matrix, and this global cache needs to provide a bandwidth corresponding to r+c read operations and c write operations per clock pulse, which is the main part of the bandwidth of the existing matrix multiplication and addition calculation accelerator, causing a bandwidth bottleneck, and also making it difficult to improve the size of the SA under the limited global cache bandwidth.

[0004] The existing solutions to this problem include: using a multi-level cache structure for the global cache to improve the bandwidth that the primary cache can provide; and separately setting up a private cache for the SA of the matrix multiplication and addition calculation accelerator, which usually includes a multi-channel cache that can simultaneously read and write to save the Y matrix, the A matrix and the C matrix, and a data transfer module for interaction between the internal cache of the matrix multiplication and addition calculation accelerator and external data. The above solutions do not solve the bandwidth problem from the root cause of the high bandwidth required by the SA. For the first of the above solutions, the increased cache system complexity is a high-cost overhead, and for the second of the above solutions, the capacity of the private cache established usually needs to meet the size of storing the entire Y matrix and A matrix, which leads to the problem of excessive capacity of the private cache of the existing matrix multiplication and addition calculation accelerator, which is also a high-cost overhead. For occasions where the size of the private cache is limited, the size of the matrix that can be calculated by the existing matrix multiplication and addition calculation accelerator in a single calculation is also limited, which limits the improvement of data multiplexing. SUMMARY

[0005] In order to overcome the data transmission bandwidth bottleneck of the existing matrix multiplication and addition calculation accelerator and optimize data multiplexing, the application provides a matrix multiplication and addition calculation acceleration system for optimizing bandwidth and reducing shared cache overhead, which can not only reduce the demand of a systolic array on the access bandwidth of an on-chip global cache, but also improve data multiplexing and reduce the energy consumption of an on-chip memory.

[0006] The technical scheme adopted by the application to solve the technical problems is:

[0007] The application first provides a matrix multiplication and addition calculation acceleration system for optimizing bandwidth and reducing shared cache overhead, which comprises a matrix multiplication and addition calculation accelerator, a global cache, a low-delay access link, a data bus, an off-chip storage and a processor core CPU, wherein the off-chip storage, the processor core CPU, the global cache and the matrix multiplication and addition calculation accelerator are connected with the data bus, a part of the storage space in the global cache is allocated as a first running cache for storing intermediate data in the matrix multiplication and addition process, the matrix multiplication and addition calculation accelerator is connected with the global cache through the low-delay access link and accesses the first running cache, and the low-delay access link has a higher priority than the data bus in accessing the global cache.

[0008] The matrix multiplication and addition calculation accelerator comprises a systolic array SA, a matrix A loading module, a matrix B loading module, a matrix C loading module, a matrix Y saving module and a PB buffer.

[0009] The systolic array SA is formed by r rows and c columns of calculation units PE in a two-dimensional array, each PE in the systolic array SA comprises a first buffer for buffering elements of a B matrix, the PB buffer is arranged at the output side of the systolic array SA and comprises an accumulator, the PB buffer is used for buffering the partial sum generated by the mutual accumulation of the outputs of the SA in adjacent clock pulses, the matrix A loading module obtains the value of an input matrix A from the data bus and inputs the value into the SA in a systolic manner, the matrix B loading module loads a coefficient matrix B stored in advance in the off-chip storage or the global cache into the SA through the data bus, the matrix C loading module loads a matrix C in the off-chip storage or the global cache into the first running cache through the data bus, the matrix Y saving module is used for saving the calculation result Y matrix in the first running cache into the off-chip storage or the global cache, and the low-delay access link is used for read-write access of the accumulator at the output side of the systolic array SA to the first running cache, write access of the matrix C loading module to the first running cache and read access of the matrix Y saving module to the first running cache.

[0010] As a preferred scheme of the present application, in the systolic array SA, each column of PE corresponds to one value of c output results outputted by the systolic array SA in each clock pulse, and the output value is propagated in the form of systolic from the top of the column to the bottom of the column; each row of PE shares one value in the column vector of the input A matrix, and the shared data is also propagated in the form of systolic from the leftmost side of the row to the rightmost side of the row; each PE in the PE array stores m*n matrix element values in the m*n B matrix, and therefore the PE contains a first buffer with a capacity of at least m*n B matrix elements for storing this part of data.

[0011] As a preferred scheme of the present application, for the case where the preloading B matrix delay needs to be concealed, the value of m*n is constrained to be less than half of the capacity of the first buffer, so that in this round of calculation, the corresponding B matrix already exists in the PE and is used for calculation, while the B matrix required for the next round of calculation can be preloaded into the space of the first buffer in the remaining PE, so as to achieve the purpose of concealing the delay of the preloaded B matrix in the clock pulses consumed by the SA calculation.

[0012] As a preferred scheme of the present application, the first buffer is a double-port memory which can be read and written simultaneously.

[0013] As a preferred scheme of the present application, the matrix A loading module divides m input channels corresponding to m A matrices into k column vectors in the column direction respectively, and traverses the column vectors of the same column in the m A matrices in sequence, and then traverses the k column vectors in each A matrix in the column direction, so as to complete the output traversal of m groups of k column vectors. The column vectors outputted by the A loading module after traversing the m A matrices are provided to the input side of the SA in the form of systolic input, wherein each column vector of the m A matrices is shared by n B matrices of the output channels, so that each column data read from the A matrix is invalidated after n clock pulses.

[0014] As a preferred scheme of the present application, when the array formed by the first buffer in the SA stores m*n B matrices of m input channels and n output channels in total, the loaded A matrix vector remains unchanged in the same time period, that is, n clock pulses, and when the partial sum outputted by the SA is accumulated in m time periods, only the n row vectors of n C matrices need to be read from the global buffer in the 0th time period, and only the n row vectors of n Y matrices need to be written into the global buffer in the m-1th time period, and the input and output data of the output side accumulator of the SA in the remaining time periods are all read from or written into the PB buffer.

[0015] As a preferred scheme of the present application, when the array of first buffers in the SA stores m*n B matrices of m input channels and n output channels, the matrix A loading module needs to load an A matrix vector from the global buffer to the SA every n clock pulses on average, the read side of the output side accumulator of the SA needs to load a C matrix vector from the global buffer to the accumulator of the PB buffer through the low-latency access link every m clock pulses on average, and the write side of the accumulator of the PB buffer needs to write a Y matrix vector to the global buffer through the low-latency access link every m clock pulses on average.

[0016] As a preferred scheme of the present application, when the array of first buffers in the SA stores m*n B matrices of m input channels and n output channels, the array of first buffers in the SA outputs a set of B matrices corresponding to the set of m B matrices of the i-th column in the i-th clock pulse in the m time periods, where i = 0, 1, …, n-1, when the array of first buffers in the SA iterates through the B matrices in m*n adjacent clock pulses.

[0017] Compared with the prior art, the present application has the beneficial effect that the access bandwidth of the matrix multiplication and addition calculation accelerator to the shared cache can be optimized in a system in which a processor core CPU and the matrix multiplication and addition calculation accelerator share the cache, and the data multiplexing of the above-mentioned accelerator is improved, so as to reduce the energy consumed by the accelerator core in the shared cache access. BRIEF DESCRIPTION OF DRAWINGS

[0018] Figure 1 is a structural diagram of a matrix multiplication and addition calculation accelerator of the prior art.

[0019] Figure 2 is a structural diagram of a matrix multiplication and addition calculation accelerator in an example of the present application.

[0020] Figure 3 is a structural diagram of a systolic array SA in an example of the present application.

[0021] Figure 4 is a system comprising a matrix multiplication and addition calculation accelerator in an example of the present application.

[0022] Figure 5 is a schematic diagram of preloading of B matrices to internal PEs of the SA in an example of the present application.

[0023] Figure 6 is a data flow schematic diagram of a matrix multiplication and addition calculation accelerator in an example of the present application.

[0024] Figure 7is a schematic diagram of data flow corresponding matrix partition in the embodiment of the present application. DETAILED DESCRIPTION

[0025] The present application will be further described and illustrated with specific embodiments. The embodiments are only examples of the present disclosure and do not define the scope of the present application. The technical features of various embodiments of the present application can be combined accordingly without conflict.

[0026] As shown in Figure 2 and 4 The matrix multiplication and addition calculation acceleration system provided by the embodiment for optimizing bandwidth and reducing shared cache overhead mainly comprises a matrix multiplication and addition calculation accelerator, a global cache, a low-latency access link, a data bus, an off-chip storage and a processor core CPU. The off-chip storage, the processor core CPU, the global cache and the matrix multiplication and addition calculation accelerator are connected with the data bus respectively. A part of the storage space in the global cache is allocated as a first running cache for storing intermediate data in the matrix multiplication and addition process. The matrix multiplication and addition calculation accelerator is connected with the global cache through the low-latency access link and accesses the first running cache. The low-latency access link has a higher priority than the data bus accessing the global cache.

[0027] The matrix multiplication and addition calculation accelerator comprises a systolic array SA, a matrix A loading module, a matrix B loading module, a matrix C loading module, a matrix Y saving module and a PB buffer. The systolic array SA is composed of r rows and c columns of calculation units PE in the form of a two-dimensional array. Each PE in the systolic array SA comprises a first buffer for buffering elements of the matrix B. The PB buffer is arranged at the output side of the systolic array SA and comprises an accumulator. The PB buffer is used for buffering partial sums generated by mutual accumulation of the outputs of the SA in adjacent clock pulses. The matrix A loading module obtains the values of the input matrix A from the data bus and inputs the values into the SA in a systolic manner. The matrix B loading module loads the coefficient matrix B stored in the off-chip storage or the global cache in advance into the SA through the data bus. The matrix C loading module loads the matrix C in the off-chip storage or the global cache into the first running cache through the data bus. The matrix Y saving module is used for saving the calculation result Y matrix in the first running cache into the off-chip storage or the global cache. The low-latency access link is used for read-write access of the accumulator at the output side of the systolic array SA to the first running cache, write access of the matrix C loading module to the first running cache and read access of the matrix Y saving module to the first running cache.

[0028] In Figure 2 The structure diagram of the matrix multiplication and addition calculation accelerator in the embodiment of the present application is similar to Figure 1The B-matrix preloading method in this context corresponds to the m*n B-matrixes for m input channels and n output channels being loaded into the PE array of the SA in the same batch. The m A-matrixes corresponding to the m input channels are read interleaved and provided to the input side of the SA in the form of pulsed inputs. Each column of the input vector of the aforementioned m A-matrixes is shared by the B-matrixes of the n output channels, ensuring that each column of data read from the A-matrix remains valid only after n clock pulses. This reduces the bandwidth of the A-matrix input side from r data points per clock pulse to... Data, such as Figure 6 The values ​​given by A during each time period do not need to be changed. For the output side of SA, since the column vectors of the A matrix corresponding to the m clock pulse inputs of the m input channels are given within adjacent clock pulses, and the outputs corresponding to these inputs need to be accumulated, the bandwidth required for this accumulation operation is provided by a small PB buffer. The introduction of PB changes the data bandwidth required by the system composed of SA and PB for the global buffer from the original read c data and write c data per clock pulse to read c data per clock pulse. Data and writing Data, such as Figure 6 As shown, within the m time periods, the C matrix needs to be read only in the 0th time period, and the Y matrix needs to be written in the (m-1)th time period.

[0029] exist Figure 3 In the structural diagram of the SA in this embodiment of the invention, the PEs (Programmers) arranged in r rows and c columns form the SA in a two-dimensional array. Each column of PE corresponds to one of the c output results of each clock pulse of the SA. The output values ​​propagate from the top of a column of PEs in a pulsating manner to the bottom of that column. Each row of PEs shares one value from the column vector of the input matrix A, and the shared data also propagates in a pulsating manner from the leftmost side of that row of PEs to the rightmost side. Each PE in the PE array stores... Figure 2 The system contains m*n B matrices, each with m*n matrix elements. Therefore, the PE (Programmer) needs a register file with a capacity of at least m*n B matrix elements to store this data. For cases where the delay of preloading the B matrices needs to be masked, the value of m*n can be constrained to be less than half the capacity of the register file. This way, while the B matrix corresponding to the current calculation is already stored in the PE and used for calculation, the B matrix needed for the next calculation can be preloaded into the remaining space of the PE's register file, thus masking the delay of preloading the B matrix within the clock pulses consumed by the SA (Self-Loading) calculation. Due to the requirement to simultaneously preload the B matrix and provide it to the MAC unit within the PE, the PE's register file needs to be a dual-port memory capable of simultaneous reading and writing.

[0030] In Figure 4 In the system containing the matrix multiplication and addition calculation accelerator in the present application example, thanks to the low-bandwidth design of the present application example for the SA, by providing the register file overhead inside each PE inside the SA and the PB buffer overhead, the present application example can save the A matrix, the C matrix and the Y matrix in the global cache, wherein the access to the C matrix and the Y matrix is directly accessed to the global cache through a low-delay access link bypassing the data bus, the access link has the highest priority to realize low delay, the bandwidth of the link should be selected according to the global cache bandwidth of the system, and for the low-bandwidth scene, the value of m can be appropriately constrained to be large to reduce the bubbles generated by the pipeline in the SA. Figure 4 In the system, the A loading obtains the A matrix data from the data bus and provides it to the SA. The B loading also obtains the B matrix from the data bus and preloads it into the register file of the PE in the SA. The C loading and Y saving module is responsible for the data transfer of the C matrix and the Y matrix in the global cache to meet the requirements of initializing the C matrix in the global cache and transferring the Y matrix. The global cache in the system is not only used as the data cache of the matrix multiplication and addition calculation accelerator in the present application example, but also as the main cache of a processor core CPU, which can save its instructions and data. For the complex algorithm requiring the cooperation of the CPU and the matrix multiplication and addition calculation accelerator, the CPU can directly access the Y matrix of the matrix multiplication and addition calculation accelerator for further processing, and thanks to the low-bandwidth design of the present application example for the SA, the bandwidth of the instruction and data request of the global cache by the CPU at runtime can be fully satisfied. Generally, the processor core CPU can be a symmetric multi-core processing subsystem, a high-bit-width vector processor with a single instruction multiple data instruction set extension, or a heterogeneous multi-core subsystem. The off-chip storage in the system containing the matrix multiplication and addition calculation accelerator is usually used to save constant data such as the coefficient matrix in matrix multiplication. The coefficient matrix is usually the B matrix. For the overhead of the register file in the PE and the PB, assuming that the size of the register file in the PE is f matrix elements, the total capacity of all the register files contained in the SA is f*r*c, and the PB needs to save the values of n output channels, the size thereof should not be less than n*c matrix elements, and since m*n≤f and m is a positive integer, the size of the PB is usually set to f*c matrix elements. Due to its small capacity characteristics, the PB is usually implemented by a register file.

[0031] In summary, the additional storage overhead in the present application example is fc(r+1) matrix elements. A commonly used f value is 16, and the overall bandwidth optimization amplitude can be represented as When m=4 and n=4, 4 times bandwidth optimization can be achieved. For a common 16 row and 16 column PE SA, the additional storage overhead is 4352 matrix elements. Assuming that the matrix multiplication and addition calculation precision is 8-bit fixed point, the 4352 matrix elements correspond to a size of about 4.3 FB. This is a relatively low overhead compared to the size of tens or hundreds of FB of global cache, but it enables 4 times bandwidth optimization when calculating tens or hundreds of FB size matrices. Moreover, the additional storage overhead required by the example of the present application is independent of the sizes of the A, B, C and Y matrices and is only related to the number of rows and columns of the SA and the value of f. For any size of global cache, the same additional storage overhead can be used to achieve the same bandwidth optimization magnitude. In addition, the bandwidth optimization magnitude can be dynamically allocated at runtime by adjusting the values of m and n. The A loading, B loading and SA output side accumulator can dynamically adjust the data loading and operation mode according to the values of m and n given by the CPU through the data bus configuration matrix multiplication and addition calculation accelerator registers to support variable m and n. For the above-mentioned 16 row and 16 column PE SA, the overall bandwidth can be reduced from the original 48B per pulse to 12B per pulse at a calculation precision of 8-bit fixed point by setting m=4 and n=4. The saved bandwidth is up to 32B per pulse, which is enough to support a vector processor with a 256-bit data bus to run at peak performance when reading and writing data in the global cache. The above discussion is only for the bandwidth of the A, C and Y matrices, but in fact the bandwidth of the preloaded B matrix remains unchanged. Compared with the existing scheme, the m*n times preloaded B matrix requires m*n times the number of clock pulses for calculation. As shown in Figure 6 the i-th column of the m A matrices needs to be traversed to output the i-th row of the Y matrix. This traversal needs to be done k times, because the A matrix is composed of k column vectors, so a total of k*m*n cycles are required to complete the calculation of the m*n preloaded B matrices. However, in the existing scheme, the k column vectors of the A matrix are traversed to generate k row Y matrix output vectors within k clock pulses.

[0032] In Figure 5The diagram illustrating the preloaded B matrix describes the construction of the PEs. Each PE contains: a B buffer storing a portion of the B matrix and its control logic; a multiply-accumulate (MAC) unit; an element of the pulsating input A matrix; pulsating input data from the upper PE in the same column; and pulsating output data to the lower PE in the same column. Writing to the B buffer depends on the input from the pulsating input B loading module above the SA. The B matrix input of the MAC inside the PE can switch within the data contained in the B buffer in each clock pulse. When the B buffer output of the two-dimensional array SA composed of the PEs switches, the result is equivalent to performing a vector-matrix multiplication operation on a vector in the input A matrix and one of the mn B matrices, outputting a partial sum, such as... Figure 6 As shown, within n clock pulses of each time period, the A matrix vector provided by the A loading module remains unchanged, while the B matrix provided by SA traverses among the n output channels, thereby generating the space of the Y matrix partially stored in the PB or directly output to the global buffer for the n output channels. The preloading of the B buffer within the PE is as follows... Figure 5 The diagram shows the physical location of PE within SA. Figure 3 The top left corner PE corresponds to Figure 5 PE in, such as Figure 5 As shown, the set of mn matrix elements in its B buffer corresponds to the top-left corner values ​​of the mn B matrices. Figure 7 This explains the combination of matrices and vector cutting; the m A matrices correspond to the I input matrices of the m input channels, therefore these m matrices in Figure 7 The matrix A is arranged in a column, and each matrix A is divided into k vectors along its columns. The m vectors in the same column constitute a vector. Figure 6 The set of vectors A that are traversed in the middle. For Figure 7 In the W matrix, mn matrices are arranged in an m x n column form. This is equivalent to dividing the large matrix W consisting of mn B matrices into m x n B matrices, where each B matrix is ​​r x c. For a set of n sub-matrices in the same row, its... Figure 6 The traversal occurs within the same time period, while the traversal of the above set for m rows occurs within a specific time period. Figure 6 It is completed within m time intervals. For matrix O, it has n Y matrices, where the set formed by the same row of each submatrix is ​​in... Figure 6 Read and write accesses are completed within the same time period, while Figure 6 This traversal process is performed a total of k times, completing the generation of k rows of the above set and finally generating the entire O matrix.

[0033] Figure 7 The operation process described in the text, which multiplies m A matrices and m*n B matrices to output n Y matrices, can be expressed as follows:

[0034]

[0035] In summary, at the system architecture level, the main control processor (CPU) and the matrix multiply-accumulate accelerator share a global cache. The matrix multiply-accumulate accelerator accesses the global cache through a low-latency link, which connects the input and output of the PB (Power Buffer). The matrix multiply-accumulate accelerator can also access the entire storage space via the data bus. This link provides bandwidth for the various loading and saving modules within the matrix multiply-accumulate accelerator. When matrix B needs to be preloaded into the PE (Power Buffer) within the SA (Structured Array), the B loading module loads the coefficient matrix pre-stored in off-chip storage or the global cache into the SA via the data bus. The coefficients of the weight matrix B loaded during the preloading process contain values ​​from multiple channels, allowing the SA to reuse the input provided by the A loading module between adjacent clock pulses, improving data reuse in matrix operations and reducing the data bus bandwidth used by the A loading module. The A loading module obtains the values ​​of the input matrix A from the data bus, and the obtained matrix itself is also multi-channel, making it possible for the SA's output data to be summed between adjacent clock pulses. This summation operation is performed within a small buffer PB, which further reduces the bandwidth required for the SA's input and output to access the global cache. The bandwidth reduction described above is similar to the bandwidth reduction process of the A loading module mentioned earlier. It relies on improving the reusability of SA output data. This improved reusability is related to the multi-channel B matrix pre-loaded within the PE in SA. It also utilizes the PB structure to read and write the reusable data, thus avoiding reading and writing this data from the more expensive global cache. In summary, by fully leveraging the data reusability during SA matrix operations, this design reduces the bandwidth accessing the global cache at both the input and output levels, improves the overall data reusability of matrix multiplication and addition calculations, and reduces the energy overhead of the accelerator core accessing the global cache.

[0036] The above-described embodiments are merely illustrative of several implementations of the present invention, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of the present invention. Those skilled in the art can make various modifications and improvements without departing from the concept of the present invention, and these modifications and improvements all fall within the scope of protection of the present invention.

Claims

1. A matrix multiply-add computation acceleration system that optimizes bandwidth reduction shared cache overhead, comprising: The application relates to a matrix multiplication and addition accelerator, which comprises a matrix multiplication and addition accelerator, a global cache, a low-delay access link, a data bus, off-chip storage and a processor core CPU, wherein the off-chip storage, the processor core CPU, the global cache and the matrix multiplication and addition accelerator are connected with the data bus, a part of the storage space in the global cache is allocated as a first running cache for storing intermediate data in a matrix multiplication and addition process, the matrix multiplication and addition accelerator is connected with the global cache through the low-delay access link and accesses the first running cache, the low-delay access link has a higher priority than the data bus accessing the global cache, The matrix multiplication and addition accelerator comprises a systolic array (SA), a matrix A loading module, a matrix B loading module, a matrix C loading module, a matrix Y saving module and a PB buffer. The systolic array SA is formed by r rows and c columns of calculation units PE in a two-dimensional array, each PE in the systolic array SA comprises a first buffer for buffering B matrix elements; the PB buffer is arranged at the output side of the systolic array SA and comprises an accumulator, the PB buffer is used for buffering the partial sum generated by mutual accumulation of the outputs of the SA in adjacent clock pulses; the matrix A loading module obtains the value of an input matrix A from the data bus and inputs the value into the SA in a systolic manner, the matrix B loading module loads a coefficient matrix B pre-stored in the off-chip storage or the global cache into the SA through the data bus, the matrix C loading module loads a matrix C in the off-chip storage or the global cache into the first running cache through the data bus, the matrix Y saving module is used for saving the calculation result Y matrix in the first running cache into the off-chip storage or the global cache, and the low-delay access link is used for read-write access of the accumulator at the output side of the systolic array SA to the first running cache, write access of the matrix C loading module to the first running cache and read access of the matrix Y saving module to the first running cache.

2. The matrix-multiply compute acceleration system that optimizes bandwidth and reduces shared cache overheads as claimed in claim 1, wherein: In the systolic array SA, each column of PEs corresponds to one value in the c output results of each clock pulse of the systolic array SA, and the output value is propagated from the top end of the column of PEs to the bottom of the column of PEs in a systolic manner; each row of PEs shares one value in the column vector of the input A matrix, and the shared data is propagated from the leftmost side of the row of PEs to the rightmost side of the row of PEs in a systolic manner; each PE in the PE array stores m*n matrix element values in the m*n B matrix, and therefore the PE comprises a first buffer with a capacity of at least m*n B matrix elements for storing the part of data.

3. The matrix-multiply compute acceleration system of claim 2, wherein, For occasions requiring to mask the delay of preloading the B matrix, the value of m*n can be constrained to be less than half of the capacity of the first buffer, so that the B matrix required by the next round of calculation can be preloaded into the space of the first buffer in the remaining PEs while the corresponding B matrix in this round of calculation already exists in the PEs and is used for calculation, thereby achieving the purpose of masking the delay of preloading the B matrix in the clock pulses consumed by the SA calculation.

4. The matrix-multiply compute acceleration system of claim 3, wherein, The first buffer is a double-port memory which can be read and written simultaneously.

5. The system of claim 1, wherein the system is configured to: The matrix A loading module divides each of the m A matrices corresponding to the m input channels into k column vectors in the column direction, and sequentially traverses the column vectors of the same column in the m A matrices, and then traverses the k column vectors in each A matrix in the column direction, so as to complete the output traversal of the m groups of k column vectors. The column vectors output by the A loading module after traversing the m A matrices are provided to the input side of the SA in the form of pulsating input. Each column vector of the m A matrices is shared by the B matrices of the n output channels, so that each column of data read from the A matrix is invalidated after n clock pulses.

6. The system of claim 1, wherein: When the array formed by the first buffers in the SA stores m*n B matrices of m input channels and n output channels, the loaded A matrix vectors remain unchanged within the same time period, that is, n clock pulses. When the partial sum output by the SA is accumulated in m time periods, only the n row vectors of the n C matrices need to be read from the global buffer in the 0th time period, and only the n row vectors of the n Y matrices need to be written into the global buffer in the m-1th time period. The input and output data of the output side accumulators of the SA in the remaining time periods are all read from or written into the PB buffer.

7. The system of claim 1, wherein: When the array formed by the first buffers in the SA stores m*n B matrices of m input channels and n output channels, the matrix A loading module needs to load an A matrix vector from the global buffer into the SA every n clock pulses on average. The read side of the output side accumulators of the SA needs to load a C matrix vector from the global buffer through the low-latency access link every m clock pulses on average to provide the accumulators of the PB buffer. The write side of the accumulators of the PB buffer needs to write a Y matrix vector into the global buffer through the low-latency access link every m clock pulses on average.

8. The system of claim 1, wherein: When the array formed by the first buffers in the SA stores m*n B matrices of m input channels and n output channels, the B matrices are traversed in m*n adjacent clock pulses. The n B matrices of the same row are traversed in adjacent n clock pulses, and then the traversal of the next row of n B matrices is continued. The time period consisting of n adjacent clock pulses has m groups in total. The set of B matrices output by the array of first buffers in the SA in the ith clock pulse in the m time periods corresponds to the set of m B matrices of the ith column, where i=0, 1, …, n-1.

Images