A high-bandwidth utilization sparse matrix vector multiplication acceleration device

Abstract

The application provides a high-bandwidth utilization rate sparse matrix vector multiplication acceleration device, which comprises a decoder, a read conflict-free input vector buffer, a calculation unit array, a write conflict-free adder tree, a ping-pong supporting accumulator group, a storage part and a result vector buffer; the decoder is used for decoding a preprocessed matrix; the decoder transmits vector elements in the matrix into the read conflict-free input vector buffer after decoding; non-zero elements in a target matrix are decoded and transmitted into the calculation unit array; the calculation unit array is used for reading corresponding vector elements from the read conflict-free input vector buffer according to column numbers of the non-zero elements, multiplying the vector elements with the non-zero data, and transmitting the multiplication results and row numbers of the non-zero elements into the write conflict-free adder tree; the write conflict-free adder tree is used for adding multiplication results with the same row numbers, and transmitting the addition results into accumulators; and the ping-pong supporting accumulator group is used for accumulating the addition results.

Description

Technical Field

[0001] This invention belongs to the field of computer architecture. Background Technology

[0002] Sparse matrix-vector multiplication (SpMV) is widely used in fields such as graphics computing, machine learning, and industrial engineering. With the development of the internet age, the scale of data processed by SpMV tasks has increased, impacting the execution time of these applications. Therefore, accelerating SpMV is crucial for improving the performance of these applications.

[0003] Compared to traditional dense matrix-vector multiplication, SpMV matrices contain a large number of zero elements. To save storage space and data transfer, the compressed storage format only stores and transmits the non-zero elements and their positions within the sparse matrix. Based on this storage method, for each non-zero element, SpMV reads the corresponding vector element based on its column number and multiplies it. Then, it sums the multiplication result with the partial sum of the result vector corresponding to the row number to obtain a new partial sum. Once all non-zero elements have been calculated, the final result is obtained. However, in sparse matrices, the distribution of non-zero elements is random, which introduces random access to vectors during SpMV task execution, causing the time required for data access to exceed the time required for floating-point computation. Furthermore, due to the characteristics of the sparse matrix compressed storage format, SpMV's access to vectors is indirect, making its memory-intensive characteristics more pronounced compared to dense matrix-vector multiplication.

[0004] For Central Processing Units (CPUs) or Graphics Processing Units (GPUs), SpMV's random access to memory is prone to cache misses. The computational throughput of CPUs and GPUs often exceeds the available memory bandwidth, which contradicts the memory-intensive nature of SpMV. Therefore, CPUs and GPUs are not ideal platforms for accelerating SpMV. FPGAs, on the other hand, with their large memory bandwidth, customizable logic units, and high-performance floating-point units, are suitable platforms for accelerating SpMV.

[0005] Because the execution time of SpMV primarily depends on the time required to load data using off-chip bandwidth, bandwidth utilization (BU) is a widely used metric for evaluating the performance of FPGA-based SpMV accelerators. Specifically, BU is determined by the effective data size and latency during task execution; therefore, reducing data redundancy and task latency is crucial for implementing SpMV accelerators on FPGAs. In accelerating SpMV tasks, parallel processing is typically employed. However, parallel operation results in multiple simultaneous random accesses. Since the on-chip Block Random Access Memory (BRAM) of an FPGA usually only has two independent ports, handling these simultaneous random accesses is a challenging task. If this problem is not well addressed, it often leads to memory port conflicts or workload imbalances, further increasing SpMV task latency and ultimately resulting in a poor BU.

[0006] Currently, there are three main solutions to the problem of random access conflicts in parallel processing. The first solution is to execute non-zero elements in the matrix sequentially within the processing elements (PEs) according to their column numbers, thus transforming random access to the input vector into sequential access. However, this execution method is prone to write-after-read conflicts in partial sums and mutual waiting between PEs in parallel mode, resulting in longer task latency and poorer performance bottlenecks (BU). The second solution is to directly replace the column numbers of non-zero elements with the vector elements required for multiplication operations when storing the position information of non-zero elements. When storing sparse matrices, column numbers are 32-bit integer data, and vector elements are 64-bit double-precision floating-point data; therefore, this solution can achieve good performance when the matrix is ​​sufficiently sparse. However, when the matrix is ​​not so sparse, the data redundancy caused by this method cannot be ignored. The last solution utilizes the large-capacity BRAM on the existing FPGA chip to load all vector elements into on-chip memory. Then, to reduce port conflicts caused by simultaneous access, the non-zero data is reordered. However, this method is limited by the distribution of non-zero data in the matrix, and cannot eliminate all port conflicts, leading to worse BU for some matrices. Furthermore, this method suffers from the high complexity of the non-zero element reordering algorithm. In summary, existing work cannot effectively handle the problem of performing multiple random accesses simultaneously in parallel mode, resulting in more data redundancy or longer latency. Based on the above analysis, reducing task latency caused by random access while minimizing data redundancy will play a crucial role in improving the BU of SpMV tasks. Summary of the Invention

[0007] The present invention aims to at least partially solve one of the technical problems in the related art.

[0008] Therefore, the purpose of this invention is to propose a high-bandwidth-utilization sparse matrix-vector multiplication acceleration device to improve the bandwidth utilization of SpMV tasks.

[0009] To achieve the above objectives, a first aspect of the present invention provides a high-bandwidth-utilization sparse matrix-vector multiplication acceleration device, comprising: a decoder, a read-conflict-free input vector cache, a computation unit array, a write-conflict-free adder tree, a ping-pong-supporting accumulator group, a storage section, and a result vector cache; wherein,

[0010] The decoder is used to decode the preprocessed target matrix, wherein the decoder decodes the vector elements in the target matrix and passes them into the read-conflict-free input vector buffer, and decodes the non-zero elements in the target matrix and passes them into the computing unit array.

[0011] The computing unit array is used to read the corresponding vector element from the read-conflict-free input vector cache according to the column number of the non-zero element, then multiply the vector element with the non-zero data, and pass the multiplication result and the row number of the non-zero element into the write-conflict-free adder tree.

[0012] The write-conflict-free adder tree is used to add the multiplication results with the same row number, and the resulting addition result is passed into the accumulator;

[0013] The accumulator group supporting ping-pong is used to accumulate the addition results.

[0014] In addition, the high bandwidth utilization sparse matrix-vector multiplication acceleration device according to the above embodiments of the present invention may also have the following additional technical features:

[0015] Furthermore, in one embodiment of the present invention, a preprocessing module is also included, which is used to preprocess the target matrix, including dividing the target matrix vertically into multiple blocks and dividing each of the multiple blocks horizontally into multiple batches.

[0016] Furthermore, in one embodiment of the present invention, the conflict-free input vector buffer is also used for:

[0017] Four Block Random Access Memory (BRAM) units are used as a group of BRAMs. Two groups of BRAMs are used to store the non-zero element and a copy of the non-zero element, respectively. For each group of BRAMs in the conflict-free input vector cache, two 4-to-1 multiplexers are used to read two non-zero elements or copies of non-zero elements from the four BRAMs in each group of BRAMs. The ports of each group of BRAMs operate in a time-division multiplexing manner.

[0018] Furthermore, in one embodiment of the present invention, the conflict-free adder tree is also used for:

[0019] The multiplication result and 0 are stored in a register;

[0020] The multiplication result is sent to four 2-to-1 multiplexers, and the other inputs of the multiplexers are 0.

[0021] The read results of the four 2-to-1 multiplexers are added together by two 10-stage pipelined adders, and the sum of the two 10-stage pipelined adders is stored in a register.

[0022] The sum of the two 10-stage pipelined adders is added to another 10-stage pipelined adder, and the sum of the other 10-stage pipelined adder is stored in a register.

[0023] By using an 8-to-4 cross array switch to select the stored data of four registers as the partial sum output, write-after-read conflicts caused by adding the multiplication results with the same row number are eliminated.

[0024] Furthermore, in one embodiment of the present invention, the ping-pong-supporting accumulator group is also used for:

[0025] After each batch of data in the target matrix is ​​calculated, the accumulated result is written into the storage portion and the cache of the result vector.

[0026] Furthermore, in one embodiment of the present invention, the ping-pong-supporting accumulator group is also used for:

[0027] The data for each batch is calculated based on the individual accumulators in the accumulator group;

[0028] The switching between batches is achieved by adding two additional registers to the independent accumulator, thereby reducing the overhead of loading and storing the hidden partial sums.

[0029] The high-bandwidth-utilization sparse matrix-vector multiplication acceleration device proposed in this invention solves the port conflict problem in parallel mode by copying a portion of the vectors, thereby increasing the number of on-chip cache read / write ports. Furthermore, the design of a conflict-free adder tree eliminates write-after-read conflicts in partial sums, further reducing SpMV task latency and achieving higher bandwidth utilization. Attached Figure Description

[0030] The above and / or additional aspects and advantages of the present invention will become apparent and readily understood from the following description of the embodiments taken in conjunction with the accompanying drawings, wherein:

[0031] Figure 1 This is a schematic diagram of a sparse matrix-vector multiplication acceleration device with high bandwidth utilization provided in an embodiment of the present invention.

[0032] Figure 2 A schematic diagram of a read-conflict-free vector cache provided in an embodiment of the present invention;

[0033] Figure 3 A schematic diagram of a write-conflict-free adder tree provided in an embodiment of the present invention;

[0034] Figure 4 This is a schematic diagram illustrating the implementation method provided in an embodiment of the present invention. Detailed Implementation

[0035] Embodiments of the present invention are described in detail below, examples of which are illustrated in the accompanying drawings, wherein the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary and intended to explain the present invention, and should not be construed as limiting the present invention.

[0036] The high-bandwidth-utilization sparse matrix-vector multiplication acceleration device of the present invention is described below with reference to the accompanying drawings.

[0037] Example 1

[0038] Figure 1 This is a schematic diagram of a sparse matrix-vector multiplication acceleration device with high bandwidth utilization provided in an embodiment of the present invention.

[0039] like Figure 1 As shown, this high-bandwidth-utilization sparse matrix-vector multiplication acceleration device includes: a decoder, a read-conflict-free input vector cache, a computation unit array, a write-conflict-free adder tree, a ping-pong-enabled accumulator group, a storage section, and a cache for the result vector; wherein,

[0040] The decoder is used to decode the preprocessed target matrix. Specifically, the decoder decodes the vector elements in the target matrix and passes them into a read-conflict-free input vector buffer, and decodes the non-zero elements in the target matrix and passes them into the computing unit array.

[0041] The computational unit array is used to read the corresponding vector element from the read-conflict-free input vector buffer according to the column number of the non-zero element, then multiply the vector element with the non-zero data, and pass the multiplication result and the row number of the zero element into the write-conflict-free adder tree.

[0042] The adder tree without write conflicts is used to add the results of multiplication with the same row number and pass the result into the accumulator;

[0043] The accumulator group that supports ping-pong is used to accumulate the summation results.

[0044] Furthermore, in one embodiment of the present invention, a preprocessing module is also included, which is used to preprocess the target matrix, including dividing the target matrix vertically into multiple blocks and dividing each of the multiple blocks horizontally into multiple batches.

[0045] This invention first divides the matrix. In SpMV, the number of vector elements is the same as the number of columns in the matrix. Therefore, when the matrix is ​​large, the on-chip storage space is insufficient to store all vector elements and their copies at once. Therefore, in this invention, the matrix is ​​vertically divided into multiple blocks, and each block requires a subset of the original vectors for SpMV operations. Furthermore, due to FPGA resource limitations, this invention horizontally divides a block of the matrix into multiple batches. During execution, the blocks are executed sequentially, and each block yields a partial sum vector. The final result vector is obtained by adding the partial sum vectors of all blocks. When a block is executed, the batches within that block are executed sequentially, and the result of each batch is a subset of the partial sum vector of that block.

[0046] The decoder decodes the data input from off-chip bandwidth. For vector elements, the decoder decodes them and passes them to a conflict-free input vector buffer. For non-zero elements of a matrix, the decoder decodes them and passes them to the computation unit array. The input vector buffer copies the input vector and stores the input vector and its copy in BRAM. Each PE in the computation unit array reads the corresponding vector element from the input vector buffer according to the column number of the input non-zero element, multiplies the retrieved vector element with the input non-zero data, and passes the multiplication result and the row number of the non-zero element to a conflict-free adder tree. In the adder tree, the parts with the same row number generated by the PE are summed, and the result is passed to the accumulator. The accumulator accumulates the input vector sums and writes the accumulated result to the result vector buffer after each batch of data is computed.

[0047] Furthermore, in one embodiment of the present invention, the conflict-free input vector buffer is also used for:

[0048] Four Block Random Access Memory (BRAM) units are used as a group of BRAMs. Two groups of BRAMs are used to store the non-zero element and a copy of the non-zero element, respectively. For each group of BRAMs in the conflict-free input vector cache, two 4-to-1 multiplexers are used to read two non-zero elements or copies of non-zero elements from the four BRAMs in each group of BRAMs. The ports of each group of BRAMs operate in a time-division multiplexing manner.

[0049] To eliminate port conflicts caused by random access, this invention designs a conflict-free input vector cache. To simplify control logic, a coordinate-based (COO) format is used to store non-zero matrix elements. For a non-zero element, the COO includes a 64-bit double-precision floating-point value, a 32-bit row number, and a 32-bit column number. Based on this format, a 512-bit off-chip bandwidth can read four non-zero elements from DRAM to the decoder per cycle. After decoding, the decoder stores the four 64-bit double-precision floating-point values ​​into four FIFOs, allowing each PE to access matrix elements independently. Because the PE array has high parallelism, the consumption rate of matrix elements is almost the same as the loading rate; therefore, the depth of the FIFOs used here is very small. This method allows the PE to load four matrix elements in parallel, but it also leads to four irregular random accesses to the vector pairs. Since BRAM on an FPGA typically only has two read / write ports, this invention designs a conflict-free vector cache to expand its read / write ports.

[0050] like Figure 3As shown, the 512-bit off-chip bandwidth can load eight 64-bit double-precision floating-point vector elements per cycle, while each BRAM can only provide two memory access ports. Therefore, this invention treats four BRAMs as a group, using two groups of BRAMs to store the incoming original vector and vector copy, respectively. For each group of BRAMs, two 4-to-1 multiplexers (MUX) are used to read two vector elements from the four BRAMs. The vector elements stored in the two groups of BRAMs are identical, and each group can provide two independent memory access ports, so the two groups of BRAMs can ultimately provide four independent read / write ports for the PE array. However, when writing vector elements, each BRAM in the vector cache also needs to provide two read / write ports, which creates the misconception that each BRAM needs four read / write ports. In reality, the BRAM ports operate in a time-division multiplexing manner. That is, the BRAM does not write and read vector elements simultaneously. When the BRAM loads vector elements from off-chip DRAM, these ports operate as write ports, and when performing multiplication operations, these ports operate as read ports.

[0051] Furthermore, in one embodiment of the present invention, the conflict-free adder tree is also used for:

[0052] The multiplication result and 0 are stored in a register;

[0053] The multiplication result is sent to four 2-to-1 multiplexers, and the other inputs of the multiplexers are 0.

[0054] The read results of the four 2-to-1 multiplexers are added together by two 10-stage pipelined adders, and the sum of the two 10-stage pipelined adders is stored in a register.

[0055] The sum of the two 10-stage pipelined adders is added to another 10-stage pipelined adder, and the sum of the other 10-stage pipelined adder is stored in a register.

[0056] By using an 8-to-4 cross array switch to select the stored data of four registers as the partial sum output, write-after-read conflicts caused by adding the multiplication results with the same row number are eliminated.

[0057] When some of the non-zero elements processed by four PEs are in the same row, the partial sum generated after multiplication of these non-zero elements in the same row will cause a write-after-read conflict. To eliminate this conflict, this invention designs a write-conflict-free adder tree. Figure 3 As shown in the figure, the boxes represent registers. The multiplication result P from the PE array... 0,0 ,P 0,1 ,P 0,2 , and P 0,3The multiplication results are then sent as input to the adder tree. The multiplication results are then sent to four 2-to-1 MUXs (M0, M1, M2, and M3), with all other inputs to the MUXs being 0. Next, the results from M0, M1, M2, and M3 are added using two 10-stage pipelined adders, and the sum is obtained using register a. 1,3 and a 2,3 Store the result of the addition. Then, use another 10-stage pipelined adder to add the previous addition results and store them in register a. 3，5 Finally, we use an 8-to-4 cross-array switch to select the four parts and P as outputs. By controlling the MUX and the cross-array switch, the adder tree can eliminate various write conflicts. For example, in the part and P... 0,0 and P 2,0 When a conflict occurs, the input of adder 1 is set to P. 0,0 And 0, while the input of adder 2 is set to P. 2,0 And 0. Then, adder 3 adds the results of adder 1 and adder 2 and stores the sum in register a. 3，5 In the middle. Finally, by selecting P 1,5 P 3,5 a 3,5 And 0 as the final output, thus eliminating P 0,0 and P 2,0 Conflict between them.

[0058] Furthermore, in one embodiment of the present invention, the ping-pong-supporting accumulator group is also used for:

[0059] After each batch of data in the target matrix is ​​calculated, the accumulated result is written into the storage portion and the cache of the result vector.

[0060] Furthermore, in one embodiment of the present invention, the ping-pong-supporting accumulator group is also used for:

[0061] The data for each batch is calculated based on the individual accumulators in the accumulator group;

[0062] The switching between batches is achieved by adding two additional registers to the independent accumulator, thereby reducing the overhead of loading and storing the hidden partial sums.

[0063] Following the adder tree, the partial sums and their corresponding row numbers are sent to the accumulator group. To eliminate the latency caused by vector addition between blocks, this invention designs a ping-pong-like accumulator group. This accumulator group contains many independent accumulators, each responsible for the partial sum of a row in each batch. Typically, an accumulator contains a register to latch the accumulated result, as shown by the white background reg in the figure, labeled R1. However, since our method involves switching between batches, the partial sums already obtained in the batch need to be read from the partial sum cache before the accumulator performs accumulation, and the partial sums of the batch need to be transferred to the partial sum cache after calculation. To hide the partial sum access overhead, we add two registers to an accumulator, such as... Figure 1 The boxes with medium-dark gray and light gray backgrounds are labeled R2 and R3, respectively. To enable fast batch switching, these three registers operate in a ping-pong-like manner. When executing the first batch, R1 and R2 are used for the current batch's accumulation calculation and loading the partial sum of the second batch from the partial sum cache, respectively, while R3 remains idle. When executing the second batch, R2, R3, and R1 are used for: performing the current batch's accumulation calculation, reading the partial sum of the third batch from the partial sum cache, and storing the partial sum obtained from the first batch into the partial sum cache, respectively. Following this switching rule, the overhead of loading and storing partial sums can be completely hidden.

[0064] Example 2

[0065] The following describes the specific implementation method of the present invention.

[0066] Step 1: Divide the matrix vertically into multiple blocks of size 2^14, meaning each block contains 2^14 columns. Then, divide its vectors into corresponding subsets, each containing 2^14 elements. Next, divide each block horizontally into multiple batches of size 64. Finally, arrange the non-zero element data of the divided matrix in the following order: first, input the corresponding vector subset for each block, then input the non-zero value information for each batch. For example... Figure 4 The figure shows an example of matrix partitioning. Since the actual batch sizes after partitioning are quite large, the figure only shows a 6×6 portion of the first batch of the partitioned matrix and a portion of its corresponding first vector subset.

[0067] Step Two: First, process the first batch of non-zero elements. Since the first batch is obtained from the first block and no batches within the first block have been processed yet, the corresponding vector subset of the first block is first passed to the chip. The vector shown in the diagram is {1,2,3,4,5,6}. The vector elements are passed to the chip via a 512-bit off-chip bandwidth; the six 64-bit vector element values ​​can be transmitted in just one cycle. The decoder decodes the vector elements and writes them to the read-conflict-free vector buffer. The read-conflict vector buffer copies and stores the passed vectors; at this time, the matrix buffer has no data written. After the vector element transmission is complete...

[0068] Begin transmitting matrix data. Note that if the batch being processed is not the first batch in the block, the transmission vector is skipped and the non-zero elements of the matrix are transmitted directly. Since a non-zero element of a matrix is ​​128 bits (64 bits for the non-zero value, 32 bits for the row number, and 32 bits for the column number), only 4 non-zero data elements can be transmitted per cycle. Figure 4 As shown in step two, the non-zero elements from 1 to 4 are first transferred to the chip. After passing through the decoder, the non-zero element values ​​and column numbers are transferred to the PE array. At this time, in addition to the data operations shown in the figure, the two registers in the accumulator begin loading the partial sum of the next batch and storing the partial sum of the previous batch.

[0069] Step 3: The PE array contains 4 PEs. Each PE obtains a non-zero element value and its column number. Based on its column number...

[0070] The address of the vector in the read-free conflict-free vector buffer is obtained. After obtaining the address, the vector buffer reads the corresponding vector value and transmits it to the corresponding PE. For example... Figure 4 As shown in cycle 2, PE1 reads the vector value 5 from the vector cache based on column number 4, and the other PEs do the same. Furthermore, the operations between each PE are completely independent. Additionally, while a PE completes this operation, the next non-...

[0071] Zero-element data is transmitted to the chip via off-chip bandwidth for the operation in step two. In this invention, a pipelined approach is used, so that when step three is performed, step two can be performed simultaneously in different hardware, and the other steps are similar.

[0072] Step 4: PE multiplies the non-zero element value with the vector element value obtained in the previous step at period 3 to obtain the multiplication result, and passes the result into the adder tree. The row number obtained from the decoder is stored in a register and, along with the non-zero element value, is also passed into the adder tree. That is, PE1 multiplies 1 and 5 to obtain a partial sum of 5, and the row number corresponding to the non-zero element 1 (0) is also passed into the adder tree.

[0073] Step 5: In step 4, the partial sums {5, 4, 3, 8} and the row numbers {0, 1, 2, 2} are passed into the write-free conflict adder tree.

[0074] The bolded {3, 8} have the same row number. In the adder tree, 3 and 8 are added together, and 8 is set to 0. The partial sum output by the adder tree is {5, 4, 11, 0}, with row numbers {0, 1, 2, 2}. The resulting partial sum and row numbers are then passed to the accumulator group.

[0075] Step 6: In the accumulator group, 5, 4, and 11 are passed to the accumulators numbered 0, 1, and 2 respectively for accumulation.

[0076] Step 7: Repeat steps 2-6 until all batches obtained after matrix partitioning are completed, and the final vector result is obtained.

[0077] The high-bandwidth-utilization sparse matrix-vector multiplication acceleration device proposed in this invention solves the port conflict problem in parallel mode by copying a portion of the vectors, thereby increasing the number of on-chip cache read / write ports. Furthermore, the design of a conflict-free adder tree eliminates write-after-read conflicts in partial sums, further reducing SpMV task latency and achieving higher bandwidth utilization.

[0078] 0 In the description of this specification, references are made to the terms "one embodiment", "some embodiments", "example", "with", etc.

[0079] The descriptions using terms such as "specific example" or "some examples" refer to specific features, structures, materials, or characteristics described in connection with that embodiment or example, which are included in at least one embodiment or example of the present invention. In this specification, illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples. Moreover, those skilled in the art can combine and integrate the different embodiments or examples described in this specification, as well as the features of different embodiments or examples, without contradiction.

[0080] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one of that feature. In the description of this invention, "a plurality of" means at least two, such as two, three, etc., unless otherwise explicitly specified.

[0081] Although embodiments of the present invention have been shown and described above, it is understood that the above embodiments are exemplary and should not be construed as limiting the present invention. Those skilled in the art can make changes, modifications, substitutions and variations to the above embodiments within the scope of the present invention.

Claims

1. A high-bandwidth-utilization sparse matrix-vector multiplication acceleration device, characterized in that, include: The system includes a decoder, a conflict-free input vector buffer, an array of computation units, a conflict-free adder tree, a ping-pong accumulator group, a storage section, and a cache for the result vector; among these, The decoder is used to decode the preprocessed target matrix, wherein the decoder decodes the vector elements in the target matrix and passes them into the read-conflict-free input vector buffer, and decodes the non-zero elements in the target matrix and passes them into the computing unit array. The computing unit array is used to read the corresponding vector element from the read-conflict-free input vector cache according to the column number of the non-zero element, then multiply the vector element with the non-zero data, and pass the multiplication result and the row number of the non-zero element into the write-conflict-free adder tree. The write-conflict-free adder tree is used to add the multiplication results with the same row number, and the resulting addition result is passed into the accumulator; The accumulator group supporting ping-pong is used to accumulate the addition results.

2. The apparatus according to claim 1, characterized in that... It also includes a preprocessing module for preprocessing the target matrix, including dividing the target matrix vertically into multiple blocks and dividing each of the multiple blocks horizontally into multiple batches.

3. The apparatus according to claim 1, characterized in that... The conflict-free input vector buffer is also used for: Four Block Random Access Memory (BRAM) units are used as a group of BRAMs. Two groups of BRAMs are used to store the non-zero elements and copies of the non-zero elements, respectively. For each group of BRAMs in the conflict-free input vector cache, two 4-to-1 multiplexers are used to read two non-zero elements or copies of non-zero elements from the four BRAMs in each group of BRAMs. The ports of each group of BRAMs operate in a time-division multiplexing manner.

4. The apparatus according to claim 1, characterized in that... The conflict-free adder tree is also used for: The multiplication result and 0 are stored in a register; The multiplication result is sent to four 2-to-1 multiplexers, and the other inputs of the multiplexers are 0. The read results of the four 2-to-1 multiplexers are added together by two 10-stage pipelined adders, and the sum of the two 10-stage pipelined adders is stored in a register. The sum of the two 10-stage pipelined adders is added to another 10-stage pipelined adder, and the sum of the other 10-stage pipelined adder is stored in a register. By using an 8-to-4 cross array switch to select the stored data of four registers as the partial sum output, write-after-read conflicts caused by adding the multiplication results with the same row number are eliminated.

5. The apparatus according to claim 1 or 2, characterized in that... The accumulator group supporting ping-pong is also used for: After each batch of data in the target matrix is ​​calculated, the accumulated result is written into the storage portion and the cache of the result vector.

6. The apparatus according to claim 5, characterized in that... The accumulator group supporting ping-pong is also used for: The data for each batch is calculated based on the individual accumulators in the accumulator group; The switching between batches is achieved by adding two additional registers to the independent accumulator, thereby reducing the overhead of loading and storing the hidden partial sums.

Images