Sparse matrix-dense matrix multiplication acceleration system and method

Abstract

The invention relates to the technical field of matrix multiplication, in particular to a sparse matrix-dense matrix multiplication acceleration system and method. The system comprises a sparse structure analyzer, a structure weighting scheduler, a plurality of PE rows and a runtime feedback controller. The sparse structure analyzer extracts features of each row of the sparse matrix and calculates a structure cost score; the structure weighting scheduler is combined with the PE row real-time load to distribute the PE row with the minimum comprehensive cost for the matrix row; executing scalar-vector multiply-accumulate by the PE array in parallel; and during operation, the feedback controller generates a blacklist bitmap and a splitting signal through performance monitoring, and dynamically adjusts task allocation. Parallel data reading and load balancing processing and calculation are carried out through a three-stage assembly line. According to the method, the calculation efficiency and the energy efficiency ratio of sparse-dense matrix multiplication are effectively improved, and then the data processing speed of the CPU / GPU is improved.

Description

Technical Field

[0001] This invention relates to the field of matrix multiplication technology, and in particular to a sparse matrix-dense matrix multiplication acceleration system and method. Background Technology

[0002] Sparse-dense matrix multiplication (SpMM) is one of the core operations in scientific computing, machine learning, neural networks, graph analysis, and other fields. Its computational efficiency directly affects the overall performance of various large-scale data processing tasks. With the explosive growth of data scale, traditional general-purpose processors (CPU / GPU) face problems such as high storage overhead, uneven load, and poor energy efficiency when processing large-scale sparse matrices. Therefore, dedicated hardware acceleration architectures have become a research hotspot.

[0003] In current general acceleration solutions based on CPU / GPU, the Processing Element (PE) is the core carrier for realizing parallel computing. Its usage logic is closely based on the thread parallelism of the hardware architecture. In particular, GPUs have become the mainstream choice due to their large-scale thread parallelism. The core acceleration logic revolves around the pipeline operation of multi-threaded collaboration. In CPU architecture, PE rows are typically mapped to physical cores of a multi-core CPU or fixed thread groups. Each PE row is assigned tasks by the CPU scheduler according to row number order, and sparse matrix row data within the allocated range is read from shared memory to prepare for computation. In GPU architecture, PE rows directly correspond to thread blocks or warps. Each thread block is a PE row, and its usage follows a four-step pipeline: First, data loading is completed by threads, loading the numerical array (val), column index array (col_idx), and row pointer array (row_ptr) of the CSR format sparse matrix together with the dense matrix into global memory. Next, row allocation is performed, with CPU multi-threads or GPU thread blocks dividing tasks by row, ensuring that each thread within a PE row focuses on processing one row of data, achieving initial task parallelism. Then, the computation phase begins, with threads within the PE row traversing each row of non-zero elements, accurately locating the corresponding column data in the dense matrix using column indices, and efficiently performing multiplication and addition operations. Finally, the results are accumulated, and part of the computation results are written to global memory. At this point, the GPU avoids multi-threaded write conflicts through atomic operations. Whether using CPU or GPU, the core logic of the PE row in existing solutions is "static task allocation by row," meaning each PE row is responsible for a fixed range of sparse matrix rows, independently performing multiplication and addition calculations between the allocated row and the dense matrix, and finally obtaining the complete product matrix by summing the results of all PE rows. While this parallel design can significantly reduce the computational pressure on a single thread and improve computational efficiency by fully utilizing the hardware's multi-threaded resources, it is not a perfect solution.

[0004] However, existing acceleration methods have significant inherent flaws, the core problem being load imbalance. Because the distribution of non-zero elements in sparse matrices is typically irregular, with significant differences in the number of non-zero elements across different rows, CPU / GPU threads assigned tasks by row face drastically different computational loads: threads processing rows with fewer non-zero elements complete their tasks quickly, while threads processing rows with more require more time, ultimately forcing faster threads to wait for slower threads, creating a computational bottleneck. This load imbalance not only leads to resource constraints but also directly increases overall latency and degrades computational performance. Furthermore, in scenarios with extremely uneven distribution of non-zero elements (such as some rows having only a few non-zero elements), traditional row-parallel schemes leave a large number of computing units in the CPU's multi-core architecture or the GPU's stream processor array idle, failing to fully utilize hardware computing resources and resulting in severe resource waste. This leads to excessively high computational latency in matrix multiplication calculations, significantly limiting the CPU / GPU's data processing efficiency. Summary of the Invention

[0005] Therefore, the technical problem to be solved by the present invention is to overcome the shortcomings of existing sparse matrix-dense matrix multiplication acceleration methods, which suffer from unbalanced load due to the unbalanced distribution of non-zero elements in the sparse matrix, resulting in increased latency in sparse matrix-dense matrix multiplication operations and reduced data processing efficiency of CPU / GPU.

[0006] To address the aforementioned technical problems, this invention provides a sparse matrix-dense matrix multiplication acceleration system, comprising: The sparse structure parser, connected to the on-chip storage unit of the sparse matrix, receives the sparse matrix to be computed. Based on the column index vector composed of the column indices of all non-zero elements in each row of the sparse matrix, it obtains the number of non-zero elements, the maximum value of the column index, the minimum value of the column index, and the total number of column index overlaps between the row and the previous row of the sparse matrix. It then calculates the structure cost score of each row of the sparse matrix and sends it to the structure weighted scheduler. Runtime feedback controller, used to collect the current load of each PE row in real time; The structure-weighted scheduler, connected to the sparse structure resolver and runtime feedback controller, receives the structure cost score of each matrix row in the sparse matrix and the current load of each PE row. It divides all matrix rows in the sparse matrix into multiple batches. For each batch, it weights and sums the structure cost score of each matrix row and the current load of each PE row in the current batch to obtain the comprehensive cost of each matrix row for each PE row. It then selects the PE row with the minimum comprehensive cost for each matrix row in the current batch as the target allocation row. After each batch is allocated, the CSR decoder sends all matrix rows in that batch to the corresponding target allocation row. Multiple PE rows, each PE row is connected to the structure-weighted scheduler, the dense matrix on-chip storage unit and the CSR decoder respectively, and is used to receive the dense matrix to be calculated and the matrix rows of the current batch assigned to the PE row output by the CSR decoder; In this process, for each matrix row assigned to its current batch, each non-zero element of the PE row is multiplied by all elements in the dense matrix that have the same column index as the non-zero element, and the results are summed to obtain a temporary row vector for that matrix row. After all PE rows have obtained the temporary row vectors for all matrix rows in their current batch, the structure-weighted scheduler begins the allocation for the next batch. When all PE rows have obtained the temporary row vectors for all matrix rows in their batches, each PE row sums the elements in the same column of the temporary row vectors for all its matrix rows to obtain the row vector corresponding to that PE row, which is then used as the row corresponding to that PE row in the resulting matrix.

[0007] Preferably, the structural cost score of each matrix row in the sparse matrix is ​​calculated based on the column index vector composed of the column indices of all non-zero elements in each matrix row, obtaining the number of non-zero elements, the maximum value of the column index, the minimum value of the column index, and the total number of column index overlaps between the current matrix row and the previous matrix row. The calculation formula is as follows: , in, The sparse matrix is ​​the th Structural cost score for each matrix row The first pre-configured weighting coefficient, The sparse matrix is ​​the th The number of non-zero elements in each row of a matrix For the second pre-configured weighting coefficient, The sparse matrix is ​​the th Maximum value of the row and column indices of a matrix The sparse matrix is ​​the th Minimum row and column indices of a matrix For the third pre-configured weighting coefficient, The sparse matrix is ​​the th The nth matrix row and the nth matrix row The total number of overlapping column indices of each matrix row.

[0008] Preferably, 10 PE rows are used, and each PE row includes 10 PE units.

[0009] Preferably, when using 10 PE rows, each PE row including 10 PE units, let .

[0010] Preferably, the sparse structure parser includes: A finite state machine is used to complete the state of a sparse matrix in one clock cycle. The parsing of an element in the column index vector of the matrix rows, and in After the nth clock cycle, the nth sparse matrix in the output The structural cost scores for each matrix row; where... For the row index of the sparse matrix, The total number of rows in the sparse matrix. The sparse matrix is ​​the th The number of non-zero elements in each row of the matrix; The in-row register set, which is connected to the finite state machine, is used to store the column index vector of the currently parsed matrix row; The structural cost cache (SRAM), connected to a finite state machine, stores the structural cost score for each row of the sparse matrix.

[0011] Preferably, the process of obtaining the total number of column indices overlapping between each row of the sparse matrix and the previous row includes: For the first row of the sparse matrix, its column index vector is stored in the row register group, and the total number of column index overlaps between this row and the previous row is set to 0. For the sparse matrix, the i-th The nth matrix row, its column index vector and the nth matrix row stored in the in-row register group are... The column index vectors of the nth row of the matrix are compared element by element, and the number of identical elements is counted, which is used as the nth column index vector in the sparse matrix. The nth matrix row and the nth matrix row The total number of overlapping column indices in each matrix row; where, , The total number of rows in the sparse matrix; After the statistics are completed, the sparse matrix is ​​used as the first... The column index vector of each matrix row updates the stored data in the in-row register group.

[0012] Preferably, the comprehensive cost of each matrix row for each PE row in the current batch is obtained by weighted summing of the structural cost score of each matrix row and the current load of each PE row in the current batch, as shown in the formula: , in, The first in the current batch For the nth matrix row, The overall cost of a PE line These are the weighting coefficients. For the first Current load of each PE row The first in the current batch The structural cost score for each matrix row.

[0013] Preferably, the runtime feedback controller further includes: The performance monitoring unit includes a performance counter mounted at the end of each PE row, used to collect the number of clock cycles consumed by each PE row in calculating the temporary row vectors of all matrix rows in the current batch, and the number of cache hits by each PE row in the current batch calculation; based on the number of clock cycles consumed by all PE rows in calculating the temporary row vectors of all matrix rows in the current batch, the average number of clock cycles of all PE rows in the current batch is obtained by a moving average method; the average number of clock cycles of all PE rows in the current batch is multiplied by a configurable fixed-point coefficient to obtain the clock cycle threshold of the current batch; for each PE row, it is determined whether the number of clock cycles consumed by it in calculating the temporary row vectors of all matrix rows in the current batch is greater than the clock cycle threshold of the current batch. If it is greater, the PE row is added to the blacklist unit graph of the next batch, and the generated blacklist unit graph of the next batch is sent to the structure weighted scheduler, so that the structure weighted scheduler skips all PE rows in the blacklist unit graph of the next batch when allocating the next batch; The feedback control unit, connected to the CSR decoder, is used to obtain all matrix rows in the current batch stored in the CSR decoder, determine whether the number of non-zero elements in each matrix row in the current batch is greater than the non-zero element number threshold, if it is greater, then determine whether the structural cost score of each matrix row in the current batch is greater than the set threshold, if it is greater, then output a split trigger signal to the CSR decoder, controlling the CSR decoder to split the matrix row into two segments and assign them to different PE rows; where the non-zero element number threshold is the product of the average number of non-zero elements in all matrix rows in the sparse matrix and a set multiple.

[0014] Preferably, the structure-weighted scheduler adopts a hardware min-heap structure. When the runtime feedback controller updates the current load of the PE row or the blacklisted unit graph, it incrementally updates the comprehensive cost of each matrix row to each PE row in the current batch. Within each clock cycle, for each matrix row in the current batch, select the PE row with the minimum overall cost.

[0015] This invention also provides a method for accelerating sparse matrix-dense matrix multiplication, applied to the aforementioned sparse matrix-dense matrix multiplication acceleration system, comprising: Using a sparse structure parser, the sparse matrix to be computed is received. Based on the column index vector composed of the column indices of all non-zero elements in each matrix row of the sparse matrix, the number of non-zero elements, the maximum value of the column index, the minimum value of the column index, and the total number of column index overlaps between the matrix row and the previous matrix row are obtained. The structure cost score of each matrix row in the sparse matrix is ​​calculated and sent to the structure weighted scheduler. The runtime feedback controller is used to collect the current load of each PE row in real time; Using a structure-weighted scheduler, all matrix rows in the sparse matrix are divided into multiple batches. For each batch, the structure cost score of each matrix row in the current batch and the current load of each PE row are weighted and summed to obtain the comprehensive cost of each matrix row in the current batch for each PE row. The PE row with the minimum comprehensive cost for each matrix row in the current batch is selected as the target allocation row. After the allocation of each batch is completed, all matrix rows in the batch are sent to the corresponding target allocation row through the CSR decoder. Using each PE row that is connected to the structure weighted scheduler, the dense matrix on-chip storage unit, and the CSR decoder respectively, the dense matrix to be calculated and the matrix rows of the current batch output by the CSR decoder and assigned to that PE row are received. In this process, for each matrix row assigned to its current batch, each non-zero element of the PE row is multiplied by all elements in the dense matrix that have the same column index as the non-zero element, and the results are summed to obtain a temporary row vector for that matrix row. After all PE rows have obtained the temporary row vectors for all matrix rows in their current batch, the structure-weighted scheduler begins the allocation for the next batch. When all PE rows have obtained the temporary row vectors for all matrix rows in their batches, each PE row sums the elements in the same column of the temporary row vectors for all its matrix rows to obtain the row vector corresponding to that PE row, which is then used as the row corresponding to that PE row in the resulting matrix.

[0016] Compared with the prior art, the above-described technical solution of the present invention has the following advantages:

[0017] The sparse matrix-dense matrix multiplication acceleration system and method described in this invention incorporates a greedy load balancing algorithm, which uses only the number of non-zero elements in each row of the sparse matrix as the basis for load allocation. This can alleviate some of the load pressure caused by traditional static row-by-row allocation, providing initial flexibility in task allocation. However, in actual CPU / GPU computing scenarios, it ignores the impact of Double Data Rate (DDR) transmission latency on data read efficiency, leading to decreased CPU cache hit rate or insufficient GPU memory bandwidth utilization. Furthermore, it does not consider the interference of PE inter-row data waiting latency on the overall computation rhythm and cannot adapt to real-time load changes in CPU cores / GPU stream processors. In scenarios with extremely uneven distribution of non-zero elements, this can exacerbate CPU thread idleness or GPU thread cluster fragmentation.

[0018] To address this, the present invention first uses a sparse structure parser to interface with the on-chip storage unit of the sparse matrix. This not only obtains the number of non-zero elements in each row but also simultaneously extracts the maximum and minimum values ​​of the column index vectors and the total number of column index overlaps with the previous row. Then, a multi-dimensional structure cost score is calculated using a formula containing three pre-configured weight coefficients, comprehensively quantifying the computational load, data transmission correlation, and overlap characteristics of each row. Next, a structure-weighted scheduler divides the matrix rows into multiple batches. Through a "batch dynamic allocation" mechanism, when allocating each batch of matrix rows, the structure-weighted scheduler combines the current real-time load of each PE row to calculate the comprehensive cost of each matrix row corresponding to different PE rows, selecting the PE row with the lowest cost for allocation. This allows task allocation to adapt to the load changes of each PE row in real time, avoiding the imbalance problem of some PE rows being overloaded and others being idle. Finally, pipelined batch processing reduces waiting overhead. After each batch of matrix rows is calculated, the next batch allocation is started immediately, achieving pipeline continuity. After all batches are completed, the results are summarized. The entire process retains the flexibility of the greedy algorithm while making up for the shortcomings of a single greedy strategy through multi-dimensional cost evaluation and real-time load adaptation, effectively improving the computational efficiency of sparse matrix-dense matrix multiplication, thereby improving the resource utilization and data processing speed of CPU / GPU. Attached Figure Description

[0019] To make the content of this invention easier to understand, the invention will be further described in detail below with reference to specific embodiments and accompanying drawings, wherein: Figure 1 This is a diagram illustrating uneven load distribution in CPU / GPU parallel processing.

[0020] Figure 2 This is a structural diagram of a sparse matrix-dense matrix multiplication acceleration system according to the present invention.

[0021] Figure 3These are schematic diagrams illustrating different matrix multiplication calculation methods. Figure 3 (a) in the diagram represents the inner product method. Figure 3 (b) in the diagram represents the outer product method. Figure 3 (c) in the diagram represents the block method.

[0022] Figure 4 This is a schematic diagram of a PE array.

[0023] Figure 5 It is a sparse adaptive sensing dynamic load balancing flow graph.

[0024] Figure 6 This is a flowchart of sparse matrix to dense matrix calculation.

[0025] Figure 7 This is a flowchart of the PE array calculation process.

[0026] Figure 8 This is a schematic diagram of a three-stage production line. Detailed Implementation

[0027] The present invention will be further described below with reference to the accompanying drawings and specific embodiments, so that those skilled in the art can better understand and implement the present invention. However, the embodiments described are not intended to limit the present invention.

[0028] like Figure 1 As shown, Figure 1 This diagram illustrates the uneven load distribution between CPU and GPU in parallel operation.

[0029] In existing general-purpose accelerated sparse matrix-dense matrix multiplication (SPMM) computation schemes based on CPU / GPU, the operation logic for processing cell arrays (PE rows) is as follows: First, in the data preparation phase, relevant threads load the val (non-zero element values), col_idx (column index), and row_ptr (row pointer) of the sparse matrix in CSR format, as well as the data of the dense matrix. This data is transferred to memory space easily accessible to PE rows, providing the data foundation for subsequent calculations. Next, in the task allocation phase, whether using CPU multithreading or GPU thread blocks, computational tasks are assigned to PE rows on a row-by-row basis; each PE row is assigned to process a specific row in the sparse matrix. During computation, each PE row responsible for a particular row iterates through the corresponding non-zero elements of that row. It uses the col_idx index to retrieve the corresponding column data in the dense matrix, then performs multiplication and addition operations, gradually obtaining a partial result after multiplying that row with the dense matrix. Finally, in the result processing phase, each PE row needs to write a portion of the calculated result to global memory. Since multiple PE rows may simultaneously write to the same area of ​​global memory, in a GPU environment, atomic operations are often necessary to avoid data write conflicts and ensure the correctness of the results.

[0030] However, this task allocation mode, which assigns each row to a single PE row, can lead to an imbalance in the computational load of different PE rows due to the difference in the number of non-zero elements in different rows of the sparse matrix. This means that after the PE row with fewer non-zero elements completes its task quickly, it has to wait for the PE row with more non-zero elements to complete its task. When threads process rows with different numbers of non-zero elements, fast threads have to wait for slow threads, resulting in idle and wasted computing resources.

[0031] To solve the above problems, refer to Figure 2 In this embodiment, a heterogeneous computing platform is built on the Xilinx Zynq chip, including the PS end and the PL end. The PS end mainly refers to the environment provided by Arm in Zynq, which can run Linux or perform bare-metal SDK development. The PL end is the FPGA part in Zynq. The PL end includes a RISC-V soft core and the sparse matrix-dense matrix multiplication acceleration system of the present invention.

[0032] This embodiment provides a sparse matrix-dense matrix multiplication acceleration system, including: The sparse structure parser (SP), connected to the sparse matrix on-chip storage unit (CSR_Buffer), receives the sparse matrix to be computed, which is stored in CSR compressed format; it is based on a column index vector consisting of the column indices of all non-zero elements in each row of the sparse matrix. To obtain the number of non-zero elements in each row of a sparse matrix and the maximum column index. Minimum value of column index And the total number of column indices that overlap between the row of this matrix and the row of the previous matrix. Calculate the structural cost score for each row of the sparse matrix. And send it to the structure-weighted scheduler; In this embodiment, specifically, the sparse structure parser includes: A finite state machine is used to complete the state of a sparse matrix in one clock cycle. The parsing of an element in the column index vector of the matrix rows, and in After the nth clock cycle, the nth sparse matrix in the output The structural cost scores for each matrix row; where... , Here, represents the row index of the sparse matrix, and N is the total number of rows in the sparse matrix. The sparse matrix is ​​the th The number of non-zero elements in each row of the matrix; The in-row register set, which is connected to the finite state machine, is used to store the column index vector of the currently parsed matrix row; The structure cost cache (SRAM), connected to a finite state machine, stores the structure cost score for each row of the sparse matrix for subsequent reading by the structure-weighted scheduler.

[0033] Before matrix computation begins, the sparse structure parser uses a hardware finite state machine to scan the column index vector of the sparse matrix in CSR format in parallel. And for each row of the matrix Get the minimum value of the column index Maximum value of column index and the column index overlap count with adjacent matrix rows A hardware finite state machine needs to complete the calculation of an element in the column index vector of the i-th row of a sparse matrix within one clock cycle. The analysis, and in Output the corresponding matrix row after one clock cycle. of Furthermore, the parsing process is executed in parallel with the CSR decoder, without adding any additional bus read latency.

[0034] in, Represents the th element in a sparse matrix. The index of an element in the column index vector of the matrix rows. Represents matrix rows The starting index of the first non-zero element in the val array (which stores the values ​​of non-zero elements) and the col_idx array (which stores the column indices of non-zero elements). Represents matrix rows The index of the position following the last non-zero element, therefore, Equal to the th in the sparse matrix Number of non-zero elements in each matrix row .

[0035] Finite state machines, inline register sets, and structural cost caches (SRAMs) are efficient implementation solutions designed for hardware-accelerated scenarios (especially suitable for the parallel logic characteristics of FPGAs, capable of parsing one element per clock cycle and executing in parallel with the CSR decoder to reduce latency), but they are not the only forms of implementing sparse structure parsers. Any combination of modules that can perform structural feature extraction and cost calculation can be used as an implementation method for sparse structure parsers, depending on the system's performance goals, hardware resources, and integration environment.

[0036] Runtime Feedback Controller (PM-FC) is used to collect the current load of each PE row in real time. ,in, For the first Current load of each PE row; The Structure Weighted Scheduler (SS), connected to the sparse structure parser and runtime feedback controller, receives the structure cost score for each row of the sparse matrix. Current load of each PE row The sparse matrix is ​​divided into multiple batches. For each batch, the structural cost score of each matrix row in the current batch and the current load of each PE row are weighted and summed to obtain the comprehensive cost of each matrix row in the current batch for each PE row. The PE row with the minimum comprehensive cost for each matrix row in the current batch is selected as the target allocation row. After the allocation of each batch is completed, all matrix rows in the batch are sent to the corresponding target allocation row through the CSR decoder. In this embodiment, preferably, the formula for obtaining the comprehensive cost of each matrix row for each PE row in the current batch by weighted summing of the structural cost score of each matrix row and the current load of each PE row in the current batch is as follows: , in, The first in the current batch For the nth matrix row, The overall cost of a PE line These are the weighting coefficients. For the first Current load of each PE row The first in the current batch The structural cost score for each matrix row.

[0037] In this embodiment, preferably, This approach avoids the hardware overhead of decimal operations and enhances the flexibility of the scheduling strategy through reasonable tier division, perfectly adapting to the hardware acceleration scenario of this system and ensuring that the overall cost calculation is efficient and controllable.

[0038] In this embodiment, specifically, the Structure-Weighted Scheduler (SS) adopts a hardware min-heap structure. During runtime, the feedback controller updates the current load of the PE row. When using a blacklisted unit graph, the comprehensive cost of each matrix row to each PE row in the current batch is updated incrementally to ensure that the scheduling results are consistent in real time. Within each clock cycle, for each matrix row in the current batch, select the row with the minimum total cost (PE). and the index of the row of the matrix Address of the allocated SRAM The data is located at [location]. , For the first The PE row with the minimum total cost corresponding to each matrix row; where the minimum heap depth is equal to the number of PE array rows. Insertion and update operations are performed by a parallel comparator-switch network, with a single latency not exceeding [a certain value]. One clock cycle, Indicates the number of rows in the PE array; Multiple PE rows are connected to a structure-weighted scheduler, a dense matrix on-chip storage unit, and a CSR decoder. Each PE row receives the dense matrix to be computed and the matrix rows of the current batch assigned to it from the CSR decoder. For each matrix row assigned to a PE row in the current batch, each non-zero element of that row is multiplied by all elements in the dense matrix with the same column index as that non-zero element, and the results are summed to obtain a temporary row vector for each matrix row in the current batch. To ensure computational correctness, a synchronization mechanism is introduced. After all PE rows have completed the calculation of the temporary row vectors for the current batch, each PE row sends a batch completion signal to the structure-weighted scheduler, instructing it to begin the allocation of the next batch. After obtaining the temporary row vectors for all matrix rows in all batches, each element in the same column of the temporary row vectors of all matrix rows assigned to each PE row is summed to obtain the row vector for each PE row, which serves as the row corresponding to that PE row in the resulting matrix.

[0039] Each PE row needs to store a list of matrix row indices. During computation, vector elements need to be broadcast to all PE rows, while the non-zero elements of the sparse matrix are stored row-by-row in each PE row. To ensure computational correctness, a synchronization mechanism is introduced, waiting for all PE rows to complete the current stage of computation before proceeding to the next step. This allocation method fully utilizes the parallelism of the PE array while avoiding data conflicts.

[0040] The calculation process for the PE line mentioned above can be further explained through a specific flowchart, such as... Figure 6 , 7 As shown, Figure 6 This is a flowchart of sparse matrix to dense matrix calculation. Figure 7 This is a flowchart of the PE array calculation process.

[0041] ① Initialize sparse matrix A, dense matrix B, and result matrix C; where the total number of rows in the sparse matrix is ​​M, and the result matrix C is the calculation result of sparse matrix A and dense matrix B; ② Traverse the val array of the sparse matrix (the array of non-zero element values ​​of sparse matrix A). For the first row PEs of the PE array, when the index e of val is equal to prtr[k] (initially k=0, i.e. the position of the first non-zero element in the 0th row of A), extract the data of the corresponding column from the dense matrix B according to cid[e] (the column index of the element); where prtr[k] represents the starting index of the first non-zero element in the kth row of A in the val / cid array; ③ Multiply and accumulate the scalar val[e] with the vector cid[e] in B; ④ Determine if the current e is equal to prtr[t+1]. If they are not equal, set e = e+1 and continue executing steps ② to ④; otherwise, execute step ⑤. ⑤ Store the accumulated and merged data in row C[k]; ⑥ Determine whether the current k is equal to M or whether k+a is greater than M (M refers to the number of rows in the matrix, and a refers to the number of rows in the PE array). If the condition is not met, let k=k+a and continue with steps ②~⑥. Otherwise, it means that the current PE has completed the calculation for this matrix, and output the result matrix C.

[0042] By employing a sparse adaptive awareness-based dynamic load balancing mechanism, the computational imbalance problem of SPMM on a fixed PE array can be effectively solved. This solution balances algorithm efficiency and hardware adaptability, making it suitable for efficient parallel computation of sparse matrices. In the future, real-time load monitoring can be combined to dynamically adjust task allocation to adapt to more complex application scenarios.

[0043] In this embodiment, preferably, the structural cost score of each matrix row in the sparse matrix is ​​calculated based on the column index vector composed of the column indices of all non-zero elements in each matrix row, the number of non-zero elements, the maximum value of the column index, the minimum value of the column index, and the total number of column index overlaps between the current matrix row and the previous matrix row. The calculation formula is as follows: , in, The sparse matrix is ​​the th Structural cost score for each matrix row The first pre-configured weighting coefficient, The sparse matrix is ​​the th The number of non-zero elements in each row of a matrix For the second pre-configured weighting coefficient, The sparse matrix is ​​the th Maximum value of the row and column indices of a matrix The sparse matrix is ​​the th Minimum row and column indices of a matrix For the third pre-configured weighting coefficient, The sparse matrix is ​​the th The nth matrix row and the nth matrix row The total number of overlapping column indices of each matrix row.

[0044] In this embodiment, specifically, the process of obtaining the total number of column indices overlapping between each row of the sparse matrix and the previous row includes: For the first row of the sparse matrix, its column index vector is stored in the row register group, and the total number of column index overlaps between this row and the previous row is set to 0. For the sparse matrix, the i-th The nth matrix row, its column index vector and the nth matrix row stored in the in-row register group are... The column index vectors of the nth row of the matrix are compared element by element, and the number of identical elements is counted, which is used as the nth column index vector in the sparse matrix. The nth matrix row and the nth matrix row Total number of column index overlaps in matrix rows ;in, , The total number of rows in the sparse matrix; After the statistics are completed, the sparse matrix is ​​used as the first... The column index vector of each matrix row updates the stored data in the in-row register group for use by the next matrix row.

[0045] In this embodiment, preferably, the runtime feedback controller further includes: The performance monitoring unit (PM) includes a performance counter attached to the end of each PE row, used to collect the number of clock cycles consumed by each PE row in calculating the temporary row vectors of all matrix rows in the current batch. That is, the number of completion cycles for each PE row and the number of cache hits for each PE row in the current batch calculation. ; Based on all PE rows, calculate the number of clock cycles consumed by the temporary row vectors of all matrix rows in the current batch, and then obtain the average number of clock cycles for all PE rows in the current batch using a moving average method. ; Multiply the average number of clock cycles for all PE rows in the current batch by the configurable fixed-point coefficient K to obtain the clock cycle threshold for the current batch. ; For each PE row Determine the number of clock cycles consumed in calculating the temporary row vectors of all matrix rows in the current batch. Is the clock cycle threshold avg_cycle×K greater than the current batch's clock cycle threshold? If it is, then remove the PE row. Add the blacklisted unit graph to the next batch, and send the blacklisted unit graph back to the structure-weighted scheduler every 64 rows through the AXI-Stream interface, so that the structure-weighted scheduler skips all PE rows in the blacklisted unit graph of the next batch when allocating. The blacklist unit map is stored in a 10-bit blacklist register. Each bit of the register is released by a 1 ms automatic decay timer or is forcibly cleared by software through the AXI-Lite interface to achieve dynamic unlocking and rescheduling.

[0046] The feedback control unit (FC), connected to the CSR decoder, is used to obtain all matrix rows in the current batch stored in the CSR decoder; The product of the average number of non-zero elements in all rows of the sparse matrix and a set multiple is used as the threshold for the number of non-zero elements; in this embodiment, the set multiple is 2. Determine whether the number of non-zero elements in each matrix row of the current batch is greater than the threshold for the number of non-zero elements. If the average NNZ×2 is greater than 2, then determine the structural cost score for each matrix row in the current batch. If the value exceeds a set threshold, a split trigger signal is output to the CSR decoder to control the CSR decoder to split the matrix row into two segments and assign them to different PE rows.

[0047] The runtime feedback controller is used to collect the number of completed cycles of each PE row in real time during the execution of sparse matrix multiplication. After comparison by moving average, a temporary blacklist unit graph is generated, so that the next round of task allocation will automatically skip the current slowest node, thereby removing system-level long-tail latency and achieving closed-loop load balancing.

[0048] In this embodiment, to further improve the parallel computing efficiency and hardware implementation adaptability of sparse matrix-dense matrix multiplication, a preferred hardware architecture design employs 10 PE rows, with each PE row containing 10 PE units, forming a 10×10 PE array structure. This array size not only improves task throughput through parallel processing of multiple PE rows but also efficiently completes the multiplication and addition operations of a single matrix row and a dense matrix column through the collaborative operation of 10 PE units per row, balancing computational resource utilization and hardware routing complexity.

[0049] Meanwhile, in the calculation of structural cost fraction, for The value selection was designed to be hardware-friendly, making... It can convert the calculation result of the structural cost score into integer operations, avoiding the precision loss and additional logic overhead that fixed-point decimal operations may introduce in hardware implementation. This simplifies hardware design, reduces latency, and ensures that the sparse structure parser can efficiently complete the parsing of column index elements and the real-time calculation of cost scores within a single clock cycle, matching the parallel scheduling rhythm of the entire system.

[0050] This invention is implemented in four main steps. First, a hardware pre-analysis module—a sparse structure parser—is set up to quickly extract the distribution characteristics of each row and column index before sparse matrix calculation, generating a structure cost score. This provides a structure-aware basis for subsequent load balancing scheduling, avoiding cache failures and DDR bandwidth waste caused by allocating only based on the number of non-zero elements. Second, a hardware min-heap engine—a structure-weighted scheduler—is adapted to dynamically select the PE row with the lowest overall cost and output an allocation index table during each matrix row allocation, thus achieving dual balancing of structure awareness and load awareness. Finally, a hardware closed-loop negative feedback unit—a runtime feedback controller—is implemented to collect the number of completion cycles and cache hit rate of each PE row in real time during sparse matrix multiplication. After comparing the moving average with a threshold, a blacklist and a fragmentation trigger signal are generated and sent back to the structure-weighted scheduler, so that the next round of task allocation automatically avoids slow or overloaded nodes, thereby continuously converging to the maximum array utilization.

[0051] For CPUs, the reduced cache misses by the sparse structure resolver decrease invalid data wait times for CPU cores, the load balancing by the structure-weighted scheduler avoids the overhead of frequent CPU thread switching, and the runtime feedback controller further reduces idle computing power, allowing the CPU to allocate more general-purpose computing power to the core tasks of the device. For GPUs, the structure-weighted scheduler adapts to the massive parallel computing units of GPUs, avoiding stream processor load imbalances caused by the irregularity of sparse matrices, and the runtime feedback controller's mechanism for avoiding slow nodes reduces GPU parallel computing wait times, effectively improving the utilization of GPU stream processors. In summary, this invention enables CPUs and GPUs to fully translate their hardware performance into actual processing efficiency in real-world applications that rely on sparse-dense matrix multiplication.

[0052] In this embodiment, the PS (Power Supply) is the core data source for the entire system. It is responsible for pre-storing the dense matrix B (stored in CSR compressed format) and the sparse matrix A (stored in CSR compressed format) in off-chip memory. The AXI_RAddr_Ctrl module controls the switching of the read address. After reading the sparse matrix A from off-chip memory, it stores it in CSR_Buffer and stores the dense matrix B in Buffer B, thus preparing the data for subsequent matrix operations.

[0053] The sparse matrix in CSR_Buffer is distributed in row-parallel using a sparse adaptive and aware dynamic load balancing mechanism, and the final result is obtained in the SPMM module. The configuration parameters of each module are uniformly distributed and configured by Configuration_Table, and the scheduling of each module is controlled by RISC-V instructions.

[0054] like Figure 3 As shown, Figure 3 This diagram illustrates different matrix multiplication methods. Figure 3 (a) in the diagram represents the inner product method. Figure 3 (b) in the diagram represents the outer product method. Figure 3 (c) in the diagram represents the block-based method. Currently, there are various methods for calculating matrix multiplication, such as inner product, outer product, row product, column product, block-based method, etc., and some are combinations of several commonly used methods. Each has its advantages and disadvantages. Selecting a suitable calculation method and adopting a reasonable optimization strategy is the focus of this invention.

[0055] like Figure 4 As shown, Figure 4 for Figure 4This is a schematic diagram of the PE array. This invention uses the Gustavson algorithm (i.e., row product) to calculate the multiplication of sparse and dense matrices. The hardware design uses a 10*10 PE array, with 10 PE units per row, considered as a group of PEs. Dense matrix data is output from Buffer B above the PE array, and sparse matrices in CSR format are output from CSR_Decoder to the left of the PE array. Based on a row-parallel allocation strategy (explained in detail below), the array is reused to calculate matrix multiplication while ensuring the absence of data bubbles. Each group of PEs multiplies the scalar elements from CSR_Encoder with the vector from Buffer B and accumulates the results. The final accumulated result is output to the corresponding row of the result matrix C.

[0056] like Figure 5 As shown, Figure 5 This is a flow graph for sparse adaptive sensing dynamic load balancing. The goal of the sparse adaptive sensing dynamic load balancing mechanism algorithm is to dynamically adjust the mapping relationship between matrix rows and PE rows at runtime based on the distribution structure of non-zero elements in the sparse matrix and the real-time load status of computing units (PE rows), thereby minimizing the maximum execution time, improving the utilization of the PE array and the system energy efficiency ratio, and thus avoiding local overload.

[0057] This invention employs a sparse adaptive awareness dynamic load balancing mechanism to address the load imbalance problem caused by the unbalanced distribution of non-zero elements in sparse matrices. It uses a pipelined approach to overcome the latency of reading data from off-chip environments and utilizes the high-speed AXI bus protocol to read DDR data, further improving computational performance. It can perform multiplication of sparse and dense matrices with arbitrary data widths and sizes, resolving load imbalance issues while reducing resource overhead to meet performance requirements. This invention can be applied to various fields such as machine learning, neural networks, and graphics analysis, providing hardware acceleration for matrix calculations in these domains.

[0058] like Figure 8 As shown, Figure 8 This is a schematic diagram of a three-stage production line. Figure 8 As we can see, each matrix calculation involves three main modules. The first module retrieves the required matrix data from memory; the second module performs sparse adaptive and dynamic load balancing on the sparse matrix; and the third module performs the matrix calculation. If these three modules are executed sequentially, the computational efficiency will obviously be very low.

[0059] The three modules are designed as a three-stage pipeline based on the module with the longest execution time, as follows: Figure 8As shown, `fetch_data` is responsible for reading the sparse matrix (CSR format) and dense matrix data to be computed from memory, providing the data foundation for subsequent processing. `load_balance` performs sparse adaptive and aware dynamic load balancing processing on the sparse matrix, distributing matrix rows to different PE rows according to matrix structure characteristics (such as the distribution of non-zero elements and row structure cost) to avoid uneven load. `SPMM` performs the core calculation of sparse matrix-dense matrix multiplication, and based on the load balancing result, completes the multiplication and accumulation operation of non-zero elements with dense matrix columns to generate the result matrix.

[0060] In traditional serial execution, the next module cannot start until the previous module has finished completely. However, the three-stage pipeline allows the three modules to be executed in parallel and interleaved (for example, while the first set of data is executing "load_balance", the second set of data can start "fetch_data" at the same time, and the third set of data can prepare "SPMM"). By overlapping the time, the utilization of hardware resources is maximized, and the overall computing throughput is significantly improved.

[0061] This invention addresses the problems of unbalanced computational load, low resource utilization, and poor scalability in large-scale sparse-dense matrix multiplication (SpMM). It proposes a high-performance hardware acceleration scheme that integrates a sparse adaptive awareness dynamic load balancing mechanism, a row product multiplication strategy, and a three-stage hardware pipeline structure. Compared to traditional fixed mapping and static allocation designs, this scheme offers the following advantages: Improved load balancing: Through a sparse adaptive and aware dynamic load balancing mechanism, the row with the highest load is moved to the row with the lowest load in the current PE array, so that the difference in computational load between each PE row does not exceed the maximum NNZ value of a single row. This improves the utilization of computational resources by more than 40% compared to the traditional static allocation scheme (based on actual test data from the Xilinx Zynq-7000 platform).

[0062] Enhanced hardware versatility: Supports configurable bit width (8 / 16 / 32 bits) to adapt to different precision requirements, avoids resource waste caused by fixed bit width, and improves chip area utilization by 25% (compared to the fixed 16-bit design of TPU).

[0063] Latency masking and bandwidth optimization: The three-stage pipeline design overlaps the serial delays of data loading, task allocation, and computation. Combined with the burst transmission mode of the AXI bus, the DDR bandwidth utilization is increased to 85% (the measured value of the theoretical peak bandwidth).

[0064] Configurable bit width and pipelined design break through performance bottlenecks, supporting 8 / 16 / 32-bit configurable data bit width. Through the bit width adaptive module of CSR_Buffer, it is compatible with scenarios with different precision requirements (such as FP16 machine learning models and INT8 quantization inference), saving 20%-40% of storage bandwidth compared to fixed bit width architectures (such as TPU).

[0065] Dual optimization of hardware cost and energy efficiency: The use of RISC-V soft cores to unify scheduling configuration parameters reduces the area of ​​dedicated control logic and reduces FPGA resource consumption by 15%; the dynamic load distribution mechanism avoids over-designing the PE array size to cope with the worst case, reducing hardware cost by 25%; in the Xilinx Zynq-7020 test, the energy efficiency ratio for processing a 1M×1K sparse matrix (sparseness 1%) reached 5.8 TOPS / W, which is better than GPU solutions of the same scale (such as NVIDIA T4's 3.2 TOPS / W).

[0066] Advantages in versatility and scalability: The PE array's working mode can be dynamically adjusted through Configuration_Table (such as block computation, row / column priority), adapting to various sparse modes (such as Block-Sparse, Diagonal-Sparse), and supporting emerging algorithm requirements without refactoring the hardware.

[0067] Based on Example 1, this Example 2 builds an FPGA verification platform using a Xilinx Zynq XC7Z020 device and conducts verification in a bare-metal SDK environment. The sparse matrix has a dimension of 512×512 (non-zero density approximately 4%), and the dense matrix has a dimension of 512×64 (using floating-point to fixed-point conversion, 16-bit width). The DDR interface uses an AXI4 bus with a clock frequency of 150MHz. The specific steps are as follows:

[0068] The PS end writes the sparse matrix A and dense matrix B in CSR format to DDR via serial port; the PL end uses a RISC-V soft core to configure the parameters of each module and control the data transfer between CSR_Buffer and Buffer_B; after the sparse adaptive sensing dynamic load module completes the row and PE mapping, the PE array begins parallel computation; after the computation is completed, the results are written back to DDR, and the PS end reads and verifies that they are consistent with the software simulation. Performance comparison shows that compared with the unaccelerated implementation on the same platform, the runtime is reduced by approximately 65.3%; compared with the conventional static PE mapping design, the array utilization is improved by 27.6%; and the average energy efficiency is improved by 51.2% (in GOPS / W).

[0069] This third embodiment focuses on low-power edge scenarios. The hardware configuration uses a Zynq-7020 (PL-side resources include 53K LUTs and 160 DSPs). The PE array size is adjusted to 8×8, and the bit width is configured to 8 bits to reduce power consumption. The matrix parameters are set as sparse matrix A (5000×5000, sparsity 1%) and dense matrix B (5000×64). The specific steps are as follows:

[0070] The PS side stores A (CSR format) and B to DDR, and configures the accelerator registers (bit width = 8, number of PE rows = 8) through AXI-lite; the sparse adaptive awareness dynamic load balancing mechanism distributes 5000 rows to 8 PE rows, achieving a balanced effect with the maximum load row NNZ of 625 and the minimum of 610 (load difference <3%); the pipeline cycle distribution is data loading (200 cycles) → task allocation (50 cycles) → computation (1200 cycles), with a total latency of 1350 cycles, which is 90% lower than the CPU serial solution, adapting to the low power consumption and low latency requirements of edge scenarios.

[0071] This fourth embodiment addresses the scenario of accelerating computation in social network graph embedding, specifically for social network graphs with hundreds of millions of nodes (adjacency matrix sparsity of 0.1%). The aim is to accelerate the multiplication operation between the adjacency matrix and the feature matrix in graph neural networks (GNNs). The specific steps are as follows:

[0072] During the data preprocessing stage, the adjacency matrix is ​​converted to CSR format (val[] stores edge weights, col_idx[] stores target nodes, and row_ptr[] is sorted by node ID); in terms of hardware configuration, the PE array bit width is set to 16 bits (to match the FP16 precision of the feature matrix), and the maximum load of a single PE row is set to ≤ total NNZ / 10+100 (tolerance for the maximum number of edges in a single row) through Configuration_Table.

[0073] The execution flow is as follows: the PS-side Arm core loads the CSR data and feature matrix into DDR; the AXI_RAddr_Ctrl module reads the data into CSR_Buffer and Buffer B in 64B burst transmission; the RISC-V core triggers the sparse adaptive awareness dynamic load balancing module, and the feature aggregation of 10 sets of PE rows of parallel computing nodes; the result matrix is ​​written back to DDR through the AXI bus, and the total time is reduced by 42% compared with the GPU solution, meeting the high-efficiency acceleration requirements of large-scale graph computing.

[0074] Embodiment 5 of the present invention provides a method for accelerating sparse matrix-dense matrix multiplication, applied to the aforementioned sparse matrix-dense matrix multiplication acceleration system, comprising: Using a sparse structure parser, the sparse matrix to be computed is received. Based on the column index vector composed of the column indices of all non-zero elements in each matrix row of the sparse matrix, the number of non-zero elements, the maximum value of the column index, the minimum value of the column index, and the total number of column index overlaps between the matrix row and the previous matrix row are obtained. The structure cost score of each matrix row in the sparse matrix is ​​calculated and sent to the structure weighted scheduler. The runtime feedback controller is used to collect the current load of each PE row in real time; Using a structure-weighted scheduler, all matrix rows in the sparse matrix are divided into multiple batches. For each batch, the structure cost score of each matrix row in the current batch and the current load of each PE row are weighted and summed to obtain the comprehensive cost of each matrix row in the current batch for each PE row. The PE row with the minimum comprehensive cost for each matrix row in the current batch is selected as the target allocation row. After the allocation of each batch is completed, all matrix rows in the batch are sent to the corresponding target allocation row through the CSR decoder. Using each PE row that is connected to the structure weighted scheduler, the dense matrix on-chip storage unit, and the CSR decoder respectively, the dense matrix to be calculated and the matrix rows of the current batch assigned to the PE row output by the CSR decoder are received. In this process, for each matrix row assigned to its current batch, each non-zero element of the PE row is multiplied by all elements in the dense matrix that have the same column index as the non-zero element, and the results are summed to obtain a temporary row vector for that matrix row. After all PE rows have obtained the temporary row vectors for all matrix rows in their current batch, the structure-weighted scheduler begins the allocation for the next batch. When all PE rows have obtained the temporary row vectors for all matrix rows in their batches, each PE row sums the elements in the same column of the temporary row vectors for all its matrix rows to obtain the row vector corresponding to that PE row, which is then used as the row corresponding to that PE row in the resulting matrix.

[0075] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0076] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this application. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart... Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0077] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0078] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0079] Obviously, the above embodiments are merely illustrative examples for clear explanation and are not intended to limit the implementation. Those skilled in the art will recognize that other variations or modifications can be made based on the above description. It is neither necessary nor possible to exhaustively list all possible implementations here. However, obvious variations or modifications derived therefrom are still within the scope of protection of this invention.

Claims

1. A sparse-matrix-dense-matrix multiplication acceleration system, comprising: include: The sparse structure parser, connected to the on-chip storage unit of the sparse matrix, receives the sparse matrix to be computed. Based on the column index vector composed of the column indices of all non-zero elements in each row of the sparse matrix, it obtains the number of non-zero elements, the maximum value of the column index, the minimum value of the column index, and the total number of column index overlaps between the row and the previous row of the sparse matrix. It then calculates the structure cost score of each row of the sparse matrix and sends it to the structure weighted scheduler. Runtime feedback controller, used to collect the current load of each PE row in real time; The structure-weighted scheduler, connected to the sparse structure parser and runtime feedback controller, is used to receive the structure cost score of each matrix row in the sparse matrix and the current load of each PE row, and divide all matrix rows of the sparse matrix into multiple batches. For each batch, the structural cost score of each matrix row and the current load of each PE row in the current batch are weighted and summed to obtain the comprehensive cost of each matrix row for each PE row in the current batch. For each matrix row in the current batch, select the PE row with the minimum overall cost as the target allocation row; After each batch of allocation is completed, all matrix rows in that batch are sent to the corresponding target allocation row via the CSR decoder; Multiple PE rows are connected to a structure-weighted scheduler, a dense matrix on-chip storage unit, and a CSR decoder, respectively. Each PE row receives the dense matrix to be computed and the matrix rows of the current batch allocated to it from the CSR decoder. For each matrix row allocated to its current batch, each PE row multiplies each non-zero element by all elements in the dense matrix with the same column index as that non-zero element, and then sums the results to obtain a temporary row vector for that matrix row. After all PE rows have obtained the temporary row vectors for all matrix rows in their current batch, the structure-weighted scheduler begins the allocation for the next batch. Once all PE rows have obtained the temporary row vectors for all matrix rows in their batches, each PE row adds the elements in the same column of the temporary row vectors for all its matrix rows to obtain the row vector corresponding to that PE row, which is then used as the row in the resulting matrix corresponding to that PE row.

2. The sparse-dense matrix multiplication acceleration system of claim 1, wherein, The method involves obtaining the number of non-zero elements, the maximum value of the column index, the minimum value of the column index, and the total overlap of column indices between the current row and the previous row in the sparse matrix, based on the column index vector composed of the column indices of all non-zero elements in each row of the sparse matrix. The structural cost score for each row of the sparse matrix is ​​then calculated using the following formula: ， in, The sparse matrix is ​​the first Structural cost score for each matrix row The first pre-configured weighting coefficient, The sparse matrix is ​​the th The number of non-zero elements in each row of a matrix For the second pre-configured weighting coefficient, The sparse matrix is ​​the th Maximum value of the row and column indices of a matrix The sparse matrix is ​​the first Minimum row and column indices of a matrix For the third pre-configured weighting coefficient, The sparse matrix is ​​the th The nth matrix row and the nth matrix row The total number of overlapping column indices of each matrix row.

3. The sparse-dense matrix multiplication acceleration system of claim 2, wherein, Ten PE rows are used, and each PE row includes 10 PE cells.

4. The sparse-dense matrix multiplication acceleration system of claim 3, wherein, When 10 PE rows are employed, each PE row including 10 PE units, let .

5. The sparse-dense matrix multiplication acceleration system of claim 1, wherein, Sparse structure parsers include: A finite state machine is used to complete the state of a sparse matrix in one clock cycle. The parsing of an element in the column index vector of the matrix rows, and in After the nth clock cycle, the nth sparse matrix in the output The structural cost scores for each matrix row; where... For the row index of the sparse matrix, The total number of rows in the sparse matrix. The sparse matrix is ​​the th The number of non-zero elements in each row of the matrix; The in-row register set, which is connected to the finite state machine, is used to store the column index vector of the currently parsed matrix row; The structural cost cache (SRAM), connected to a finite state machine, stores the structural cost score for each row of the sparse matrix.

6. The sparse-dense matrix multiplication acceleration system of claim 5, wherein, The process of obtaining the total number of column indices overlapping between each row of a sparse matrix and the row of the previous matrix includes: For the first matrix row in the sparse matrix, store its column index vector into the row register bank, and set the column index overlap number of the matrix row with the last matrix row to 0; For the j-th matrix row in the sparse matrix, compare the column index vector thereof with the column index vector of the i-th matrix row stored in the row register bank, count the number of same elements, and take the number as the total number of column index overlaps between the j-th matrix row and the i-th matrix row in the sparse matrix; wherein, , , , , , is the total number of rows of the sparse matrix. After the statistics are completed, the storage data of the in-register group is updated with the column index vector of the matrix row in the sparse matrix number 1.

7. The sparse-dense matrix multiplication acceleration system of claim 1, wherein, The structure cost score of each matrix row in the current batch is weighted and summed to obtain the comprehensive cost of each matrix row to each PE row in the current batch, and the formula is: ， wherein, is the overall cost of the th matrix row for the th PE row, is the weight coefficient, is the current load of the th PE row, is the structure cost fraction of the th matrix row in the current batch.

8. The sparse-dense matrix multiplication acceleration system of claim 1, wherein, The runtime feedback controller further comprises: The performance monitoring unit comprises a performance counter mounted at the tail of each PE row, which is used to collect the number of clock cycles consumed by each PE row to calculate the temporary row vector of all matrix rows in the current batch, the number of cache hits accessed by each PE row in the current batch calculation; based on the number of clock cycles consumed by all PE rows to calculate the temporary row vector of all matrix rows in the current batch, the average number of clock cycles of all PE rows in the current batch is obtained by a sliding average method; the average number of clock cycles of all PE rows in the current batch is multiplied by a configurable fixed-point coefficient to obtain the clock cycle threshold of the current batch; for each PE row, it is judged whether the number of clock cycles consumed by the PE row to calculate the temporary row vector of all matrix rows in the current batch is greater than the clock cycle threshold of the current batch, if greater, the PE row is added to the blacklist unit graph of the next batch, and the generated blacklist unit graph of the next batch is sent to the structure weighted scheduler, so that the structure weighted scheduler skips all PE rows in the blacklist unit graph of the batch in the next batch allocation; The feedback control unit is connected with the CSR decoder, and is used to obtain all matrix rows in the current batch stored by the CSR decoder, judge whether the number of non-zero elements of each matrix row in the current batch is greater than the non-zero element number threshold, if greater, judge whether the structure cost score of each matrix row in the current batch is greater than a set threshold, if greater, output a splitting trigger signal to the CSR decoder to control the CSR decoder to split the matrix row into two segments and assign them to different PE rows; wherein the non-zero element number threshold is the product of the average number of non-zero elements of all matrix rows in the sparse matrix and a set multiple.

9. The sparse-dense matrix multiplication acceleration system of claim 8, wherein, The structure weighted scheduler adopts a hardware minimum heap structure, and when the runtime feedback controller updates the current load of the PE row or the blacklist unit graph, the comprehensive cost of each matrix row to each PE row in the current batch is updated by increment. In each clock cycle, for each matrix row in the current batch, the PE row with the minimum comprehensive cost corresponding to the matrix row is selected.

10. A sparse-matrix-dense-matrix multiplication acceleration method, characterized by, The application is applied to a sparse matrix-dense matrix multiplication acceleration system as claimed in any one of claims 1 to 9, and comprises: The sparse structure parser is used to receive a sparse matrix to be calculated, obtain the number of non-zero elements, the maximum column index and the minimum column index of each matrix row in the sparse matrix based on the column index vector composed of the column indexes of all non-zero elements in each matrix row, calculate the structure cost score of each matrix row in the sparse matrix, and send the structure cost score to the structure weighted scheduler; The runtime feedback controller is used to collect the current load of each PE row in real time. The structure weighted scheduler is utilized to divide all matrix rows in the sparse matrix into multiple batches; for each batch, the structure cost score of each matrix row in the current batch and the current load of each PE row are weighted and summed to obtain the comprehensive cost of each matrix row in the current batch for each PE row; the PE row with the minimum comprehensive cost is selected as the target allocation row for each matrix row in the current batch; after the allocation of each batch is completed, all matrix rows in the batch are sent to the corresponding target allocation row through the CSR decoder; The PE rows connected with the structure weighted scheduler, the on-chip storage unit of the dense matrix and the CSR decoder respectively are utilized to receive the dense matrix to be calculated and each matrix row in the current batch allocated to the PE row by the CSR decoder; Each PE row multiplies each non-zero element of each matrix row allocated to the current batch of the PE row with all elements in the dense matrix with the same column index as the non-zero element, and then accumulates to obtain a temporary row vector of the matrix row; after all PE rows obtain the temporary row vectors of all matrix rows in the current batch, the structure weighted scheduler starts the allocation of the next batch; when all PE rows obtain the temporary row vectors of all matrix rows in all batches, each PE row adds elements in the same column in the temporary row vectors of all matrix rows of the PE row to obtain a row vector corresponding to the PE row as a row in the result matrix corresponding to the PE row.

Images