Hardware accelerator, device and application method for sparse matrix vector multiplication

Abstract

This invention discloses a hardware accelerator, device, and application method for sparse matrix-vector multiplication. The hardware accelerator includes an off-chip storage system, an on-chip network for data exchange routing, and an on-chip processing system for performing data fetching, multiplication calculations, and matrix row element merging. The off-chip storage system includes HBM channels for storing five types of data: column indices, row indices, vector values, matrix values, and result vectors. Each HBM channel includes an HBM stack and a memory controller. The HBM channels for storing vector values ​​are connected to the on-chip network via a L2 cache, while the remaining HBM channels are directly connected to the on-chip processing system. This invention aims to improve on-chip data reuse in hardware accelerators for sparse matrix-vector multiplication, reduce off-chip memory accesses, and improve the performance and energy efficiency of the hardware accelerator.

Description

Technical Field

[0001] This invention relates to hardware accelerators in the field of high-performance computing and architecture, and more specifically to a hardware accelerator, device, and application method for sparse matrix-vector multiplication. Background Technology

[0002] In the field of high-performance computing (HPC), solving partial differential equations (PDEs) and their discretized sparse linear equation systems is a core computational task in many scientific and engineering applications. With increasing problem complexity and computational scale, traditional solution methods face significant computational and storage challenges, especially in fields such as fluid mechanics, power system simulation, and climate prediction. In these applications, solving sparse linear equation systems often becomes a performance bottleneck. To address this issue, iterative solution methods such as the conjugate gradient method (CG algorithm) have become one of the commonly used methods for efficiently solving large-scale sparse linear equation systems. The CG algorithm is an iterative method based on Krylov subspaces and is widely used to solve linear equation systems with symmetric positive definite matrices. In practical applications, the performance and efficiency of the CG algorithm are usually affected by multiple factors, including the sparsity and size of the matrix, as well as the computational environment. The core of the CG algorithm lies in iteratively optimizing the solution process, with each iteration continuously reducing the error of the solution. However, the computational speed of the CG algorithm largely depends on the sparse matrix-vector multiplication (SpMV) in each iteration; typically, SpMV computation accounts for more than 70% of the computation time of each iteration. In scientific computing and engineering applications, the CG algorithm is not only used to solve linear equations, but also widely used in other high-performance computing tasks, such as graph computing and finite element analysis. In the field of graph computing, the multiplication operation of sparse matrices and vectors is also a basic operation of many graph algorithms (such as PageRank, graph convolutional networks, etc.). For example, in the calculation of the product of the adjacency matrix and eigenvector of a graph, SpMV is used for tasks such as graph traversal, path finding and community detection. With the widespread application of neural networks, SpMV is no longer limited to the traditional field of scientific computing, but also occupies an important position in modern algorithms such as deep learning and graph neural networks (GNN). However, the implementation of efficient SpMV computing on general-purpose CPU / GPU platforms faces the following two major challenges: (1) a large number of random and discontinuous memory accesses. The non-zero element distribution pattern of highly sparse matrices and the row-by-row operation method result in huge memory access spans and make it difficult to achieve efficient data reuse on general-purpose architectures. This leads to extremely high access latency and significantly reduces hardware computing efficiency. (2) The distribution of non-zero elements often varies greatly between rows, meaning the number of non-zero elements in different rows is not similar. However, general-purpose platforms typically divide tasks by row, which may lead to significant load imbalance and reduce overall parallelism and resource utilization. Most current SpMV optimization research is based on software solutions implemented on general-purpose computing platforms. Common methods include matrix data preprocessing, load partitioning algorithms, dynamic scheduling algorithms, and migration to new batch processing computing units. For the first method, preprocessing the matrix data and reconstructing its data storage structure can significantly optimize the memory access efficiency of SpMV, but preprocessing itself requires a large amount of time overhead.The latter three methods all focus on computational optimization, offering limited granularity for memory access optimization and thus limited performance improvement for large-scale and highly sparse matrices in SpMV. Therefore, there is an urgent need for an accelerator architecture capable of optimizing SpMV memory access performance at the hardware level to improve the performance and energy efficiency of SpMV in scientific computing, graph computing, and deep learning applications. Summary of the Invention

[0003] The technical problem to be solved by this invention is to provide a hardware accelerator, device and application method for sparse matrix-vector multiplication, which addresses the above-mentioned problems of the prior art. This invention aims to improve on-chip data reuse of hardware accelerators, reduce off-chip memory accesses, and improve the performance and energy efficiency of hardware accelerators for sparse matrix-vector multiplication.

[0004] To solve the above-mentioned technical problems, the technical solution adopted by the present invention is as follows:

[0005] A hardware accelerator for sparse matrix-vector multiplication includes an off-chip memory system, an on-chip network, and an on-chip processing system for performing data fetching, multiplication calculations, and matrix row element merging. The off-chip memory system includes HBM channels for storing five types of data: column indices, row indices, vector values, matrix values, and result vectors. Each HBM channel includes an HBM stack and a memory controller. The HBM channels for storing vector values ​​are connected to the on-chip network via a secondary cache, while the remaining HBM channels are directly connected to the on-chip processing system.

[0006] Optionally, the on-chip processing system comprises multiple computing units, each including a sequential read module, a processing unit (PE), a random read module, and a sequential write module. Each PE includes multiple pipelines, each pipeline being the smallest unit executing a computational task. Each pipeline performs computational tasks during sparse matrix-vector multiplication. The sequential read module initiates a memory access request to the HBM channel storing column indices, row indices, and matrix values ​​based on the address stored in the address register, distributing the transmitted data to the temporary storage area of ​​each pipeline within the PE. The random read module initiates a memory access request to the HBM channel storing vector values, using an internal memory access record buffer to merge and record memory access requests initiated by different pipelines, and loads the returned data into the corresponding cache within each PE. The sequential write module writes the result vector into the HBM channel used to store the result vector.

[0007] Optionally, the pipeline includes a temporary storage area, a cache, and a merging unit. The temporary storage area consists of two storage blocks and a computing unit. The two storage blocks alternately input and output data to avoid data flow congestion. Each storage block includes 64 storage cells. Each storage cell temporarily stores a column index value, a matrix value, and an intermediate result of multiplying the matrix value and the vector value. When the storage block is empty, data is input through a sequential reading module. After the initial data of the 64 storage cells is in place, the corresponding vector value is requested from the cache according to the column index value. After the vector value is returned, a floating-point multiplication operation is performed with the corresponding matrix value in the computing unit. When the intermediate results in all storage cells are returned, the intermediate results are passed to the merging unit with a vector width of 4.

[0008] Optionally, the cache consists of a cache line direct-mapped memory and a cache miss status register MSHR. The cache line direct-mapped memory is used to cache the vector values ​​returned by the random access module, and the cache miss status register MSHR is used to record multiple cache miss access requests to achieve out-of-order access at a specified depth. The cache miss status register MSHR is a fully connected structure and uses a first-in-first-out replacement strategy to store access requests.

[0009] Optionally, the merging unit includes a 4-width sequence merger, used to merge the received intermediate results within rows according to the received row index value, and output the result vector sequentially after merging.

[0010] Optionally, the HBM channels used to store the five types of data—column index, row index, vector value, matrix value, and result vector—include four HBM channels for storing column indexes, four HBM channels for storing row indexes, four HBM channels for storing matrix values, eight HBM channels for storing vector values, and eight HBM channels for storing result vectors.

[0011] The present invention also provides a computing device including a processor and a hardware accelerator interconnected, wherein the hardware accelerator is the hardware accelerator for sparse matrix-vector multiplication.

[0012] This invention also provides an application method for the hardware accelerator for sparse matrix-vector multiplication, comprising the following steps: before computation, performing static load planning on the sparse matrix X stored in CSR format to determine the address for each processing unit (PE) to retrieve data; loading the address into the address register of the sequential read module in each processing unit (PE); uniformly dividing and loading the sparse matrix X stored in CSR format into eight different HBM channels for storing vector values; and executing the computation task in the sparse matrix-vector multiplication process through a two-stage parallel structure consisting of multiple processing units (PE) and pipelines within each processing unit (PE).

[0013] Optionally, the step of performing static load planning on the sparse matrix X stored in CSR format before computation to determine the address for each processing unit PE to retrieve data includes: based on the matrix row index of the sparse matrix X stored in CSR format and the number of subtasks to be divided, performing another traversal on the row index to divide the sparse matrix X stored in CSR format into multiple consecutive data blocks, so that one pipeline in each processing unit PE undertakes the computation task of one data block.

[0014] Optionally, when the computation task in the sparse matrix-vector multiplication process is executed by the two-level parallel structure consisting of multiple processing units (PEs) and pipelines within each PE, the steps for any pipeline to execute the computation task include: the sequential read module obtains non-zero metadata from the HBM channels storing column indices, matrix values, and row indices respectively, and then transmits it to the temporary storage area of ​​the corresponding pipeline; each cycle, the temporary storage area converts the index value of a non-zero element into an address through the computation unit and submits a memory access request to the cache. If a hit occurs, the value of the corresponding vector X is immediately retrieved from the cache; if a miss occurs, a memory access request is submitted to the HBM channel used to store vector values ​​through the random read module; the computation unit multiplies the retrieved vector value with the corresponding matrix value, and saves the intermediate result obtained by multiplication to the temporary storage area; when the intermediate results of all storage blocks in a storage block of the temporary storage area are ready, all the intermediate results in this storage block are transmitted sequentially to the merging unit with a vector width of 4, while at the same time, another storage block submits a memory access request to the cache; the result vector value obtained by the merging unit after merging is written to the HBM channel used to store the result vector through the sequential write module.

[0015] Traditional processors suffer from severe performance bottlenecks when handling large-scale and highly discrete sparse matrix-vector multiplication (SpMV), due to numerous irregular memory accesses and load balancing limitations. Compared with existing technologies, this invention offers the following advantages: 1. The two-level parallel structure, consisting of multiple processing units (PEs) and the pipelines within each PE, achieves high parallelism. 2. The off-chip storage system includes HBM channels for storing five types of data: column indices, row indices, vector values, matrix values, and result vectors. By grouping HBM channels and separating read and write channels, memory access behavior becomes more controllable, resulting in better bandwidth utilization. 3. The off-chip storage system constructs a large-scale cache system that can contain multiple (e.g., 64) independent caches, making fuller use of the data locality of the matrix. 4. This invention utilizes a static load planning algorithm to load data, achieving good load balancing and enhancing memory access independence. Attached Figure Description

[0016] Figure 1This is a schematic diagram of the hardware accelerator in an embodiment of the present invention.

[0017] Figure 2 This is a schematic diagram of the on-chip processing system in an embodiment of the present invention.

[0018] Figure 3 This is a schematic diagram of the data-hardware mapping relationship for sparse matrix-vector multiplication in an embodiment of the present invention. Detailed Implementation

[0019] To enable those skilled in the art to better understand the technical solutions of the present invention, the technical solutions of the present invention will be further described in detail below with reference to the accompanying drawings in the embodiments of the present invention.

[0020] like Figure 1 As shown, the hardware accelerator for sparse matrix-vector multiplication in this embodiment includes an off-chip storage system composed of an HBM stack, a network-on-chip (NOC) for data exchange routing, and an on-chip processing system for performing data fetching, multiplication calculations, and merging of matrix row elements. The off-chip storage system includes HBM channels for storing five types of data: column indices, row indices, vector values, matrix values, and result vectors. Each HBM channel includes an HBM (High Bandwidth Memory) stack and a memory controller. The HBM channels storing vector values ​​are connected to the NOC via a secondary cache, while the remaining HBM channels are directly connected to the on-chip processing system.

[0021] To improve system memory access bandwidth and parallelism, enhance cross-channel data access capabilities, and reduce the limitations of memory access bandwidth on sparse matrix-vector multiplication, this invention statically groups multiple HBM channels for storage: HBM channels are divided into five groups for storing column indices, row indices, vector values, matrix values, and result vectors. The HBM channels within the column index, row index, matrix value, and result vector groups are independent of each other, with each channel independently sending or receiving data to or from the processing unit. The HBM channels within the vector value group are connected to the NOC (Network Controller) for cross-channel access to vector values ​​by different processing units. To enhance data reuse, an additional level-two cache is connected between each HBM channel and the NOC. Specifically, in this embodiment, the HBM channels used to store the five types of data—column indices, row indices, vector values, matrix values, and result vectors—include 4 HBM channels for storing column indices, 4 HBM channels for storing row indices, 4 HBM channels for storing matrix values, and 8 HBM channels for storing vector values. Figure 1 The 8 HBM channels, denoted as HBM_0 to HBM_7, are used to store the result vector. Figure 1 (represented as HBM_8~HBM_15) Figure 1 The other HBM stacks include 4 HBM channels for storing column indices, 4 HBM channels for storing row indices, and 4 HBM channels for storing matrix values, for a total of 12 HBM channels.

[0022] like Figure 2 As shown, the on-chip processing system in this embodiment consists of multiple computing units. Each computing unit includes a sequential read module, a processing unit (PE), a random read module, and a sequential write module. The processing unit (PE) includes multiple pipelines (e.g., 8 pipelines; the 8 pipelines of the first processing unit (PE_0) are represented as pipelines 0-7, and so on, with the number adjustable as needed). Each pipeline is the smallest unit for completing a computational task, and each pipeline performs the computational task in the sparse matrix-vector multiplication (SpMV) process. The sequential read module initiates a memory access request to the HBM channel storing column indices, row indices, and matrix values ​​based on the address stored in the address register, distributing the transmitted data to the temporary storage area of ​​each pipeline within the processing unit (PE). The random read module initiates a memory access request to the HBM channel storing vector values, using an internal memory access record buffer to merge and record memory access requests initiated by different pipelines, and loads the returned data into the corresponding cache within each processing unit (PE). The sequential write module writes the result vector into the HBM channel used to store the result vector.

[0023] like Figure 2 As shown, the pipeline in this embodiment includes a temporary storage area, a cache, and a merging unit. The temporary storage area consists of two storage blocks and a computing unit. The two storage blocks alternately input and output data to avoid data flow congestion. Each storage block includes 64 storage cells. Each storage cell temporarily stores a column index value, a matrix value, and an intermediate result after multiplying the matrix value and the vector value. When the storage block is empty, data is input through the sequential reading module. After the initial data of the 64 storage cells is in place, the corresponding vector value is requested from the cache according to the column index value. After the vector value is returned, it is multiplied by the corresponding matrix value in the computing unit. When the intermediate results in all storage cells are returned, the intermediate results are passed to the merging unit with a vector width of 4.

[0024] like Figure 2As shown, the cache in this embodiment consists of a cache line direct-mapped memory and a cache miss status register (MSHR). The cache line direct-mapped memory is used to cache the vector values ​​returned by the random access module. The cache miss status register (MSHR) is used to record multiple cache miss access requests (MSHR entries) to achieve out-of-order access at a specified depth. The cache miss status register (MSHR) is a fully associative structure and uses a first-in-first-out replacement strategy to store access requests. The depth is 8 to store 8 cache miss access requests (MSHR entries). The cache capacity is small, only 2KB.

[0025] In this embodiment, the merging unit includes a 4-width sequence merger, used to merge the received intermediate results within a row according to the received row index value, and output the result vector sequentially after merging. Specifically, based on the row index value transmitted by the sequential reading module, the intermediate results are merged within a row, the result vector is output sequentially after merging, and the data is sent to the sequential writing module.

[0026] This embodiment of a hardware accelerator for sparse matrix-vector multiplication optimizes storage access and accelerates parallel processing at the hardware level. It leverages the data locality within the matrix structure through an on-chip caching system, while rearranging and merging external memory access requests to reduce unnecessary memory accesses. Furthermore, it partitions the matrix data, mapping each group to a specific hardware pipeline for execution, thereby achieving highly parallel computation and significantly improving the computational performance and energy efficiency of sparse matrix-vector multiplication. The key to this embodiment is fully utilizing the small-range data locality and large-range data reuse of large-scale sparse matrices. The caching system migrates most random memory accesses to the chip, and deep merging and rearrangement of off-chip random memory accesses improves on-chip data reuse and reduces the number of off-chip memory accesses. Combined with a load planning algorithm, the computational load can be evenly distributed across each processing unit.

[0027] In addition, this embodiment also provides a computing device, including a processor and a hardware accelerator interconnected, wherein the hardware accelerator is the hardware accelerator for sparse matrix-vector multiplication.

[0028] Furthermore, this embodiment also provides an application method for the aforementioned hardware accelerator for sparse matrix-vector multiplication, including the following steps: before computation, static load planning is performed on the sparse matrix X stored in CSR format to determine the address for each processing unit PE to retrieve data; the address is loaded into the address register of the sequential read module in each processing unit PE; the sparse matrix X stored in CSR format is evenly divided and loaded into eight different HBM channels for storing vector values; the computation task in the sparse matrix-vector multiplication process is executed through a two-level parallel structure consisting of multiple processing units PE and pipelines inside each processing unit PE.

[0029] In this embodiment, the hardware accelerator for sparse matrix-vector multiplication performs parallel computations on multiple pipelines of multiple processing units (PEs). Since this architecture is a dataflow-based accelerator and does not integrate load control logic, static load planning is performed before computation. The core objective of load planning is to divide the computational task into parallelizable subtasks with the most balanced load possible, and then map these subtasks to the basic execution units within the architecture for computation. This embodiment employs a load planning algorithm called "merge path." Before computation, static load planning is performed on the sparse matrix X stored in CSR format to determine the address for each processing unit (PE) to retrieve data. This includes: based on the matrix row index of the sparse matrix X stored in CSR format and the number of subtasks to be divided, a single traversal of the row indexes divides the sparse matrix X into multiple contiguous data blocks (e.g., 64 blocks). Each pipeline in each processing unit (PE) undertakes the computation task of one data block, and SpMV computation is performed independently within each data block. To ensure load balancing among each PE, the boundaries of the data blocks need to be determined based on the computational load. For a data block, the computational load consists of two parts: one part is the calculation of multiplying each non-zero element with a vector element, and the other part is the calculation of merging elements within the row. This embodiment uses the merge path load planning algorithm, which initially inputs the matrix row index and the number of subtasks to be divided. After calculating the computational load for each data block, a single traversal of the row indexes is sufficient to divide the data blocks. Furthermore, this algorithm itself can be executed in parallel with minimal overhead. Taking the hardware accelerator in this embodiment, which includes seven processing units PE_0 to PE_7, as an example, for sparse matrix-vector multiplication of y = AX, the data-hardware mapping relationship of sparse matrix-vector multiplication is as follows: Figure 3 As shown in the figure, the top of the figure is vector X, and the rectangular squares in the lower left corner are the processing parts (subtasks) of processing units PE_0 to PE_7 respectively. Finally, the output results of processing units PE_0 to PE_7 are combined to obtain the output matrix y.

[0030] In this embodiment, the two-level parallel structure consisting of multiple processing units (PEs) and pipelines within each PE is as follows: (1) The global design of multiple processing units (PEs) constitutes the first level of parallelism. Each PE and the HBM channel with the corresponding numbered storage vector value are designed with a direct data path in the on-chip network (NOC). Through the load planning algorithm, the load that accesses a certain HBM the most is allocated to the corresponding processing unit (PE) for execution, reducing horizontal cross-HBM access within the NOC, enhancing the independence of HBM access between processing units (PEs), and improving overall parallelism and memory access bandwidth. (2) The second level of parallelism comes from the multi-pipeline parallelism within the processing unit (PE). Each pipeline contains the basic computation and logic units required for SpMV computation and is the smallest execution unit for load allocation. The random read module of the processing unit (PE) is equipped with a memory access record buffer, which can merge and record memory access requests issued by different pipelines within a processing unit (PE) to avoid making duplicate memory access requests to HBM. This design reduces memory access conflicts and improves memory access efficiency and local parallelism. Specifically, when performing the computation task in the sparse matrix-vector multiplication process through a two-level parallel structure consisting of multiple processing units (PEs) and pipelines within each PE, the steps for any pipeline to perform the computation task include: the sequential read module obtains non-zero metadata from the HBM channels storing column indices, matrix values, and row indices respectively, and then passes it to the temporary storage area of ​​the corresponding pipeline; each cycle, the temporary storage area converts the index value of a non-zero element into an address through the computation unit and submits a memory access request to the cache. If a cache hit occurs, the corresponding vector X value is immediately retrieved from the cache; if a cache miss occurs, a memory access request is submitted to the HBM channel used to store vector values ​​through the random read module; the computation unit multiplies the retrieved vector value with the corresponding matrix value and saves the intermediate result obtained from the multiplication to the temporary storage area; when the intermediate results of all storage blocks in a storage block of the temporary storage area are ready, all the intermediate results in this storage block are transmitted sequentially to the merging unit with a vector width of 4, while at the same time, another storage block submits a memory access request to the cache; the result vector value obtained after merging by the merging unit is written to the HBM channel used to store the result vector through the sequential write module.

[0031] Through the above architecture and method, the hardware accelerator for sparse matrix vector multiplication in this embodiment divides a large-scale SpMV problem into multiple parallel subtasks. Each subtask is computed in a highly pipelined manner in terms of hardware structure. Furthermore, through load planning algorithm, a large number of independent caches, and a two-level parallel hardware structure, the three major performance bottlenecks of unbalanced load, irregular memory access, and low parallelism are effectively solved, thereby achieving performance and energy efficiency improvement. The hardware accelerator for sparse matrix vector multiplication in this embodiment executes the computation task in the sparse matrix vector multiplication process through a two-level parallel structure consisting of multiple processing units (PE) and pipelines inside each processing unit (PE). This enables out-of-order non-blocking execution within the pipeline, which has the following advantages: (1) The design of the temporary storage area with dual memory blocks requires simultaneous input and output to the temporary storage area during the computation process. However, for the hardware structure, simultaneous vectorized input and output will bring a large amount of additional control logic overhead. If control logic is not added, it is impossible to perform input and output simultaneously, causing system congestion and reducing throughput. To overcome the above problems, each temporary storage area is configured with dual storage blocks. When one storage block is input, the other storage block is output. The two storage blocks alternately input and output, which ensures that the system can continuously perform data calculation and transmission without blocking. (2) Non-blocking read-only cache. The cache is designed with an 8-depth cache miss status register MSHR. Through the cache miss status register MSHR, out-of-order memory access with a depth of 8 can be realized, which can effectively reduce system blocking caused by memory access blocking. (3) Full data path backpressure mechanism. Data transmission is based on the handshake protocol from the sequential read module, the sequential write module and various components in PE, realizing the full data path backpressure mechanism. This ensures that when the HBM channel returns data slowly and the data flow in the temporary storage area is blocked, the sequential read module will not raise a memory access request, preventing data loss and system deadlock.

[0032] The above description is merely a preferred embodiment of the present invention. The scope of protection of the present invention is not limited to the above embodiments. All technical solutions falling within the scope of the present invention's concept are within the scope of protection of the present invention. It should be noted that for those skilled in the art, any improvements and modifications made without departing from the principles of the present invention should also be considered within the scope of protection of the present invention.

Claims

1. A hardware accelerator for sparse matrix-vector multiplication, characterized in that, The system includes an off-chip storage system, an on-chip network for data exchange routing, and an on-chip processing system for performing data fetching, multiplication, and matrix element merging. The off-chip storage system includes HBM channels for storing five types of data: column indices, row indices, vector values, matrix values, and result vectors. Each HBM channel includes an HBM stack and a memory controller. The HBM channels storing vector values ​​are connected to the on-chip network via a secondary cache, while the remaining HBM channels are directly connected to the on-chip processing system. The on-chip processing system consists of multiple computing units, each including a sequential read module, a processing unit (PE), a random read module, and a sequential write module. The module; the processing unit PE includes multiple pipelines, each pipeline being the smallest unit for executing computational tasks, and each pipeline performing computational tasks in the sparse matrix-vector multiplication process; the sequential read module is used to initiate memory access requests to the HBM channel storing column indices, row indices, and matrix values ​​according to the address stored in the address register, and distributes the transmitted data to the temporary storage area of ​​each pipeline within the processing unit PE; the random read module is used to initiate memory access requests to the HBM channel storing vector values, and uses an internal memory access record buffer to merge and record memory access requests initiated by different pipelines, and loads the returned data into the corresponding cache within each processing unit PE; The sequential write module is used to write the result vector into the HBM channel used to store the result vector. The pipeline includes a temporary storage area, a cache, and a merging unit. The temporary storage area consists of two storage blocks and a computing unit. The two storage blocks alternately input and output data to avoid data flow congestion. Each storage block includes 64 storage cells. Each storage cell temporarily stores a column index value, a matrix value, and an intermediate result after multiplying the matrix value and the vector value. When the storage block is empty, data is input through the sequential read module. After the initial data of the 64 storage cells is in place, the corresponding vector value is requested from the cache according to the column index value. After the vector value is returned, a floating-point multiplication operation is performed with the corresponding matrix value in the computing unit. When the intermediate results in all storage cells are returned, the intermediate results are passed into the merging unit with a vector width of 4.

2. The hardware accelerator for sparse matrix-vector multiplication according to claim 1, characterized in that, The cache consists of a cache line direct-mapped memory and a cache miss status register (MSHR). The cache line direct-mapped memory is used to cache the vector values ​​returned by the random access module. The cache miss status register (MSHR) is used to record multiple cache miss access requests to achieve out-of-order access at a specified depth. The cache miss status register (MSHR) is a fully connected structure and uses a first-in-first-out replacement strategy to store access requests.

3. The hardware accelerator for sparse matrix-vector multiplication according to claim 2, characterized in that, The merging unit includes a 4-width sequence merger, which is used to merge the received intermediate results within a row according to the received row index value, and output the result vector sequentially after the merging is completed.

4. The hardware accelerator for sparse matrix-vector multiplication according to claim 3, characterized in that, The HBM channels, which are used to store five types of data—column indexes, row indexes, vector values, matrix values, and result vectors—include four HBM channels for storing column indexes, four HBM channels for storing row indexes, four HBM channels for storing matrix values, eight HBM channels for storing vector values, and eight HBM channels for storing result vectors.

5. A computing device comprising a processor and a hardware accelerator interconnected, characterized in that, The hardware accelerator is the hardware accelerator for sparse matrix-vector multiplication as described in any one of claims 1 to 4.

6. An application method of the hardware accelerator for sparse matrix-vector multiplication as described in claim 4, characterized in that, The process includes the following steps: Before computation, static load planning is performed on the sparse matrix X stored in CSR format to determine the address for each processing unit (PE) to retrieve data; the address is loaded into the address register of the sequential read module in each processing unit (PE); the sparse matrix X stored in CSR format is evenly divided and loaded into eight different HBM channels for storing vector values; the computation task in the sparse matrix-vector multiplication process is executed through a two-stage parallel structure consisting of multiple processing units (PE) and pipelines within each processing unit (PE).

7. The application method of the hardware accelerator for sparse matrix-vector multiplication according to claim 6, characterized in that, The step of performing static load planning on the sparse matrix X stored in CSR format before computation to determine the address for each processing unit PE to retrieve data includes: based on the matrix row index of the sparse matrix X stored in CSR format and the number of subtasks to be divided, the row index is traversed once to divide the sparse matrix X stored in CSR format into multiple consecutive data blocks, so that one pipeline in each processing unit PE undertakes the computation task of one data block.

8. The application method of the hardware accelerator for sparse matrix-vector multiplication according to claim 6, characterized in that, When the computation task in the sparse matrix-vector multiplication process is executed by the two-level parallel structure consisting of multiple processing units (PEs) and pipelines within each PE, the steps for any pipeline to execute the computation task include: the sequential read module obtains non-zero metadata from the HBM channels storing column indices, matrix values, and row indices respectively, and then transmits it to the temporary storage area of ​​the corresponding pipeline; each cycle, the temporary storage area converts the index value of a non-zero element into an address through the computation unit and submits a memory access request to the cache. If a hit occurs, the corresponding vector value is immediately retrieved from the cache; if a miss occurs, a memory access request is submitted to the HBM channel used to store vector values ​​through the random read module; the computation unit multiplies the retrieved vector value with the corresponding matrix value, and saves the intermediate result obtained by multiplication to the temporary storage area; when the intermediate results of all storage blocks in a storage block of the temporary storage area are ready, all the intermediate results in this storage block are transmitted sequentially to the merging unit with a vector width of 4, while at the same time, another storage block submits a memory access request to the cache; the result vector value obtained by the merging unit after merging is written to the HBM channel used to store the result vector through the sequential write module.

Images