An influenza-aware high-bandwidth memory FPGA sparse LU decomposition acceleration method

Abstract

The application discloses a kind of flow-aware high bandwidth memory FPGA sparse LU decomposition acceleration method, belong to LU decomposition hardware acceleration field, to solve the performance bottleneck and efficiency challenge faced by sparse LU decomposition on FPGA acceleration.The acceleration method includes the following steps: data stream preprocessing: based on sparse storage format, matrix data and metadata are stored and packaged;Custom FPGA hardware accelerator: HBM channel allocation, collaborative design of control unit and memory management are carried out;Parallel data stream management and scheduling, execute sparse LU decomposition: ensure timely supply of data through multi-stage pipeline prefetch mechanism;Combine flow-aware synchronous scheduling strategy to transmit task stream and data stream to dedicated parallel processing engine for MAC / DIV calculation.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of LU decomposition hardware acceleration, and particularly relates to a flow-aware high-bandwidth memory FPGA sparse LU decomposition acceleration method. BACKGROUND

[0002] Solving large sparse linear systems Ax = b is a fundamental computational task in scientific and engineering applications, involving circuit simulation, finite element analysis, power system modeling, etc. In practice, direct solvers based on sparse LU decomposition are preferred over iterative methods because they provide reliable accuracy without the need to select and tune preconditioners. As the problem size grows, the sparse LU decomposition phase has become a bottleneck of computation time, and its acceleration has become a key issue in high-performance computing. Therefore, it is of great significance to develop a special sparse LU decomposition accelerator. Field programmable gate array (FPGA) provides an extremely attractive platform for hardware acceleration through customized parallel architecture and data flow design. Compared with central processing unit (CPU) and graphics processing unit (GPU), FPGA not only can achieve lower latency and power consumption, but also can provide higher resource utilization for sparse matrix workloads. In addition, sparse matrix computation is essentially memory-intensive. High-bandwidth memory (HBM) has higher bandwidth and more independent memory channels than traditional double data rate memory (DDR), which enables HBM-equipped FPGA to effectively address the memory access challenges in the sparse LU decomposition process.

[0003] Sparse LU decomposition has irregular data dependencies and memory access patterns due to its inherent high sparsity and random distribution of non-zero elements, making its efficient acceleration scheme on FPGA still not fully explored. Although the high concurrency capability of HBM has brought new possibilities for the acceleration of sparse LU decomposition in recent years, the inherent random dependencies and complex computation patterns of the algorithm make it difficult to achieve efficient bandwidth utilization on the HBM platform.

[0004] The key technical challenges faced by sparse LU decomposition acceleration on HBM FPGA mainly include the following four aspects:

[0005] (1) The mismatch between existing sparse storage formats and HBM access patterns leads to low HBM bandwidth utilization. The pointer array of CSR (row compressed format) / CSC (column compressed format) cannot achieve full streaming access to non-zero elements. The accelerator must access the pointer array before reading the non-zero elements. In addition, the highly random distribution of non-zero elements in sparse matrices hinders effective data merging, thereby reducing bandwidth utilization.

[0006] (2) On-chip memory resource limitation and insufficient data reuse. When processing large-scale sparse matrices, simple on-chip data allocation strategies can cause dependent memory access data to be removed prematurely, even though these data still have reuse value afterwards, thereby reducing on-chip memory access efficiency and effective utilization of memory.

[0007] (3) High access delay of HBM affects off-chip memory communication efficiency. Although HBM provides huge bandwidth, its inherent high access delay limits the performance of sparse LU decomposition. Traditional prefetching strategies can cause prefetched data to be unnecessary when facing irregular off-chip sparse memory access patterns, resulting in inefficient prefetching and unnecessary HBM access, which seriously affects off-chip memory communication efficiency.

[0008] (4) Complex data dependencies make parallel scheduling difficult. In sparse LU decomposition, the dependency relationship between columns is complex and irregular, which makes it extremely difficult to perform parallel scheduling on FPGA. SUMMARY

[0009] In order to make up for the shortcomings of the prior art, the present application aims to provide a flow-aware high-bandwidth memory FPGA sparse LU decomposition acceleration method to solve the performance bottleneck and efficiency challenge faced by sparse LU decomposition on FPGA.

[0010] The technical problems to be solved by the present application can be solved by the following specific technical solutions:

[0011] The flow-aware high-bandwidth memory FPGA sparse LU decomposition acceleration method comprises the following steps:

[0012] Data flow preprocessing: store and package matrix data and metadata based on sparse storage format;

[0013] Custom FPGA hardware architecture: HBM channel allocation, co-design of control unit and memory management;

[0014] Parallel data flow management and scheduling, and execution of sparse LU decomposition: ensure timely data supply through multi-stage pipeline prefetching mechanism; combine flow-aware synchronous scheduling strategy to transmit task flow and data flow to dedicated parallel processing engine for MAC / DIV calculation.

[0015] Further, the matrix data adopts COO (coordinate format) explicit storage of non-zero elements and their row and column indexes, and through vectorized packaging, multiple elements are combined into an HBM data word; the input data is reordered according to the row index module, each element is allocated to a target slot, if the slot is occupied, the data word is filled with dummy elements, and a new data word packaging cycle is started.

[0016] Further, the metadata includes dependency metadata, which uses the non-zero structure of the matrix U obtained after the static symbolic decomposition of the matrix A to construct a dependency graph, which is represented as a directed acyclic graph, and for nodes without dependency relationships, they are grouped into the same level according to their dependency depth; wherein the dependency relationship is organized into the following four linearization arrays:

[0017] Level pointer: store the starting offset of each level;

[0018] Level column index: store the column index in each level;

[0019] Dependency pointer: store the offset of each column in the dependency list, and the difference between adjacent two items represents the number of dependencies of the specified column;

[0020] Dependency index: store the dependency column index of the specified column.

[0021] Further, the metadata also includes matrix metadata, which adopts a data word layout format, which contains two arrays, namely data word offset and data word count, which store the starting offset and occupied data words of each column respectively.

[0022] Further, the FPGA hardware accelerator customization includes channel allocation, control unit and hierarchical memory module, wherein,

[0023] For channel allocation: adopt HBM channel allocation strategy, a total of 26 HBM channels are allocated for data transmission;

[0024] For the control unit: a scheduler is designed, which includes a data prefetcher and a controller, used to coordinate the data and task flow of the accelerator; the data prefetcher pre-fetches the requested raw data through the matrix A loader to avoid unnecessary HBM access; at the same time, the controller obtains and sorts the metadata from the metadata loader to form a processing element (PE) task package;

[0025] For the hierarchical memory module: it contains local memory and shared memory, the local memory is used to provide local access, and the shared memory is used to provide shared storage space, both the local memory and the shared memory use the first-in-first-out replacement strategy for management.

[0026] Further, the multi-stage pipeline simultaneously manages and processes multiple HBM data prefetch requests, and carefully divides the HBM access operation into multiple stages to achieve high overlap in time.

[0027] Further, the scheduler adopts a column index modulo strategy to distribute the column tasks of the initial level (for example, level 0) and their corresponding data streams to different processing element groups (PEGs) for parallel processing; within each PEG, the PEs cooperatively complete the analysis of the dependency relationship and perform the multiply-accumulate (MAC) or division (DIV) operation on the current column. The stream-aware synchronous scheduling strategy strictly follows the level processing sequence: the stream-aware synchronous scheduling strategy strictly follows the level processing sequence: after all the column tasks of the current level are processed, the scheduler starts the stream computing of the next level.

[0028] Further, the content of performing the MAC operation includes: each processing element (PE) receives multiply-add input data packets and performs the multiply-accumulate operation of sparse LU decomposition; the PE performs parallel multiply-add operations using 8 floating-point values and their packed elements through vectorization processing; the floating-point multiplication and addition operations utilize the digital signal processing resources inside the FPGA and adopt a deep pipeline design to maximize resource utilization, and through internal loop unrolling optimization, an ideal start interval of 1 is achieved.

[0029] Further, the content of performing the DIV operation includes: after the completion of the MAC phase, the PEs inside the PEG cooperatively complete the processing and normalization of the diagonal elements of the current column; the operation adopts a pipelined floating-point divider to effectively optimize its inherent high delay characteristics, and the process also handles the diagonal elements close to zero to ensure numerical stability; after all the multiply-add operations are completed, the diagonal element A jj of the current column j becomes U jj Then, the processing elements in the division unit divide all the elements A rj below the diagonal in column j in parallel by U jj to generate the final L factor L rj , where r>j.

[0030] Compared with the prior art, the present application has the following advantages:

[0031] (1) The sparse LU decomposition accelerator (SCLAER) on the HBM FPGA proposed in the present application adopts a sparse storage format and combines data through vectorization packing, customizes a data stream compatible with HBM to improve bandwidth utilization;

[0032] (2) The two-layer hierarchical memory module proposed in the present application optimizes memory management and reduces redundant transmission to improve data access efficiency and on-chip memory reuse rate;

[0033] (3) In order to manage the complex dependency relationship of sparse LU decomposition and maximize parallelism, the application designs a flow-aware synchronous scheduling strategy, which converts irregular dependency relationship into hierarchical flow access; combined with a multi-stage pipeline data prefetching mechanism, the delay is hidden by overlapping HBM access stages, improving the efficiency of off-chip communication. BRIEF DESCRIPTION OF DRAWINGS

[0034] Figure 1 For the original matrix example of the application;

[0035] Figure 2 For the matrix data storage and packaging flowchart of the application (based on the original matrix example);

[0036] Figure 3 For the metadata storage and packaging flowchart of the application (based on the original matrix example);

[0037] Figure 4 For the hardware architecture diagram of the FPGA hardware accelerator of the application;

[0038] Figure 5 For the parallel LU decomposition process in the PEG of the application. DETAILED DESCRIPTION

[0039] In order to make the purpose, technical scheme and advantages of the application clearer, the application will be further described in detail below in combination with the drawings and specific embodiments. It should be understood that the specific embodiments described herein are only used to explain the application and do not limit the application.

[0040] The application provides a flow-aware high-bandwidth memory FPGA sparse LU decomposition acceleration method, which includes the following contents:

[0041] (I) Data stream preprocessing: based on the sparse storage format, the matrix data and metadata are stored and packaged.

[0042] Sparse LU decomposition has irregular non-zero distribution and complex inter-column dependency relationship. Without special storage and packaging design, it will result in low bandwidth utilization and serious limitation of parallelism. In order to solve this problem, the application proposes a sparse data format compatible with HBM and a vectorized packaging scheme, especially for the original matrix and metadata, a sparse storage format is customized.

[0043] (1) Matrix data storage and packaging

[0044] For matrix data stream preprocessing, the application stores non-zero elements and their row and column indexes in coordinate format, which can effectively retrieve scattered non-zero elements and is suitable for irregular access mode of sparse matrix. For example, Figure 1 and Figure 2As shown, each non-zero element is encoded as 64 bits: a 16-bit column index, a 16-bit row index, and a 32-bit single-precision floating-point value. By vectorized packing, multiple 64-bit elements are merged into a 512-bit HBM data word. To prevent on-chip block random access memory (BRAM) conflicts, the input data is reordered according to the row index modulo, and each element is assigned to a target slot. If the slot is already occupied, the data word is padded with dummy elements, and a new data word packing cycle is initiated. Despite the slight storage overhead, this strategy ensures conflict-free BRAM access and fixed-timing PE processing, which is acceptable considering the performance advantage.

[0045] (2) Metadata storage and packing

[0046] Metadata includes two parts: dependency metadata (hierarchy information and fine-grained dependency list) and matrix metadata (data word offset and count of matrix A and L / U data).

[0047] Dependency metadata is crucial for guiding the computation order and maintaining the pipeline efficiency in FPGA. As shown in (a) of Figure 3 , the dependency graph is constructed using the non-zero structure of matrix U obtained by static symbolic factorization of matrix A. The dependency graph is represented as a directed acyclic graph (DAG) , where the node set corresponds to all columns, and the edge set represents the dependency relationship from column j to column k. For nodes with no dependency relationship, they are grouped into the same hierarchy according to their dependency depth, where the hierarchy of node k is calculated as follows:

[0048] ,

[0049] where denotes the row index of all non-diagonal non-zero elements in the kth column of U (i.e. ). Considering the irregular storage pattern of the adopted dependency graph, it is difficult to efficiently utilize the memory by traversing the dependency graph layer by layer, and it will further lead to non-continuous data access. Therefore, the present application organizes these dependency relationships into the following four linearized arrays, as shown in (b) of Figure 3 , including hierarchy pointers, hierarchy column indices, dependency pointers, and dependency indices. The hierarchy pointers are used to store the starting offset of each hierarchy; the hierarchy column indices are used to store the column indices in each hierarchy; the dependency pointers are used to store the offset of each column in the dependency list, and the difference between adjacent two entries represents the number of dependencies of the specified column; the dependency indices are used to store the dependency column indices of the specified column.

[0050] To convert the logical structure of sparse matrix into physically contiguous and efficiently accessible data blocks in HBM, the invention introduces matrix metadata. With a data word layout format, which has two arrays, as shown in (c), i.e. data word offset and data word count, stores the starting offset and occupied data words of each column. After that, all metadata entries (32-bit unsigned integer) are packed into 512-bit HBM data words to optimize the transmission efficiency of HBM. Figure 3

[0051] (II) Customized FPGA hardware accelerator: HBM channel allocation, control unit and memory management.

[0052] The invention realizes a customized hardware architecture on the FPGA side, which is designed to efficiently perform sparse LU decomposition and fully utilize the bandwidth of HBM and the parallel processing capability of FPGA. The following is the specific architecture design:

[0053] (1) Channel allocation

[0054] To solve the bandwidth competition problem that may exist when different types of data (matrix A, metadata, L / U factor) are accessed in HBM, the invention adopts a fine-grained HBM channel allocation strategy, allocating a total of 26 HBM channels for data transmission. Since the space occupied by the two types of core calculation data (matrix A and L / U factor) is large, the invention maps them to 12 HBM channels respectively. To isolate the bandwidth competition between metadata access and core calculation data, 2 independent HBM channels are allocated for all metadata (one for scheduling control and matrix A layout information, and the other for L / U factor layout information). Matrix A loader, metadata loader and L / U writer all support highly concurrent streaming access, ensuring that data can be efficiently transmitted between HBM and internal modules in FPGA.

[0055] (2) Control unit

[0056] The high access latency of HBM and the complex data dependency are challenges in sparse LU decomposition. To solve these problems, the invention designs a scheduler, which includes a data prefetcher (as shown in (e)) and a controller (as shown in (f)), to coordinate the data and task flow of the accelerator. The data prefetcher efficiently prefetches the requested raw data through the matrix A loader (as shown in (a)) to avoid unnecessary HBM access. At the same time, the controller (as shown in (f)) coordinates the data and task flow of the accelerator. Figure 4 Figure 4 Figure 4 Figure 4 Figure 4 ​​​​​The metadata is obtained and organized to form PE task packages in (b) and (c). Finally, the scheduler distributes these tasks and their corresponding column data in parallel to the dedicated parallel processing engine (such as Figure 4 The 12 processing element groups (PEGs) in (g) are used to perform numerical column calculations by cooperating MAC and DIV units inside each PEG, thereby achieving efficient sparse LU decomposition.

[0057] (3) Hierarchical memory module

[0058] L / U factors have random and frequent access patterns in sparse LU decomposition. If only relying on the limited capacity of local memory, its coverage is insufficient to effectively capture all related dependent factors, which will lead to frequent memory misses. In addition, when the required L / U factors are located in the local memory of different processing element groups (PEGs), it faces significant challenges to achieve efficient cross-PEG direct communication and data forwarding. Using an all-to-all crossbar communication structure to exchange data directly between PEGs is not feasible, because this scheme will lead to a quadratic increase in circuit complexity, significantly increasing the difficulty of hardware implementation and reducing the working frequency. However, the high access delay caused by direct access to HBM will seriously slow down the calculation pipeline and affect the overall performance.

[0059] To solve the above problems and effectively improve the on-chip data reuse rate, the present application proposes a two-layer hierarchical memory module. The hierarchical memory module includes a very fast local memory and a larger capacity shared memory. Among them, the local memory provides very fast local access; while the shared memory provides a large capacity of shared storage space to capture a wider range of dependent factors, improve on-chip data reuse rate and reduce memory access delay. Both memory units use a first-in-first-out (FIFO) replacement strategy for management.

[0060] ① Local memory

[0061] L / U factors are stored in a sparse column memory entry array constructed by ultra-high-speed random access memory (URAM). The column index mapping table (recording the column ID corresponding to each memory slot) and the validity flag array (indicating whether the data in the slot is valid) are stored in the corresponding array using BRAM. By full partition design, each memory slot is mapped to different physical storage units (independent memory banks of BRAM / URAM). The local memory provides very fast local access, stores the L / U factor columns recently calculated by the PEG or obtained from the shared memory / HBM, and maximizes the use of calculation locality. By using the dual-port feature of BRAM, the local memory supports parallel PE access and effectively prevents irregular dependency access conflicts.

[0062] ② Shared memory

[0063] To avoid the acquisition of far dependent data across PEGs, improve the overall memory hit rate and reduce HBM access, shared memory is introduced as a large-capacity cache shared by all PEGs. The L / U factor is stored using URAM resources, and the global column index mapping table and validity flag array are stored using BRAM resources; the internal storage unit also adopts a complete partition design, supports high-concurrency access, and maximizes time and space locality. After calculation and storage in the local memory, the L / U factor is synchronously copied to the shared memory and written back to the HBM asynchronously through the L / U writer (as shown in (d) of FIG. 8). Figure 4 The HBM write-back operation overlaps with the computation pipeline of the PEG, hiding the delay.

[0064] (Three) Parallel data flow management and scheduling, and sparse LU decomposition: through a multi-stage pipeline prefetching mechanism to ensure timely data supply; in combination with a flow-aware synchronous scheduling strategy, the task flow and data flow are transmitted to a dedicated parallel processing engine for MAC / DIV calculation.

[0065] The present application proposes an efficient data flow calculation and scheduling strategy, aiming to accelerate sparse LU decomposition on HBM FPGA. Inside the PEG, the PE efficiently performs numerical calculation of left-looking sparse LU decomposition: first, through a series of vectorized MAC operations, the accumulated contribution of the product of the L / U factor of all left-decomposed columns is subtracted from the original value of the current processing column, thereby updating the dense representation of the current column; then, through the pipeline DIV, the diagonal element is normalized, and the final L / U factor of the elements below the diagonal line in the column is generated.

[0066] (1) Multi-stage pipeline prefetching

[0067] Although HBM provides tremendous bandwidth, its inherent high access latency constitutes a serious bottleneck for sparse LU decomposition performance. Conventional prefetching strategies often result in mismatch between prefetched data and actual computation needs when facing irregular off-chip sparse memory access patterns, thus reducing efficiency. To effectively alleviate this problem, the present application proposes a multi-stage pipeline prefetching mechanism. Specifically, the data prefetcher is configured with an asynchronous interface, which can access all 12 HBM channels in parallel to obtain the original matrix A; under the guidance of the metadata of the controller, data is obtained strictly according to the column sequence of the processing level to which it belongs, thus effectively avoiding unnecessary HBM access. The multi-stage pipeline simultaneously manages and processes multiple HBM read requests, carefully divides the HBM access operation into multiple stages (e.g., address sending, internal processing, data returning), to achieve high overlap in time. For example, when the data of the first HBM request is still being processed internally by HBM, the data prefetcher can already send the address of the second request to HBM; when the data of the second request starts to return, the address of the third request may have been sent. In addition, the data prefetcher accurately tracks HBM requests and responses to ensure data integrity. This mechanism ensures that multiple parallel processing columns of the same level can access the required data in a timely manner without causing congestion, enhancing off-chip memory communication and parallelism.

[0068] (2) Stream-aware synchronous scheduling

[0069] Due to the inherent random dependency relationship of sparse LU decomposition, it is extremely challenging to achieve efficient parallel scheduling. To maximize concurrent execution efficiency, the present application proposes a stream-aware synchronous scheduling strategy, which aims to ensure that data stream scheduling strictly follows the predefined level order. Specifically, the scheduler uses a column index modulo strategy to distribute column tasks of the initial level (e.g., level 0) and their corresponding data streams to different PEGs for parallel processing. Within each PEG, PEs cooperate to resolve dependency relationships and perform MAC or DIV operations for the current column. This stream-aware synchronous scheduling strategy strictly follows the level processing order: it ensures that only after all column tasks of the current level are processed, the stream computing of the next level can be started. This mechanism ensures that when a PEG processes its current level column, all related dependent column data from the previous level has been computed and can be efficiently obtained through the hierarchical memory module, thus effectively solving the data dependency problem.

[0070] ① MAC operation: Each PE receives the multiply-add input data packet and performs the multiply-add operation of sparse LU decomposition. The PE performs parallel multiply-add operations using 8 floating-point values and their packed elements through vectorization processing. The floating-point multiplication and addition operations use the digital signal processing (DSP) resources inside the FPGA and adopt a deep pipeline design to maximize resource utilization. Through internal loop unrolling optimization, an ideal initiation interval (II) of 1 is achieved. The core numerical calculation process is shown in FIG. 8, and the parallel computation of the PEs inside the PEG is shown in FIG. 9. Figure 5 ji ×U ij and L rk ×U kj , and subtract them from the corresponding elements A jj and A rj of the current column j to update their dense representations.

[0071] ② DIV operation: After the completion of the MAC phase, the PEs inside the PEG will cooperatively complete the processing and normalization of the diagonal elements of the current column. This operation uses a pipelined floating-point divider (usually configured with 1 to 2 instances) to effectively optimize its inherent high latency characteristics. This process also handles diagonal elements close to zero to ensure numerical stability. After all multiply-add operations are completed, the diagonal element A jj of the current column j becomes U jj . Then, the processing units in the division unit will divide all elements A rj below the diagonal in column j (where r > j) by U jj in parallel to produce the final L factor L rj .

[0072] (Four) Experiment

[0073] The sparse LU decomposition accelerator SCALER proposed in the present application is specifically implemented on an Xilinx Alveo U280 FPGA board. Its development uses Xilinx HLS C++ language, and hardware implementation is completed through Vitis 2022.1 tool chain. Among them, the HLS C++ code is constructed based on the streaming processing framework TAPA.

[0074] In order to evaluate the performance of the proposed sparse LU decomposition accelerator, the present application selects 7 sparse matrices from the SuiteSparse dataset for evaluation. These matrices cover multiple application fields such as circuit simulation, computational fluid dynamics, and two-dimensional / three-dimensional problems, ensuring the wide applicability of the evaluation results. Table 1 lists the key information of these evaluation matrices in detail, including but not limited to their dimension size, number of non-zero elements, and matrix density.

[0075] Table 1 Information of evaluation matrices​

[0076]

[0077] To quantitatively evaluate the performance of the SCALER accelerator of the present application, the present application employs industry-standard GPU platforms as benchmarks for comparison. Specifically, the benchmark tests are executed on NVIDIA RTX 4090 and NVIDIA Tesla V100 GPUs, and utilize the CUDSS_PHASE_FACTORIZATION and CUDA12.4 environment of the secondary application programming interface (API) of the NVIDIA cuDSS library for sparse LU decomposition computation. cuDSS is recognized as a state-of-the-art solution for sparse LU decomposition on current GPUs. On the GPU platforms, the execution time of the LU decomposition phase is measured by cudaEventElapsedTime, and the power consumption is obtained by the nvidia-smi tool.

[0078] As for the SCALER accelerator, the kernel execution time is measured by the Xilinx Runtime (XRT), and the power consumption is obtained by the xbutil tool. To comprehensively evaluate the performance of the SCALER, the present application employs the following three key indicators: throughput (measured in GFLOP / s, i.e., the number of trillion floating-point operations per second), bandwidth efficiency (measured in MFLOP / s / (GB / s), i.e., the throughput per unit of memory bandwidth), and energy efficiency (measured in MFLOP / s / W, i.e., the throughput per watt).

[0079] Table 2 presents the detailed comparison results of SCALER and cuDSS running on NVIDIA V100 and RTX 4090 GPUs in terms of throughput, bandwidth efficiency, and energy efficiency. Among them, “performance improvement” is defined as the gain of SCALER relative to the best performance in the compared GPU. In addition, Table 3 reports the resource utilization of SCALER on the Xilinx Alveo U280 FPGA board in detail.

[0080] Table 2 Comparison of performance of SCLAER and cuDUSS running on NVIDIA V100 and RTX 4090 GPUs

[0081]

[0082] Table 3 Resource utilization of SCALER on Xilinx U280 FPGA board

[0083]

[0084] As shown in Table 2, 1) in terms of throughput, the geometric mean throughput gain of the SCALER accelerator proposed in the present application is 2.93 times higher than that of the V100 GPU and 1.68 times higher than that of the RTX 4090 GPU, respectively. On the matrix "poli4", the throughput of SCALER further achieves a peak speedup of up to 2.00 times on these GPUs. Although the frequencies of the RTX 4090 and V100 GPUs are much higher (nearly or more than 3 times) than that of the SCALER, the SCLAER accelerator still achieves significant improvement in overall throughput by managing dependencies and utilizing on-chip parallelism through flow-aware synchronous scheduling, making up for the lower frequency through high-concurrency data processing capability, and thus exceeding the two GPUs; 2) the geometric mean energy efficiency of SCALER is 6.58 times and 3.48 times higher than that of the V100 GPU and the RTX 4090 GPU, respectively. It can be seen that SCALER achieves higher throughput while maintaining lower power consumption. This is mainly due to the efficient data prefetching and optimized on-chip memory allocation strategy of SCALER, which significantly reduces the high-latency HBM access, thereby reducing the data movement energy consumption. Bandwidth utilization is crucial for sparse LU decomposition (memory-intensive task); 3) the SCALER's customized data stream compatible with high-bandwidth memory and dedicated vectorized data packing make its bandwidth efficiency 6.29 times and 4.57 times higher than that of the NVIDIA V100 GPU and the RTX 4090 GPU, respectively.

[0085] Finally, it should be noted that: the above embodiments are only used to illustrate the technical solutions of the present application, and not to limit them; although the present application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that they can still modify the technical solutions recorded in the foregoing embodiments, or make equivalent replacement for part or all of the technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the scope of the technical solutions of the embodiments of the present application.

Claims

1. A method for accelerating sparse LU decomposition in a flow-aware high-bandwidth memory FPGA, characterized in that, Includes the following steps: Data stream preprocessing: Storing and packaging matrix data and metadata based on a sparse storage format; Customized FPGA hardware architecture: Co-design of HBM channel allocation, control unit and memory management; Parallel data stream management and scheduling, execution of sparse LU decomposition: ensuring timely data supply through a multi-stage pipeline prefetching mechanism; Combining a stream-aware synchronous scheduling strategy, task streams and data streams are transmitted to a dedicated parallel processing engine for MAC / DIV computation. The matrix data uses a coordinate format to explicitly store non-zero elements and their row and column indices. Multiple elements are merged into HBM data words through vectorized packing. The input data is reordered according to the row index modulus, and each element is assigned to a target slot. If the slot is already occupied, the data word is filled with a dummy element and a new data word packing cycle is started. Metadata includes dependency metadata, which uses the non-zero structure of matrix U obtained by static symbolic decomposition of matrix A to construct a dependency graph. This dependency graph is represented as a directed acyclic graph (DAG). Nodes without dependencies are grouped into the same level based on their dependency depth. The dependencies are organized into the following four linearized arrays: Hierarchy pointer: Stores the starting offset of each hierarchy; Hierarchical column index: Stores the column indexes for each level; Dependency pointer: Stores the offset of each column in the dependency list. The difference between two adjacent entries indicates the number of dependencies of the specified column. Dependent index: Stores the index of the dependent columns of a specified column; The metadata also includes matrix metadata, which uses a data word layout format containing two arrays: data word offset and data word count, which store the starting offset and the number of data words occupied for each column, respectively.

2. The method for accelerating sparse LU decomposition in a flow-aware high-bandwidth memory FPGA according to claim 1, characterized in that, Customized FPGA hardware accelerators include channel allocation, control units, and hierarchical memory modules, among which... Regarding channel allocation: The HBM channel allocation strategy is adopted, and a total of 26 HBM channels are allocated for data transmission; For the control unit: a scheduler was designed, which includes a data prefetcher and a controller to coordinate the data and task flow of the accelerator; the data prefetcher prefetches the requested raw data through the matrix A loader to avoid unnecessary HBM access; at the same time, the controller obtains and organizes metadata from the metadata loader to form PE task packages; For the hierarchical memory module: it includes local memory and shared memory. The local memory is used to provide local access, and the shared memory is used to provide shared storage space. Both local memory and shared memory are managed using a first-in-first-out replacement strategy.

3. The method for accelerating sparse LU decomposition in a flow-aware high-bandwidth memory FPGA according to claim 2, characterized in that, A multi-stage pipeline manages and processes multiple HBM data prefetch requests simultaneously, carefully dividing HBM access operations into multiple stages to achieve a high degree of temporal overlap.

4. The method for accelerating sparse LU decomposition in a flow-aware high-bandwidth memory FPGA according to claim 2, characterized in that, The scheduler employs a column index modulo strategy to distribute the task list of columns at the same level and their corresponding data streams to different PEGs for parallel processing. Within each PEG, PEs work together to resolve dependencies and perform MAC or DIV operations on the current column. The stream-aware synchronous scheduling strategy strictly follows the hierarchical processing order: ensuring that streaming computation at the next level can only be started after all column tasks at the current level have been processed.

5. The method for accelerating sparse LU decomposition in a flow-aware high-bandwidth memory FPGA according to claim 4, characterized in that, The MAC operation includes: each PE receives multiply-accumulate input data packets and performs multiply-accumulate operations with sparse LU decomposition; the PE performs parallel multiply-accumulate operations using 8 floating-point values ​​and their packing elements through vectorization processing; floating-point multiplication and addition operations utilize the digital signal processing resources inside the FPGA and adopt a deep pipeline design to maximize resource utilization. Through internal loop unrolling optimization, an ideal startup interval of 1 is achieved.

6. The method for accelerating sparse LU decomposition in a flow-aware high-bandwidth memory FPGA according to claim 4, characterized in that, The DIV operation includes: after the MAC phase is complete, the PE within PEG will collaboratively process and normalize the diagonal elements of the current column; this operation uses a pipelined floating-point divider to effectively optimize its inherent high latency characteristics, and also processes diagonal elements close to zero to ensure numerical stability; after all multiply-accumulate operations are completed, the diagonal elements A of the current column j are normalized. jj Change to U jj Then, the processing unit in the division unit will process all elements A below the diagonal in column j. rj Parallel division by U jj To produce the final L factor L rj , where r > j.

Images