An FPGA Superposition Processor Acceleration System and Method Based on State-Space Duality

Abstract

The present invention belongs to the field of machine learning, and discloses an FPGA overlay processor acceleration system and method based on state space duality, including modules such as a sparse predefined data acquirer, a reconfigurable systolic array, a partial sum cache, an element-wise operation cache, function calculation, and on-chip memory management. The sparse predefined data acquirer eliminates zero elements to reduce redundant data, significantly reducing redundant calculations; the reconfigurable systolic array flexibly supports multiple calculation modes, improving the utilization rate of hardware resources; the partial sum cache realizes cross-cycle accumulation, the element-wise operation cache integrates the results, the function calculation module completes non-linear operations, and the on-chip memory management module optimizes the result cache to ensure the on-chip implementation of SSD calculations, significantly reducing off-chip memory access, improving inference efficiency and energy efficiency; using this system effectively reduces memory occupancy, improves element-wise calculation efficiency, reduces sparse calculation redundancy, and thus significantly accelerates the inference process of the Mamba2 model.

Description

Technical Field

[0001] The present invention belongs to the technical field of machine learning, and in particular relates to an FPGA stacking processor acceleration system and method based on state-space duality. Background Art

[0002] In today's digital age, the field of large-scale language models is developing rapidly. Models such as the Transformer have been widely used in a wide range of scenarios, including intelligent assistants, content creation, and personalized recommendations. They demonstrate excellent performance in processing long-distance dependencies in sequential data. With the rise of edge computing, higher requirements are being placed on the inference performance of models in resource-constrained environments. Against this backdrop, the Mamba model, based on a state-space model, emerged. It achieves near-linear computational complexity through parallel scanning and loop computation, making it suitable for resource-constrained deployments. The subsequent Mamba2 model introduced state-space duality (SSD), further improving computational efficiency on parallel hardware such as GPUs. This model has attracted widespread attention within the industry, providing a new solution for processing long-sequence data.

[0003] However, the Mamba2 model has exposed a series of serious issues in actual deployment. These issues are particularly evident in the following areas: Regarding memory usage, the SSD mechanism uses a large number of broadcast operations to expand small tensors into larger ones for element-by-element multiplication. This significantly increases memory usage and bandwidth requirements during inference. In dual-attention mode, memory requirements reach 41.55GB, and in loop mode, 10.5GB, placing significant strain on memory resources. Regarding hardware performance, mainstream GPUs and traditional FPGA accelerators are designed for tensor compression and dense matrix computation, and lack support for element-by-element computation after broadcast expansion. Under the SSD mechanism, element-by-element operations account for 83.88% of the overall computational latency during inference, significantly slowing down inference speed. Computational redundancy is also a prominent issue. SSD computations involve numerous sparse matrix multiplications, and traditional computing units are unable to effectively skip zero-valued regions within the sparse matrices, resulting in a large number of redundant operations that significantly impact overall computational performance. Consequently, traditional sparse optimization and memory optimization methods suffer from memory usage, computational efficiency, and redundant computation, making them difficult to adapt to the unique structure of the Mamba model.

[0004] It can be seen that the existing hardware platform has problems such as huge memory overhead, low efficiency of element-by-element calculation, and serious redundancy in sparse calculation. Summary of the Invention

[0005] The present invention provides an FPGA stacking processor acceleration system and method based on state-space duality. The system reduces memory overhead during state-space duality calculations, improves computing efficiency, inference speed, and energy efficiency.

[0006] To achieve the above object, the present invention adopts the following technical solutions:

[0007] An FPGA superposition processor acceleration system based on state - space duality, comprising:

[0008] A sparse predefined data acquirer module, configured to generate a valid data access address sequence according to a predefined sparse pattern, and extract valid data according to the valid data access address sequence; wherein, the valid data includes input data and weights with zero elements removed;

[0009] A reconfigurable systolic array module, configured to receive the valid data transmitted by the sparse predefined data acquirer module, and perform dense matrix operations, sparse matrix operations, element - by - element operations, and segmented multiplication operations on the valid data;

[0010] A partial sum cache module, configured to store intermediate calculation results after the reconfigurable systolic array module performs dense matrix operations, sparse matrix operations, and segmented multiplication operations, and perform cross - cycle accumulation operations on the intermediate calculation results;

[0011] An element - by - element operation cache module, configured to receive the element - by - element operation results after the reconfigurable systolic array module performs element - by - element operations and the accumulated intermediate calculation results after the cross - cycle accumulation operations of the partial sum cache module;

[0012] A function calculation module, configured to perform non - linear operations on the element - by - element operation results and the accumulated intermediate calculation results transmitted by the element - by - element operation cache module to obtain non - linear operation results;

[0013] An on - chip memory management module, configured to directly cache the non - linear operation results output by the function calculation module into the FPGA internal cache for subsequent calculations or transmit them back to the DDR memory as the final results.

[0014] Further, it further includes an input cache and weight cache module, configured to provide access addresses corresponding to the input data and weights to be calculated;

[0015] The sparse predefined data acquirer module extracts valid data from the access addresses of the input cache and weight cache module according to the valid data access address sequence.

[0016] Further, it further includes an instruction control unit module, configured to receive an instruction sequence transmitted by the host CPU, and control the sparse predefined data acquirer module, the reconfigurable systolic array module, the partial sum cache module, the element - by - element operation cache module, the function calculation module, and the on - chip memory management module to work together according to the instruction sequence.

[0017] Further, the sparse predefined data acquirer module includes an address sequence generator, a sparse data register group, and a distributed queue unit;

[0018] The address sequence generator generates a valid data access address sequence according to a predefined sparse pattern to eliminate zero-element data; extracts valid data according to the valid data access address sequence, and stores the valid data in the sparse data register bank; and transmits the valid data to the reconfigurable systolic array module through the distributed queue unit.

[0019] Furthermore, the reconfigurable systolic array module includes a plurality of cascaded processing units; each processing unit includes an input selection multiplexer, a data input register, a control logic unit, a matrix multiply-accumulate unit, and an element-wise calculation unit.

[0020] Furthermore, there are bidirectional connections both between the element-wise operation cache module and the reconfigurable systolic array module and between the element-wise operation cache module and the function calculation module.

[0021] A control method for an FPGA overlay processor acceleration system based on state space duality, based on the above-mentioned FPGA overlay processor acceleration system based on state space duality, includes:

[0022] Through the sparse predefined data acquirer module, a valid data access address sequence is generated according to a predefined sparse pattern, and valid data is extracted according to the valid data access address sequence; wherein, the valid data includes the input data and weights with zero elements eliminated.

[0023] Through the reconfigurable systolic array module, receive the valid data transmitted by the sparse predefined data acquirer module, and perform dense matrix operations, sparse matrix operations, element-wise operations, and segmented multiplication operations on the valid data.

[0024] Store the intermediate calculation results after the reconfigurable systolic array module performs dense matrix operations, sparse matrix operations, and segmented multiplication operations through the partial sum cache module, and perform cross-cycle accumulation operations on the intermediate calculation results.

[0025] Through the element-wise operation cache module, receive the element-wise operation results after the reconfigurable systolic array module performs element-wise operations and the accumulated intermediate calculation results after the cross-cycle accumulation operations of the partial sum cache module.

[0026] Through the function calculation module, perform non-linear operations on the element-wise operation results and the accumulated intermediate calculation results transmitted by the element-wise operation cache module to obtain non-linear operation results.

[0027] Through the on-chip memory management module, directly cache the non-linear operation results output by the function calculation module into the FPGA internal cache for subsequent calculations or return them to the DDR memory as the final results.

[0028] Further, before the valid data access address sequence is generated according to the predefined sparse pattern by the sparse predefined data acquirer module and the valid data is extracted according to the valid data access address sequence, it further includes:

[0029] Based on the data flow graph formed by each operator in the state space duality calculation process, the state space duality calculation process is decomposed into multiple basic operator sequences that can be parallelized and reordered.

[0030] Further, during the process of the reconfigurable systolic array module receiving the valid data transmitted by the sparse predefined data acquirer module and performing dense matrix operations, sparse matrix operations, element-wise operations, and segmented multiplication operations on the valid data, it further includes:

[0031] Using the operation fusion framework, the broadcast matrix multiplication and the summation operation are combined into a vector matrix multiplication operation; the segmented multiplication in the causal matrix generation is moved to the element-wise operation stage;

[0032] Select the calculation mode according to the operator type and generate the corresponding instruction sequence for the reconfigurable systolic array module to execute the corresponding operator operation.

[0033] Further,

[0034] When the operator type is a dense matrix operation, the general matrix multiplication is adopted to optimize the data block reading through the address mapping list;

[0035] When the operator type is a sparse matrix operation, the sparse matrix multiplication is adopted to eliminate the redundant calculation of zero elements by using the tensor rearrangement algorithm.

[0036] Compared with the prior art, the present invention has the following beneficial effects:

[0037] The present invention provides an FPGA overlay processor acceleration system based on state - space duality, including modules such as a sparse predefined data acquirer, a reconfigurable systolic array, a partial - sum cache, an element - by - element operation cache, function calculation, and on - chip memory management. The sparse predefined data acquirer eliminates zero elements to reduce redundant data, significantly reducing redundant calculations; the reconfigurable systolic array flexibly supports multiple calculation modes, improving the utilization rate of hardware resources; the partial - sum cache realizes cross - cycle accumulation, the element - by - element operation cache integrates results, the function calculation module completes non - linear operations, and the on - chip memory management module optimizes the result cache, ensuring the on - chip implementation of SSD calculations, significantly reducing off - chip memory access, and improving the inference efficiency and energy efficiency; this system addresses the problems of large memory occupation, low element - by - element calculation efficiency, and sparse calculation redundancy in the Mamba2 model. By reducing redundant data transmission and calculations, optimizing memory access, and leveraging the reconfigurable characteristics of FPGAs to efficiently execute various operations, the system effectively reduces memory occupation, improves element - by - element calculation efficiency, reduces sparse calculation redundancy, and thus significantly accelerates the inference process of the Mamba2 model.

[0038] Preferably, in the present invention, by adding an input cache and a weight cache module, a unified source of input data and weight access addresses is provided for the system, enabling the sparse predefined data acquirer module to more efficiently extract valid data from a fixed location, avoiding data source confusion and additional address lookup operations, further improving the efficiency and accuracy of data extraction, and contributing to the more stable and rapid operation of the entire acceleration system.

[0039] Preferably, in the present invention, an instruction control unit module is introduced. By receiving the instruction sequence transmitted by the host CPU, it realizes the coordinated control of each module in the system. This enables the entire acceleration system to work according to a predetermined order and logic, avoiding conflicts and confusion between modules, improving the overall operation efficiency and stability of the system, and ensuring that complex calculation tasks can be completed accurately and efficiently.

[0040] Preferably, in the present invention, the sparse predefined data acquirer module specifically includes an address sequence generator, a sparse data register group, and a distributed queue unit. The address sequence generator can accurately generate an effective data access address sequence according to the predefined sparse pattern, effectively eliminating zero - element data; the sparse data register group provides a dedicated storage space for valid data for subsequent processing; the distributed queue unit realizes the efficient transmission of valid data. These components work together, enabling the sparse predefined data acquirer module to more quickly and accurately extract valid data, providing strong support for subsequent calculations.

[0041] Preferably, in the present invention, the reconfigurable systolic array module includes a number of cascaded processing units. The number of cascaded processing units provides powerful computing capabilities and can process multiple data simultaneously; the input selection multiplexer realizes flexible selection and input of data; the data input register temporarily stores the input data to ensure the stability of data processing; the control logic unit precisely controls the operation of the processing unit; the matrix multiply-accumulate unit and the element-wise calculation unit respectively implement dense matrix operations and element-wise operations. These designs enable the reconfigurable systolic array module to efficiently and flexibly complete various computing tasks.

[0042] Preferably, in the present invention, the two-way connection mode of the element-wise operation cache module with the reconfigurable systolic array module and the function calculation module. This two-way connection enables the element-wise operation cache module to simultaneously receive the element-wise operation results from the reconfigurable systolic array module and the accumulated intermediate calculation results from the partial sum cache module, and transmit these results to the function calculation module in a timely manner for non-linear operations. At the same time, the function calculation module can also feedback the calculation results to the element-wise operation cache module to achieve flexible data interaction and sharing, improving the overall computing efficiency and flexibility of the system.

[0043] The present invention also provides a control method for the FPGA overlay processor acceleration system based on state-space duality. Based on the above acceleration system, this method eliminates zero elements through a sparse predefined data acquirer to reduce the data volume, the reconfigurable systolic array flexibly executes various matrix operations, the partial sum cache realizes cross-cycle accumulation of intermediate results, the element-wise operation cache integrates element-wise operations and accumulated results, the function calculation module completes non-linear operations, and the on-chip memory management module optimizes the result cache. Aiming at the problems of large memory occupation, low element-wise calculation efficiency, and sparse calculation redundancy in the Mamba2 model, by reducing redundant data and calculations, optimizing memory access, and utilizing the reconfigurable characteristics of the FPGA to improve the calculation efficiency. Using this method significantly reduces the memory occupation, improves the element-wise calculation efficiency, reduces the sparse calculation redundancy, thus greatly accelerating the inference process of the Mamba2 model and effectively solving the key problems faced by existing hardware platforms.

[0044] Preferably, in the present invention, before the sparse predefined data acquirer module extracts the valid data, a decomposition step of the data flow graph in the state-space duality calculation process is added. The beneficial effect is that the duality calculation process is decomposed into multiple basic operator sequences that can be parallelized and reordered, providing a basis for subsequent parallel computing and operator reordering, helping to improve the computing efficiency and flexibility of the system, enabling the system to more efficiently utilize hardware resources, and further enhancing the overall performance.

[0045] Preferably, in the present invention, an operation fusion framework is used to combine broadcast matrix multiplication and summation operations into vector matrix multiplication operations, reducing the number of calculation steps and latency; the segmented multiplication in causal matrix generation is moved to the element-wise operation stage, optimizing the calculation process; the calculation mode is selected according to the operator type and the corresponding instruction sequence is generated, enabling the reconfigurable systolic array module to execute the corresponding operator operations more precisely, improving the calculation accuracy and efficiency, enabling the system to better adapt to different calculation tasks, and giving full play to the hardware advantages of the FPGA.

[0046] Preferably, in the present invention, for dense matrix operations, general matrix multiplication is adopted and data block reading is optimized through an address mapping list, which can improve data reading efficiency and reduce memory access time; for sparse matrix operations, sparse matrix multiplication is adopted and the tensor rearrangement algorithm is used to eliminate redundant zero element calculations, reducing the amount of calculation and memory occupancy. These optimization methods further improve the calculation performance and resource utilization rate of the system, enabling the system to process various types of calculation tasks more efficiently, and effectively solving the problems of the Mamba2 model in terms of memory and calculation efficiency. Brief Description of the Drawings

[0047] Figure 1 It is a schematic diagram of the software stack and hardware microarchitecture of the FPGA overlay processor acceleration system based on state space duality provided by the embodiment of the present invention;

[0048] Figure 2 It is a schematic diagram of the structure of the sparse predefined data acquirer and processing unit provided by the present invention;

[0049] Figure 3 It is a schematic diagram of the on-chip cache management and data interaction mechanism provided by the present invention;

[0050] Figure 4 It is a detailed schematic diagram of the calculation process of sparse matrix multiplication provided by the present invention;

[0051] Figure 5 It is a detailed schematic diagram of the calculation process of segmented multiplication provided by the present invention;

[0052] Figure 6 It is a schematic diagram of the structure of an FPGA overlay processor acceleration system based on state space duality provided by the present invention. Detailed Embodiments

[0053] To further understand the content of the present invention, the following provides a detailed description of the present invention in combination with the accompanying drawings and specific embodiments. It should be understood that the embodiments are only for explaining the present invention and not for limiting it.

[0054] The following explains the technical terms involved in the present invention:

[0055] FPGA (Field-Programmable Gate Array), a field-programmable gate array, is an integrated circuit that can be programmed to configure hardware logic and is commonly used in high-performance computing and hardware acceleration fields.

[0056] DDR (Double Data Rate), double data rate, is a random access memory (RAM) technology widely used in computers and electronic devices for high-speed data transmission.

[0057] SSD (State Space Duality), state space duality, is a key technology introduced in the Mamba2 model to improve the computational efficiency on parallel hardware such as GPUs.

[0058] GPU (Graphics Processing Unit), a graphics processing unit, was originally used for graphics rendering and is now widely used in parallel computing, artificial intelligence, etc. fields to accelerate large-scale data processing.

[0059] Mamba is a machine learning model, specifically a sequence data processing model based on the state space model, with near-linear computational complexity and suitable for inference in resource-constrained environments.

[0060] Mamba2 is an improved version of the Mamba model, introducing state space duality (SSD) to further improve computational efficiency.

[0061] CPU (Central Processing Unit) is the arithmetic and control core of a computer manufactured using very large scale integrated circuits.

[0062] ASG (Address Sequence Generator), an address sequence generator, is used to generate a valid data access address sequence according to a predefined sparse pattern, thereby eliminating zero-element data, and then extracting valid data according to this sequence and storing it in the sparse data register bank.

[0063] BRAM (Block Random Access Memory), that is, block random access memory, exists in units of blocks and has an independent storage structure and read / write ports.

[0064] The PCIe (Peripheral Component Interconnect Express) interface is a high-speed serial computer expansion bus standard used to connect various hardware devices inside a computer, such as graphics cards, solid-state drives, network cards, sound cards, etc.

[0065] As described in the background art, traditional sparse optimization strategies such as pruning, quantization, and memory optimization methods (model partitioning, computational graph fusion, and data flow optimization) cannot be directly and effectively applied to the unique structure of the Mamba model. Therefore, there is an urgent need to design a brand-new acceleration optimization scheme that can simultaneously solve the problems of high memory occupancy, low per-element calculation performance, and sparse calculation redundancy, and achieve efficient acceleration of the Mamba2 model. It can be seen that the existing hardware platforms have problems such as huge memory overhead, low per-element calculation efficiency, and serious sparse calculation redundancy.

[0066] To solve the above problems, as Figure 6 shown, this embodiment provides an FPGA stacked processor acceleration system based on state-space duality, including a sparse predefined data acquirer module for generating a valid data access address sequence according to a predefined sparse pattern and extracting valid data according to the valid data access address sequence. The valid data includes input data and weights with zero elements removed; a reconfigurable systolic array module for receiving the valid data transmitted by the sparse predefined data acquirer module and performing dense matrix operations, sparse matrix operations, per-element operations, and segmented multiplication operations on the valid data; a partial sum cache module for storing the intermediate calculation results after the reconfigurable systolic array module performs dense matrix operations, sparse matrix operations, and segmented multiplication operations and performing cross-cycle accumulation operations on the intermediate calculation results; a per-element operation cache module for receiving the per-element operation results after the reconfigurable systolic array module performs per-element operations and the accumulated intermediate calculation results after the cross-cycle accumulation operations of the partial sum cache module; a function calculation module for performing non-linear operations on the per-element operation results and the accumulated intermediate calculation results transmitted by the per-element operation cache module to obtain non-linear operation results; and an on-chip memory management module for directly caching the non-linear operation results output by the function calculation module into the FPGA internal cache for subsequent calculations or transmitting them back to the DDR memory as the final results.

[0067] Exemplarily, this embodiment provides an FPGA stacked processor acceleration system based on state-space duality, which simultaneously improves two parts: the software stack and the hardware microarchitecture. The specific implementation of this acceleration system includes:

[0068] The hardware microarchitecture of this FPGA stacked processor acceleration system includes: an instruction control unit module, an input cache and weight cache module, a sparse predefined data acquirer module, a reconfigurable systolic array module, a partial sum cache module, a per-element operation cache module, a function calculation module, and an on-chip memory management module. The specific structure and functions are as follows:

[0069] Instruction control unit module:

[0070] Composition: It includes an instruction decoding unit, an instruction cache unit, and a timing control logic.

[0071] Function: Receive the instruction sequence generated by the host CPU compilation and control the coordinated operation of each module of the entire processor system.

[0072] Input cache and weight cache module:

[0073] Composition: The input cache and the weight cache are respectively composed of multiple high-speed on-chip storage units.

[0074] Connection: They are respectively connected to the sparse predefined data fetcher module to provide the data and weights to be calculated.

[0075] Function: Provide fast data and weight access to meet the high-bandwidth and low-latency computing requirements.

[0076] Sparse predefined data fetcher module:

[0077] Composition: Address sequence generator (ASG), sparse data register bank, distributed queue unit.

[0078] Connection: Input cache and weight cache module and reconfigurable systolic array module.

[0079] Function: Generate an address sequence according to the predefined sparse pattern, exclude zero-element data loading in advance, thereby precisely controlling the transmission of sparse matrix data and minimizing redundant operations to the greatest extent.

[0080] Reconfigurable systolic array module:

[0081] This module is composed of several processing elements (PEs) cascaded. The structure of each processing element is as follows:

[0082] Composition of processing element (PE): It is composed of an input selection multiplexer, a data input register, a control logic unit, and a computing unit (including a matrix multiply-accumulate unit MAC and an element-wise computing unit).

[0083] Connection method: The PEs are cascaded through horizontal and vertical data paths to support efficient cross-PE data transmission; at the same time, each PE unit is connected to the sparse predefined data fetcher module to achieve precise data loading; the output end of the PE is connected to the partial sum cache module and the element-wise operation cache module to store the intermediate results of each stage of calculation.

[0084] Function and Computational Mode Support: The processing unit supports General Matrix Multiplication (GEMM) for dense matrix operations, Sparse Matrix Multiplication (SpMM) to improve computational efficiency by dynamically skipping zero elements, element-wise operations to achieve efficient parallel point-wise calculations, and Segment-Mul to efficiently perform causal masking-related operations.

[0085] Partial Sum Cache Module:

[0086] Composition: A multi-level cache register array.

[0087] Connection: Connected to the output of the reconfigurable systolic array module and the element-wise operation cache module.

[0088] Function: Stores intermediate results during calculations, supports cross-cycle accumulation for multiple rounds of calculations, and ensures efficient and stable partial sum operations.

[0089] Element-Wise Operation Cache Module

[0090] Composition: Two dedicated high-speed BRAM cache arrays (EleOps Buffer A, B).

[0091] Connection: This module is bidirectionally connected to the reconfigurable systolic array module and the function calculation module (SFU module), thus supporting efficient interaction for element-wise operations.

[0092] Function: Provides high-bandwidth caches required for element-wise calculations, ensuring efficient element-wise data transfer.

[0093] Function Calculation Module:

[0094] Composition: A pipelined calculation structure, including exponential function (EXP), activation function (SiLU), normalization calculation unit (RMSNorm), etc.; the above functions are all dedicated functions.

[0095] Connection: Connected to the element-wise operation cache module and the on-chip memory management module respectively.

[0096] Function: Efficiently completes non-linear operations in the SSD algorithm, significantly reducing the calculation latency of non-linear functions.

[0097] On-Chip Memory Management Module:

[0098] Composition: This module consists of a width-first buffer unit, a depth-first cache unit, and data interaction control logic.

[0099] Connection: Connected to the element-wise cache module, SFU module, and systolic array module.

[0100] Function: Through the intelligent allocation of caches, the SSD calculation process is fully executed within the on-chip cache, reducing the number of off-chip memory accesses and improving the utilization rate of data transfer bandwidth.

[0101] Exemplarily, based on the hardware microarchitecture of the above FPGA overlay processor acceleration system, the overall data flow and signal transmission relationship of this system are as follows:

[0102] Data input process: Data is read by the input cache and weight cache modules and passed into the reconfigurable systolic array module through the sparse predefined data acquirer module for sparse calculation.

[0103] Calculation process: After the reconfigurable systolic array module completes the GEMM operation, SpMM operation, element-wise operation, and Segment-mul operation respectively, it outputs partial results to the partial sum cache module or the element-wise operation cache module. The results of the element-wise operation cache module are further passed into the function calculation module for non-linear operation. Among them, the calculation results received by the partial sum cache module are the intermediate calculation results after the reconfigurable systolic array module executes dense matrix operations, sparse matrix operations, and segmented multiplication operations, and perform cross-cycle accumulation operations on the intermediate calculation results; the element-wise operation cache module receives the element-wise operation results after the reconfigurable systolic array module executes the element-wise operation and the accumulated intermediate calculation results after the cross-cycle accumulation operation of the partial sum cache module.

[0104] Result output process: The final result processed by the function calculation module is directly cached on the chip through the on-chip memory management module, waiting for the next stage of calculation or transmitted back to the DDR memory through a dedicated path.

[0105] Exemplarily, in order to further improve the performance of this system, this embodiment also performs optimization and improvement on the software stack, that is, provides a control method for the FPGA overlay processor acceleration system based on state space duality, specifically as follows:

[0106] The software stack design provided in this embodiment is dedicated to reducing the memory overhead in the SSD calculation process and improving the resource utilization efficiency of the hardware microarchitecture. The software stack runs entirely on the host CPU and mainly consists of the following key steps:

[0107] First, Data-flow Graph Unrolling:

[0108] In order to efficiently map the Mamba model to the FPGA hardware platform, in this embodiment, first, the data flow graph (DFG) composed of each operator in the SSD calculation is analyzed and unfolded, and the complex SSD calculation is divided into several basic operator sequences that can be parallelized and reordered, so as to perform more efficient hardware mapping subsequently.

[0109] Second, adopt the Operator Fusion framework:

[0110] In SSD calculations, there are a large number of broadcast element-wise multiplication operations, causing huge memory pressure. To alleviate this problem, this embodiment proposes a software-hardware collaborative Operator Fusion framework.

[0111] Exemplarily, the specific fusion strategy includes the following two important aspects:

[0112] (1) Operator Merge: The broadcast matrix multiplication and the subsequent summation operation can essentially be uniformly represented as a simpler linear operator calculation. This embodiment proposes to fuse the two into a unified descriptor and express it through vector matrix multiplication. In this way, through Operator Merge, the original two-step operation (broadcast multiplication and summation) is unified into a single matrix multiplication operation, effectively shortening the calculation path and significantly reducing the memory overhead of the intermediate feature tensor.

[0113] (2) Operator Backward Shift: In the calculation of State Space Duality (SSD), the generation of the Causal Matrix M is involved, usually implemented through Segment-mul, which involves segment summation and exponentiation operations. In this method, by utilizing the properties of the SSD calculation itself, some operations are executed in advance, that is, the segment multiplication is embedded into the subsequent element-wise operation process, thereby shortening the overall calculation path. The implementation of Operator Backward Shift effectively advances this segment calculation in advance, reducing the memory burden and latency of the complex calculation stage.

[0114] Third, adopt the Data Allocation algorithm:

[0115] This embodiment also proposes a set of efficient data allocation algorithms to support the tile-granularity multiply-accumulate (MAC) calculation of matrix operations.

[0116] Specifically, it includes two calculation modes:

[0117] (1) GEMM mode (General Matrix Multiplication): The GEMM mode supports linear matrix multiplication and the fused broadcast MAC calculation. By precisely defining the address mapping list before and after the broadcast through software, the required data blocks are efficiently read every cycle and sent to the systolic array for vectorized MAC calculation after reordering, thereby significantly reducing the memory occupancy.

[0118] (2)SpMM mode (Sparse Matrix Multiplication): To address the redundancy issue in sparse computing, this embodiment also designs a set of Tensor-reorder-and-group transformation algorithms. First, a zero-element mask is applied to the sparse matrix to eliminate the invalid calculations of zero elements. Then, the non-zero element data is regrouped and sorted in a compact manner to facilitate efficient loading into the systolic array computing unit, greatly reducing redundant calculations.

[0119] Fourth, Computation Mode Selection:

[0120] The software stack of this embodiment integrates high-level instructions through the compiler, supporting flexible and diverse computation mode selections, including but not limited to:

[0121] A. Load / Store mode: Responsible for data transfer between DDR and on-chip cache;

[0122] B. Fetch mode: Scheduling data to adapt to systolic array computing;

[0123] C. GEMM mode: General Matrix Multiplication;

[0124] D. SpMM mode: Sparse Matrix Multiplication;

[0125] E. SegMul mode: Piecewise multiplication of causal masks;

[0126] F. EleOps mode: Element-wise calculation;

[0127] G. SFU mode: Non-linear function calculation, such as EXP, SiLU, RMSNorm, etc.

[0128] By reasonably selecting the above different computation modes, this control method can be optimized and configured according to the requirements of different computation stages, thereby further significantly improving the utilization rate of hardware computing resources.

[0129] In summary, the FPGA overlay processor acceleration system based on state-space duality provided in this embodiment has the following advantages:

[0130] In terms of hardware microarchitecture:

[0131] First, efficient sparse computing: The sparse data predefined module effectively skips zero elements, significantly reducing redundant calculations;

[0132] Second, flexible multi-mode computing: The reconfigurable systolic array supports multiple computation modes through dynamic configuration, improving the utilization rate of hardware resources;

[0133] Third, on-chip cache optimization: An intelligent cache management strategy to ensure on-chip SSD computing, significantly reducing off-chip memory access and improving inference efficiency and energy efficiency;

[0134] Fourth, efficient non-linear computing: An efficient implementation of the function computing module to further improve the overall computing performance.

[0135] In terms of software stack:

[0136] First, significantly reduce memory overhead: Through the operation fusion framework and precise data allocation, greatly reduce the amount of intermediate tensor data and lower the memory bandwidth requirements;

[0137] Second, improve computing efficiency: Data rearrangement and sparse computing mode selection significantly reduce redundant zero-element calculations and improve the effective utilization rate of hardware resources;

[0138] Third, improve inference speed and energy efficiency: The software scheduling strategy precisely controls the computing process to ensure the efficient operation of FPGA hardware units, effectively improving the overall inference throughput and energy efficiency.

[0139] Next, for the FPGA overlay processor acceleration system based on state space duality provided in this embodiment, specific implementation will be carried out and described in conjunction with the accompanying drawings:

[0140] As Figure 1 shown, it is a schematic diagram of the software stack and hardware microarchitecture of this system; Figure 1 On the left side in the figure shows the step composition of the software stack of the FPGA overlay processor acceleration system (MambaOPU), including key steps such as data flow graph expansion, operation fusion, data allocation, array mode selection, memory scheduling, and event generation. Data and instructions are sent from the host CPU to the acceleration system through the PCIe interface. On the right side shows the hardware microarchitecture composition of the FPGA overlay processor acceleration system, including an instruction control unit module, an input cache and weight cache module, a sparse predefined data acquirer module, a reconfigurable systolic array module, a partial sum cache module, an element-wise operation cache module, a function computing module, and an on-chip memory management module. Each module is interconnected through an efficient data path to achieve the efficient execution of SSD computing.

[0141] As Figure 2 shown, Figure 2 on the left side of [figure number] is the sparse predefined data acquirer module, whose core consists of an address sequence generator, a register bank, and a distributed queue, precisely controlling the data flow by generating folded indexes to support different computing modes (including sparse computing in the Sa1, Sa3, and Sa5 stages). Figure 2On the right is a detailed structural diagram of the processing element PE, showing the input and output signals of the multiplexer switch, data register, and computing and control logic of the PE (including the current input CUR IN, weight input W IN, element-wise multiplication ELE MUL, cumulative multiplication CAS MUL, element-wise addition ELE ADD, cumulative calculation CAS MAC, bypass signal BYPASS, and the next input NEXT IN), reflecting the high flexibility of the processing element and its ability to support multiple computing modes.

[0142] As Figure 3 shown, it clearly describes the data interaction process during the fused computing and element-wise computing stages. The fused computing results are efficiently stored in the URAM-based cache and can further interact seamlessly with the element-wise cache. The left part of the figure shows the data interaction strategies between fused computing and element-wise computing, including four cases: fused to fused, fused to element-wise, element-wise to element-wise, and element-wise to fused, to optimize the on-chip data flow and minimize off-chip memory access; Figure 3 The right part shows the access strategies and timelines of the on-chip cache and off-chip cache during different computing stages (such as Sa1 - Sa6, Sr1 - Sr8, So1 - So4) in SSD computing, reflecting the efficient data management scheme of this system and ensuring that the SSD computing process is always efficiently completed in the on-chip cache.

[0143] As Figure 4 shown, it describes the calculation process of sparse matrix multiplication (SpMM), showing the specific operation process for tensors B ∈ R(1×3×3) and tensor C ∈ R(3×3×1). In each computing cycle, each processing element (PE) loads two elements into register REG2 through the current input port and one element into register REG1 through the weight input port. Under the control of the multiplexer, the current computing unit PE[i,j] extracts a specific element from REG2 and performs a multiply-accumulate operation with the loaded element b_ij. The resulting partial product is passed vertically down to the next row of processing elements PE[i + 1,j] in the subsequent cycle. At the same time, the input elements pre-stored in register REG2 are passed horizontally to the adjacent processing element PE[i,j + 1]. Each processing element (PE) alternates the above calculations until all partial accumulations are completed. The final calculation result is output by the PE units in the last row.

[0144] As Figure 5As shown, each processing unit loads the causal vector elements {1, ai} after exponential operation processing into register REG2 through the current input port. At the same time, the masked intermediate activation value {b} is loaded into register REG1 through the weight input port for performing element-wise multiplication. The partial product results generated inside the PE unit PE[i, j] are stored back into register REG2 or register REG3 according to specific requirements:

[0145] If stored in register REG2, this result will be multiplied by the element bij newly loaded into the weight input port in the next calculation cycle;

[0146] If stored in register REG3, this result is passed downward through the CAS_MUL path to the next row of PE units PE[i + 1, j] for cumulative multiplication with the element a_(i, j + 1).

[0147] When each PE completes the calculation at a specific position, its calculation result is output to an external register through a bypass channel (BYPASS channel) controlled by control signal CTRL3 and control signal CTRL4.

[0148] The above embodiments are merely one of the implementation manners capable of implementing the technical solution of the present invention. The scope of protection required by the present invention is not limited solely by this embodiment, but also includes any changes, substitutions, and other implementation manners that are easily conceivable by those skilled in the art within the technical scope disclosed by the present invention.

[0149] Finally, it should be noted that: the above embodiments are only used to illustrate the technical solution of the present invention and are not intended to limit it. Although the present invention has been described in detail with reference to the above embodiments, those of ordinary skill in the art should understand that: it is still possible to modify the specific implementation manners of the present invention or make equivalent substitutions, and any modification or equivalent substitution that does not depart from the spirit and scope of the present invention should be covered within the scope of the claims of the present invention.

Claims

1. An FPGA superposition processor acceleration system based on state space duality, characterized in that Including: A sparse predefined data acquirer module, which is used to generate a sequence of valid data access addresses according to a predefined sparse pattern and extract valid data according to the sequence of valid data access addresses; wherein the valid data includes input data and weights from which zero elements have been removed. A reconfigurable systolic array module, which is used to receive the valid data transmitted by the sparse predefined data acquirer module and perform dense matrix operations, sparse matrix operations, element-wise operations, and segmented multiplication operations on the valid data; during the process of receiving the valid data transmitted by the sparse predefined data acquirer module and performing dense matrix operations, sparse matrix operations, element-wise operations, and segmented multiplication operations on the valid data, it also includes: Using an operation fusion framework to combine broadcast matrix multiplication and summation operations into vector matrix multiplication operations; moving the segmented multiplication in causal matrix generation to the element-wise operation stage. Selecting a calculation mode according to the operator type and generating a corresponding instruction sequence for the reconfigurable systolic array module to execute the corresponding operator operation. A partial sum cache module, which is used to store the intermediate calculation results after the reconfigurable systolic array module performs dense matrix operations, sparse matrix operations, and segmented multiplication operations, and perform cross-cycle accumulation operations on the intermediate calculation results. An element-wise operation cache module, which is used to receive the element-wise operation results after the reconfigurable systolic array module performs element-wise operations and the accumulated intermediate calculation results after the cross-cycle accumulation operations of the partial sum cache module. A function calculation module, which is used to perform non-linear operations on the element-wise operation results and the accumulated intermediate calculation results transmitted by the element-wise operation cache module to obtain non-linear operation results. An on-chip memory management module, which is used to directly cache the non-linear operation results output by the function calculation module into the internal cache of the FPGA for subsequent calculations or return them to the DDR memory as the final results.

2. The FPGA overlay processor acceleration system based on state space duality according to claim 1, wherein It also includes an input cache and weight cache module, which is used to provide access addresses corresponding to the input data and weights to be calculated. The sparse predefined data acquirer module extracts valid data from the access addresses of the input cache and weight cache module according to the sequence of valid data access addresses.

3. A FPGA overlay processor acceleration system based on state space duality according to claim 1, characterized in that, It also includes an instruction control unit module, which is used to receive the instruction sequence transmitted by the host CPU to control the collaborative work of the sparse predefined data acquirer module, the reconfigurable systolic array module, the partial sum cache module, the element-wise operation cache module, the function calculation module, and the on-chip memory management module according to the instruction sequence.

4. The FPGA overlay processor acceleration system based on state space duality according to claim 1, wherein The sparse predefined data acquirer module includes an address sequence generator, a sparse data register group, and a distributed queue unit. The address sequence generator generates a sequence of valid data access addresses according to the predefined sparse pattern to remove zero-element data; extracts valid data according to the sequence of valid data access addresses and stores the valid data in the sparse data register group. Transmits the valid data to the reconfigurable systolic array module through the distributed queue unit.

5. The FPGA superposition processor acceleration system based on state space duality according to claim 1, characterized in that The reconfigurable systolic array module includes several cascaded processing units; each processing unit includes an input selection multiplexer, a data input register, a control logic unit, a matrix multiply-accumulate unit, and an element-wise calculation unit.

6. The FPGA overlay processor acceleration system based on state space duality according to claim 1, wherein A two-way connection is adopted between the per-element operation cache module and the reconfigurable systolic array module, and between the per-element operation cache module and the function calculation module.

7. A control method for an FPGA superposition processor acceleration system based on state space duality, characterized in that, The FPGA superposition processor acceleration system based on state space duality according to any one of claims 1-6, comprising: Through a sparse predefined data acquirer module, generating a valid data access address sequence according to a predefined sparse pattern, and extracting valid data according to the valid data access address sequence; wherein, the valid data includes input data and weights with zero elements removed. Through a reconfigurable systolic array module, receiving the valid data transmitted by the sparse predefined data acquirer module, and performing dense matrix operations, sparse matrix operations, per-element operations, and segmented multiplication operations on the valid data; during the process of receiving the valid data transmitted by the sparse predefined data acquirer module and performing dense matrix operations, sparse matrix operations, per-element operations, and segmented multiplication operations on the valid data, it further includes: Using an operation fusion framework to combine broadcast matrix multiplication and summation operations into vector matrix multiplication operations; moving the segmented multiplication in causal matrix generation to the per-element operation stage. Selecting a calculation mode according to the operator type and generating a corresponding instruction sequence for the reconfigurable systolic array module to execute the corresponding operator operation. Storing the intermediate calculation results after the reconfigurable systolic array module performs dense matrix operations, sparse matrix operations, and segmented multiplication operations through a partial sum cache module, and performing cross-cycle accumulation operations on the intermediate calculation results. Through a per-element operation cache module, receiving the per-element operation results after the reconfigurable systolic array module performs per-element operations and the accumulated intermediate calculation results after the cross-cycle accumulation operations of the partial sum cache module. Through a function calculation module, performing non-linear operations on the per-element operation results and the accumulated intermediate calculation results transmitted by the per-element operation cache module to obtain non-linear operation results. Through an on-chip memory management module, directly caching the non-linear operation results output by the function calculation module into the FPGA internal cache for subsequent calculations or transmitting them back to the DDR memory as the final results.

8. The control method of an FPGA overlay processor acceleration system based on state space duality according to claim 7, characterized in that, Before the step of generating a valid data access address sequence according to a predefined sparse pattern and extracting valid data according to the valid data access address sequence through the sparse predefined data acquirer module, it further includes: Based on the data flow graph formed by each operator in the state space duality calculation process, decomposing the state space duality calculation process into multiple basic operator sequences that can be parallelized and reordered.

9. The control method of an FPGA superposition processor acceleration system based on state space duality according to claim 7, characterized in that When the operator type is a dense matrix operation, general matrix multiplication is adopted to optimize data block reading through an address mapping list. When the operator type is a sparse matrix operation, sparse matrix multiplication is adopted to eliminate redundant zero element calculations using a tensor rearrangement algorithm.

Images