An FPGA large model HBM high-speed interconnection acceleration method based on bloom parallel attention

Abstract

The embodiment of the application discloses an FPGA large model HBM high-speed interconnection acceleration method based on Bloom parallel attention, the embodiment of the application decomposes the calculation process of Bloom parallel attention into multiple sub-modules with parallel potential, and designs FPGA hardware operators suitable for the burst transmission characteristics of HBM in a targeted manner. An HBM multi-channel data scheduling module is constructed, channel resources are dynamically allocated according to the real-time load of parallel operators, and the bandwidth utilization rate is optimized. The operator and the scheduling module are integrated into the FPGA on-chip network, the transmission paths of large and small data are distinguished through an intelligent routing strategy, and the network layout is optimized to reduce the delay. The attention acceleration modules of each layer are cascaded in a pipeline, and the overlap of interlayer calculation and parameter loading is realized by using the HBM prefetch mechanism. The application significantly improves the calculation speed and energy efficiency ratio of large model training and reasoning on the FPGA platform, and can be widely applied to intelligent speech recognition, automatic driving, natural language processing and other scenes that require efficient processing of large-scale data.

Description

Technical Field

[0001] This invention relates to the field of large model computing technology, specifically to a high-speed interconnect acceleration method for FPGA large model HBM based on Bloom parallel attention. Background Technology

[0002] With the rapid development of artificial intelligence technology, large models are widely used in fields such as natural language processing, image recognition, and autonomous driving. However, the computational complexity of large models is extremely high, posing severe challenges to hardware computing power, memory bandwidth, and energy efficiency. Traditional computing architectures, when processing large models, are often limited by data transmission bandwidth and the coordination efficiency between computing units, making it difficult to fully realize the computational potential of the models.

[0003] Currently, the main acceleration solutions for large model computation include the following:

[0004] CPU-based general computing methods: While CPUs possess versatility and flexibility, their multi-core parallel capabilities are limited, making it difficult to handle large-scale matrix operations and parallel attention computations. Furthermore, the low data transfer bandwidth between the CPU and external storage (such as HBM) can easily become a system bottleneck, failing to meet the high throughput and low latency requirements of large model training and inference.

[0005] GPU-based parallel acceleration solutions: GPUs are widely used in large model training due to their powerful parallel computing capabilities. However, GPU solutions suffer from high power consumption and high cost. Furthermore, the general-purpose parallel architecture of GPUs may not be fully adapted to certain specific model structures (such as Bloom parallel attention), leading to insufficient utilization of computing resources or even resource waste.

[0006] Simple FPGA-based porting solutions: Some studies attempt to directly map large-scale model computation tasks onto FPGAs, but these often lack in-depth optimization of the computation process and fail to fully leverage the reconfigurability of FPGAs and the high bandwidth advantage of HBM. These solutions typically do not perform parallel reconfiguration of computation modules or design efficient data scheduling mechanisms, resulting in limited overall acceleration.

[0007] Data scheduling methods for static or fixed channels:

[0008] Some existing FPGA acceleration solutions use a fixed HBM channel allocation strategy, which cannot be adjusted according to dynamic changes in computing load. This can easily lead to some channels being overloaded and having uneven bandwidth utilization, while other channels are idle, thus affecting the overall system performance. Summary of the Invention

[0009] To address this issue, this invention provides a high-speed interconnect acceleration method for large-scale FPGA models (HBM) based on Bloom parallel attention, thereby resolving the technical problem of inefficiency caused by the mismatch between computation and data transmission in existing technologies, which leads to computational units waiting for data.

[0010] To achieve the above objectives, the embodiments of the present invention provide the following technical solutions:

[0011] This invention provides a method for accelerating high-speed interconnection of large-scale HBM models in FPGAs based on Bloom parallel attention. The method includes:

[0012] The computational flow of Bloom parallel attention is decomposed into multiple parallel sub-modules, the parallel computation potential of each sub-module is marked, and the sub-modules that can be piped or multi-instance parallelized are identified.

[0013] For each submodule that can be processed in parallel, a corresponding FPGA hardware operator is designed and implemented, and the interface of the hardware operator is adapted to match the burst transmission characteristics of HBM to form a parallel operator.

[0014] A multi-channel HBM data scheduling module is constructed to dynamically allocate HBM channels according to the real-time computing load of each parallel operator, monitor the bandwidth utilization of each channel, and cyclically adjust the allocation strategy to match the real-time computing requirements.

[0015] Parallel operators and HBM scheduling modules are integrated into the on-chip network of the FPGA. Routing rules are set so that data with a volume below a preset threshold is transmitted through the on-chip buffer, while data with a volume above the preset threshold is transmitted through HBM. Transmission delay under different layouts is tested and routing paths are optimized iteratively.

[0016] The Bloom attention acceleration modules of each layer are pipelined and cascaded, so that the output of the previous layer is directly used as the input of the next layer. The parameters required for the next layer's calculation are prefetched using HBM, and the inter-layer pipeline is executed cyclically to complete the calculation of the entire large model.

[0017] Furthermore, the plurality of parallel sub-modules include at least a QKV projection sub-module, an attention score calculation sub-module, and an output projection sub-module; wherein the QKV projection sub-module and the output projection sub-module are marked as supporting multi-instance parallelism, and the attention score calculation sub-module is marked as supporting pipelined processing.

[0018] Furthermore, for each parallelizable submodule, a corresponding FPGA hardware operator is designed and implemented, and the interface of the hardware operator is adapted to match the burst transmission characteristics of HBM, forming a parallel operator, including:

[0019] The FPGA hardware operator designed for the QKV projection submodule is implemented using a vector multiplier array; the FPGA hardware operator designed for the attention score calculation submodule is implemented using a parallel accumulating tree architecture.

[0020] Furthermore, an HBM multi-channel data scheduling module is constructed to dynamically allocate HBM channels based on the real-time computing load of each parallel operator, monitor the bandwidth utilization of each channel, and iteratively adjust the allocation strategy to match real-time computing requirements, including:

[0021] The process of dynamically allocating HBM channels includes:

[0022] Real-time collection of computational load metrics for each parallel operator;

[0023] Based on the real-time load of each operator, the corresponding target channel allocation quantity is calculated proportionally.

[0024] The target channel allocation number is discretized into integers and resource constraints are adjusted to obtain the actual channel allocation scheme.

[0025] Channel allocation is accomplished in real time by writing to the memory controller register.

[0026] Furthermore, the cyclic adjustment allocation strategy adopts a feedback control mechanism, which dynamically adjusts the number of channels allocated in the next cycle based on the deviation between the current channel allocation and the target allocation, as well as the error between the system average bandwidth utilization and the target utilization.

[0027] Furthermore, the process of cyclically optimizing the routing path adopts an iterative optimization algorithm. Based on the feedback of the delay results obtained from the test, the weights of different paths or routing table entries in the on-chip network are dynamically updated until the overall delay meets the preset conditions or converges.

[0028] Furthermore, the Bloom attention acceleration modules of each layer are pipelined and cascaded, so that the output of the previous layer is directly used as the input of the next layer. HBM is used to prefetch the parameters required for the next layer's computation, and the inter-layer pipeline is executed iteratively to complete the computation of the entire large model, including:

[0029] The operation of prefetching parameters of the next layer using HBM is executed asynchronously and in parallel with the attention calculation process of the current layer, so as to hide the access latency of loading parameters from HBM into the on-chip cache.

[0030] Furthermore, the method is applied to training or inference scenarios of large models, including environmental perception and decision-making in natural language processing, image recognition, intelligent speech recognition, or autonomous driving.

[0031] Furthermore, the method is executed on a hardware system consisting of an FPGA, HBM, and an on-chip network, wherein the HBM multi-channel data scheduling module and the routing rules are implemented in the programmable logic resources of the FPGA.

[0032] Furthermore, the parallel operator and the HBM scheduling module are integrated into the on-chip network of the FPGA, and routing rules are set so that data with a data volume below a preset threshold is transmitted through the on-chip cache, and data with a data volume above the preset threshold is transmitted through HBM. The transmission delay under different layouts is tested and the routing path is optimized cyclically. The preset threshold is determined by experimental calibration based on the bandwidth of the on-chip cache, access delay and system conflict probability.

[0033] The embodiments of the present invention have the following advantages:

[0034] This invention decomposes the computational process of Bloom parallel attention into multiple sub-modules with parallel potential and designs FPGA hardware operators specifically adapted to the burst transmission characteristics of HBM. An HBM multi-channel data scheduling module is constructed to dynamically allocate channel resources and optimize bandwidth utilization based on the real-time load of the parallel operators. The operators and scheduling module are integrated into the FPGA on-chip network, using intelligent routing strategies to distinguish the transmission paths of large and small data and optimizing the network layout to reduce latency. Attention acceleration modules at each layer are pipelined and cascaded, and the HBM prefetch mechanism is used to achieve overlap between inter-layer computation and parameter loading. This invention effectively solves the problems of insufficient data transmission bandwidth and low computational unit efficiency in large model computation, significantly improving the computational speed and energy efficiency of large model training and inference on the FPGA platform. It can be widely applied in scenarios requiring efficient processing of large-scale data, such as intelligent speech recognition, autonomous driving, and natural language processing. Attached Figure Description

[0035] To more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are merely exemplary, and those skilled in the art can derive other embodiments based on the provided drawings without creative effort.

[0036] The structures, proportions, sizes, etc. illustrated in this specification are only for the purpose of assisting those skilled in the art in understanding and reading the content disclosed herein, and are not intended to limit the conditions under which the present invention can be implemented. Therefore, they have no substantial technical significance. Any modifications to the structure, changes in the proportions, or adjustments to the size, without affecting the effects and objectives that the present invention can produce, should still fall within the scope of the technical content disclosed in the present invention.

[0037] Figure 1 A flowchart illustrating a high-speed interconnect acceleration method for large FPGA models based on Bloom parallel attention, provided in an embodiment of the present invention;

[0038] Figure 2 This is a schematic diagram illustrating the application process of a high-speed interconnect acceleration method for large FPGA models based on Bloom parallel attention, provided in an embodiment of the present invention. Detailed Implementation

[0039] The following specific embodiments illustrate the implementation of the present invention. Those skilled in the art can easily understand other advantages and effects of the present invention from the content disclosed in this specification. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0040] In today's field of artificial intelligence, large models are increasingly widely used, with applications such as natural language processing and image recognition relying heavily on their support. However, large models involve extremely large computational demands, placing very high demands on hardware computing power and data transmission speed. Traditional computing architectures often encounter data transmission bottlenecks when processing large models, leading to low computational efficiency.

[0041] Bloom's parallel attention mechanism is an important technique in large models, enabling parallel processing of attention calculations and improving model efficiency. However, in practical applications, how to efficiently transfer relevant data to the computing unit for processing is a key factor affecting its performance.

[0042] FPGAs (Field-Programmable Gate Arrays) are characterized by their flexible programmability, allowing for customized designs to suit different computing tasks and exhibiting extremely high efficiency when handling specific computational tasks. HBMs (High-Bandwidth Memory), on the other hand, possess ultra-high bandwidth, enabling rapid data read and write operations and providing ample data support for the computation process.

[0043] In the field of high-speed interconnect acceleration of large-scale HBM models on FPGAs, common related technical solutions and their shortcomings are as follows. Traditional CPU computing methods are slow, and for large-scale model computations, the multi-core parallel capabilities of CPUs are limited, failing to meet the demands of rapid processing of large-scale data. Furthermore, the data transfer bandwidth between the CPU and HBM is low. While GPU acceleration methods offer strong computing power, they are also power-intensive and costly. Moreover, when processing certain specific large-scale model structures, the parallel computing architecture of GPUs may not fully leverage its advantages, leading to resource waste. Some simple FPGA acceleration methods merely port computational tasks to the FPGA without optimizing the computational flow or fully considering the characteristics of HBM, resulting in low data transfer efficiency and an inability to achieve efficient large-scale model computation. Additionally, some fixed data scheduling methods cannot dynamically adjust HBM channel allocation based on the real-time load of operators, easily leading to situations where some channels have excessively high bandwidth utilization while others are idle.

[0044] This invention combines the Bloom parallel attention mechanism with FPGA and HBM technologies. The Bloom parallel attention computation task is optimized and accelerated using FPGA, while HBM's high-speed data transfer capabilities enable rapid data interaction between memory and computing units. This high-speed interconnect acceleration method effectively reduces data transfer latency and improves the computation speed and overall performance of large models.

[0045] The method proposed in this invention can be applied to various scenarios requiring large models for processing, such as intelligent customer service systems, image and environmental perception in autonomous driving, and risk assessment in the financial sector. In these scenarios, the efficient operation of large models can lead to more accurate results and faster responses, thereby improving the performance and competitiveness of the entire system.

[0046] refer to Figure 1 and Figure 2 This invention discloses a method for accelerating high-speed interconnection of large FPGA model HBM based on Bloom parallel attention, including: decomposing the computation process of Bloom parallel attention into multiple parallel sub-modules, marking the parallel computing potential of each sub-module, and determining the sub-modules that can be piped or multi-instance parallelized.

[0047] For each submodule that can be processed in parallel, a corresponding FPGA hardware operator is designed and implemented, and the interface of the hardware operator is adapted to match the burst transmission characteristics of HBM to form a parallel operator.

[0048] A multi-channel HBM data scheduling module is constructed to dynamically allocate HBM channels according to the real-time computing load of each parallel operator, monitor the bandwidth utilization of each channel, and cyclically adjust the allocation strategy to match the real-time computing requirements.

[0049] Parallel operators and HBM scheduling modules are integrated into the on-chip network of the FPGA. Routing rules are set so that data with a volume below a preset threshold is transmitted through the on-chip buffer, while data with a volume above the preset threshold is transmitted through HBM. Transmission delay under different layouts is tested and routing paths are optimized iteratively.

[0050] The Bloom attention acceleration modules of each layer are pipelined and cascaded, so that the output of the previous layer is directly used as the input of the next layer. The parameters required for the next layer's calculation are prefetched using HBM, and the inter-layer pipeline is executed cyclically to complete the calculation of the entire large model.

[0051] 1) Decompose the computational flow of Bloom parallel attention into sub-modules such as QKV projection, attention score calculation, and output projection, mark the parallel computation potential of each sub-module, and determine the modules that can be piped or parallelized by multiple instances. The specific implementation method of this step is as follows:

[0052] First, the QKV projection submodule processes the input sequence matrix. ,in It is the sequence length. It is the input dimension, which generates queries through three independent linear layers. ,key Sum Matrix. Formula: , , Weight matrix , It is the key dimension.

[0053] This module has high parallel computing potential, supporting data parallelism (parallel computation on different batch samples) and multi-instance parallelism (e.g., deploying multiple projection layer copies on multiple GPUs to process independent sequences), because matrix multiplication operations have no data dependencies and can efficiently partition computing resources.

[0054] Secondly, the attention score calculation submodule executes the core attention mechanism, from Calculate the weighted output. Formula: Calculate the point integral. Scaling Stabilize the gradient, then apply softmax normalization. Finally, attention output is generated. .

[0055] This submodule has moderate parallelism potential; softmax suffers from inter-row dependency limitations on sequence-dimensional parallelism, but it can process different attention heads or partial fractional blocks in parallel; the computation steps are as follows: arrive It can be pipelined, meaning that scaling and softmax are performed in stages for overlapping computation.

[0056] The output projection submodule will Mapping to the final output ,formula: Weight matrix , It is a value dimension.

[0057] This module has high parallel potential, similar to QKV projection, and is suitable for data parallelism and multi-instance parallelism (e.g., parallelizing the output layer to process different batches).

[0058] Overall, the parallel potential is evident: QKV and output projection modules can be parallelized across multiple instances; attention score computation can be pipelined (e.g., performing scaling, softmax, and weighting in stages), and the entire process can be optimized for throughput using sequential parallel strategies.

[0059] 2) Design FPGA hardware operators for each parallel submodule. For example, QKV projection uses a vector multiplier array, and attention score calculation uses a parallel accumulator tree, adapting the operator interface to the burst transmission characteristics of HBM. The specific implementation method of this step is as follows:

[0060] In FPGA hardware design, operator optimization is performed on key parallel sub-modules of the Transformer model to improve computational efficiency and throughput. The QKV projection module is implemented using a vector multiplier array, which consists of multiple parallel processing units (PEs). Each PE is responsible for calculating the dot product of the projection weight matrix and the input vector.

[0061] For the input vector and weight matrix Projection output The i-th element is passed The calculation, in which the array structure allows h PEs to be executed in parallel, maximizes resource utilization; the array uses DSP slicing to implement multiply-accumulate operations and reduces latency through pipelined scheduling to ensure that a vector output is completed in each clock cycle.

[0062] The attention score calculation module deploys a parallel accumulating tree architecture for query vectors. and key vector Summation of dot products; first calculate element-wise products. Then, summation is performed using a binary tree structure. Each node of the tree uses an adder (PE) for parallel accumulation, with a depth of [insert depth here]. Significantly reduces the summation delay to The tree structure supports multi-path parallel input, adapting to batch processing requirements.

[0063] Adapting to the operator interface and HBM burst transmission characteristics, data flow is optimized through the design of a FIFO buffer and AXI bus protocol; the interface implements burst read / write mode, with a data block size of [missing information]. Bytes, bandwidth formula To meet HBM peak bandwidth requirements, To minimize burst cycles, register-aligned data bursts are inserted between pipeline stages to reduce stall events and ensure that operator inputs and outputs are directly connected to the HBM controller, thereby increasing the overall system throughput to its theoretical maximum. The overall operator design was verified using the Vivado HLS tool, and the comprehensive report confirmed resource usage and timing convergence.

[0064] 3) Construct an HBM multi-channel data scheduling module to dynamically allocate HBM channels based on the real-time load of each parallel operator, monitor channel bandwidth utilization, and iteratively adjust the allocation strategy to match computing demands. The specific implementation method for this step is as follows:

[0065] Deploy a hardware performance monitoring unit (PMU) and software probes to collect real-time computational load metrics for each parallel operator (e.g., GPU cores or AI processing units). Load is defined as the computational intensity of operator i at time t. This includes floating-point operations per second (FLOPS) and memory request frequency. Dynamic load data is obtained by reading internal processor counters and the operating system scheduler interface.

[0066] During system initialization, the total number of HBM channels N is set to a fixed value to ensure resource constraints. Average load is calculated as follows:

[0067]

[0068] This is used to assess the overall computational requirements, where M is the total number of parallel operators. This provides a normalized baseline for subsequent allocation.

[0069] A dynamic channel allocation algorithm is implemented based on real-time load data. A proportional allocation strategy is adopted to map HBM channels to the allocation number of operator i. The goal is to be with Proportional. Calculate the normalization factor. Generate ideal continuous allocation The actual allocation needs to be discretized: using a rounding function.

[0070]

[0071] And apply constraints To avoid resource overflow, the allocation process is implemented through writing to the memory controller registers, ensuring low-latency execution.

[0072] An integrated bandwidth utilization monitoring mechanism is used to measure the actual bandwidth usage of each HBM channel j using the PMU's bandwidth counter. Maximum channel bandwidth Fixed by HBM specifications. Calculate the utilization rate of channel j:

[0073]

[0074] And aggregated into the system average utilization rate:

[0075]

[0076] Set target utilization threshold To optimize performance and avoid overload, monitoring data is updated in real time via service interruption routines or polling, and written to a shared memory log for policy adjustments.

[0077] Design a cyclical adjustment strategy and employ feedback control to optimize allocation. Define the current allocation deviation. And introduce average utilization error Adjust channel allocation and usage ratio control: Update:

[0078]

[0079] in and The learning rate parameter requires empirical calibration. Apply normalization constraints. To maintain a constant total number of channels, the strategy loop period is set to milliseconds and triggered by a timer interrupt to achieve adaptive matching calculation requirements.

[0080] 4) Integrate the parallel operator and HBM scheduling module into the FPGA on-chip network, set routing rules so that small data is passed through the on-chip buffer and large data is transmitted through HBM, test the latency of different layouts and iteratively optimize the routing path. The specific implementation method of this step is as follows:

[0081] Integrating parallel operators and an HBM scheduling module into a network-on-a-chip (NoC) involves modifying the NoC router architecture and adding custom logic units to manage the interaction between operators and HBM. The scheduling module is implemented using an FPGA hardware description language (such as VHDL or Verilog). This module dynamically allocates the computational tasks of parallel operators to designated processing units and interfaces with the HBM controller to handle high-bandwidth data streams.

[0082] Key parameters include HBM access latency and on-chip cache latency ,in , Indicates the initial access time. This refers to data transmission time. The scheduler needs to optimize bandwidth utilization and define the maximum bandwidth. These are constraints.

[0083] Configure routing rules based on packet size thresholds This implements the logic of prioritizing on-chip cache paths for small data and forcing HBM transmission for large data. A decision-making algorithm is embedded in the NoC routing table, such as when the data packet size... satisfy When routing is active, it is routed to the on-chip cache module; otherwise, the HBM path is enabled, and low latency is ensured by configuring the router's priority queue. Threshold Through experimental calibration, the calculation formula was determined. ,in It's cache bandwidth. It is a regulating factor, optimized to reduce conflicts.

[0084] Latency testing for different layouts was conducted using on-chip network simulation tools (such as BookSim or custom models), with layouts including Mesh, Torus, and other topologies. A test bench was deployed on the FPGA, injecting variable-size packet sequences to measure end-to-end latency. Its model is:

[0085]

[0086] It is a delay in transmission. It's queue delay. This refers to the switch processing time. Layout variations are automatically generated via scripts, and the average latency is recorded. And peak latency.

[0087] Iterative optimization of routing paths uses iterative algorithms (such as gradient descent or genetic algorithms) with the goal of minimizing overall latency. Each iteration adjusts the routing table entries, updating path weights based on latency test results, using the following formula:

[0088]

[0089] in It is path weight. This is the learning rate. The loop continues until the latency reduction rate converges or reaches a preset threshold.

[0090] 5) Cascade the Bloom attention acceleration module pipeline for each layer, with the output of the previous layer directly used as the input of the next layer. Utilize HBM to prefetch the parameters of the next layer and iteratively execute the inter-layer pipeline to complete the large model computation. The specific implementation method for this step is as follows:

[0091] When implementing pipeline cascading, the model is divided into L consecutive Bloom attention layers, each containing a parameter matrix. , and These correspond to the query, key, and value weights, respectively. HBM stores parameters for all layers, while on-chip memory is used to cache the data and parameters needed for the current computation. During initialization, the parameters for the first layer are loaded. , and To the on-chip cache. Input data. The initial activation value is typically derived from the input embedding layer. The pipeline scheduler manages the execution flow, enabling inter-layer overlapping computation and prefetching through asynchronous operations. For each layer l ( Once the calculation begins, the input is read directly from the output buffer of the previous layer. This avoids data replication delays. Attention output calculation includes: , , Next, the attention weights are calculated:

[0092]

[0093] Final output As input for the next layer In calculation During the process, an asynchronous prefetch command is triggered to load the next layer of parameters from HBM. , and In on-chip caches, prefetching operations utilize hardware queues (such as CUDA streams) to execute in parallel with computation, thus hiding memory access latency. Output Immediately write to the buffer for use by layer l+1, forming a wait-free cascade. Execute loops from l=1 to l=L, with the output of layer L serving as the final activation; simultaneously process multiple micro-batches to fill the pipeline, improving hardware utilization. Optimizations include adjusting pipeline depth and prefetch timing, such as predicting the prefetch window based on computational load to ensure… To avoid pauses. The overall formula derivation is coherent:

[0094]

[0095] And complete the following in the process:

[0096]

[0097] The key to performance improvement lies in reducing the number of HBM accesses. By using prefetching, the average latency can be brought close to the computation time, thus achieving end-to-end acceleration.

[0098] This invention is applied to large-scale model computation scenarios, thereby improving the computation speed and data transmission efficiency of large models on FPGAs. Utilizing FPGA large-scale model HBM high-speed interconnect acceleration technology based on Bloom parallel attention, in large-scale model training and inference scenarios requiring the processing of massive amounts of data and complex calculations, such as intelligent speech recognition, it enables speech recognition systems to process speech data faster, achieving more real-time and accurate speech-to-text conversion. In the field of autonomous driving, it can accelerate the processing speed of road images, sensor data, etc., allowing vehicles to make decisions more quickly. This invention has wide applications in fields such as artificial intelligence and big data processing.

[0099] The core idea of ​​this invention is to decompose the Bloom parallel attention computation process, explore its parallel computing potential, design hardware operators adapted to HBM, dynamically schedule HBM channels, optimize on-chip network routing, and finally complete the large model computation through pipeline cascading. In practice, the computation process is first decomposed, parallel potential is marked, and parallelizable modules are identified. Then, FPGA hardware operators are designed for each parallel sub-module to adapt to the burst transmission characteristics of HBM. An HBM multi-channel data scheduling module is constructed to dynamically allocate channels. The operators and scheduling module are integrated into the FPGA on-chip network to optimize routing paths. Finally, each layer of acceleration modules is pipeline cascaded. The highlight is that it fully explores the parallelism of Bloom parallel attention, improving HBM utilization and data transmission efficiency while reducing computational latency through dynamic scheduling and optimized routing.

[0100] The embodiments of this invention offer significant advantages over these methods. Compared to CPU-based computation methods, the embodiments of this invention utilize the parallel computing capabilities of FPGAs, greatly improving computation speed, and address the low data transmission bandwidth issue by optimizing HBM data transmission. Compared to GPU-accelerated methods, the embodiments of this invention have lower power consumption and cost, and are specifically optimized for Bloom parallel attention, better leveraging the advantages of parallel computing. Compared to simple FPGA acceleration methods, the embodiments of this invention deeply decompose and optimize the computation process, fully considering the burst transmission characteristics of HBM, thus improving data transmission efficiency. Compared to fixed data scheduling methods, the dynamic scheduling module of the embodiments of this invention can adjust channel allocation according to real-time load, improving the overall utilization rate of HBM channels.

[0101] Although the present invention has been described in detail above with general descriptions and specific embodiments, modifications or improvements can be made to it, which will be obvious to those skilled in the art. Therefore, all such modifications or improvements made without departing from the spirit of the present invention fall within the scope of protection claimed by the present invention.

Claims

1. A method for accelerating high-speed interconnection of large-scale FPGA models based on Bloom parallel attention, characterized in that, The method includes: The computational flow of Bloom parallel attention is decomposed into multiple parallel sub-modules, the parallel computation potential of each sub-module is marked, and the sub-modules that can be piped or multi-instance parallelized are identified. For each submodule that can be processed in parallel, a corresponding FPGA hardware operator is designed and implemented, and the interface of the hardware operator is adapted to match the burst transmission characteristics of HBM to form a parallel operator. A multi-channel HBM data scheduling module is constructed to dynamically allocate HBM channels according to the real-time computing load of each parallel operator, monitor the bandwidth utilization of each channel, and cyclically adjust the allocation strategy to match the real-time computing requirements. Parallel operators and HBM scheduling modules are integrated into the on-chip network of the FPGA. Routing rules are set so that data with a volume below a preset threshold is transmitted through the on-chip buffer, while data with a volume above the preset threshold is transmitted through HBM. Transmission delay under different layouts is tested and routing paths are optimized iteratively. The Bloom attention acceleration modules of each layer are pipelined and cascaded, so that the output of the previous layer is directly used as the input of the next layer. The parameters required for the next layer's calculation are prefetched using HBM, and the inter-layer pipeline is executed cyclically to complete the calculation of the entire large model.

2. The method for accelerating high-speed interconnection of large-scale FPGA models based on Bloom parallel attention as described in claim 1, characterized in that, The plurality of parallel submodules include at least a QKV projection submodule, an attention score calculation submodule, and an output projection submodule; wherein, the QKV projection submodule and the output projection submodule are marked as supporting multi-instance parallelism, and the attention score calculation submodule is marked as supporting pipelined processing.

3. The FPGA large-model HBM high-speed interconnect acceleration method based on Bloom parallel attention as described in claim 2, characterized in that, For each parallelizable submodule, a corresponding FPGA hardware operator is designed and implemented, and the interface of the hardware operator is adapted to match the burst transmission characteristics of HBM, forming a parallel operator, including: The FPGA hardware operator designed for the QKV projection submodule is implemented using a vector multiplier array; the FPGA hardware operator designed for the attention score calculation submodule is implemented using a parallel accumulating tree architecture.

4. The method for accelerating high-speed interconnection of large-scale FPGA models based on Bloom parallel attention as described in claim 1, characterized in that, A multi-channel HBM data scheduling module is constructed to dynamically allocate HBM channels based on the real-time computing load of each parallel operator, monitor the bandwidth utilization of each channel, and iteratively adjust the allocation strategy to match real-time computing requirements, including: The process of dynamically allocating HBM channels includes: Real-time collection of computational load metrics for each parallel operator; Based on the real-time load of each operator, the corresponding target channel allocation quantity is calculated proportionally. The target channel allocation number is discretized into integers and resource constraints are adjusted to obtain the actual channel allocation scheme. Channel allocation is accomplished in real time by writing to the memory controller register.

5. The FPGA large-model HBM high-speed interconnect acceleration method based on Bloom parallel attention as described in claim 4, characterized in that, The cyclical adjustment allocation strategy employs a feedback control mechanism, dynamically adjusting the number of channels allocated in the next cycle based on the deviation between the current channel allocation and the target allocation, as well as the error between the system's average bandwidth utilization and the target utilization.

6. The FPGA large-model HBM high-speed interconnect acceleration method based on Bloom parallel attention as described in claim 4, characterized in that, The process of cyclically optimizing the routing path adopts an iterative optimization algorithm. Based on the feedback of the delay results obtained from the test, the weights or routing table entries of different paths in the on-chip network are dynamically updated until the overall delay meets the preset conditions or converges.

7. The method for accelerating high-speed interconnection of large-scale FPGA models based on Bloom parallel attention as described in claim 1, characterized in that, The Bloom attention acceleration modules at each layer are pipelined and cascaded, so that the output of the previous layer is directly used as the input of the next layer. HBM is used to prefetch the parameters required for the next layer's computation. This inter-layer pipeline is executed iteratively to complete the computation of the entire large model, including: The operation of prefetching parameters of the next layer using HBM is executed asynchronously and in parallel with the attention calculation process of the current layer, so as to hide the access latency of loading parameters from HBM into the on-chip cache.

8. The method for accelerating high-speed interconnection of large-scale FPGA models based on Bloom parallel attention as described in claim 1, characterized in that, The method is applied to training or inference scenarios of large models, including environmental perception and decision-making in natural language processing, image recognition, intelligent speech recognition, or autonomous driving.

9. The FPGA large-model HBM high-speed interconnect acceleration method based on Bloom parallel attention as described in claim 1, characterized in that, The method is executed on a hardware system consisting of an FPGA, HBM, and an on-chip network, wherein the HBM multi-channel data scheduling module and the routing rules are implemented in the programmable logic resources of the FPGA.

10. The FPGA large-model HBM high-speed interconnect acceleration method based on Bloom parallel attention as described in claim 1, characterized in that, Parallel operators and HBM scheduling modules are integrated into the on-chip network of the FPGA. Routing rules are set so that data with a volume below a preset threshold is transmitted through the on-chip cache, while data with a volume above the preset threshold is transmitted through HBM. Transmission delays under different layouts are tested and routing paths are iteratively optimized. The preset threshold is determined through experimental calibration based on the bandwidth of the on-chip cache, access delay, and system conflict probability.

Images