Multi-FPGA collaborative deep neural network parallel acceleration method

Abstract

The invention relates to the technical field of computers, and discloses a multi-FPGA collaborative deep neural network parallel acceleration method. According to the method, unified compiling and automatic calculation graph segmentation mapping based on a dynamic programming algorithm are carried out on a deep neural network model through a host control node, a heterogeneous strategy fusing data, the model and assembly line parallel is generated, and the problems of single FPGA computing power resource limitation and memory wall are effectively solved; direct communication between the FPGAs is achieved through a high-speed point-to-point internet, an asynchronous assembly line execution mechanism and a hardware-level data packaging technology are combined, communication delay is reduced, assembly line bubbles are eliminated, and a multi-FPGA cluster reaches a near-linear speed-up ratio; computing distribution is optimized in real time through a global load balancing unit, and the equipment utilization rate is increased; and meanwhile, a transparent deployment interface is provided through the hardware abstraction layer, so that the super-large-scale neural network can efficiently run on a multi-FPGA platform with low delay, and the overall calculation efficiency is improved.

Description

Technical Field

[0001] This invention relates to the field of computer technology, specifically to a method for parallel acceleration of deep neural networks using multi-FPGA collaborative approaches. Background Technology

[0002] In recent years, deep neural networks (DNNs) have achieved revolutionary success in fields such as computer vision and natural language processing. However, the explosive growth in the size of DNN models has placed extremely high demands on their computing hardware. Field-programmable gate arrays (FPGAs), due to their customizable parallel computing architecture and high energy efficiency, have become an important platform for accelerating DNN inference and training.

[0003] Currently, FPGA-based DNN acceleration solutions mainly face the following technical bottlenecks: The limited on-chip logic resources, DSP units, and on-chip memory (BRAM) capacity of a single FPGA chip make it difficult to efficiently deploy ultra-large-scale models (such as Transformers with tens of billions of parameters and large convolutional networks). The models have to be compressed or split, resulting in accuracy loss or frequent off-chip memory access, forming a performance bottleneck.

[0004] Even if the model can be accommodated by a single FPGA, its huge number of parameters and intermediate calculation results require frequent interaction with off-chip DRAM (such as DDR). The bandwidth and latency of DRAM severely limit the efficiency of the computing unit, which is often in a state of waiting for data.

[0005] In existing technologies, multi-FPGA solutions often employ simple data parallelism (e.g., different FPGAs process different input data) or coarse model parallelism (e.g., dividing the model by layer). These methods have serious limitations: data parallelism cannot solve the problem that a single FPGA cannot accommodate extremely large models; traditional inter-layer model parallelism introduces a large number of pipeline bubbles because data cannot be transferred until the previous FPGA has completed the computation of the entire layer, resulting in low equipment utilization; and the interconnection and communication between multiple FPGAs often rely on host CPU scheduling or shared memory, resulting in high communication overhead, high latency, and inability to achieve efficient fine-grained collaboration.

[0006] Therefore, there is an urgent need in this field for a collaborative acceleration method that can efficiently, transparently, and with low latency map ultra-large deep neural networks onto multiple FPGA clusters to overcome the resource and computing power limitations of a single chip, while hiding communication overhead to the maximum extent and improving overall computing efficiency. To this end, a multi-FPGA collaborative deep neural network parallel acceleration method is proposed. Summary of the Invention

[0007] To address the shortcomings of existing technologies, this invention provides a parallel acceleration method for multi-FPGA collaborative deep neural networks to solve the problems mentioned in the background.

[0008] To achieve the above objectives, the present invention provides the following technical solution: a multi-FPGA collaborative deep neural network parallel acceleration method, the method comprising the following steps: The host control node compiles the deep neural network model to be accelerated into a unified intermediate representation; Based on the hardware resource information of each FPGA node in a multi-FPGA cluster, a dynamic programming algorithm is used to automatically segment and map the computation graph represented by the intermediate representation, generating a heterogeneous parallel strategy that integrates data parallelism, model parallelism and pipeline parallelism. According to the parallel strategy, the subgraphs of the computation graph are allocated to the corresponding FPGA nodes, and a specific hardware configuration bitstream is generated for each FPGA node; During runtime, each FPGA node configures its internal computing engine and communication controller based on the bit stream to collaboratively execute the forward propagation or backpropagation computation during the training process of the deep neural network in a pipelined parallel and synchronous manner.

[0009] Preferably, the multiple FPGA clusters are interconnected through a high-speed point-to-point communication network, which is an Ethernet, Infiniband, or dedicated interconnection protocol based on SerDes, and the communication controller directly manages the data transmission from FPGA to FPGA without the need for relaying through the host memory.

[0010] Preferably, the automatic segmentation and mapping of the computational graph specifically includes: Identify the operator nodes and data dependencies in the computation graph; With the optimization objectives of minimizing the amount of cross-FPGA communication data and maximizing the utilization of computing resources, the computation graph is divided into multiple computation subgraphs; Adjacent operator nodes with close data dependencies are preferentially assigned to the same FPGA node; Insert data rearrangement and communication primitives between subgraphs spanning different FPGA nodes.

[0011] Preferably, the pipeline parallelism employs an asynchronous pipeline execution mechanism, wherein: The computation graph is divided into multiple stages, each stage being deployed on one or more FPGA nodes; After completing the processing of a micro-batch, the FPGA node in the previous stage immediately sends the intermediate results to the next stage via a high-speed interconnect network, without waiting for the entire batch to be processed. Meanwhile, the next FPGA node begins computation immediately after receiving enough data from a micro-batch, thus filling the pipeline with multiple micro-batches to effectively hide cross-node communication latency.

[0012] Preferably, the method further includes a global synchronization and load balancing unit, used for: Real-time monitoring of the computational load and communication latency of each FPGA node; When an FPGA node is detected as a performance bottleneck, the allocation strategy of subsequent micro-batches is dynamically adjusted or the remapping of the computation subgraph is triggered to achieve dynamic load balancing within the cluster.

[0013] Preferably, the communication controller integrates a hardware data packaging module, which aligns and encapsulates the tensor data to be transmitted according to a specific data width and address offset to generate a data frame conforming to a custom data packet format. The data frame includes a header and a data body, and the header includes tensor dimensions, data type, and destination node information.

[0014] Preferably, the host control node runs hardware abstraction layer software that provides a set of application programming interfaces (APIs) for receiving user-specified neural network models and parameters and translating these high-level instructions into scheduling and management commands for underlying multi-FPGA hardware resources.

[0015] Preferably, the method further includes a configuration and management unit, used to automatically discover the cluster topology and collect resource information of each FPGA node when the system starts up or the FPGA node is hot-swapped, so as to provide hardware basic information for the automatic segmentation and mapping steps.

[0016] Preferably, the unified intermediate representation is a multi-level intermediate representation, wherein the high-level representation is used to describe the neural network operators and computation graph structure, and the low-level representation is used to describe the hardware instructions and memory operations for a specific FPGA architecture; the compilation process includes performing operator fusion and constant folding optimization on the high-level representation, and converting the optimized computation graph into the low-level representation.

[0017] Compared with the prior art, the present invention has the following beneficial effects: This invention unifies the compilation of deep neural network models through a host control node and performs automated computation graph partitioning and mapping based on dynamic programming algorithms. This generates a heterogeneous strategy that integrates data, models, and pipeline parallelism, effectively solving the problems of limited computing power and memory walls on single FPGAs. Direct communication between FPGAs is achieved through a high-speed point-to-point interconnection network. Combined with asynchronous pipeline execution mechanisms and hardware-level data packaging technology, communication latency is reduced and pipeline bubbles are eliminated, enabling multi-FPGA clusters to achieve near-linear speedup. Real-time optimization of computation allocation by a global load balancing unit improves equipment utilization. Simultaneously, a transparent deployment interface is provided through a hardware abstraction layer, enabling ultra-large-scale neural networks to run efficiently and with low latency on multiple FPGA platforms, improving overall computational efficiency.

[0018] Other features and advantages of the invention will be set forth in the description which follows, and will be apparent in part from the description, or may be learned by practicing the invention. The objects and other advantages of the invention may be realized and obtained by means of the structures pointed out in the description, claims and drawings. Attached Figure Description

[0019] Figure 1 This is a schematic diagram illustrating the overall system architecture and workflow of the present invention; Figure 2 This invention unifies the compilation and intermediate representation (IR) generation process; Figure 3 This is a schematic diagram of the computational graph segmentation and mapping process of the present invention; Figure 4 This is a schematic diagram of the hardware data frame format of the present invention; Figure 5 This is a schematic diagram of the global load balancing mechanism of the present invention. Detailed Implementation

[0020] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0021] Please see Figures 1-5 The multi-FPGA collaborative deep neural network parallel acceleration method of the present invention is based on the fact that a set of hardware and software collaborative automated systems can efficiently and transparently deploy large-scale deep neural network models on a multi-FPGA cluster interconnected by a high-speed network, so as to overcome the limitations of a single FPGA in terms of computing resources and memory bandwidth.

[0022] 1. System Initialization and Hardware Abstraction Layer Technical execution process: After the system powers on, the configuration and management unit (a resident daemon) running on the host control node (an x86 server) begins operation. This unit traverses the PCIe bus to send a specific discovery packet to each FPGA board.

[0023] Pseudocode illustration: # Configuration and Management Snap-in Pseudocode def config_manager(): # 1. Hardware Discovery fpga_list = [] for pci_device in scan_pci_bus(): # Scan the PCIe bus if pci_device.vendor_id == FPGA_VENDOR_ID: # Handshake communication with the firmware on the FPGA board fpga_node = FPGANode(pci_device) # 2. Resource Information Collection resource_info = fpga_node.read_resource_info() # resource_info contains: # { # 'LUTs': total_luts, # 'DSPs': total_dsps, # 'BRAMs': total_brams, # 'DRAM_BW': dram_bandwidth, # 'NET_BW': network_bandwidth #} fpga_list.append((fpga_node, resource_info)) # 3. Topology Discovery (Based on Peer-to-Peer Networks) # Assume each FPGA node is pre-configured with network port MAC / IP address mapping topology_graph = build_topology_graph(fpga_list) # Build the topology graph by exchanging LLDP messages # Register cluster hardware information to the Hardware Abstraction Layer (HAL) hal.register_hardware_topology(fpga_list, topology_graph) The configuration and management unit is also responsible for monitoring hot-plug events of FPGA nodes. When a node insertion or removal is detected, the unit rescans the PCIe bus, updates the cluster topology and resource information, and notifies the Hardware Abstraction Layer (HAL) to re-evaluate the parallel strategy, triggering dynamic remapping if necessary.

[0024] This step enables the system to be plug-and-play. The configuration and management unit automatically identifies all FPGA nodes in the cluster, collects detailed hardware resource information for each node (number of LUTs, DSPs, BRAMs, DRAM, and network bandwidth), and constructs an interconnection topology graph between nodes. This information provides crucial hardware constraints for subsequent computation graph partitioning, ensuring that the generated parallel strategy is feasible. The Hardware Abstraction Layer (HAL) provides a unified programming interface, allowing users to submit their models without needing to concern themselves with the underlying hardware details.

[0025] 2. Model compilation and unified intermediate representation (IR) generation Technical execution process: After receiving the model, HAL calls its internal compiler front-end (such as a framework based on LLVM / MLIR) to parse the ONNX model into a unified multi-level intermediate representation (IR).

[0026] High-level IR: is a directed acyclic graph (DAG), denoted as G = (V, E). Where:

[0027]

[0028] IR optimization: The compiler performs several optimizations at the high-level IR level: Operator fusion combines consecutive operators that can be performed within the same computational unit into a single kernel. For example, matching the pattern Conv2D -> BatchNorm -> ReLU and fusing it into a single FusedConvBNReLU node. This significantly reduces inter-operator interactions and the number of global memory accesses.

[0029] Pseudocode for fusion rules: def operator_fusion(graph): for node in graph.nodes: if node.op_type == 'Conv2D': next_node = graph.get_next_node(node) if next_node.op_type == 'BatchNorm': next_next_node = graph.get_next_node(next_node) if next_next_node.op_type == 'ReLU': # Create a merge node fused_node = create_fused_node('FusedConvBNReLU', [node, next_node, next_next_node]) # Replace the atom graph with a fusion node graph.replace_subgraph([node, next_node, next_next_node], fused_node) return graph Constant folding: Pre-calculate all nodes whose values ​​can be statically determined (such as the dimension parameters of Shape and Transpose), solidify the results as constants, and reduce runtime calculations.

[0030] Lower-level IR: Optimized higher-level IRs are lowered to lower-level IRs. Lower-level IRs are specific to a particular FPGA architecture and describe specific hardware instructions, memory allocation (such as allocating tensors to BRAM or DDR), and data transfer operations.

[0031] A unified intermediate representation serves as a bridge connecting the software model and the hardware architecture. Optimization of high-level IR, particularly operator fusion, significantly reduces the number of nodes in the computation graph and the write / read operations for intermediate results, thereby lowering runtime overhead and communication requirements. The low-level IR, in turn, provides precise input for subsequent code generation and optimization specific to the FPGA.

[0032] 3. Computational Graph Partitioning and Mapping Based on Dynamic Programming Technical execution process: This is the core of the invention. The compiler uses the collected hardware resource information... The underlying IR computation graph G is segmented using a dynamic programming (DP) algorithm.

[0033] Objective function: The goal of the algorithm is to find a graph partitioning scheme. ,Will Divided into Sub-image (M is less than or equal to the number of FPGA nodes), and mapped onto the FPGA cluster to minimize the total execution time, which is mainly achieved by minimizing communication costs and balancing the computational load.

[0034] We define a cost function. :

[0035] in: It is the sum of the communication costs between all adjacent subgraphs; ; It is the size of the transfer tensor. and Determined by network topology; It is an indicator of the balance of resource utilization. and It is a weighting factor used to adjust the importance of communication and computing resources.

[0036] DP algorithm process: Identify data dependencies: Perform topological sorting on the computation graph G to obtain the node sequence. .

[0037] State definition: Definition For the future Each node is assigned to The minimum cost obtained on each FPGA, and the corresponding partitioning scheme.

[0038] State transition:

[0039] in From node arrive The resulting subgraph. This includes the computational cost of the subgraph on a single FPGA (estimated execution time).

[0040] Constraints: Subgraphs must be checked during state transitions. resource requirements Is it less than the available resources of the k-th FPGA node? .express This ensures the feasibility of the segmentation.

[0041] Backtracking solution: Ultimately, The minimum cost and optimal partitioning scheme are given. .

[0042] Insert communication primitives: in the final segmentation scheme In each cross-FPGA edge The compiler will automatically insert data rearrangement (such as Tensor Reshape / Transpose) and communication primitive (Send / Recv) nodes.

[0043] Dynamic programming algorithms systematically explore all possible partitioning methods to find the optimal deployment strategy for communication and computation costs under given hardware constraints. It aggregates operators with close data dependencies in the computation graph, minimizing cross-node communication; simultaneously, through load balancing, it avoids individual nodes becoming performance bottlenecks, thereby maximizing the overall computational efficiency of the entire cluster.

[0044] 4. Bitstream generation and node configuration Technical execution process: For each subgraph The compiler generates a hardware configuration bitstream for its target FPGA node.

[0045] Computation Engine: The compiler uses High-Level Synthesis (HLS) or RTL templates to transform subgraphs. The computational logic generates highly parallel customized computing units (such as DSP arrays and pipelined processing units).

[0046] Communication Controller: Each bitstream contains a hardened communication controller IP core. This controller includes: Hardware data packaging module: This module operates on the sending end. When a tensor T (assuming dimensions [N, C, H, W]) needs to be sent, this module will: 1) Alignment: Align tensor data to addresses of 64 bytes or 128 bytes (matching the network bus width).

[0047] 2) Encapsulation: Generate a custom data frame.

[0048] 3) Packet Header: Contains control information such as destination node ID, source node ID, tensor dimensions [N,C,H,W], data type (e.g., FP32), and sequence number.

[0049] Payload: The aligned tensor data.

[0050] DMA transfer: The encapsulated data frame is directly pushed to the Ethernet MAC core via DMA and enters the high-speed network.

[0051] At the receiving end, the communication controller parses the packet header information, unpacks the data body according to the tensor dimension and data type, and writes it to a pre-allocated memory address (BRAM or DDR) via DMA. It may also trigger an interrupt or set a flag to notify the computing engine that the data is ready.

[0052] The host computer downloads each bitstream file to the corresponding FPGA via PCIe to complete the configuration.

[0053] This step generates a tailored hardware-accelerated image for each FPGA node. The compute engine is highly optimized to perform specific subgraph tasks. Crucially, the hardware communication controller enables direct data exchange (P2P) between FPGAs, completely bypassing the host CPU and memory, resulting in extremely low communication latency. Custom packet formats and hardware-level packing / unpacking operations maximize network bandwidth utilization.

[0054] 5. Runtime asynchronous pipelined execution and dynamic load balancing Technical execution process: During runtime, the input data is divided into multiple micro-batches.

[0055] Asynchronous pipeline execution: 1) FPGA0 (pipeline stage 1) begins processing micro-batch 1.

[0056] 2) After FPGA0 finishes processing micro-batch 1, its computing engine will convert the intermediate result tensor... The address is provided to the local communication controller.

[0057] 3) The communication controller immediately initiates DMA, and... The package is then sent directly to FPGA1 (point-to-point communication, bypassing host memory). FPGA0 then immediately begins processing micro-batch 2, without waiting for FPGA1 to complete processing micro-batch 1.

[0058] 4) The communication controller of FPGA1 receives the data packet of micro-batch 1, unpacks it, and writes the data into its input buffer. Once enough data is received (e.g., a complete data block), its computing engine immediately begins to compute micro-batch 1. After the computation is complete, it sends the result to FPGA2 and begins to process the data of micro-batch 2.

[0059] 5) This process repeats, with multiple micro-batches being processed simultaneously at different stages of the pipeline, creating pipeline parallelism. Communication latency is effectively hidden by computation time.

[0060] To ensure data consistency across micro-batches, the system inserts a synchronization barrier mechanism between each stage of the pipeline to ensure the correctness of gradient calculation during backpropagation training.

[0061] Theoretically, the throughput of a pipeline can reach:

[0062] in Throughput is a key metric for measuring system performance, representing the number of micro-batches the system can process per unit of time. This represents the total number of micro-batches. That is, the total number of micro-batches that need to be processed in a single training or inference task. The time required to process a microbatch for the i-th pipeline stage, where i ranges from 0 to M, representing different stages in the pipeline (each stage may be handled by one or more FPGAs). This time includes the computation time of the stage and the communication time for waiting for its required input data. The processing time of the slowest stage in the entire pipeline is the maximum value that determines the "heartbeat" or basic rhythm of the entire pipeline, also known as the pipeline cycle. The speed of the entire pipeline cannot be faster than its slowest stage. Fill time for the production line. This refers to the time from when the first micro-batch enters the production line until it is finally output from the end of the production line. During this period, the production line is not at full capacity; only the earlier stages are busy, while the later stages are idle and waiting.

[0063] Global synchronization and load balancing: The load balancing unit uses a scheduling algorithm based on heuristics or reinforcement learning to evaluate the load of each node in real time, dynamically adjust the micro-batch allocation strategy, or trigger subgraph migration and local reconfiguration when a performance bottleneck is detected.

[0064] A separate monitoring unit (which can run on the host or a specific FPGA) continuously collects performance counter data from each node: : The average computation time to process a micro-batch.

[0065] :from arrive The average communication time.

[0066] : The average queue length of the input buffer within the time window.

[0067] : The percentage of time the input buffer is empty.

[0068] This unit continuously calculates the throughput of the entire pipeline. If a node is detected... of If the value is significantly higher than other nodes, or if its input queue is consistently empty (indicating a communication bottleneck), it is identified as a performance bottleneck.

[0069] The load balancing unit uses a scheduling algorithm based on heuristics or reinforcement learning to evaluate the load of each node in real time, dynamically adjust the micro-batch allocation strategy, or trigger subgraph migration and local reconfiguration when a performance bottleneck is detected.

[0070] Dynamic adjustment strategy: Micro-batch adjustment: The monitoring unit can instruct the front end of the pipeline to temporarily reduce the rate at which micro-batches are allocated to bottleneck nodes, or dynamically adjust the micro-batch size.

[0071] Remapping: In more severe cases, the monitoring unit can report to the host, triggering a lightweight re-segmentation and remapping. For example, ... Some operators (subgraphs) on the FPGA are migrated to adjacent FPGAs with lighter loads. This process may require dynamic reconfiguration of some bitstreams.

[0072] The runtime system is the ultimate guarantee of performance. The asynchronous pipelined execution mechanism allows for full overlap between computation and communication, hiding most of the unavoidable communication latency and greatly improving device utilization and system throughput. The global synchronization and load balancing unit gives the system dynamic adaptability, enabling it to cope with uneven computational loads between different layers, network fluctuations, and other practical situations, avoiding long-term performance bottlenecks that may result from static partitioning, and ensuring that the system always operates at a high performance level.

[0073] This method first performs unified compilation and optimization of the neural network model through the Hardware Abstraction Layer (HAL) of the host node to generate a hardware-independent intermediate representation (IR). Then, based on the dynamic programming algorithm and according to the real-time collected cluster hardware resource topology information, the computation graph is automatically segmented and mapped into a heterogeneous parallel strategy that integrates data parallelism, model parallelism, and pipeline parallelism. Subsequently, a hardware bitstream containing a customized computing engine and an efficient communication controller is generated for each FPGA node. Finally, at runtime, the overlap of computation and communication is achieved through an asynchronous pipeline execution mechanism and a global load balancing unit, hiding latency and enabling multi-FPGA systems to achieve a near-linear speedup and high device utilization when performing ultra-large-scale neural network inference or training tasks.

Claims

1. A multi-FPGA collaborative deep neural network parallel acceleration method, characterized in that, The method includes the following steps: The host control node compiles the deep neural network model to be accelerated into a unified intermediate representation; Based on the hardware resource information of each FPGA node in a multi-FPGA cluster, a dynamic programming algorithm is used to automatically segment and map the computation graph represented by the intermediate representation, generating a heterogeneous parallel strategy that integrates data parallelism, model parallelism, and pipeline parallelism. According to the parallel strategy, the subgraphs of the computation graph are allocated to the corresponding FPGA nodes, and a specific hardware configuration bitstream is generated for each FPGA node; During runtime, each FPGA node configures its internal computing engine and communication controller based on the bit stream to collaboratively execute the forward propagation or backpropagation computation during the training process of the deep neural network in a pipelined parallel and synchronous manner.

2. The multi-FPGA collaborative deep neural network parallel acceleration method according to claim 1, characterized in that, The multiple FPGA clusters are interconnected through a high-speed point-to-point communication network, which is based on SerDes Ethernet, Infiniband, or a dedicated interconnection protocol. The communication controller directly manages the data transmission from FPGA to FPGA without the need for relaying through the host memory.

3. The multi-FPGA collaborative deep neural network parallel acceleration method according to claim 1, characterized in that, The automatic segmentation and mapping of the computational graph specifically includes: Identify the operator nodes and data dependencies in the computation graph; With the optimization objectives of minimizing the amount of cross-FPGA communication data and maximizing the utilization of computing resources, the computation graph is divided into multiple computation subgraphs; Adjacent operator nodes with close data dependencies are preferentially assigned to the same FPGA node; Insert data rearrangement and communication primitives between subgraphs spanning different FPGA nodes.

4. The multi-FPGA collaborative deep neural network parallel acceleration method according to claim 1, characterized in that, The pipeline parallelism employs an asynchronous pipeline execution mechanism, wherein: The computation graph is divided into multiple stages, each stage being deployed on one or more FPGA nodes; After completing the processing of a micro-batch, the FPGA node in the previous stage immediately sends the intermediate results to the next stage via a high-speed interconnect network, without waiting for the entire batch to be processed. Meanwhile, the next FPGA node begins computation immediately after receiving enough data from a micro-batch, thus filling the pipeline with multiple micro-batches to effectively hide cross-node communication latency.

5. The multi-FPGA collaborative deep neural network parallel acceleration method according to claim 1, characterized in that, The method further includes a global synchronization and load balancing unit, used for: Real-time monitoring of the computational load and communication latency of each FPGA node; When an FPGA node is detected as a performance bottleneck, the allocation strategy of subsequent micro-batches is dynamically adjusted or the remapping of the computation subgraph is triggered to achieve dynamic load balancing within the cluster.

6. The multi-FPGA collaborative deep neural network parallel acceleration method according to claim 2, characterized in that, The communication controller integrates a hardware data packaging module, which aligns and encapsulates the tensor data to be transmitted according to a specific data width and address offset, generating a data frame that conforms to a custom data packet format. The data frame includes a header and a data body, and the header includes tensor dimensions, data type, and destination node information.

7. The multi-FPGA collaborative deep neural network parallel acceleration method according to claim 1, characterized in that, The host control node runs a hardware abstraction layer software that provides a set of application programming interfaces (APIs) for receiving user-specified neural network models and parameters and translating these high-level instructions into scheduling and management commands for the underlying multi-FPGA hardware resources.

8. The multi-FPGA collaborative deep neural network parallel acceleration method according to claim 1, characterized in that, The method also includes a configuration and management unit, which is used to automatically discover the cluster topology and collect resource information of each FPGA node when the system starts up or the FPGA node is hot-swapped, so as to provide hardware basic information for the automatic segmentation and mapping steps.

9. The multi-FPGA collaborative deep neural network parallel acceleration method according to claim 1, characterized in that, The unified intermediate representation is a multi-level intermediate representation. The high-level representation is used to describe neural network operators and computation graph structures, while the low-level representation is used to describe hardware instructions and memory operations for a specific FPGA architecture. The compilation process includes performing operator fusion and constant folding optimization on the high-level representation and converting the optimized computation graph into the low-level representation.

Images