A heterogeneous computing low latency communication method and system

Abstract

This application relates to the field of computer technology and discloses a low-latency communication method and system for heterogeneous computing, aiming to solve the high latency problems caused by high communication protocol overhead, lack of dynamic scheduling coordination, redundant memory migration, and inconsistent cross-node communication abstractions in existing heterogeneous computing. The method includes: receiving task scheduling requests and parsing task dependency graphs; dynamically allocating tasks based on node load and link status; adaptively selecting RDMA, NVLink, or PCIe passthrough protocols to establish communication channels according to node type; achieving zero-copy data exchange through a shared memory mapping buffer; and utilizing hardware timestamps and PTP synchronization to provide latency feedback and optimize scheduling. The system includes a heterogeneous computing node cluster, a unified communication scheduling controller, a low-latency communication protocol stack, a shared memory mapping buffer, and a communication latency-aware task allocator. Through the above scheme, this application significantly reduces communication latency and improves throughput and task execution efficiency.

Description

Technical Field

[0001] This invention belongs to the field of computer technology, specifically relating to a heterogeneous computing low-latency communication method and system. Background Technology

[0002] With the rapid development of artificial intelligence, high-performance computing, and edge computing, heterogeneous computing architecture has become a core technology path to support low-latency, high-throughput data processing tasks due to its ability to integrate the advantages of multiple computing units such as CPU, GPU, FPGA, and ASIC. It is widely used in scenarios with stringent requirements for response timeliness, such as autonomous driving, industrial control, real-time video analysis, and financial transactions. Its core goal is to achieve the optimal balance between task processing performance and energy efficiency through the collaborative scheduling and efficient communication of computing resources.

[0003] Low-latency communication, as a key component for the performance of heterogeneous computing systems, directly determines the efficiency and real-time performance of data exchange between different computing units. This technological direction focuses on establishing efficient, reliable, and deterministic data transmission channels between heterogeneous hardware to minimize communication overhead and ensure the continuous execution of task pipelines.

[0004] Existing communication mechanisms in heterogeneous computing environments still suffer from significant shortcomings: First, general communication protocols (such as TCP / IP or standard PCIe transmission) are not optimized for the data flow characteristics of heterogeneous computing tasks, resulting in high protocol stack overhead and high transmission latency. Second, they lack dynamic perception capabilities of data dependencies between computing units, making it impossible to achieve collaborative optimization of communication scheduling and computing tasks. Third, existing systems typically employ static memory mapping or fixed bandwidth allocation strategies, which are difficult to adapt to changes in communication demands caused by fluctuations in task load, easily leading to resource contention or bandwidth waste. Finally, in multi-device collaborative scenarios, cross-node communication and local communication are not uniformly abstracted, resulting in high software stack complexity and poor portability. These problems are particularly prominent in applications requiring millisecond-level response, such as autonomous driving perception fusion and real-time industrial control, severely restricting the overall performance release of heterogeneous computing systems. Therefore, a low-latency communication method and system that can deeply integrate computing characteristics and communication scheduling is urgently needed. Summary of the Invention

[0005] The purpose of this invention is to overcome the shortcomings of the prior art and provide a heterogeneous computing low-latency communication method and system, which can effectively solve the problems in the background art.

[0006] To achieve the above objectives, the present invention provides the following technical solution: On one hand, a heterogeneous computing low-latency communication system, comprising the following components: a heterogeneous computing node cluster for executing task processing under different computing architectures, the heterogeneous computing node cluster including CPU nodes, GPU nodes, FPGA nodes, and dedicated AI accelerator nodes; a unified communication scheduling controller for receiving task scheduling requests and generating communication scheduling instructions, coordinating data interaction between the heterogeneous computing nodes; a low-latency communication protocol stack deployed within each heterogeneous computing node for encapsulating and parsing communication data packets, and dynamically selecting the optimal communication path and transmission protocol based on the communication scheduling instructions; a shared memory mapping buffer configured between the heterogeneous computing nodes for achieving zero-copy data exchange, reducing redundant data migration between host memory and device memory; and a communication latency-aware task allocator for dynamically allocating computing tasks and optimizing the execution order of the task dependency graph based on the real-time load status, communication link bandwidth, and latency characteristics of each heterogeneous computing node.

[0007] On the other hand, a low-latency communication method for heterogeneous computing is provided, comprising the following steps: Step S110, receiving a task scheduling request from an upper-layer application through a unified communication scheduling controller, the task scheduling request including a task dependency graph, data input specifications, and quality of service requirements; Step S120, a communication latency-aware task allocator performs topological sorting of the task dependency graph based on the real-time load status, communication link bandwidth, and latency characteristics of each heterogeneous computing node and allocates it to the optimal heterogeneous computing node; Step S130, establishing a communication channel between the source heterogeneous computing node and the target heterogeneous computing node through a low-latency communication protocol stack, and dynamically selecting one of RDMA, NVLink, or PCIe passthrough as the underlying transmission protocol according to the communication scheduling instructions; Step S140, writing the data to be transmitted into a shared memory-mapped buffer at the source heterogeneous computing node, and notifying the target heterogeneous computing node to read it directly through a memory address mapping mechanism to achieve zero-copy data exchange; Step S150, after receiving the data ready signal, the target heterogeneous computing node immediately starts its local computing task and sends the computing results back to the next dependent node or upper-layer application through the same mechanism.

[0008] Preferably, the unified communication scheduling controller includes a task parsing unit, a resource status monitoring unit, and a scheduling strategy generation unit. The task parsing unit is used to parse the task dependency graph in the task scheduling request and extract the predecessor-successor relationship and data dependency between tasks. The resource status monitoring unit is used to periodically collect the CPU utilization, GPU memory usage, FPGA logic unit utilization, and AI accelerator queue depth of each heterogeneous computing node, and simultaneously obtain the current bandwidth utilization and end-to-end latency of each communication link. The scheduling strategy generation unit generates communication scheduling instructions based on the task dependency graph and resource status information using a heuristic scheduling algorithm. The heuristic scheduling algorithm preferentially selects nodes with communication latency below a threshold and computing resource idle rate above 80% as task allocation targets.

[0009] Furthermore, the low-latency communication protocol stack includes a protocol adaptive selection module, a data packet encapsulation module, and an error recovery module. The protocol adaptive selection module dynamically selects the RDMA protocol for communication between the CPU and GPU, the NVLink protocol for communication between GPUs, and the PCIe passthrough protocol for communication between the FPGA and the AI ​​accelerator, based on the source node and target node types specified in the communication scheduling instruction. The data packet encapsulation module encapsulates the data to be transmitted according to the frame format of the target protocol and adds a checksum and sequence number. When the error recovery module detects data packet loss or checksum failure, it triggers a selective retransmission mechanism to retransmit only the lost or damaged data packets, avoiding the additional latency introduced by full retransmission.

[0010] Furthermore, the shared memory-mapped buffer employs a unified virtual address space technology. During the initialization phase, the operating system kernel allocates contiguous virtual address segments to all heterogeneous computing nodes and maps these virtual address segments to the physical device address space of each node via the IOMMU (Input / Output Memory Management Unit). Each heterogeneous computing node can perform read and write operations on shared data by accessing the same virtual address, without needing to execute explicit memory copy instructions. The size of the shared memory-mapped buffer is dynamically adjusted according to the maximum data transfer volume declared in the task scheduling request, with an initial allocation of 64MB and a maximum expandable size of 4GB.

[0011] Preferably, the communication latency-aware task allocator adopts a critical path-based scheduling strategy. First, it identifies the critical path in the task dependency graph, which is the path with the longest cumulative computation and communication latency from the start task to the end task. Then, it prioritizes assigning tasks on the critical path to the heterogeneous computing node pairs with the lowest communication latency for execution. For tasks on non-critical paths, it performs load balancing based on computing resource utilization. The identification of the critical path is achieved through a dynamic programming algorithm, with a time complexity of O(N+M), where N is the number of task nodes and M is the number of dependency edges.

[0012] Furthermore, each node in the heterogeneous computing node cluster is equipped with a hardware timestamp unit to record nanosecond-level precision timestamps at the moment of data packet transmission and reception. The unified communication scheduling controller calculates the end-to-end communication delay in real time based on the timestamp difference between the source node and the target node, and feeds back the delay value to the communication delay-aware task allocator for decision-making in the next round of task allocation. The hardware timestamp unit achieves clock synchronization between nodes through PTP (Precise Time Protocol), with the synchronization error controlled within ±50 nanoseconds.

[0013] Compared with the prior art, the present invention has the following beneficial effects:

[0014] By constructing a collaborative mechanism between a unified communication scheduling controller and a low-latency communication protocol stack, dynamic adaptive selection of communication protocols between heterogeneous computing nodes is achieved, reducing the average communication latency to less than 1 / 10 of that of the traditional TCP / IP protocol stack.

[0015] By utilizing shared memory-mapped buffers and zero-copy data exchange technology, redundant data migration between host memory and device memory is completely eliminated, increasing data transmission throughput by more than 3 times.

[0016] By introducing a communication latency-aware task allocation strategy, combined with critical path identification and real-time resource monitoring, the overall execution time of the task dependency graph is effectively shortened, and the end-to-end task completion latency is reduced by 45%.

[0017] By using hardware timestamps and PTP clock synchronization mechanisms, nanosecond-level precision in communication delay measurement and feedback is achieved, providing a high-precision decision-making basis for dynamic scheduling and significantly enhancing system scheduling stability. Attached Figure Description

[0018] Figure 1 This is a schematic diagram of the overall technical solution architecture of the present invention. Detailed Implementation

[0019] Please refer to Figure 1 To further illustrate the technical means and effects of the present invention in order to achieve the intended purpose, the following detailed description of the specific implementation methods, structures, features and effects of the present invention, in conjunction with the accompanying drawings and preferred embodiments, is provided below.

[0020] Example 1

[0021] This embodiment describes a heterogeneous computing low-latency communication method and system applied in high-performance computing (HPC) clusters, with a particular focus on scientific simulation and data analysis scenarios. In this scenario, tasks are typically decomposed into multiple subtasks, which may be executed on CPUs, GPUs, FPGAs, or dedicated AI accelerators, respectively. Furthermore, there are complex data dependencies between these tasks, placing extremely high demands on communication latency and throughput to ensure the overall efficiency of large-scale parallel computing.

[0022] First, this invention provides a heterogeneous computing low-latency communication system, which consists of multiple core components working together, including a heterogeneous computing node cluster, a unified communication scheduling controller, a low-latency communication protocol stack, a shared memory mapping buffer, and a communication latency-aware task allocator.

[0023] Heterogeneous computing node clusters form the physical foundation of the system, consisting of a series of interconnected computing nodes, each integrating processors with different computing architectures internally or across nodes. Specifically, the cluster contains multiple CPU nodes, each typically equipped with a multi-core processor (e.g., Intel Xeon or AMD EPYC series processors), large-capacity DDR5 or DDR4 memory, and running a general-purpose operating system (such as Linux). CPU nodes are primarily responsible for control logic, serial computationally intensive tasks, and data preprocessing and post-processing tasks unsuitable for parallelization. In addition, the cluster includes GPU nodes, which typically feature one or more high-performance graphics processors (e.g., NVIDIA H100 or AMD Instinct series) and high-bandwidth HBM or GDDR memory. GPU nodes excel at large-scale parallel computing, particularly suitable for data-parallel tasks such as matrix operations, numerical integration, and deep learning training. FPGA nodes integrate programmable gate arrays (e.g., Xilinx Versal or Intel Agilex series), offering high flexibility and reconfigurability, and can be used to implement customized hardware accelerators, such as high-speed packet processing, encryption / decryption, or hardware offloading of domain-specific algorithms. Finally, dedicated AI accelerator nodes (e.g., Google TPU, Graphcore IPU, or Huawei Ascend series) are optimized for AI workloads, providing extremely high computing efficiency and suitable for deep learning inference and some training tasks. These heterogeneous computing nodes are physically connected via high-speed interconnect networks (such as InfiniBand EDR / HDR / NDR, RoCEv2, or PCIe Gen4 / Gen5 fabric) to ensure basic data exchange capabilities. Each heterogeneous computing node comes pre-installed with the necessary drivers, runtime libraries, and communication agents to support integration with low-latency communication protocol stacks.

[0024] The unified communication scheduling controller is the intelligent central hub of the system, responsible for receiving global task requests from upper-layer applications and transforming them into fine-grained communication scheduling instructions. This controller is an independent software service or microservice cluster deployed on a dedicated control node to ensure high availability and responsiveness. The controller consists of a task parsing unit, a resource status monitoring unit, and a scheduling policy generation unit. The task parsing unit receives and deeply parses the "task scheduling request" from the upper-layer application. This request is typically encapsulated in a structured data format (e.g., Protobuf or JSON), containing a "directed acyclic graph" (DAG) describing the task execution logic. This unit identifies the "predecessor-successor relationships" between tasks using graph traversal algorithms (e.g., depth-first search or breadth-first search), determining which task must complete before it can begin execution. Simultaneously, it extracts the "data dependencies" of each task, including the number of bytes of input data, the data structure type (e.g., floating-point arrays, image frames), and the specifications of the expected output data. The resource status monitoring unit periodically (e.g., every 100 milliseconds) collects detailed real-time runtime status data from all heterogeneous computing nodes in the cluster. For CPU nodes, it monitors CPU utilization (e.g., average idle rate obtained by reading / proc / stat or using the perf tool) to determine its computational load. For GPU nodes, it collects GPU memory utilization (e.g., the percentage of allocated memory obtained via the nvidia-smi or AMD ROCm SMI API) and the utilization of compute units. For FPGA nodes, it monitors logic unit (LUTs, FFs) utilization and DSP block and BRAM utilization, typically obtained via runtime APIs provided by the FPGA vendor. For AI accelerator nodes, it monitors their internal queue depth (e.g., the current length of the instruction queue and data buffer queue) to reflect their task backlog. Furthermore, this unit synchronously acquires the current bandwidth utilization of each communication link (e.g., send / receive byte rates obtained via network interface card statistics or InfiniBand ibstat) and end-to-end latency (measured via a hardware timestamp unit synchronized with PTP). All collected data is normalized and aggregated, then stored in an in-memory database for quick querying by the scheduling policy generation unit. Based on these real-time task dependency graphs and resource status data, the scheduling strategy generation unit uses a heuristic scheduling algorithm to generate specific communication scheduling instructions. This heuristic algorithm is designed for multi-objective optimization, and its core strategy is to prioritize node pairs that meet specific communication latency constraints (e.g., links with communication latency below 1 microsecond) and have a computing resource idle rate (e.g., CPU idle rate, GPU computing unit idle rate) higher than 80% as task allocation targets.When searching for the optimal allocation, this algorithm considers the computational load of the task, the data transfer volume, the computing power of the target node, the current load of the communication link, and the expected latency to maximize system throughput and minimize critical path latency. The generated communication scheduling instructions include the source node ID, target node ID, the size of the data to be transferred, the recommended transmission protocol (RDMA, NVLink, or PCIe passthrough), and the memory address range and priority for transmission, and are sent to the corresponding heterogeneous computing nodes.

[0025] The low-latency communication protocol stack is deployed within each heterogeneous computing node, tightly integrated with hardware drivers as an operating system kernel module or user-space library. Its core function is to optimize the encapsulation, transmission, and parsing of data packets according to instructions issued by the unified communication scheduler, ensuring data delivery with minimal latency. This protocol stack consists of a protocol adaptive selection module, a data packet encapsulation module, and an error recovery module. The protocol adaptive selection module is the intelligent decision-making part of the stack. It dynamically selects the most suitable underlying transmission protocol based on the source and target node types and data transmission characteristics specified in the communication scheduling instructions. Specifically, when the communication scheduling instructions indicate data exchange between CPU and GPU nodes, the protocol adaptive selection module prioritizes RDMA (Remote Direct Memory Access) protocols, such as those based on InfiniBand or RoCE (RDMA over Converged Ethernet). Leveraging its zero-copy and kernel bypass characteristics, it directly transfers data from the source node's memory to the target node's GPU memory, avoiding CPU involvement and the overhead of operating system kernel context switching. When high-speed data transfer is required between GPU nodes, if they are on the same server or interconnected via NVLink, this module will select the NVLink protocol, leveraging its high-bandwidth, low-latency GPU-to-GPU direct data transfer capabilities, such as through NVIDIA GPUDirect RDMA. For communication between FPGA nodes and AI accelerator nodes or CPU nodes, especially when exchanging data between custom hardware accelerators and host memory, the protocol adaptive selection module will choose the PCIe passthrough protocol. By configuring the DMA (Direct Memory Access) engine, data can be transferred directly between device memory and host memory, minimizing CPU intervention. The packet encapsulation module is responsible for efficiently encapsulating the raw data to be transmitted according to the frame format of the selected underlying transport protocol. For example, for RDMA transmission, data will be encapsulated into InfiniBand or RoCE transport packets, including the necessary headers, source address, destination address, queue pair identifier, and type of transport service. For NVLink or PCIe passthrough, data may be divided into PCIe Transaction Layer Packets (TLPs) or NVLink-specific data units, with corresponding protocol headers and control information appended. To ensure data integrity, this module calculates and appends a checksum (e.g., CRC-32 or a more complex error-correcting code) and a sequence number during the encapsulation process. The sequence number is used for out-of-order detection and acknowledgment at the receiving end, while the checksum is used to detect data corruption during transmission. The error recovery module immediately initiates a selective retransmission mechanism upon detecting packet loss or checksum failure.When the receiving end detects discontinuous sequence numbers (indicating lost data packets) or mismatched checksums (indicating corrupted data packets), it does not request the sending end to retransmit the entire data stream. Instead, it sends a precise NACK (Negative Acknowledgment) message to the sending end, specifying the sequence number of the specific data packet that needs to be retransmitted. Upon receiving the NACK, the sending end only retransmits the lost or corrupted data packets, thus avoiding the additional latency introduced by full retransmission and significantly improving the real-time performance and efficiency of communication.

[0026] Shared memory-mapped buffers are a key technological component for achieving zero-copy data exchange. This buffer employs Unified Virtual Address (UVA) technology, where the operating system kernel allocates a contiguous, globally visible virtual address segment to all heterogeneous computing nodes during system initialization. While logically unified, this virtual address segment may physically map to different physical memory regions (e.g., host memory, GPU memory, FPGA onboard memory, or AI accelerator cache). The Input Output Memory Management Unit (IOMMU) plays a crucial role, acting as a hardware auxiliary unit responsible for mapping this global virtual address segment to the actual physical device address spaces of each heterogeneous computing node. Through this mechanism, different types of heterogeneous computing nodes can directly read and write shared data by accessing the same virtual address, eliminating the need for explicit memory copy instructions and thus completely eliminating redundant data migration between host memory and device memory. For example, the GPU can directly read memory regions written by the CPU without the CPU needing to copy data from host memory to GPU memory. This design greatly simplifies the programming model and significantly reduces data transfer latency and CPU load. The size of the shared memory-mapped buffer is dynamically adjusted based on the maximum data transfer volume declared in the task scheduling request. Initially, to ensure fast startup and reduce resource consumption, the system typically allocates a small default value, such as 64MB. However, when a large-scale data transfer requirement is detected, the buffer can be dynamically expanded as needed, for example, up to 4GB, to accommodate large datasets or multiple concurrent tasks. This dynamic adjustment mechanism ensures efficient resource utilization and avoids over-allocation or under-allocation of memory.

[0027] The communication latency-aware task allocator is the core intelligent module for the system's global optimized scheduling. It employs a critical path-based scheduling strategy to shorten the overall execution time of the task dependency graph. First, upon receiving the task dependency graph, the allocator analyzes it using a dynamic programming algorithm to identify the "critical path." The critical path is defined as the path with the longest cumulative computation and communication latency among all possible paths from the start task to the end task. This dynamic programming algorithm typically uses topological sorting to calculate the Earliest Start Time (EST) and Earliest Finish Time (EFT) for each task, and determines the critical path accordingly. Its time complexity is O(N+M), where N is the number of task nodes and M is the number of dependency edges, ensuring efficient computation in large-scale task graphs. After identifying the critical path, the allocator prioritizes assigning tasks on the critical path to heterogeneous computing node pairs with the lowest communication latency. For example, if a task on the critical path needs to transfer data from the CPU to the GPU, the allocator will prioritize assigning the task to the CPU node connected to the GPU with the lowest end-to-end communication latency. For tasks on non-critical paths, since their impact on overall task completion time is relatively small, the allocator performs load balancing based on computing resource utilization to fully utilize idle computing resources in the system and avoid local overload. This strategy ensures that tasks with the greatest impact on overall performance receive optimal scheduling, while also ensuring full utilization of system resources.

[0028] Each heterogeneous computing node in the cluster is equipped with a hardware timestamp unit, a set of dedicated hardware counters that record nanosecond-level timestamps at the moment of packet transmission and reception. These timestamps are generated based on a high-precision hardware clock and are unaffected by operating system scheduling jitter, ensuring the accuracy of the measurement results. The unified communication scheduling controller calculates the precise end-to-end communication delay in real time by comparing the timestamp difference between the source node and the target node during the same packet transmission process. These delay values ​​are immediately fed back to the communication delay-aware task allocator. To ensure the synchronization of timestamps across all nodes, the system employs the PTP (Precision Time Protocol, IEEE 1588) mechanism. PTP is a high-precision time synchronization protocol that synchronizes clocks in the network through the interaction of master and slave clocks, controlling the clock synchronization error between nodes within ±50 nanoseconds, providing a reliable basis for accurate measurement of communication delays. Based on this high-precision real-time delay data, the task allocator can make more accurate judgments in the next round of task allocation decisions, dynamically adjust scheduling strategies, and further optimize task execution efficiency.

[0029] Next, the present invention further provides a heterogeneous computing low-latency communication method, which achieves efficient task scheduling and execution through the collaborative work of the above-mentioned system components.

[0030] Step S110: Receive a task scheduling request from the upper-layer application through the unified communication scheduling controller. The task scheduling request includes a task dependency graph, data input specifications, and quality of service requirements.

[0031] In this step, the upper-layer application (e.g., a large-scale image processing pipeline, a financial risk model computation service, or a molecular dynamics simulation program) generates a task scheduling request and sends it to the unified communications scheduling controller via a standard API interface (e.g., gRPC, RESTful API, or a dedicated message queue interface). This request is a structured data object, the core component of which is a task dependency graph (DAG). The task dependency graph represents independent computational tasks (e.g., "image preprocessing," "feature extraction," "model inference," "data aggregation") as nodes, and directed edges represent data flow and execution order dependencies between tasks. Each task node also includes an estimate of its computational load (e.g., number of floating-point operations, memory access patterns) and the required heterogeneous computing resource type (CPU, GPU, FPGA, AI accelerator, and their version requirements). The "Data Input Specification" details the characteristics of each task's input data, including the data source (e.g., local storage, remote file system, or real-time data stream), data type (e.g., 8-bit integer image, 32-bit floating-point array), data dimensions and format (e.g., 1080p RGB image, NxM matrix), and expected data size (e.g., MB or GB). The "Quality of Service (QoS) Requirements" specify the task's performance targets, such as the end-to-end latency ceiling (e.g., 10 milliseconds), the throughput floor (e.g., 1 GB / s), and error tolerance. The unified communications scheduling controller's task parsing unit receives and rigorously verifies the format and integrity of this request. It uses a high-performance parsing library to perform syntactic and semantic analysis on the task dependency graph, ensuring the graph's validity (e.g., no circular dependencies) and extracting all task nodes and their attributes, all predecessor-successor relationships between tasks, and the direction and expected amount of data transmission. The parsed information is stored in the controller's internal data structure, providing a basis for subsequent scheduling decisions.

[0032] Step S120: The communication latency-aware task allocator sorts the task dependency graph topology based on the real-time load status, communication link bandwidth and latency characteristics of each heterogeneous computing node and allocates the task to the best heterogeneous computing node.

[0033] This step is the core intelligent decision-making stage for achieving low-latency scheduling. The communication latency-aware task allocator first obtains the current cluster's global real-time operational data from the resource status monitoring unit of the unified communication scheduling controller. This includes:

[0034] Real-time load status of each heterogeneous computing node: including average load, CPU core utilization, and free memory for CPU nodes; VRAM utilization, compute unit utilization, and temperature for GPU nodes; logic resource (LUT, FF, DSP, BRAM) utilization for FPGA nodes; and inference queue depth and processing unit utilization for AI accelerator nodes. This data is obtained through periodic hardware sensor reads and operating system API calls, for example, refreshed every 50 milliseconds.

[0035] Communication link bandwidth: The current actual available bandwidth of the link (e.g., MB / s) obtained through the network interface controller (NIC) counter, InfiniBand, or RoCE performance monitoring interface (PMI), as well as the port error rate.

[0036] Latency characteristics: The end-to-end communication latency between all heterogeneous node pairs, measured using the aforementioned hardware timestamp unit and PTP synchronization mechanism, achieves nanosecond-level accuracy. These latency values ​​are updated in real time in matrix form.

[0037] After acquiring this information, the task allocator performs topological sorting on the received task dependency graph, generating a task execution sequence that ensures all predecessor tasks complete before their dependent tasks begin. Based on this, it employs a reinforcement learning or heuristic algorithm combined with critical path analysis for task allocation. First, it identifies the critical path in the task dependency graph—the path with the longest sum of computational and communication latency among all possible execution paths from the start task to the end task. For tasks on the critical path, the allocator prioritizes assigning them to heterogeneous computing node pairs that can provide the lowest communication latency and meet computational resource requirements. For example, if there is a large amount of data transfer between a GPU task and a previous CPU task on the critical path, the allocator will prioritize scheduling the CPU-GPU pair with the fastest connection speed (lowest latency). For tasks on non-critical paths, the allocator uses a load balancing strategy, assigning them to nodes with high current computing resource idle rates (e.g., CPU idle rate greater than 80% or GPU memory utilization less than 50%) to avoid hotspots and maximize the overall throughput of the cluster. Finally, the allocator generates a detailed execution plan, specifying on which specific heterogeneous computing node each task will be executed, the data transfer path between tasks, the selected communication protocol, and the expected communication parameters.

[0038] Step S130: Establish a communication channel between the source heterogeneous computing node and the target heterogeneous computing node through a low-latency communication protocol stack, and dynamically select one of RDMA, NVLink or PCIe passthrough as the underlying transmission protocol according to the communication scheduling instructions.

[0039] After the task allocation decision is made, the unified communication scheduling controller generates specific communication scheduling instructions and distributes them to the source heterogeneous computing nodes and target heterogeneous computing nodes involved in the communication. Within each node, the protocol adaptive selection module of the low-latency communication protocol stack receives and parses these instructions.

[0040] RDMA Protocol Establishment: When a scheduling instruction instructs large-scale data transfer between a CPU node and a GPU node, or between any two nodes interconnected via an InfiniBand / RoCE network, the protocol adaptive selection module initializes the RDMA communication channel. This includes creating or reusing RDMA queue pairs (QPs), each QP containing a send queue (SQ) and a receive queue (RQ). It then registers memory segments for data transfer in the memory regions of both the source and target nodes, marking them as accessible to RDMA. Once memory registration is complete, the source node can directly write data from its local registered memory to the target node's registered memory via RDMA write operations, without intervention from the target CPU. The entire process is kernel-bypassed and zero-copy, significantly reducing latency.

[0041] Establishing the NVLink protocol: If the scheduling instruction specifies communication between two NVIDIA GPUs within the same server, or between multiple GPUs connected via NVLink Fabric, the protocol adaptive selection module activates the NVLink communication channel. This is typically achieved through the NVIDIA GPUDirect RDMA mechanism, allowing one GPU to directly access the memory of another GPU. The process involves memory mapping and address space translation, enabling the source GPU to directly read and write to the target GPU's memory, avoiding redundant data transfers through the CPU and host memory.

[0042] PCIe passthrough protocol establishment: When communication occurs between FPGA nodes or AI accelerator nodes and host CPU memory, or between custom hardware accelerators, the protocol adaptive selection module configures the PCIe passthrough channel. This is typically achieved by configuring the DMA engines of each device. The DMA controller of the source device (e.g., the FPGA) is programmed to read data directly from its internal memory and transfer it via the PCIe bus to the physical memory address of the target device (e.g., the host CPU) and vice versa. During this process, the operating system provides the IOMMU mapping service to ensure the translation from virtual address to physical address, but the data transfer itself is completed by the hardware DMA engine without involving the CPU, achieving low latency and high throughput.

[0043] During the channel establishment process, the protocol stack will also perform necessary handshakes and resource reservations to ensure the exclusivity or priority of the data path and prepare for subsequent data transmission.

[0044] Step S140: The source heterogeneous computing node writes the data to be transmitted into the shared memory-mapped buffer and notifies the target heterogeneous computing node to read it directly through the memory address mapping mechanism, thereby realizing zero-copy data exchange.

[0045] Once the communication channel is established, the application or kernel module within the source heterogeneous computing node writes the data to be transmitted into a pre-configured shared memory-mapped buffer. Crucially, this write operation is not a traditional memory copy to the send queue; instead, the data is placed in a memory region mapped by the operating system kernel to the physical device address space of all relevant heterogeneous computing nodes using unified virtual address space technology and the IOMMU.

[0046] The specific execution process is as follows:

[0047] Data Writing: After completing their partial computation, the computational tasks on the source heterogeneous computing nodes directly write the generated data (e.g., an image feature vector, a simulation result matrix) to the shared memory-mapped buffer address in their local view. Since this is part of a unified virtual address space, this write operation effectively places the data in a physical memory region directly accessible to all participating nodes (this could be host memory, GPU memory, FPGA onboard DDR, etc., depending on the configuration). For example, if the shared buffer is mapped to host DDR, the GPU can directly access this DDR region via DMA on PCIe without copying the data to GPU memory and then transferring it.

[0048] Notification Mechanism: Once the data has been fully written to the shared memory-mapped buffer and is ready, the source heterogeneous computing node notifies the target heterogeneous computing node that the data is available through a lightweight notification mechanism. This notification can take various forms, depending on the chosen underlying protocol:

[0049] RDMA Immediate Value: An Immediate Value can be appended to an RDMA Write operation. The target node receives this immediate value simultaneously with the received data, serving as a signal that the data is ready. Alternatively, the source node can initiate a separate RDMA Send / Recv operation, sending only a control message (such as an event notification) to instruct the target node to read data from a specific address.

[0050] NVLink Atomic Operations / Semaphores: Between GPUs interconnected by NVLink, notifications can be communicated through atomic operations or hardware semaphores in a shared memory region (such as a specific register or a segment of video memory). For example, the source GPU writes a flag variable to "1" to indicate that data is ready, and the target GPU polls for or waits for this flag.

[0051] PCIe Doorbell Mechanism: In a PCIe passthrough scenario, a device (such as an FPGA) can trigger an interrupt or event by writing a value to a specific register of the host or another device (i.e., "ringing the doorbell"), notifying the other party that data is ready in shared memory.

[0052] Zero-copy read: Upon receiving a data-ready signal, the target heterogeneous computing node does not need to copy data from the network interface or bus to its local memory. Due to the unified virtual address space and IOMMU mapping of the shared memory-mapped buffer, the target node can directly access the data in the shared buffer through its local physical address space. For example, a GPU's compute core can directly read data from the GPU memory address mapped to the shared buffer, and FPGA's custom logic can directly read data from the mapped FPGA onboard DDR or host DDR area. This eliminates at least one (and often multiple) memory copy operations in traditional communication, significantly reducing data transmission latency, increasing data throughput, and freeing the CPU from heavy data copying tasks.

[0053] Step S150: After receiving the data ready signal, the target heterogeneous computing node immediately starts the local computing task and sends the computing results back to the next dependent node or upper-layer application through the same mechanism.

[0054] Once the data is successfully received and the data is confirmed to be accessible in the shared memory-mapped buffer, the target heterogeneous computing node will immediately start its local computing task.

[0055] Initiating a local computation task: The scheduler or runtime environment on the target node (e.g., the CUDA runtime on a GPU, the OpenCL runtime on an FPGA, or the MACC-specific runtime on an AI accelerator) loads and executes the computation kernel on the corresponding heterogeneous hardware based on pre-assigned task instructions. Since the data is already prepared in a shared buffer, the computation task can directly access this data without any additional copy latency. For example, the GPU starts its kernel function, directly reading input data from the shared memory region mapped to its video memory address and performing computations.

[0056] Result Feedback: After the computation task is completed, the target heterogeneous computing node writes the computation result to another shared memory-mapped buffer area specified by the unified communications scheduler. Then, using the same mechanism described in step S140 (e.g., RDMA Immediate Value, NVLink semaphore, or PCIe doorbell), it notifies the next heterogeneous computing node or upper-layer application that the data is ready. This process ensures smooth pipeline flow, with all intermediate results flowing between heterogeneous nodes in a zero-copy manner. If the computation result is the final output, it will be fed back to the upper-layer application via a similar mechanism, potentially involving a final copy from the shared buffer to the application memory (if the application is not running in the shared memory space), but this copy is typically accomplished by efficient DMA or RDMA operations.

[0057] Example 2

[0058] This embodiment further describes the application of the present invention in an edge computing platform for autonomous vehicles. Autonomous driving scenarios place extremely stringent demands on the system's real-time performance, reliability, and energy efficiency. Sensor data (such as radar, lidar, and camera data) needs to be preprocessed, fused, and inferred with extremely low latency to support millisecond-level decision-making and control. Unlike the general HPC scenario in Embodiment 1, autonomous driving platforms are typically resource-constrained embedded systems with additional requirements for power consumption, space, and shock resistance. Therefore, their heterogeneous node clusters may be more compact, and the choice of communication protocols tends to focus on intra-device or board-level interconnects.

[0059] The system architecture of this embodiment is consistent with that of Embodiment 1 in terms of core components, but its implementation details, parameter configuration and protocol selection strategy will be adjusted according to the specific needs of autonomous driving.

[0060] In autonomous driving scenarios, heterogeneous computing node clusters may include: one or more automotive-grade high-performance CPUs (e.g., ARM Cortex-A series or specific automotive-grade SoCs) responsible for the complex control logic of the operating system, high-level path planning, and environmental perception algorithms; multiple embedded GPUs (e.g., NVIDIA Drive series or Qualcomm Snapdragon Auto) for image processing, feature extraction, and deep learning inference; and dedicated ASIC accelerators (such as TPUs or self-developed NPUs) focused on efficiently executing neural network models. These nodes are typically connected via onboard high-speed interconnects (such as PCIe Switch Fabric) and custom on-chip interconnects (NoC) to minimize physical distance and transmission loss.

[0061] In this scenario, the unified communications scheduling controller's task parsing unit receives task scheduling requests from multiple onboard sensors and actuators. These requests may include radar point cloud processing tasks, multi-camera video stream fusion tasks, path prediction tasks, and communication tasks with the vehicle control module. The task dependency graph is more complex, often containing hard real-time constraints (e.g., LiDAR data processing must be completed within 50 milliseconds). In addition to monitoring conventional metrics such as CPU utilization and GPU memory usage, the resource status monitoring unit pays special attention to sensor frame rates (e.g., 30 frames per second for cameras), packet loss rates, and computing unit power consumption. The scheduling strategy generation unit employs a hybrid scheduling strategy, incorporating hard real-time scheduling algorithms such as Earliest Deadline First (EDF) and Rate Monotonic Scheduling (RMS) based on critical path analysis to ensure that time-critical tasks are completed on time. For example, for low-latency, high-frequency tasks like lane line recognition, the scheduler will prioritize allocating dedicated ASIC accelerators and transmitting data via high-speed onboard interconnects. For tasks with communication latency below the threshold and extremely high real-time requirements, the scheduling policy generation unit will attempt to assign them to the node with the closest physical distance or a dedicated direct connection channel. When generating communication scheduling instructions, the priority and deadline of the task will be specified in addition.

[0062] In this scenario, the protocol adaptive selection module of the low-latency communication protocol stack will tend to leverage the characteristics of embedded hardware. For example, for data transfer between the CPU and the onboard ASIC accelerator, it may prioritize using a zero-copy mechanism mapped to the on-chip bus (such as AXI or a custom NoC), or configure a specific PCIe Root Port for direct device access. For image data transfer or feature map exchange between multiple GPUs, if they are connected via internal NVLink or proprietary high-speed interconnects, the protocol stack will maximize the use of these links. In the packet encapsulation module, in addition to checksums and serial numbers, timestamps and security authentication information may be added to meet the high requirements of data time synchronization and information security in autonomous driving scenarios. When handling packet loss or corruption, the error recovery module may introduce faster redundancy coding or forward error correction (FEC) mechanisms, especially for critical sensor data streams, to reduce unpredictable delays caused by retransmissions. For example, for visual perception data, even if some packets are lost, FEC can recover them to some extent, avoiding retransmissions and ensuring real-time performance.

[0063] Shared memory-mapped buffers are of particular importance in autonomous driving platforms. They are not only used for zero-copy data exchange but also frequently serve as a "blackboard" mechanism for real-time shared state and intermediate computation results. All sensor data, the fused environmental model, and perception results (e.g., identified vehicles, pedestrians, and lane line information) can be stored in this unified virtual address space. The operating system kernel maps this virtual address segment to the CPU's L3 cache, GPU's video memory, and ASIC's on-chip SRAM or external DDR via the IOMMU. This design allows different perception, planning, and control modules to concurrently access the latest shared state on different heterogeneous nodes without additional data transfer. To ensure data consistency, hardware cache coherency protocols (such as ARM CCIX or CXL) or lock-free data structures at the software level (such as circular buffers or read-write locks) may be introduced to manage concurrent access to shared memory. Dynamic adjustment of the buffer size is also applicable, but for hard real-time systems, a larger fixed area may be pre-allocated to reduce the uncertainty caused by dynamic memory allocation at runtime.

[0064] In autonomous driving scenarios, the communication latency-aware task allocator employs more complex critical path analysis and scheduling strategies. In addition to a dynamic programming algorithm with a time complexity of O(N+M), it introduces a "resource reservation" mechanism. Once a task on the critical path is assigned, its required computing resources (e.g., the number of stream processors in a GPU, the MAC unit in an ASIC) and communication bandwidth (e.g., a specific DMA engine for a PCIe channel) are explicitly reserved to ensure that it is not preempted by other tasks during execution. For identified critical path tasks, the allocator considers not only the node pairs with the lowest communication latency but also their power consumption characteristics and temperature conditions to ensure stable system operation in embedded environments. Furthermore, since autonomous driving tasks are typically periodic, the allocator leverages these periodic characteristics to combine static scheduling with dynamic adjustments, establishing a fixed scheduling table and fine-tuning it at runtime to handle unexpected events.

[0065] Hardware timestamp units and PTP synchronization mechanisms are crucial in autonomous driving because the fusion and timestamp alignment of all sensor data are central to perception. Each sensor (radar, lidar, camera) may integrate a high-precision timestamp unit, which is synchronized to the vehicle's master clock via PTP or NTP (Network Time Protocol). Hardware timestamp units on heterogeneous computing nodes synchronize with these sensors to ensure that the timestamp recorded when receiving data packets has consistent nanosecond-level accuracy with the timestamp when the sensors acquired the data. The unified communications scheduling controller uses these timestamps to accurately calculate the end-to-end latency of sensor data from acquisition to entry into the heterogeneous computing nodes, processing, and transmission to the next node. These latency values ​​are used not only for task allocation but also to evaluate the real-time performance of the entire autonomous driving system and trigger warnings or degradation strategies when latency exceeds a threshold, such as switching to a simpler perception model under specific latency constraints. PTP clock synchronization error requirements may need to be controlled within ±20 nanoseconds to meet the stringent requirements of multi-sensor time synchronization fusion.

[0066] Through the optimized combination of the above system components and methods, this invention can significantly reduce the end-to-end communication latency of heterogeneous computing tasks and improve the real-time performance of sensor data processing in autonomous driving edge computing scenarios, thereby enhancing the vehicle's perception, decision-making, and control capabilities and providing solid technical support for achieving safer and more reliable autonomous driving.

[0067] The above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention in any way. Although the present invention has been disclosed above with reference to preferred embodiments, it is not intended to limit the present invention. Any person skilled in the art can make some modifications or alterations to the above-disclosed technical content to create equivalent embodiments without departing from the scope of the present invention. Any simple modifications, equivalent changes and alterations made to the above embodiments based on the technical essence of the present invention without departing from the scope of the present invention shall still fall within the scope of the present invention.

Claims

1. A heterogeneous computing low-latency communication system, characterized in that, It includes the following components: A heterogeneous computing node cluster is used to execute task processing under different computing architectures. The heterogeneous computing node cluster includes CPU nodes, GPU nodes, FPGA nodes and dedicated AI accelerator nodes; a unified communication scheduling controller is used to receive task scheduling requests and generate communication scheduling instructions to coordinate data interaction between heterogeneous computing nodes. A low-latency communication protocol stack is deployed inside each heterogeneous computing node to encapsulate and parse communication data packets and dynamically select the optimal communication path and transmission protocol based on communication scheduling instructions. A shared memory-mapped buffer, configured between heterogeneous computing nodes, is used to achieve zero-copy data exchange and reduce redundant data migration between host memory and device memory; a communication latency-aware task allocator is used to dynamically allocate computing tasks and optimize the execution order of the task dependency graph based on the real-time load status, communication link bandwidth and latency characteristics of each heterogeneous computing node. The unified communication scheduling controller includes a task parsing unit, a resource status monitoring unit, and a scheduling strategy generation unit. The task parsing unit is used to parse the task dependency graph in the task scheduling request and extract the predecessor-successor relationship and data dependency between tasks. The resource status monitoring unit is used to periodically collect the CPU utilization, GPU memory usage, FPGA logic unit utilization, and AI accelerator queue depth of each heterogeneous computing node, and simultaneously obtain the current bandwidth utilization and end-to-end latency of each communication link. The scheduling strategy generation unit generates communication scheduling instructions based on the task dependency graph and resource status information using a heuristic scheduling algorithm. The heuristic scheduling algorithm prioritizes nodes with communication latency below a threshold and computational resource idle rate above 80% as task allocation targets. The low-latency communication protocol stack includes a protocol adaptive selection module, a data packet encapsulation module, and an error recovery module. The protocol adaptive selection module dynamically selects the RDMA protocol for communication between the CPU and GPU, the NVLink protocol for communication between GPUs, and the PCIe passthrough protocol for communication between the FPGA and the AI ​​accelerator, based on the source node and target node types specified in the communication scheduling instruction. The data packet encapsulation module encapsulates the data to be transmitted according to the frame format of the target protocol and adds a checksum and sequence number. When the error recovery module detects a data packet loss or verification failure, it triggers a selective retransmission mechanism to retransmit only the lost or damaged data packets. Each node in the heterogeneous computing node cluster is equipped with a hardware timestamp unit, which is used to record timestamps with nanosecond precision at the time of data packet transmission and reception. The unified communication scheduling controller calculates the end-to-end communication delay in real time based on the timestamp difference between the source node and the target node, and feeds back the delay value to the communication delay-aware task allocator. The hardware timestamp unit achieves clock synchronization between nodes through PTP, with the synchronization error controlled within ±50 nanoseconds.

2. The heterogeneous computing low-latency communication system according to claim 1, characterized in that, The shared memory-mapped buffer adopts a unified virtual address space technology. During the initialization phase, the operating system kernel allocates a continuous virtual address segment to all heterogeneous computing nodes and maps this virtual address segment to the physical device address space of each node through the IOMMU. Each heterogeneous computing node performs read and write operations on shared data by accessing the same virtual address. The size of the shared memory-mapped buffer is dynamically adjusted according to the maximum data transfer volume declared in the task scheduling request. The initial allocation is 64MB, and it can be expanded to a maximum of 4GB.

3. The heterogeneous computing low-latency communication system according to claim 1, characterized in that, The communication latency-aware task allocator adopts a critical path-based scheduling strategy. First, it identifies the critical path in the task dependency graph, which is the path with the longest cumulative computation and communication latency from the start task to the end task. Then, it prioritizes assigning tasks on the critical path to the heterogeneous computing node pairs with the lowest communication latency for execution. For tasks on non-critical paths, it performs load balancing based on computing resource utilization. The identification of the critical path is achieved through a dynamic programming algorithm, with a time complexity of O(N+M), where N is the number of task nodes and M is the number of dependency edges.

4. A communication method applied to the heterogeneous computing low-latency communication system according to any one of claims 1-3, characterized in that, Includes the following steps: Step S110: Receive a task scheduling request from the upper-layer application through the unified communication scheduling controller. The task scheduling request includes a task dependency graph, data input specifications, and quality of service requirements. Step S120: The communication latency-aware task allocator performs topological sorting of the task dependency graph based on the real-time load status, communication link bandwidth, and latency characteristics of each heterogeneous computing node, and allocates the task to the optimal heterogeneous computing node. Step S130: Establish a communication channel between the source heterogeneous computing node and the target heterogeneous computing node through a low-latency communication protocol stack, and dynamically select one of RDMA, NVLink, or PCIe passthrough as the underlying transmission protocol according to the communication scheduling instructions. Step S140: The source heterogeneous computing node writes the data to be transmitted into a shared memory-mapped buffer and notifies the target heterogeneous computing node to read it directly through a memory address mapping mechanism, achieving zero-copy data exchange. Step S150: After receiving the data ready signal, the target heterogeneous computing node immediately starts its local computing task and sends the computing results back to the next dependent node or upper-layer application through the same mechanism.

5. The communication method according to claim 4, characterized in that, In step S120, the communication latency-aware task allocator identifies the critical path in the task dependency graph and prioritizes the allocation of tasks on the critical path to the heterogeneous computing node pairs with the lowest communication latency; tasks on non-critical paths are load-balanced based on the idle rate of computing resources; the critical path identification adopts a dynamic programming algorithm with a time complexity of O(N+M), where N is the number of task nodes and M is the number of dependency edges.

6. The communication method according to claim 4, characterized in that, In step S130, when the source node and the target node are CPU and GPU, the RDMA protocol is selected; when both the source node and the target node are GPUs and interconnected via NVLink, the NVLink protocol is selected; when the source node or the target node is FPGA or AI accelerator, the PCIe pass-through protocol is selected. The selected protocol enables direct data transmission through kernel bypass or DMA engine, avoiding CPU involvement in data copying.

7. The communication method according to claim 4, characterized in that, In step S140, the shared memory mapping buffer is allocated a unified virtual address segment by the operating system kernel and mapped to the physical device address space of each node through the IOMMU. After the source node writes data into the buffer, it sends a data ready signal to the target node through RDMA Immediate Value, NVLink semaphore or PCIe doorbell mechanism. The target node reads the data directly from the mapped address without explicit memory copying.

8. The communication method according to claim 4, characterized in that, In step S110, the task scheduling request is encapsulated in a structured data format, including a task dependency graph, an estimate of the computational load of each task, the required heterogeneous resource types, data input specifications, and service quality requirements. The task parsing unit of the unified communication scheduling controller performs syntax and semantic verification on the task dependency graph to ensure that there are no circular dependencies, and extracts the predecessor-successor relationships and data transmission volume between tasks.

Images