An edge computing method and system capable of realizing direct exchange of remote memory data

Abstract

The present application relates to the technical field of edge computing architecture, and more particularly to an edge computing method and system capable of realizing direct exchange of remote memory data, in the FPGA of a heterogeneous processing unit, innovative integration of key components such as a hardware RoCE v2 RDMA engine, a high-speed memory pool, a reconfigurable calculator, and a flow control and congestion manager, with the aid of a QDMA IP core, a high-speed PCIe data channel is constructed with an edge inference module containing a CPU, a GPU and an AI accelerator, overcoming the disadvantages of low system integration and heavy hardware dependence of traditional RDMA solutions, realizing efficient collaboration of computing acceleration and RDMA function, at the same time, the problem of limited on-chip resources in large-scale connection scenarios can be solved, ensuring stable and efficient data transmission, supporting preprocessing when data flows, deeply integrating with AI inference, breaking the separation pattern of computing and communication, and having strong flow control and scheduling capabilities.

Description

Technical Field

[0001] This invention relates to the field of edge computing architecture technology, and in particular to an edge computing method and system that enables direct exchange of remote memory data. Background Technology

[0002] With the rapid development of emerging fields such as autonomous driving, industrial IoT, and augmented reality, the data interaction requirements between edge computing nodes are showing a trend towards high bandwidth, low latency, and high concurrency. Traditional data transmission methods mostly rely on the operating system kernel and CPU interrupt and copy processing, resulting in long data transmission paths, high latency, and excessive CPU utilization. In high-concurrency and large-scale data interaction scenarios, it is difficult to meet the requirements of real-time performance and energy efficiency.

[0003] To address the aforementioned issues, Remote Direct Memory Access (RDMA) technology emerged. RDMA establishes a zero-copy data channel between hosts, allowing direct data transfer between the memory of different nodes, bypassing operating system kernel and CPU intervention, and significantly reducing transmission latency and CPU load. However, existing RDMA solutions primarily rely on dedicated high-end RDMA network interface cards (RNICs), which have the following shortcomings in edge computing scenarios:

[0004] Low system integration and heavy hardware dependency:

[0005] Dedicated RDMA network cards, as independent hardware devices, need to be used in conjunction with host servers. They are difficult to work closely with computing acceleration units (such as AI accelerators and FPGA logic) in edge computing nodes, resulting in limited system integration and high overall power consumption and cost.

[0006] On-chip resources are limited in large-scale connectivity scenarios:

[0007] In high-concurrency scenarios such as autonomous driving and industrial IoT, it is necessary to maintain hundreds or thousands of RDMA connections. In existing solutions, all Queue Pair (QP) context information and retransmission buffers are stored on-chip BRAM resources. When the number of connections is too large, it will lead to shortage of on-chip storage resources, increased transmission latency, and decreased throughput.

[0008] Lack of coordinated optimization between computation and communication:

[0009] Traditional RDMA solutions focus solely on data transmission and cannot perform preprocessing operations such as compression, encryption, and feature extraction directly during data flow. Furthermore, they cannot be deeply integrated with AI inference computing, resulting in the separation of computing and communication and limiting the overall efficiency of the system.

[0010] Insufficient flow control and priority scheduling capabilities:

[0011] In scenarios with high traffic and high concurrency, the existing RDMA flow control mechanism cannot dynamically adjust bandwidth allocation according to task priority, making it difficult to guarantee low-latency and high-reliability transmission of critical tasks.

[0012] Therefore, there is an urgent need for an RDMA architecture for edge computing scenarios to solve the above problems. Summary of the Invention

[0013] The purpose of this invention is to address the shortcomings of existing technologies by providing an edge computing method and system that enables direct remote memory data exchange.

[0014] To achieve the above objectives, the present invention employs an edge computing system capable of direct remote memory data exchange, comprising a heterogeneous processing unit, wherein the heterogeneous processing unit includes an FPGA and an edge inference module;

[0015] The FPGA integrates a hardware-based RoCE v2 RDMA engine, a high-speed memory pool, a reconfigurable calculator, and a flow control and congestion manager.

[0016] The edge inference module includes a CPU, GPU, and AI accelerator, used to run applications and operating systems, perform edge computing tasks, and manage the control plane of RDMA communication;

[0017] The FPGA and the edge inference module establish a high-speed PCIe data channel through the QDMA IP core to achieve efficient data transmission and collaborative processing.

[0018] The hardware-based RoCE v2 RDMA engine integrates an IP / UDP protocol offloading module and an RDMA transport layer protocol processing module, which are used to directly connect to the high-speed serial transceiver of the FPGA to realize hardware-level parsing, encapsulation and flow control of network packets.

[0019] The high-speed memory pool consists of a large-capacity DDR4 memory connected to the FPGA via an external bus. It is used to store metadata and buffer resources required for RDMA communication, including a send queue, a receive queue, a completion queue, a retransmission queue, QP context information, and memory region information.

[0020] The high-speed memory pool adopts a hierarchical storage management strategy, including on-chip cache and off-chip DRAM storage pool, which are used to store high-priority and low-priority QP contexts and data buffers, respectively.

[0021] The flow control and congestion manager integrates a priority flow control mechanism and a data center quantitative congestion notification mechanism to achieve end-to-end hardware-based flow control and congestion management, ensuring low latency and high reliability for high-priority traffic.

[0022] The reconfigurable calculator is implemented based on FPGA programmable logic and directly accesses off-chip DRAM and host memory. It is used for data preprocessing or hardware-accelerated inference of some AI models during data transmission, realizing the coordinated offloading of computing and communication.

[0023] The edge computing system capable of direct remote memory data exchange also includes a software architecture, which comprises:

[0024] The virtual-physical address mapping module is used to map application-layer virtual addresses to a unified memory management view and allocate physical addresses according to QP priority.

[0025] The RDMA communication driver mechanism is based on the Verbs API to implement user-mode / kernel-mode interfaces, connection management and WQE distribution, and supports QP priority-aware operations.

[0026] The dynamic memory management module is used for memory region registration and access control, and implements priority-based QP context management and storage strategies.

[0027] The task scheduling engine is used to schedule computation tasks and RDMA data transfer tasks, and optimizes the execution strategy by combining QP priority and context position.

[0028] This invention also provides an edge computing method that enables direct remote memory data exchange, applicable to the edge computing system described above that enables direct remote memory data exchange.

[0029] Includes the following steps:

[0030] Register the memory region through the RDMA driver, establish the QP connection, and complete the QP state transition;

[0031] Submitting WQE triggers the hardware to automatically capture data, package and reassemble the data into segments;

[0032] Priority-aware flow control and congestion management are achieved through PFC and DCQCN mechanisms;

[0033] Pre-commit RECV WQE, parses the data packet and writes it directly into memory;

[0034] Generate a CQE and automatically send an ACK to complete the RDMA operation;

[0035] By using a reconfigurable calculator to preprocess received data or perform AI inference, collaborative execution of computation and communication can be achieved.

[0036] The QP connection is established as follows: the two hosts exchange the information required for RDMA communication through an out-of-band connection (TCP / IP), including: QP context and priority, remote operation key (R_KEY), operation address and length.

[0037] In the step of "registering memory regions through RDMA driver, establishing QP connection, and completing QP state transition", high-priority QP / MR information is stored in on-chip cache, and low-priority QP / MR information is dynamically mapped to off-chip DRAM.

[0038] This invention discloses an edge computing method and system capable of direct remote memory data exchange. In practical application, this invention cleverly integrates key components such as a hardware-based RoCE v2 RDMA engine, a high-speed memory pool, a reconfigurable calculator, and a flow control and congestion manager into the FPGA of a heterogeneous processing unit. It also establishes a high-speed PCIe data channel with the edge inference module, which includes a CPU, GPU, and AI accelerator, using a QDMA IP core. This not only overcomes the problems of low system integration and heavy hardware dependence in traditional RDMA solutions, achieving efficient collaboration between the computing acceleration unit and RDMA functionality, but also effectively addresses the challenges of limited on-chip resources in large-scale connection scenarios, ensuring the stability and efficiency of data transmission. Furthermore, preprocessing operations can be performed during data flow, and the system is deeply integrated with AI inference computing, breaking down the separation between computing and communication. In addition, powerful flow control and priority scheduling capabilities ensure low-latency and high-reliability transmission of critical tasks in high-traffic, high-concurrency scenarios, comprehensively improving the data transmission and processing efficiency in edge computing scenarios. Attached Figure Description

[0039] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0040] Figure 1 This is a block diagram of the principle of the edge computing system of the present invention, which enables direct exchange of remote memory data.

[0041] 1-Heterogeneous processing unit, 2-FPGA, 3-Edge inference module, 4-Hardware RoCE v2 RDMA engine, 5-High-speed memory pool, 6-Reconfigurable calculator, 7-Flow control and congestion manager, 8-CPU, 9-GPU, 10-AI accelerator, 11-IP / UDP protocol offloading module, 12-RDMA transport layer protocol processing module, 13-Priority flow control mechanism, 14-Data center quantized congestion notification mechanism, 15-Software architecture, 16-Virtual-physical address mapping module, 17-RDMA communication driver mechanism, 18-Dynamic memory management module, 19-Task scheduling engine, 20-System monitoring module, 21-Power management module. Detailed Implementation

[0042] The embodiments of the present invention are described in detail below. Examples of the embodiments are shown in the accompanying drawings. The embodiments described below with reference to the accompanying drawings are exemplary and intended to explain the present invention, but should not be construed as limiting the present invention.

[0043] Please see Figure 1 , Figure 1 This is a block diagram of the principle of the edge computing system of the present invention, which enables direct exchange of remote memory data.

[0044] This invention provides an edge computing system that enables direct remote memory data exchange, comprising a heterogeneous processing unit 1, wherein the heterogeneous processing unit 1 includes an FPGA 2 and an edge inference module 3;

[0045] The FPGA2 integrates a hardware-based RoCE v2 RDMA engine 4, a high-speed memory pool 5, a reconfigurable calculator 6, and a flow control and congestion manager 7.

[0046] The edge inference module 3 includes a CPU8, a GPU9, and an AI accelerator 10, which are used to run applications and operating systems, perform edge computing tasks, and manage the control plane of RDMA communication.

[0047] The FPGA2 and the edge inference module 3 establish a high-speed PCIe data channel through the QDMA IP core to achieve efficient data transmission and collaborative processing.

[0048] For this specific implementation method,

[0049] The hardware-based RoCE v2 RDMA engine 4 integrates an IP / UDP protocol offloading module 11 and an RDMA transport layer protocol processing module 12, which are used to directly connect to the high-speed serial transceiver of FPGA2 to realize hardware-level parsing, encapsulation and flow control of network packets.

[0050] Secondly, the high-speed memory pool 5 consists of a large-capacity DDR4 memory connected to the FPGA via an external bus, used to store metadata and buffer resources required for RDMA communication, including the transmit queue, receive queue, completion queue, retransmission queue, QP context information and memory area information.

[0051] The high-speed memory pool 5 adopts a hierarchical storage management strategy, including on-chip cache and off-chip DRAM storage pool, which are used to store high-priority and low-priority QP contexts and data buffers, respectively.

[0052] Meanwhile, the flow control and congestion manager 7 integrates a priority flow control mechanism 13 and a data center quantitative congestion notification mechanism 14 to achieve end-to-end hardware-based flow control and congestion management, ensuring low latency and high reliability of high-priority traffic.

[0053] Furthermore, the reconfigurable calculator 6 is implemented based on FPGA2 programmable logic, directly accessing off-chip DRAM and host memory, and is used for data preprocessing or hardware-accelerated inference of some AI models during data transmission, thereby achieving collaborative offloading of computation and communication.

[0054] Furthermore, the edge computing system capable of enabling direct remote memory data exchange also includes a software architecture 15, which includes:

[0055] The virtual-physical address mapping module 16 is used to map application-layer virtual addresses to a unified memory management view and allocate physical addresses according to QP priority;

[0056] RDMA communication driver mechanism 17, based on Verbs API, implements user-mode / kernel-mode interface, connection management and WQE distribution, and supports QP priority-aware operation;

[0057] The dynamic memory management module 18 is used for memory region registration and access control, and implements priority-based QP context management and storage strategies.

[0058] The task scheduling engine 19 is used to schedule computation tasks and RDMA data transfer tasks, and optimizes the execution strategy by combining QP priority and context position.

[0059] Furthermore, the edge computing system capable of direct remote memory data exchange also includes a system monitoring module 20, which is connected to the heterogeneous processing unit 1. The system monitoring module 20 employs a rule-based fault diagnosis algorithm and a machine learning-based anomaly detection algorithm. The rule-based algorithm judges the system's operating status based on preset fault rules and thresholds, issuing an alarm when an anomaly occurs. The machine learning-based algorithm, through training and learning from historical data, establishes a normal operating model of the system. When the deviation between the actual operating data and the model exceeds a certain range, it is determined to be an anomaly and fault diagnosis is performed.

[0060] Finally, the edge computing system enabling direct remote memory data exchange also includes a power management module 21, which employs a dynamic power management algorithm. This algorithm dynamically adjusts the output voltage and current of the power supply based on the system's operating state (e.g., idle, low load, high load) and the power consumption requirements of each hardware module, achieving on-demand power allocation. Simultaneously, when the system enters an idle state, it automatically reduces the supply voltage of some hardware modules or puts them into a low-power mode to save energy.

[0061] Using the edge computing system of this invention that enables direct remote memory data exchange, in practical application, this invention cleverly integrates key components such as a hardware-based RoCE v2 RDMA engine 4, a high-speed memory pool 5, a reconfigurable calculator 6, and a flow control and congestion manager 7 into the FPGA2 of the heterogeneous processing unit 1. It also establishes a high-speed PCIe data channel with the edge inference module 3, which includes a CPU 8, GPU 9, and AI accelerator 10, using a QDMA IP core. This not only overcomes the problems of low system integration and heavy hardware dependence in traditional RDMA solutions, achieving efficient collaboration between the computing acceleration unit and RDMA functions, but also effectively addresses the challenges of limited on-chip resources in large-scale connection scenarios, ensuring the stability and efficiency of data transmission. Simultaneously, preprocessing operations can be performed during data flow, and it is deeply integrated with AI inference computing, breaking down the separation between computing and communication. Furthermore, powerful flow control and priority scheduling capabilities ensure low-latency and high-reliability transmission of critical tasks in high-traffic, high-concurrency scenarios, comprehensively improving the data transmission and processing efficiency in edge computing scenarios.

[0062] This invention also provides an edge computing method that enables direct remote memory data exchange, applicable to the edge computing system described above that enables direct remote memory data exchange.

[0063] Includes the following steps:

[0064] Register the memory region through the RDMA driver, establish the QP connection, and complete the QP state transition;

[0065] Submitting WQE triggers the hardware to automatically capture data, package and reassemble the data into segments;

[0066] Priority-aware flow control and congestion management are achieved through PFC and DCQCN mechanisms;

[0067] Pre-commit RECV WQE, parses the data packet and writes it directly into memory;

[0068] Generate a CQE and automatically send an ACK to complete the RDMA operation;

[0069] The reconfigurable calculator 6 can preprocess the received data or perform AI inference to achieve collaborative execution of computing and communication.

[0070] The QP connection is established as follows: the two hosts exchange the information required for RDMA communication through an out-of-band connection (TCP / IP), including: QP context and priority, remote operation key (R_KEY), operation address and length.

[0071] In the step of "registering memory regions through RDMA driver, establishing QP connection, and completing QP state transition", high-priority QP / MR information is stored in on-chip cache, and low-priority QP / MR information is dynamically mapped to off-chip DRAM.

[0072] Furthermore, a zero-copy mechanism is used to directly transfer data from the host memory to the FPGA2 transmit buffer;

[0073] Furthermore, legitimate data is written directly to the application buffer via the QDMA channel, bypassing CPU8 and the kernel protocol stack.

[0074] Using the edge computing method and system of this invention that enables direct remote memory data exchange, in specific applications, this invention cleverly integrates key components such as a hardware-based RoCE v2 RDMA engine 4, a high-speed memory pool 5, a reconfigurable calculator 6, and a flow control and congestion manager 7 into the FPGA2 of the heterogeneous processing unit 1. It also establishes a high-speed PCIe data channel with the edge inference module 3, which includes a CPU 8, GPU 9, and AI accelerator 10, using a QDMA IP core. This not only overcomes the problems of low system integration and heavy hardware dependence in traditional RDMA solutions, achieving efficient collaboration between the computing acceleration unit and RDMA functions, but also effectively addresses the challenges of limited on-chip resources in large-scale connection scenarios, ensuring the stability and efficiency of data transmission. Simultaneously, preprocessing operations can be performed during data flow, and it is deeply integrated with AI inference computing, breaking down the separation between computing and communication. Furthermore, powerful flow control and priority scheduling capabilities ensure low-latency and high-reliability transmission of critical tasks in high-traffic, high-concurrency scenarios, comprehensively improving the data transmission and processing efficiency in edge computing scenarios.

[0075] Beneficial effects:

[0076] (1) Extremely low latency and high-performance transmission;

[0077] The RoCE v2 RDMA offloading engine, implemented in hardware via FPGA2, combined with a zero-copy data channel, enables end-to-end memory-to-memory direct access.

[0078] Data is written directly from remote nodes to local memory without CPU intervention or operating system kernel copying, and remote memory access latency can reach the microsecond level.

[0079] Hardware-based protocol parsing, segmentation and reassembly, and completion notification mechanisms ensure high data throughput while maintaining extremely low CPU utilization.

[0080] (2) High integration and low system cost;

[0081] The RDMA network function, on-chip / off-chip cache, internal data exchange channel (QDMA), and reconfigurable computing unit are integrated into a single FPGA2, tightly coupled with the edge inference module 3.

[0082] It eliminates the need for a separate dedicated network card and complex external storage, resulting in a smaller system size, lower power consumption, and reduced overall system cost.

[0083] The tiered storage architecture (on-chip cache + off-chip DRAM) solves the problem of limited on-chip resources in large-scale Queue Pair (QP) scenarios, ensuring the stability of high-concurrency data exchange.

[0084] (3) Priority awareness and reliable flow control;

[0085] It integrates PFC (Priority Flow Control) and DCQCN (Data Center Quantized Congestion Notification) hardware modules to achieve priority-based flow scheduling and end-to-end congestion control.

[0086] High-priority RDMA traffic maintains low-latency transmission even during network congestion, while making full use of network bandwidth to improve the stability and reliability of data transmission.

[0087] Hardware-based flow control and congestion management eliminate the need for CPU8 intervention, avoiding software stack latency and congestion response delays.

[0088] (4) Highly flexible and customizable;

[0089] FPGA2's reconfigurability allows users to customize the number of RDMA ports, queues, and data path strategies.

[0090] Data preprocessing, encryption / decryption, AI inference and other computational offloading tasks can be deployed inside the FPGA2, achieving deep integration of computing and communication.

[0091] The Reconfigurable Calculator 6 (RCU), which supports dynamic reconfiguration, directly accesses off-chip DRAM and host memory, enabling real-time computation of edge intelligent tasks.

[0092] Compared to fixed-function ASIC network cards, this solution has significant advantages in flexibility and functional scalability, and can adapt to the needs of different application scenarios.

[0093] (5) High reliability and scalability;

[0094] Layered QP / MR storage, hardware completion queues (CQ), and automatic retransmission mechanisms ensure data integrity and task reliability under large-scale concurrent communication.

[0095] Multi-node edge computing collaboration can execute computing tasks in parallel based on shared memory, maintain global state consistency through a lightweight message synchronization mechanism, and support the horizontal scaling of edge computing clusters.

[0096] The above description discloses only one preferred embodiment of the present invention, and should not be construed as limiting the scope of the present invention. Those skilled in the art will understand that all or part of the processes of the above embodiments can be implemented, and equivalent changes made in accordance with the claims of the present invention are still within the scope of the invention.

Claims

1. An edge computing system capable of enabling direct remote memory data exchange, characterized in that, It includes a heterogeneous processing unit, which includes an FPGA and an edge inference module; The FPGA integrates a hardware-based RoCE v2 RDMA engine, a high-speed memory pool, a reconfigurable calculator, and a flow control and congestion manager. The edge inference module includes a CPU, GPU, and AI accelerator, used to run applications and operating systems, perform edge computing tasks, and manage the control plane of RDMA communication; The FPGA and the edge inference module establish a high-speed PCIe data channel through the QDMAIP core to achieve efficient data transmission and collaborative processing; The flow control and congestion manager integrates a priority flow control mechanism and a data center quantitative congestion notification mechanism to achieve end-to-end hardware-based flow control and congestion management, ensuring low latency and high reliability for high-priority traffic. The reconfigurable calculator is implemented based on FPGA programmable logic and directly accesses off-chip DRAM and host memory. It is used for data preprocessing or hardware-accelerated inference of some AI models during data transmission, realizing the coordinated offloading of computing and communication. The edge computing system enabling direct remote memory data exchange also includes a software architecture, which comprises: The virtual-physical address mapping module is used to map application-layer virtual addresses to a unified memory management view and allocate physical addresses according to QP priority. The RDMA communication driver mechanism is based on VerbsAPI to implement user-mode / kernel-mode interfaces, connection management and WQE distribution, and supports QP priority-aware operations. The dynamic memory management module is used for memory region registration and access control, and implements priority-based QP context management and storage strategies. The task scheduling engine is used to schedule computation tasks and RDMA data transfer tasks, and optimizes the execution strategy by combining QP priority and context position.

2. The edge computing system capable of direct remote memory data exchange as described in claim 1, characterized in that, The hardware-based RoCE v2 RDMA engine integrates an IP / UDP protocol offloading module and an RDMA transport layer protocol processing module, which are used to directly connect to the high-speed serial transceiver of the FPGA to realize hardware-level parsing, encapsulation and flow control of network packets.

3. The edge computing system capable of direct remote memory data exchange as described in claim 2, characterized in that, The high-speed memory pool consists of a large-capacity DDR4 memory connected to the FPGA via an external bus. It is used to store metadata and buffer resources required for RDMA communication, including transmit queue, receive queue, complete queue, retransmission queue, QP context information and memory region information. The high-speed memory pool adopts a hierarchical storage management strategy, including on-chip cache and off-chip DRAM storage pool, which are used to store high-priority and low-priority QP contexts and data buffers, respectively.

4. An edge computing method enabling direct remote memory data exchange, applied to the edge computing system enabling direct remote memory data exchange as described in claim 3. Its features are, Includes the following steps: Register the memory region through the RDMA driver, establish the QP connection, and complete the QP state transition; Submitting WQE triggers the hardware to automatically capture data, package and reassemble the data into segments; Priority-aware flow control and congestion management are achieved through PFC and DCQCN mechanisms; Pre-commit RECVWQE, parses the data packet and writes it directly into memory; Generate a CQE and automatically send an ACK to complete the RDMA operation; By using a reconfigurable calculator to preprocess received data or perform AI inference, collaborative execution of computation and communication can be achieved. The QP connection is established as follows: the two hosts exchange the information required for RDMA communication through an out-of-band connection, including: QP context and priority, remote operation key, operation address and length.

5. The edge computing method for enabling direct remote memory data exchange as described in claim 4, characterized in that, In the step "Register memory region through RDMA driver, establish QP connection and complete QP state transition", high-priority QP / MR information is stored in on-chip cache, and low-priority QP / MR information is dynamically mapped to off-chip DRAM.

Images