Edge computing method and system capable of realizing direct exchange of remote memory data

Abstract

The invention relates to the technical field of edge computing architectures, in particular to an edge computing method and system capable of achieving direct exchange of remote memory data, in an FPGA of a heterogeneous processing unit, a hardware RoCE v2 RDMA engine, a high-speed memory pool, a reconfigurable calculator, a flow control and congestion manager and other key components are creatively integrated, and by means of a QDMA IP core, the RDMA engine, the high-speed memory pool, the reconfigurable calculator, the flow control and congestion manager and other key components are integrated. A high-speed PCIe data channel is constructed with an edge reasoning module containing a CPU, a GPU and an AI accelerator, the defects that a traditional RDMA scheme is low in system integration degree and heavy in hardware dependence are overcome, efficient cooperation of calculation acceleration and the RDMA function is achieved, meanwhile, the problem that on-chip resources are limited in a large-scale connection scene can be solved, stable and efficient data transmission is guaranteed, preprocessing is supported during data flowing, and the data flow speed is increased. The method is deeply fused with AI reasoning, a calculation and communication separation pattern is broken, and the flow control and scheduling capability is strong.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of edge computing architecture, and in particular to an edge computing method and system capable of realizing direct exchange of remote memory data. BACKGROUND

[0002] With the rapid development of emerging fields such as autonomous driving, industrial Internet of Things, and augmented reality, the demand for data interaction between edge computing nodes presents a trend of high bandwidth, low latency, and high concurrency. Traditional data transmission methods mostly rely on the interruption and copy processing of the operating system kernel and CPU, resulting in a long data transmission path, high latency, and excessive CPU occupancy. In high-concurrency and large-scale data interaction scenarios, it is difficult to meet the requirements of real-time performance and energy efficiency ratio.

[0003] To solve the above problems, the Remote Direct Memory Access (RDMA) technology has emerged. RDMA establishes a zero-copy data channel between hosts, allowing data to be directly transmitted between the memories of different nodes, bypassing the operating system kernel and CPU intervention, greatly reducing transmission delay and CPU load. However, existing RDMA solutions mainly rely on dedicated high-end RDMA network cards (RNICs), which have the following shortcomings in edge computing scenarios: Low system integration and heavy hardware dependence: As a standalone hardware device, the dedicated RDMA network card needs to be used with the host server, making it difficult to closely collaborate with computing acceleration units (such as AI accelerators, FPGA logic) in edge computing nodes, limiting system integration, and increasing overall power consumption and cost.

[0004] On-chip resource constraints in large-scale connection scenarios: In high-concurrency scenarios such as autonomous driving and industrial Internet of Things, hundreds or thousands of RDMA connections need to be maintained. In existing solutions, all Queue Pair (QP) context information and retransmission buffers are stored on-chip BRAM resources. When the connection scale is too large, it will cause on-chip storage resource shortage, transmission delay increase, and throughput decrease.

[0005] Lack of collaborative optimization of computing and communication: Traditional RDMA solutions only focus on data transmission and cannot directly perform preprocessing operations such as compression, encryption, feature extraction, etc. during data flow, nor can they deeply integrate with AI inference computing, resulting in separation of computing and communication, limiting the overall efficiency of the system.

[0006] Inadequate flow control and priority scheduling capabilities: In a multi-flow and high-concurrency scenario, the existing RDMA flow control mechanism cannot dynamically adjust the bandwidth allocation according to the task priority, and it is difficult to guarantee the low-delay transmission and high-reliability transmission of critical tasks.

[0007] Therefore, there is an urgent need for an RDMA architecture for edge computing scenarios to solve the above problems. SUMMARY

[0008] The present application aims to overcome the shortcomings of the prior art and provide an edge computing method and system that can realize direct exchange of remote memory data.

[0009] To achieve the above-mentioned purpose, the edge computing system that can realize direct exchange of remote memory data comprises a heterogeneous processing unit, which includes an FPGA and an edge inference module; The FPGA is integrated with a hardware RoCE v2 RDMA engine, a high-speed memory pool, a reconfigurable calculator, and a flow control and congestion manager; The edge inference module includes a CPU, a GPU, and an AI accelerator, which are used to run application programs and operating systems, execute edge computing tasks, and manage the control plane of RDMA communication; The FPGA and the edge inference module are connected through a QDMA IP core to establish a high-speed PCIe data channel, realizing efficient data transmission and collaborative processing.

[0010] The hardware RoCE v2 RDMA engine is integrated with an IP / UDP protocol offload module and an RDMA transport layer protocol processing module, which are used to directly connect to the high-speed serial transceiver of the FPGA, realize hardware-level parsing, packaging, and flow control of network packets.

[0011] The high-speed memory pool is composed of a large-capacity DDR4 memory connected to the FPGA through an external bus, which is used to store the metadata and buffer resources required for RDMA communication, including sending queues, receiving queues, completion queues, retransmission queues, QP context information, and memory region information. The high-speed memory pool adopts a hierarchical storage management strategy, including on-chip cache and off-chip DRAM storage pool, which are used to store high-priority and low-priority QP context and data buffer, respectively.

[0012] The flow control and congestion manager integrates a priority flow control mechanism and a data center quantization congestion notification mechanism, realizing end-to-end hardware flow control and congestion management, and guaranteeing low delay and high reliability of high-priority traffic.

[0013] The reconfigurable calculator is based on FPGA programmable logic implementation, directly accesses off-chip DRAM and host memory, and is used for data preprocessing or hardware acceleration inference of a partial AI model in a data transmission process, so that the cooperative offloading of calculation and communication is realized.

[0014] The edge computing system capable of realizing direct exchange of remote memory data further comprises a software architecture, and the software architecture comprises: A virtual-physical address mapping module is configured to map an application layer virtual address into a unified memory management view and assign a physical address according to a QP priority. An RDMA communication driving mechanism is configured to realize a user state / kernel state interface, connection management and WQE issuing based on a Verbs API, and support QP priority awareness operation. A dynamic memory management module is configured to register and control the authority of a memory region, realize QP context management and storage strategy based on priority, and realize the scheduling between a computing task and an RDMA data carrying task, and optimize an execution strategy in combination with a QP priority and a context position. A task scheduling engine is configured to realize the scheduling between a computing task and an RDMA data carrying task, and optimize an execution strategy in combination with a QP priority and a context position.

[0015] The application further provides an edge computing method capable of realizing direct exchange of remote memory data, applied to the edge computing system capable of realizing direct exchange of remote memory data. The method comprises the following steps: Register a memory region through an RDMA driver, establish a QP connection, and complete QP state migration. Submit a WQE, trigger hardware to automatically grab data, and perform data packaging and segmentation and recombination. Realize priority-aware traffic control and congestion management through a PFC and a DCQCN mechanism. Pre-submit a RECV WQE, analyze a data packet and directly write into a memory. Generate a CQE and automatically send an ACK, and complete an RDMA operation. Preprocess or perform AI inference on received data through the reconfigurable calculator, and realize cooperative execution of calculation and communication. The method for establishing a QP connection is that: two hosts exchange information required for RDMA communication through an out-of-band connection (TCP / IP), including: a QP context and a priority, a remote operation key (R_KEY), an operation address and a length.

[0016] In the step of registering a memory region through an RDMA driver, establishing a QP connection, and completing QP state migration, high-priority QP / MR information is stored in an on-chip cache, and low-priority QP / MR information is dynamically mapped to an off-chip DRAM.

[0017] The edge computing method and system capable of realizing direct exchange of remote memory data of the application can overcome the problems of low system integration and heavy hardware dependence of the traditional RDMA scheme, realize efficient cooperation of the computing acceleration unit and the RDMA function, effectively cope with the challenge of limited on-chip resources in large-scale connection scenarios, guarantee the stability and efficiency of data transmission, simultaneously perform preprocessing operations in the data flow process and deeply integrate with AI inference calculation, break the separation situation of calculation and communication, and ensure low delay and high reliability transmission of key tasks in multi-traffic and high-concurrency scenarios, thereby comprehensively improving the data transmission and processing efficiency in the edge computing scenario. BRIEF DESCRIPTION OF DRAWINGS

[0018] In order to more clearly illustrate the technical solutions in the embodiments of the application or the prior art, the following will briefly introduce the drawings needed to be used in the embodiment or prior art description. Obviously, the drawings in the following description are only some embodiments of the application, and for those skilled in the art, other drawings can also be obtained without creative labor based on these drawings.

[0019] Figure 1 is the principle diagram of the edge computing system capable of realizing direct exchange of remote memory data of the application.

[0020] 1-heterogeneous processing unit, 2-FPGA, 3-edge inference module, 4-hardware RoCE v2 RDMA engine, 5-high-speed memory pool, 6-reconfigurable calculator, 7-flow control and congestion manager, 8-CPU, 9-GPU, 10-AI accelerator, 11-IP / UDP protocol offload module, 12-RDMA transport layer protocol processing module, 13-priority flow control mechanism, 14-data center quantization congestion notification mechanism, 15-software architecture, 16-virtual-physical address mapping module, 17-RDMA communication driving mechanism, 18-dynamic memory management module, 19-task scheduling engine, 20-system monitoring module, 21-power management module. DETAILED DESCRIPTION

[0021] The embodiments of the application will be described in detail below, examples of which are shown in the drawings. The embodiments described below by referring to the drawings are exemplary and are intended to explain the application, and cannot be understood as a limitation of the application.

[0022] Referring to Figure 1 , Figure 1 is a principle block diagram of the edge computing system of the present application capable of realizing direct exchange of remote memory data.

[0023] The present application provides an edge computing system capable of realizing direct exchange of remote memory data, comprising a heterogeneous processing unit 1, which comprises an FPGA 2 and an edge inference module 3. The FPGA 2 is integrated with a hardware RoCE v2 RDMA engine 4, a high-speed memory pool 5, a reconfigurable calculator 6 and a flow control and congestion manager 7. The edge inference module 3 comprises a CPU 8, a GPU 9 and an AI accelerator 10, which are used to run application programs and operating systems, execute edge computing tasks and manage the control plane of RDMA communication. The FPGA 2 and the edge inference module 3 are connected through a QDMA IP core to establish a high-speed PCIe data channel, realizing efficient transmission and cooperative processing of data.

[0024] For this specific embodiment, The hardware RoCE v2 RDMA engine 4 is integrated with an IP / UDP protocol offload module 11 and an RDMA transport layer protocol processing module 12, which are used to be directly connected to the high-speed serial transceiver of the FPGA 2, realizing hardware-level analysis, encapsulation and flow control of network messages.

[0025] Secondly, the high-speed memory pool 5 is composed of a large-capacity DDR4 memory connected to the FPGA through an external bus, which is used to store metadata and buffer resources required for RDMA communication, including sending queues, receiving queues, completion queues, retransmission queues, QP context information and memory region information. The high-speed memory pool 5 adopts a hierarchical storage management strategy, including on-chip cache and off-chip DRAM memory pool, which are respectively used to store high-priority and low-priority QP context and data buffer.

[0026] Meanwhile, the flow control and congestion manager 7 is integrated with a priority flow control mechanism 13 and a data center quantization congestion notification mechanism 14, realizing end-to-end hardware flow control and congestion management, guaranteeing low delay and high reliability of high-priority traffic.

[0027] And the reconfigurable calculator 6 is realized based on programmable logic of the FPGA 2, directly accessing off-chip DRAM and host memory, which is used for data preprocessing or hardware-accelerated inference of part of AI models during data transmission, realizing cooperative offloading of computation and communication.

[0028] Further, the edge computing system capable of direct exchange of remote memory data further comprises a software architecture 15, which comprises: a virtual-physical address mapping module 16 for mapping application layer virtual addresses into a unified memory management view and assigning physical addresses according to QP priorities; an RDMA communication driving mechanism 17 for realizing user state / kernel state interface, connection management and WQE issuing based on Verbs API, and supporting QP priority-aware operation; a dynamic memory management module 18 for registration and permission control of memory regions, and realizing priority-based QP context management and storage strategy; a task scheduling engine 19 for scheduling between computing tasks and RDMA data carrying tasks, and combining QP priority and context location to optimize execution strategy.

[0029] Thirdly, the edge computing system capable of direct exchange of remote memory data further comprises a system monitoring module 20 connected with the heterogeneous processing unit 1, which adopts a rule-based fault diagnosis algorithm and a machine learning-based anomaly detection algorithm. The rule-based algorithm judges the running state of the system according to preset fault rules and thresholds, and issues an alarm when an abnormal situation occurs; the machine learning-based algorithm establishes a normal running model of the system through training and learning of historical data, and determines an abnormal situation and performs fault diagnosis when the deviation between actual running data and the model exceeds a certain range.

[0030] Finally, the edge computing system capable of direct exchange of remote memory data further comprises a power management module 21, which adopts a dynamic power management algorithm. The algorithm dynamically adjusts the output voltage and current of the power supply according to the working state (such as idle, low load and high load) of the system and the power consumption demand of each hardware module, realizes on-demand allocation of the power supply, and at the same time, when the system enters an idle state, automatically reduces the supply voltage of part of the hardware modules or makes them enter a low-power mode to save energy.

[0031] The edge computing system capable of realizing direct exchange of remote memory data provided by the application can overcome the problems of low system integration and heavy hardware dependence of the traditional RDMA scheme, realize efficient cooperation between the computing acceleration unit and the RDMA function, effectively cope with the challenge of limited on-chip resources in a large-scale connection scenario, guarantee the stability and efficiency of data transmission, simultaneously perform preprocessing operations in the data flow process and deeply integrate with AI inference calculation, break the separation situation of calculation and communication, and further ensure the low delay and high reliability transmission of key tasks in a multi-traffic and high-concurrency scenario, and comprehensively improve the data transmission and processing efficiency in the edge computing scenario.

[0032] The application further provides an edge computing method capable of realizing direct exchange of remote memory data, applied to the edge computing system capable of realizing direct exchange of remote memory data, including the following steps: registering a memory area through an RDMA driver, establishing a QP connection, and completing QP state migration; submitting a WQE, triggering automatic data grabbing of hardware, and performing data packaging and segmentation and recombination; realizing priority-aware traffic control and congestion management through PFC and DCQCN mechanisms; pre-submitting a RECV WQE, analyzing a data packet and directly writing into a memory; generating a CQE and automatically sending an ACK, and completing an RDMA operation; performing preprocessing or AI inference on received data through the reconfigurable calculator 6, and realizing cooperative execution of calculation and communication; The method for establishing a QP connection is that: both hosts exchange information required for RDMA communication through an out-of-band connection (TCP / IP), including: QP context, priority, remote operation key (R_KEY), operation address and length.

[0033] In the step of "registering a memory area through an RDMA driver, establishing a QP connection, and completing QP state migration", high-priority QP / MR information is stored in an on-chip cache, and low-priority QP / MR information is dynamically mapped to off-chip DRAM.

[0034] And, a zero-copy mechanism is adopted to directly carry data from the host memory to the FPGA2 sending buffer; Further, the legal data is directly written into the application buffer through the QDMA channel, bypassing the CPU8 and the kernel protocol stack.

[0035] The edge computing method and system of the application can realize direct exchange of remote memory data. In specific use, the application ingeniously integrates key components such as a hardware RoCE v2 RDMA engine 4, a high-speed memory pool 5, a reconfigurable calculator 6, and a flow control and congestion manager 7 in the FPGA2 of the heterogeneous processing unit 1, and establishes a high-speed PCIe data channel with the edge inference module 3 containing the CPU8, GPU9, and AI accelerator 10 by means of the QDMA IP core. Not only does it overcome the problems of low system integration and heavy hardware dependence of traditional RDMA schemes, but also realizes efficient collaboration between the computing acceleration unit and the RDMA function. It can effectively cope with the challenge of limited on-chip resources in large-scale connection scenarios, ensuring the stability and efficiency of data transmission. At the same time, it can perform preprocessing operations during data flow and deeply integrate with AI inference calculation, breaking the separation between computation and communication. In addition, the powerful flow control and priority scheduling capability can ensure low latency and high reliability transmission of key tasks in multi-traffic and high-concurrency scenarios, and comprehensively improve the data transmission and processing efficiency in edge computing scenarios.

[0036] Advantages: (1) Extremely low latency and high-performance transmission; The RoCE v2 RDMA offload engine realized by FPGA2, combined with the zero-copy data channel, realizes end-to-end memory-to-memory direct access.

[0037] Data is directly written from the remote node to the local memory without CPU8 intervention and operating system kernel copying, and the remote memory access delay can reach microseconds.

[0038] The hardware protocol analysis, segmentation and recombination, and completion notification mechanism ensure high data throughput while keeping CPU8 occupancy extremely low.

[0039] (2) High integration and low system cost; The RDMA network function, on-chip / off-chip cache, internal data exchange channel (QDMA), and reconfigurable computing unit are integrated in a single FPGA2, tightly coupled with the edge inference module 3.

[0040] Without independent dedicated network cards and complex external storage, the system is smaller in size, lower in power consumption, and lower in overall system cost.

[0041] The hierarchical storage architecture (on-chip cache + off-chip DRAM) solves the problem of on-chip resource shortage in large-scale Queue Pair (QP) scenarios, ensuring the stability of high-concurrency data exchange.

[0042] (3) Priority awareness and reliable flow control; Integrating PFC (Priority Flow Control) and DCQCN (Data Center Quantized Congestion Notification) hardware modules, priority-based traffic scheduling and end-to-end congestion control are achieved.

[0043] High-priority RDMA traffic still maintains low-latency transmission when the network is congested, while fully utilizing network bandwidth, improving the stability and reliability of data transmission.

[0044] Hardware flow control and congestion management do not require CPU8 intervention, avoiding software stack delay and congestion response lag.

[0045] (4) High flexibility and customizability; The reconfigurable nature of FPGA2 allows users to customize the number of RDMA ports, queue numbers, and data path strategies.

[0046] Data preprocessing, encryption / decryption, AI inference, and other computational offloading tasks can be deployed within FPGA2 to achieve deep integration of computation and communication.

[0047] The reconfigurable calculator 6 (RCU) with dynamic reconfiguration supports direct access to off-chip DRAM and host memory, enabling real-time computation of edge intelligence tasks.

[0048] Compared with fixed-function ASIC network cards, this solution has significant advantages in flexibility and functional scalability, and can adapt to different application scenarios.

[0049] (5) High reliability and scalability; Hierarchical QP / MR storage, hardware-completed queue (CQ), and automatic retransmission mechanisms ensure data integrity and task reliability in large-scale concurrent communication.

[0050] Multi-node edge computing collaboration can perform computing tasks in parallel based on shared memory, maintain global state consistency through a lightweight message synchronization mechanism, and support horizontal expansion of edge computing clusters.

[0051] The above disclosed is only a preferred embodiment of the present application, of course, cannot be limited by this to limit the scope of the present application, the person skilled in the art can understand that the implementation of all or part of the above-mentioned processes, and according to the equivalent changes of the claims of the present application, still belong to the scope covered by the present application.

Claims

1. An edge computing system capable of remote memory data direct exchange, characterized in that, it comprises a heterogeneous processing unit comprising an FPGA and an edge inference module; the FPGA is integrated with a hardwareized RoCE v2 RDMA engine, a high-speed memory pool, a reconfigurable calculator, and a flow control and congestion manager; the edge inference module comprises a CPU, a GPU, and an AI accelerator, for running application programs and operating systems, performing edge computing tasks, and managing the control plane of RDMA communication; a high-speed PCIe data channel is established between the FPGA and the edge inference module through a QDMA IP core to achieve efficient data transmission and collaborative processing.

2. The edge computing system capable of remote memory data direct exchange according to claim 1, characterized in that, the hardwareized RoCE v2 RDMA engine is integrated with an IP / UDP protocol offload module and an RDMA transport layer protocol processing module, for directly connecting to the high-speed serial transceiver of the FPGA to achieve hardware-level parsing, encapsulation, and flow control of network packets.

3. The edge computing system capable of remote memory data direct exchange according to claim 2, characterized in that, the high-speed memory pool is composed of a large-capacity DDR4 memory connected to the FPGA through an external bus, for storing metadata and buffer resources required for RDMA communication, including sending queues, receiving queues, completion queues, retransmission queues, QP context information, and memory region information; the high-speed memory pool adopts a layered storage management strategy, including on-chip cache and off-chip DRAM storage pool, for storing high-priority and low-priority QP contexts and data buffers, respectively.

4. The edge computing system capable of remote memory data direct exchange according to claim 3, characterized in that, the flow control and congestion manager integrates a priority flow control mechanism and a data center quantized congestion notification mechanism to achieve end-to-end hardwareized flow control and congestion management, ensuring low latency and high reliability of high-priority traffic.

5. The edge computing system capable of remote memory data direct exchange according to claim 4, characterized in that, the reconfigurable calculator is implemented based on the programmable logic of the FPGA, directly accessing off-chip DRAM and host memory, for data preprocessing or hardware-accelerated inference of partial AI models during data transmission, achieving collaborative offloading of computation and communication.

6. The edge computing system capable of remote memory data direct exchange according to claim 5, characterized in that, the edge computing system capable of remote memory data direct exchange further comprises a software architecture, which comprises: a virtual-physical address mapping module for mapping application layer virtual addresses to a unified memory management view and allocating physical addresses according to QP priority; an RDMA communication driving mechanism based on Verbs API to implement user / kernel interfaces, connection management, and WQE delivery, supporting QP priority-aware operations; A dynamic memory management module is used for registration and permission control of memory regions, and priority-based QP context management and storage strategies are implemented. A task scheduling engine is used for scheduling between computing tasks and RDMA data carrying tasks, and a QP priority and context position optimized execution strategy is combined.

7. An edge computing method capable of realizing direct exchange of remote memory data, applied to the edge computing system capable of realizing direct exchange of remote memory data according to claim 6. Its characteristics are that The method comprises the following steps: Registering a memory region through an RDMA driver, establishing a QP connection, and completing QP state migration; Submitting a WQE, triggering automatic data grabbing by hardware, and performing data packaging and segmented recombination; Priority-aware traffic control and congestion management are achieved through PFC and DCQCN mechanisms; Pre-submission of a RECV WQE, analysis of a data packet, and direct writing into memory; A CQE is generated and an ACK is automatically sent, and an RDMA operation is completed; Received data is preprocessed or AI inference is performed through a reconfigurable calculator, and cooperative execution of computing and communication is achieved; The method for establishing a QP connection is as follows: both hosts exchange information required for RDMA communication through an out-of-band connection, including: QP context and priority, remote operation key, operation address, and length.

8. The edge computing method capable of realizing remote memory data direct exchange according to claim 7, characterized in that In the step "registering a memory region through an RDMA driver, establishing a QP connection, and completing QP state migration", high-priority QP / MR information is stored in an on-chip cache, and low-priority QP / MR information is dynamically mapped to off-chip DRAM.

Images