A method for accelerating processing of data between memories in a heterogeneous processor

Abstract

The application provides a method for accelerating data processing between memories of a heterogeneous processor, relates to the technical field of intelligent computing, and is applied to a distributed heterogeneous computing platform. The platform comprises a sensing node and a computing node. The computing node is equipped with an accelerator and an intelligent network card. The method comprises the following steps: initializing the intelligent network card and the accelerator, and obtaining intelligent network card configuration information; registering and configuring the memory address of the accelerator; sending the intelligent network card configuration information and the memory address of the accelerator to the intelligent network card; configuring the intelligent network card, establishing a listening queue, and listening to the request information of remote data transmission; when the computing node receives request information from a remote end, the intelligent network card receives and pre-processes the data, and then sends the data to the memory address of the accelerator; the accelerator stores the data into a circular cache queue, performs data calculation through a calculation unit, sends the calculated result data to the host memory, and the operating system reads the result data. The application improves the computing efficiency and meets the real-time computing requirement.

Description

Technical Field

[0001] This invention relates to the field of intelligent computing technology, and specifically to a method for accelerating data processing between heterogeneous processor memories. Background Technology

[0002] The distributed heterogeneous computing platform comprises perception nodes and compute nodes. The compute nodes are equipped with various accelerators, such as FPGAs, domestically produced GPUs, and domestically produced neural network processors (NPUs), as well as network interface cards (NICs). These accelerators and NICs in the compute nodes can be directly connected or interconnected via PCIe switches. The perception nodes are responsible for transmitting data to the compute nodes' accelerators (XPUs) for processing.

[0003] Traditional data transmission methods suffer from performance bottlenecks and high latency issues in this architecture. Typically, data is transmitted from the sensing node to the network interface card (NIC) of the computing node, and then transferred to host memory via DMA technology. Next, the CPU controls the data transfer to accelerators such as GPUs, NPUs, and FPGAs mounted on the host for computation. This process leads to increased latency, and this traditional approach is no longer suitable, especially in large-scale data processing and real-time computing scenarios.

[0004] Therefore, there is an urgent need for a technological means to accelerate tasks running on distributed XPUs in order to reduce transmission latency and improve computing efficiency. Summary of the Invention

[0005] In view of this, embodiments of this application provide a method for accelerating data processing between heterogeneous processor memories. This method can bypass the host CPU and directly transmit NIC data to XPU memory, enabling the XPU to perform tasks such as neural network inference and transmit the results back to the CPU, thereby reducing transmission latency and improving computing efficiency.

[0006] This application provides the following technical solution: a method for accelerating data processing between heterogeneous processor memories, the method being applied to a distributed heterogeneous computing platform, the distributed heterogeneous computing platform including sensing nodes and computing nodes, wherein the computing nodes are equipped with accelerators and smart network interface cards, the method including the following steps:

[0007] (1) Initialize the smart network card and accelerator on the computing node, configure the protocol and programmable computing module of the smart network card, and obtain the smart network card configuration information; register and configure the memory address of the accelerator;

[0008] (2) The CPU sends the smart network card configuration information and the accelerator memory address to the smart network card driver.

[0009] (3) The smart network card driver configures the smart network card according to the provided smart network card configuration information. The smart network card establishes a listening queue to listen for remote data transmission request information.

[0010] (4) When a request message is sent to the computing node from the remote end, the smart network card receives the request message, performs preprocessing, and then sends the preprocessed data to the memory address of the accelerator.

[0011] (5) The accelerator receives the data and stores it in the ring data management cache queue. Then, it performs data calculation through the computing unit and sends the calculated result data to the host memory.

[0012] (6) The operating system reads the result data and performs subsequent calculations.

[0013] According to one embodiment of this application, in step (1), the application obtains the physical memory address space of the accelerator through the accelerator driver, and converts the physical memory address space into a host memory address that meets the registration requirements of the RDMA protocol, thereby completing the registration and configuration of the accelerator memory address.

[0014] According to one embodiment of this application, in step (1), the smart network card configuration information includes computing function configuration information, which includes data encryption, data decryption, and data preprocessing.

[0015] According to one embodiment of this application, in step (3), the smart network card driver configures the smart network card programmable computing module, and the configuration includes data encryption, data decryption, and data preprocessing.

[0016] According to one embodiment of this application, in step (4), the smart network card sends the preprocessed data to the memory address of the accelerator through the PCIe P2P mechanism.

[0017] According to one embodiment of this application, in step (5), the accelerator receives the data and stores the data in the head of the circular data management cache queue, and then uses the YOLO algorithm of the computing unit to sequentially retrieve data from the tail of the circular data management cache queue for data computation.

[0018] According to one embodiment of this application, in step (5), the calculated result data is sent to the host memory via a DMA engine.

[0019] Compared with the prior art, the beneficial effects that at least one of the technical solutions adopted in the embodiments of this specification can achieve include: the method for accelerating data processing between heterogeneous processors proposed in the embodiments of this invention can accelerate the process of calculating remote data locally and reduce computational latency. Simultaneously, it can bypass the CPU, reducing the consumption of CPU computing resources during the transmission process and alleviating the problem of CPU computing resource constraints in embedded environments. Attached Figure Description

[0020] To more clearly illustrate the technical solutions of the embodiments of this application, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0021] Figure 1 This is a hardware architecture diagram of the accelerated processing method according to an embodiment of the present invention. Detailed Implementation

[0022] The embodiments of this application will now be described in detail with reference to the accompanying drawings.

[0023] The following specific examples illustrate the implementation of this application. Those skilled in the art can easily understand other advantages and effects of this application from the content disclosed in this specification. Obviously, the described embodiments are only a part of the embodiments of this application, and not all of them. This application can also be implemented or applied through other different specific embodiments, and the details in this specification can also be modified or changed based on different viewpoints and applications without departing from the spirit of this application. It should be noted that, in the absence of conflict, the following embodiments and features in the embodiments can be combined with each other. Based on the embodiments in this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0024] like Figure 1 As shown, this embodiment of the invention provides a method for accelerating data processing between heterogeneous processor memories. The method is applied to a distributed heterogeneous computing platform, which includes sensing nodes and computing nodes, wherein the computing nodes are equipped with accelerators and smart network interface cards (NICs). The method includes the following steps:

[0025] (1) Initialize the smart network card and accelerator on the computing node, configure the protocol and programmable computing module of the smart network card, and obtain the smart network card configuration information; register and configure the memory address of the accelerator;

[0026] (2) The CPU sends the smart network card configuration information and the accelerator memory address to the smart network card driver.

[0027] (3) The smart network card driver configures the smart network card according to the provided smart network card configuration information. The smart network card establishes a listening queue to listen for remote data transmission request information.

[0028] (4) When a request message is sent to the computing node from the remote end, the smart network card receives the request message, performs preprocessing, and then sends the preprocessed data to the memory address of the accelerator.

[0029] (5) The accelerator receives the data and stores it in the ring data management cache queue. Then, it performs data calculation through the computing unit and sends the calculated result data to the host memory.

[0030] (6) The operating system reads the result data and performs subsequent calculations.

[0031] This invention provides an accelerated data processing method for heterogeneous processor memory, aiming to improve computational efficiency and reduce data transmission latency. The method involves a smart network interface card (NIC) working in conjunction with an accelerator to directly transmit data to the accelerator's memory, bypassing the host CPU and thus reducing transmission latency. The method includes NIC initialization, accelerator memory address registration, NIC configuration, data reception and processing, and data transmission to host memory. Through this method, the accelerator can efficiently execute computational tasks and transmit the results to the host for further processing. This method is applied to distributed heterogeneous computing platforms, improving computational efficiency and meeting real-time computing requirements.

[0032] The hardware architecture of the acceleration processing method in this embodiment of the invention mainly consists of three parts: host driver and runtime, smart network card, and smart accelerator receiving module.

[0033] The host driver is divided into smart NIC drivers and accelerator drivers. The smart NIC driver can configure and use the NIC's functions, including hardware management, device identification, network protocol support, and packet processing. The accelerator driver mainly includes hardware management, device identification, general computing support, and performance optimization.

[0034] The host runtime is also divided into two parts: the smart NIC runtime and the accelerator runtime. The smart NIC runtime provides a programming interface for the smart NIC, including network protocol settings, network transmission interfaces, and programmable computing module configuration. The accelerator runtime provides a programming interface for the accelerator, including accelerator device management, memory management, computing core management, and data transmission.

[0035] The smart network interface card (NIC) is based on FPGA hardware, on which a portion of the RDMA software stack and a computable module are deployed. The RDMA software stack on the smart NIC offloads CPU network processing tasks and includes an RDMA engine, a network offload engine, and a DMA engine.

[0036] RDMA Engine: The RDMA engine is a hardware component that supports remote memory access. It allows data to be transferred directly between the host and the network without intervention from the host CPU. This helps reduce network communication latency and alleviate the burden on the host CPU.

[0037] Network offload engine: The network offload engine performs network protocols and packet processing instead of delegating these tasks to the host CPU. This includes protocol processing, packet grouping, traffic classification and processing, etc.

[0038] DMA Engine: The DMA (Direct Memory Access) engine is used for high-speed data transfer, transferring data from the network interface to memory or other devices.

[0039] The programmable computing module allows users to perform custom programming at runtime to achieve computing tasks such as data compression, encryption / decryption, and data parsing.

[0040] Intelligent accelerators, represented by general-purpose accelerators like GPUs and FPGAs, possess hardware capable of accelerating general-purpose computing while also executing complex logical instructions (loops, branches, jumps). This invention designs a network receiving unit based on existing accelerator functions, performing buffering and flow control on data transmitted from the intelligent network interface card (NIC). For GPU accelerators, this solution utilizes existing programming libraries to implement a data receiving queue on the GPU's computing cores. For FPGA accelerators, this solution designs a data receiving hardware module outside the existing FPGA computing module, interconnecting it with the computing module on the FPGA. It receives data from the NIC and transmits it to the computing module.

[0041] The data receiving queue adopts a producer-consumer pattern design, constructing a circular queue to achieve heterogeneous decoupling of data reception and transmission from the GPU to the computing module, thus solving the problem of mismatch between transmission and computing speeds. The specific design is as follows:

[0042] A producer module is implemented on the GPU, whose task is to receive data from the network card's DMA and write it to a circular buffer. The producer needs to keep track of the current write position to ensure that the data is stored correctly. If the buffer is full, the producer will wait until there is enough space available.

[0043] A consumer module is implemented on the GPU, whose task is to read data from a circular buffer and process it. The consumer needs to keep track of the current read position to ensure that data is read in order. If the buffer is empty, the consumer will wait for new data to arrive.

[0044] To ensure synchronization between producers and consumers, this scheme introduces synchronization mechanisms such as semaphores and mutexes. These mechanisms are used to avoid race conditions, ensuring that producers and consumers do not access the same data block simultaneously.

[0045] Its main operating process is as follows:

[0046] 1. The application obtains the physical memory address space of the accelerator through the accelerator driver and converts it into a host memory address that conforms to the registration requirements of the RDMA protocol. Then, the network card driver initializes the RDMA transmission, completes the registration of the accelerator memory address, and completes the configuration of the optional computing functions of the smart network card (encryption, decryption, data preprocessing, etc.).

[0047] 2. The smart network card receives data sent from the remote end, performs preprocessing calculations on the data according to the network card configuration, such as encryption and decryption, and then uses the PCIe P2P mechanism to send the data to the registered accelerator memory address mentioned above.

[0048] 3. The accelerator-side data receiving module receives data sent by the network card and caches it in the circular data management queue. Then, the computing unit reads the data from the data management queue and performs calculations. After the calculations are completed, the accelerator sends the data results to the main memory through the DMA engine.

[0049] Next, the implementation process of this embodiment of the invention will be described using the example of a sensing node equipped with sensors sending images to a computing node, and the computing unit performing calculations.

[0050] The perception node is equipped with a smart NIC supporting RDMA networks, while the compute node is equipped with a smart NIC and an accelerator. The compute node runs a target recognition application using the YOLO algorithm. The compute node first initializes the NIC and accelerator, and then allocates a block of memory on the accelerator via an API. The operating system allocates a virtual address space for the accelerator, which contains the virtual addresses of the accelerator memory; this virtual address space is separate from the virtual address space of the host memory.

[0051] The accelerator driver maintains a mapping table that maps the virtual addresses of the accelerator memory to the physical addresses of the accelerator.

[0052] The application uses the accelerator driver to convert the virtual address space allocated by the accelerator into a host memory address that conforms to the registration requirements of the RDMA protocol, and provides it to the smart NIC driver. The smart NIC driver registers the address, then checks the configuration status of the smart NIC's computing function, and sends the accelerator address information and smart NIC configuration information to the smart NIC. The smart NIC configures the accelerator based on this information, and then establishes a listening queue to listen for messages from the remote end.

[0053] When data is sent from a remote location, the smart network card first receives the data, then decrypts it, and sends the decrypted data to the accelerator based on the previously registered accelerator memory address.

[0054] After receiving the data, the accelerator stores it at the head of a circular buffer queue. Then, the computation module running the YOLO algorithm retrieves data from the tail of the buffer queue and performs functional calculations. Finally, the calculated results are sent to the host memory via the DMA engine. After the DMA engine completes the transfer, it notifies the operating system that the data is available. Subsequently, the host reads the recognized data and performs subsequent calculations.

[0055] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A method for accelerating data processing between heterogeneous processor memories, the method being applied to a distributed heterogeneous computing platform, the distributed heterogeneous computing platform including sensing nodes and computing nodes, wherein the computing nodes are equipped with accelerators and smart network interface cards, characterized in that, The method includes the following steps: (1) Initialize the smart network card and accelerator on the computing node, configure the protocol and programmable computing module of the smart network card, and obtain the smart network card configuration information; register and configure the memory address of the accelerator; (2) The CPU sends the smart network card configuration information and the accelerator memory address to the smart network card driver. (3) The smart network card driver configures the smart network card according to the provided smart network card configuration information, and the smart network card establishes a listening queue to listen for remote data transmission request information; (4) When a request is sent to the computing node from the remote end, the smart network card receives the request and preprocesses it, and then sends the preprocessed data to the memory address of the accelerator. (5) The accelerator receives the data and stores it in the head of the ring data management cache queue. Then, the YOLO algorithm of the computing unit sequentially retrieves data from the tail of the ring data management cache queue, performs data calculation, and sends the calculated result data to the host memory through the DMA engine. (6) The operating system reads the result data and performs subsequent calculations.

2. The method for accelerating data processing between heterogeneous processor memories according to claim 1, characterized in that, In step (1), the application obtains the physical memory address space of the accelerator through the accelerator driver, and converts the physical memory address space into a host memory address that meets the registration requirements of the RDMA protocol, thereby completing the registration and configuration of the accelerator memory address.

3. The method for accelerating data processing between heterogeneous processor memories according to claim 1, characterized in that, In step (1), the smart network card configuration information includes computing function configuration information, which includes data encryption, data decryption, and data preprocessing.

4. The method for accelerating data processing between heterogeneous processor memories according to claim 1, characterized in that, In step (3), the smart network card driver configures the smart network card programmable computing module, including data encryption, data decryption, and data preprocessing.

5. The method for accelerating data processing between heterogeneous processor memories according to claim 1, characterized in that, In step (4), the smart network card sends the preprocessed data to the memory address of the accelerator through the PCIe P2P mechanism.

Images