Computing engine communication method and apparatus

Abstract

The application discloses a computing engine communication method and device, and applies to the technical field of computing engine communication, and comprises the following steps: obtaining device-side program data sent by a host-side computing engine; wherein the device-side program data is program data obtained by compiling program code containing a communication function and a computing task function; the communication function is a function created based on communication demand; the device-side program data is executed by using a device-side computing engine, target data is calculated based on the computing task function to obtain a calculation result, and the calculation result is sent to a target receiving end based on the communication function, so that communication between the device-side computing engine and a computing engine in the target receiving end is realized. The host-side computing engine can be avoided from allocating a communication task, the computing power of the host-side computing engine is released, and the communication scene of different computing engines in the same node and between different nodes is adapted.

Description

Technical Field

[0001] This application relates to the field of computing engine communication technology, and in particular to computing engine communication methods and apparatus. Background Technology

[0002] With the development of Moore's Law, computing power has evolved from increasing the clock speed of individual processors, increasing the number of processors within a single chip, and then increasing the number of chips within a single computing network (such as large-scale cloud computing networks). Currently, the benefits brought by chip manufacturing processes have almost disappeared, and relying on simply increasing the number of chips has encountered a series of problems such as computing power mismatch and high energy consumption. For the diverse and massive computing tasks currently being undertaken, deploying multiple computing engines with different architectures within the same system—such as CPUs (Central Processing Units), GPUs (Graphics Processing Units), and FPGAs (Field Programmable Gate Arrays)—and maximizing computing power utilization by leveraging the characteristics of each architecture—either serving different stages of the same task or serving different tasks—has become an industry consensus and development trend.

[0003] Due to legacy issues in traditional computer architecture systems' network software protocol stacks and hardware devices, in a system with diverse, heterogeneous computing engines, the engine chips, PCIe (Peripheral Component Interconnect Express) or local interconnect buses, and network cards of various computing nodes operate independently. This results in highly inefficient communication and computational performance, with limited freedom and flexibility. Consequently, large-scale computing networks not only face challenges related to the computing power of individual devices but also the bottleneck indirectly caused by inefficient communication between computing engines of different architectures. Existing communication schemes between different computing engines are typically divided into intra-node communication schemes and inter-node communication schemes. Furthermore, communication tasks usually require the host-side computing engine, leaving the host-side computing workload significantly unmet. Summary of the Invention

[0004] In view of this, the purpose of this application is to provide a computing engine communication method and apparatus that can avoid the host-side computing engine from allocating communication tasks, freeing up the computing power of the host-side computing engine, and is suitable for communication scenarios between different computing engines within the same node and between computing engines of different nodes. The specific solution is as follows:

[0005] In a first aspect, this application discloses a computing engine communication method, applied to a first device, comprising:

[0006] Obtain device-side program data sent by the host-side computing engine; wherein, the device-side program data is program data obtained by compiling program code containing communication functions and computing task functions, and the communication functions are functions created based on communication requirements;

[0007] The device-side computing engine executes the device-side program data, performs calculations on the target data based on the computing task function to obtain the calculation result, and sends the calculation result to the target receiving end based on the communication function, so as to realize the communication between the device-side computing engine and the computing engine in the target receiving end.

[0008] Optionally, obtaining the device-side program data sent by the host-side computing engine includes:

[0009] The device-side program data sent by the host-side computing engine is obtained through the RDMA module; wherein, the RDMA module and the device-side computing engine are integrated in the same chip;

[0010] Accordingly, sending the calculation result to the target receiving end based on the communication function includes: sending the calculation result to the target receiving end based on the communication function and through the RDMA module.

[0011] Optionally, the step of obtaining device-side program data sent by the host-side computing engine through the RDMA module includes:

[0012] The RDMA module acquires the target data, parameter information, and device program data sent by the host computing engine; wherein, the parameter information includes the starting address and length information of the target data written into the memory of the first device.

[0013] The target data is written into the memory based on the parameter information;

[0014] Accordingly, the step of using the device-side computing engine to execute the device-side program data and performing calculations on the target data based on the computing task function to obtain the calculation result includes:

[0015] The device-side computing engine executes the device-side program data, reads the target data from the memory based on the parameter information, and performs calculations on the target data based on the computing task function to obtain the calculation result.

[0016] Optionally, the step of sending the calculation result to the target receiving end based on the communication function and through the RDMA module includes:

[0017] Based on the communication function, the identification information of the target receiving end is written into the communication table, and the table parsing engine is notified;

[0018] The target receiver's identification information is obtained from the communication table through the table parsing engine, and the corresponding IP address is obtained.

[0019] The RDMA module sends the calculation result to the target receiving end based on the IP address.

[0020] Optional, also includes:

[0021] The calculation result is written into the data area in memory, and the write address and data length are written into the communication table.

[0022] The write address and data length are obtained from the communication table by the table parsing engine, and the write address, data length, and IP address are written into the memory of the RDMA module.

[0023] Optionally, sending the calculation result to the target receiving end via the RDMA module based on the IP address includes:

[0024] The RDMA module detects the memory, and when the memory is not empty, it reads the write address, the data length, and the IP address from the memory. Based on the write address and the data length, it reads the calculation result from the memory and sends the calculation result to the target receiving end based on the IP address.

[0025] Optionally, obtaining the IP address corresponding to the identification information includes:

[0026] Obtain the IP address corresponding to the identification information from the IP identifier lookup table.

[0027] Optional, also includes:

[0028] The RDMA module sends a discovery message to the host, so that the host can assign an IP address and identification information to each device based on the discovery message.

[0029] Obtain the IP addresses and identification information of each device sent by the host, and the IP address and identification information of the host.

[0030] The IP addresses and identification information of each device, and the IP addresses and identification information of the host are saved to the IP identification lookup table.

[0031] Optionally, after sending the discovery message to the host via the RDMA module, the method further includes:

[0032] The RDMA module obtains the IP address and identification information allocated by the host to the first device and replies with confirmation information to the host. After receiving the confirmation information from each device, the host saves the IP address and identification information of each device and sends the IP address and identification information of each device, as well as the IP address and identification information of the host, to each device.

[0033] The confirmation information includes the IP address and identification information assigned by the host to the corresponding device.

[0034] Optionally, the target receiving end is a host end or a second device end;

[0035] If the target receiving end is multiple second devices, then the step of writing the identification information of the target receiving end into the communication table based on the communication function includes:

[0036] Based on the communication function, the identification information of multiple second device terminals is written into the communication table.

[0037] Optionally, if the target receiving end is a second device, the second device receives the calculation result through its own RDMA module and stores the calculation result in its own memory.

[0038] Optionally, after sending the calculation result to the second device based on the communication function, the method further includes:

[0039] Send a transmission end information to the second device so that the RDMA module of the second device can return the starting address and length information of the calculation result in memory to the calculation engine of the second device after receiving the transmission end information.

[0040] Optional, also includes:

[0041] During the execution of the device-side program data, if the target program instruction is executed, the system waits to receive data from the third device and continues to execute subsequent operations after receiving the data from the third device.

[0042] Secondly, this application discloses a computing engine communication method applied to the host side, including:

[0043] The host-side computing engine sends device-side program data to the device-side, so that the device-side computing engine on the device-side executes the device-side program data, performs calculations on the target data based on the calculation task function to obtain the calculation result, and sends the calculation result to the target receiving end based on the communication function.

[0044] The device-side program data is program data compiled from program code containing communication functions and computation task functions, and the communication functions are functions created based on communication requirements.

[0045] Thirdly, this application discloses a computing engine communication device, applied to a first device, comprising:

[0046] A communication engine is used to acquire device-side program data sent by the host-side computing engine; wherein, the device-side program data is program data obtained by compiling program code containing communication functions and computing task functions, and the communication functions are functions created based on communication requirements;

[0047] The device-side computing engine is used to execute the device-side program data, perform calculations on the target data based on the computing task function to obtain the calculation result, and send the calculation result to the target receiving end based on the communication function.

[0048] As can be seen, this application first obtains device-side program data sent by the host-side computing engine; wherein, the device-side program data is program data obtained by compiling program code containing communication functions and computing task functions, the communication functions are functions created based on communication requirements, then the device-side computing engine executes the device-side program data, calculates the target data based on the computing task functions to obtain the calculation results, and sends the calculation results to the target receiving end based on the communication functions, so as to realize communication between the device-side computing engine and the computing engine in the target receiving end. In other words, in this application, a communication function is created based on communication requirements, and the program code containing the communication function and the computation task function is compiled to obtain device-side program data. After the device receives the device-side program code sent by the host computing engine, the device-side computing engine completes the computation and communication tasks during the execution of the device-side program code and sends the computation results to the target receiving end. This realizes communication between the computing engine in the host and the computing engine in the device, as well as between the computing engine in the device and the computing engine in the target receiving end. There is no need for the host computing engine to allocate communication tasks, thus freeing up the computing power of the host computing engine. Furthermore, it is suitable for communication scenarios between different computing engines within the same node and between computing engines in different nodes. Attached Figure Description

[0049] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of this application. For those skilled in the art, other drawings can be obtained based on the provided drawings without creative effort.

[0050] Figure 1 This is a flowchart of a computing engine communication method disclosed in this application;

[0051] Figure 2 This is a schematic diagram of a specific distributed communication engine architecture disclosed in this application;

[0052] Figure 3 This is a schematic diagram of the structure of a computing engine communication device disclosed in this application. Detailed Implementation

[0053] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0054] Due to legacy issues in traditional computer architecture systems' network software protocol stacks and hardware devices, in a system with diverse, heterogeneous computing engines, the engine chips, PCIe or local interconnect buses, and network cards of various computing nodes operate independently. This results in highly inefficient communication and computational performance, with limited freedom and flexibility. Consequently, large-scale computing networks not only face challenges related to the computing power of individual devices but also the bottlenecks indirectly caused by inefficient communication between computing engines of different architectures. Taking the common communication between a traditional CPU and GPU (and similar issues with other xPUs) as an example, within a single device, when data needs to be transferred between application memory space and GPU memory space, it must cross the system kernel space and the PCIe bus, requiring an additional copy—a task typically handled by the CPU. When data needs to be transferred between the application memory spaces of device A and device B, the data first needs to be copied from the application memory space of device A to the system kernel space of device A, and then sent from the system kernel space of device A to the network device space of device A. After receiving the data, device B needs to send it from its network device space to its system kernel space, and then copy it from its system kernel space to its application memory space. When data needs to be transferred between the GPU memory spaces of device A and device B, the data must first be copied from the GPU memory space of device A to its system kernel space, and then sent from its system kernel space to its network device space. After receiving the data, device B needs to send it from its network device space to its system kernel space, and then copy it from its system kernel space to its GPU memory space. Typically, this requires four data transfers and copies. It can be seen that the communication architecture of traditional computing networks is a significant factor contributing to the inefficiency and low speed of heterogeneous computing networks. It is evident that in traditional solutions, the repetitive and unnecessary data copying between computing engines of different architectures leads to inefficiency. Furthermore, although a single device contains multiple computing engines, data transmission control is entirely dominated by the CPU, with the GPU merely acting as a coprocessor to share some computational workload. There are two key points to solving these problems: first, minimize the number of unnecessary cross-chip data copies; second, fully leverage the initiative of each computing engine during data transmission. Each computing engine should no longer be a coprocessor; each engine should be able to freely communicate autonomously with any other type of computing engine, freeing up CPU computing power to serve the true core business.

[0055] Currently, several technical solutions have emerged to address the aforementioned issues:

[0056] For example, regarding communication between the GPU and CPU within a single node, NVIDIA launched GPUDirect Shared Memory technology in 2010, gradually developing GPUDirect P2P (Peer-to-Peer), NVLink, and the latest NVSwitch technology. NVLink is a bus protocol developed by NVIDIA to address the PCIe transfer rate limitations when transmitting data between GPUs within a single node. Based on this protocol, a corresponding hardware bus implementation was also developed for physical connections. However, because this method primarily addresses high-speed one-to-one interconnection between GPUs within a single node, it heavily relies on circuit customization and is ineffective for communication between GPUs across nodes. Furthermore, the implementation of NVLink shows that its data transmission depends on the direct physical circuit link between GPUs, making long-distance communication impossible. NVSwitch is a standalone communication chip designed by NVIDIA. It integrates multiple NVLink chips to enable many-to-many GPU communication within a single node at the high speed of NVLink, thereby further improving interconnect performance. For example, 12 NVSwitch chips can connect 16 GPUs, enabling zero-hop arbitrary interconnection between these GPUs. NVSwitch is a dedicated data chip based on NVLink, so the limitations of NVLink still exist in NVSwitch. The node machines are completely customized, and the nodes are connected in close proximity, making distributed deployment impossible.

[0057] For communication between GPUs on different nodes, current solutions primarily utilize RDMA networks. RDMA allows the hardware network card to directly read the data to be sent from user space, bypassing the system kernel space and reducing data copying. This significantly reduces network latency and CPU overhead during data copying, thereby increasing transmission speed. Among multi-node GPU communication solutions based on RDMA (Remote Direct Memory Access), NVIDIA released the latest GPU Direct RDMA Async technology in 2017. This technology allows direct synchronization between the GPU and third-party devices, without the CPU participating in the critical communication path of the GPU application. Data is sent directly from GPU memory to the RDMA NIC (Network Interface Controller), and the receiving RDMA NIC directly transmits it to GPU memory, further reducing CPU involvement and the number of data copies. The working process is as follows: 1) The CPU dispatches computing and communication tasks to the GPU through NVIDIA's CUDA API (Application Programming Interface); 2) After completing the computing task, the GPU automatically executes the communication task to directly trigger communication operations to the RDMA NIC; 3) The RDMA NIC copies data directly from the GPU's memory or the host memory to the RDMA NIC through the RDMA network; 4) The RDMA NIC sends the data. As can be seen from this process, except for the initial stage where the host distributes computing and communication tasks, the remaining processes do not require host involvement. In step 3, during data transmission, the data is copied only once. However, this solution has a problem: the GPU and RDMA NIC are two independent devices. The GPU needs to trigger data movement to the RDMA NIC via PCIe. PCIe, as a physical link, imposes certain physical distance limitations and bindings on the GPU and RDMA NIC. Although data transmission reduces the number of temporary storage operations, the physical PCIe cross-node signal transmission path remains unchanged. Furthermore, as a closed-source solution from a commercial company, this technology is difficult to modify based on actual circumstances.

[0058] Besides GPUs, another common heterogeneous computing engine is the FPGA. For FPGA-based computing engines, Intel developed IKL (Inter-Kernel Links) to enable heterogeneous communication between FPGAs within or between nodes. The User Kernel is a module written by the user in OpenCL to implement specific computing tasks. When writing the User Kernel, the following two main functions are used: `write_channel_intel(channel_id, data)` and `read_channel_intel(channel_id)` to implement communication between different FPGAs. These two functions send the data from the User Kernel to the Inter-Kernel Logic RTL IP via the IKL I / O Channel. The IP then packages the data and forwards it to the Ethernet Switch (network interface card). The IKL method for inter-FPGA communication has several drawbacks: 1) The `write_channel_intel` and `read_channel_intel` functions in OpenCL, used for sending and receiving data, are based on `channel_id`, making configuration cumbersome and each channel only supports fixed point-to-point communication; 2) To achieve reliable communication, the Inter-Kernel Logic RTL IP needs to implement complex control functions similar to the TCP / IP protocol stack, such as timeout retransmission, fragmentation, and packet loss retransmission, consuming a large amount of FPGA resources. This resource consumption increases with the number of `channel_id` values, currently ranging from 48 to 256 depending on the FPGA board model; this communication method only supports network communication between FPGAs, not between the FPGA and the host, and when communicating between FPGAs, only data on the FPGA board can be exchanged, not data in the host's memory.

[0059] Based on the above, the main technical solutions currently available have the following drawbacks: 1) Communication between computing engines within a single node is achieved through traditional PCIe or NVLink-like methods with physical link distance limitations, making large-scale distributed deployment impossible; 2) For communication between computing engines between nodes, even GPU Direct RDMA Async technology requires the host to allocate communication tasks in the first step. Furthermore, it fails to completely decouple the physical link between the GPU and the RDMA NIC, still requiring data transfer to be triggered via the PCIe link. Deploying such GPUs requires significant physical space and higher power consumption. FPGA-based Intel IKL tends to achieve point-to-point transmission between similar devices, and due to the limited number of channel_ids, the number of objects that can exchange data between devices is severely limited, resulting in limited transmission and extremely poor efficiency. Additionally, this solution does not support the NIC within a node directly reading the host's memory and sending it to the peer; 3) Whether based on GPU or FPGA computing engines, most solutions are closed-source and specialized, difficult to modify, and lack versatility.

[0060] See Figure 1 As shown in the figure, this application discloses a computing engine communication method, applied to a first device, including:

[0061] Step S11: Obtain device-side program data sent by the host computing engine; wherein, the device-side program data is program data obtained by compiling program code containing communication functions and computing task functions, and the communication functions are functions created based on communication requirements.

[0062] In this embodiment, the host-side computing engine is a CPU. Furthermore, in this embodiment, the specific computing tasks described using the OpenCL language are used to obtain computing task functions. In addition, the difference between the device-side program data and ordinary device-side OpenCL programs in this embodiment is that a custom communication function library is integrated. When writing the code, the corresponding communication functions in this custom communication function library are added to the OpenCL code of the cl file according to the communication requirements of the task. The functions are compiled together with the cl file to form the final device-side binary code, that is, the device-side program data.

[0063] Furthermore, embodiments of this application can acquire device-side program data sent by the host-side computing engine through an RDMA module; wherein, the RDMA module and the device-side computing engine are integrated in the same chip. In this way, by integrating the module responsible for RDMA network communication with the computing engine into a single chip, the connection between the computing engine and the communication engine is further accelerated, eliminating the physical link distance limitation problem in the prior art and facilitating large-scale distributed deployment. The device-side computing engine is an xPU (i.e., a general-purpose computing engine), a collective term for various computing engines including GPUs, NPUs (i.e., Neural-network Processing Units), DPUs (i.e., Data Processing Units), and IPUs (i.e., Infrastructure Processing Units). Embodiments of this application can integrate the RDMA module and the computing engine in the same FPGA chip.

[0064] Furthermore, in one implementation, the target data, parameter information, and device-side program data sent by the host-side computing engine can be obtained through the RDMA module; wherein, the parameter information includes the starting address and length information of the target data to be written into the memory of the first device; and the target data is written into the memory based on the parameter information. The memory can be DDR (i.e., DDR SDRAM, Double Data Rate Synchronous Dynamic Random Access Memory).

[0065] Step S12: Execute the device-side program data using the device-side computing engine, perform calculations on the target data based on the computing task function to obtain the calculation result, and send the calculation result to the target receiving end based on the communication function, so as to realize the communication between the device-side computing engine and the computing engine in the target receiving end.

[0066] In one implementation, the device-side computing engine can be used to execute the device-side program data, read the target data from the memory based on the parameter information, and perform calculations on the target data based on the computing task function to obtain the calculation result. The calculation result is then sent to the target receiving end via the RDMA module using the communication function.

[0067] Specifically, the calculation result, based on the communication function and transmitted to the target receiving end via the RDMA module, includes the following steps:

[0068] Step 00: Write the identification information of the target receiver into the communication table based on the communication function, and notify the table parsing engine.

[0069] In this application, both the table parsing engine and the RDMA module are underlying hardware IP (intellectual property). In one implementation, custom IP can be written using HDL (Hardware Description Language) to obtain the table parsing engine and the RDMA module. Therefore, in this application, the custom communication library of the upper-layer software can be implemented based on common C / C++ languages, while the underlying hardware IP is implemented using HDL code. The solution provided in this application has good versatility and portability.

[0070] Furthermore, in this embodiment of the application, the corresponding target receiving end can be specified when creating the communication function. Specifically, it can be specified based on the device's identification information.

[0071] Step 01: Obtain the identification information of the target receiving end from the communication table through the table parsing engine, and obtain the IP address corresponding to the identification information.

[0072] Furthermore, in this embodiment, the calculation result is written into the data area in memory, and the write address and data length are written into the communication table; the write address and data length are obtained from the communication table through the table parsing engine, and the write address, data length, and IP address are written into the memory in the RDMA module. The memory can be a FIFO (First Input First Output) memory.

[0073] In this embodiment of the application, the IP address corresponding to the identification information can be obtained from the IP identification lookup table.

[0074] Furthermore, in a specific implementation, during the driver initialization phase, the RDMA module can send a discovery message to the host, so that the host can allocate IP addresses and identification information to each device based on the discovery messages from each device. The host then obtains the IP addresses and identification information of each device and the host itself, and saves these information to the IP identification lookup table. After sending the discovery message to the host via the RDMA module, the RDMA module can obtain the IP address and identification information allocated by the host for the first device and send a confirmation message back to the host. Upon receiving the confirmation messages from each device, the host saves the IP addresses and identification information of each device and sends the IP addresses and identification information of each device and the host itself to each device. The confirmation message carries the IP address and identification information allocated by the host for the corresponding device.

[0075] It's important to note that upon power-up, each device is essentially a DHCP (Dynamic Host Configuration Protocol) client. The RDMA module sends DHCP discovery messages to find a DHCP server, specifically broadcasting a message to the IP address 255.255.255.255. The host, acting as the DHCP server, receives these messages and assigns each device its IP address, device identifier, and other information. Each device's RDMA module, upon receiving the information from the host, replies with an acknowledgment message containing all the information assigned to it, declaring its intention to use this information for communication. The host, upon receiving these acknowledgments, sends the assigned information, along with its own IP address and identifier, to each device in a custom format, while also retaining a copy on its own. Finally, each device's RDMA module, upon receiving the communication information from the host, stores it in an IP address lookup table.

[0076] Furthermore, in this embodiment of the application, a node may include a host and at least one device. If the same network includes multiple nodes, the host in a node can be assigned IP address and identification information. That is, the IP address and identification information of other hosts and all devices in the network are assigned. Finally, each host and each device has an IP identification lookup table that includes the IP address and identification information of all hosts and all devices in the entire network.

[0077] In this way, any device in the same network can automatically discover and obtain the communication information of all devices in the network without manual intervention. It has scalability and flexibility, and provides a physical basis for realizing the expansion of arbitrary distributed computing networks with high degree of freedom.

[0078] Furthermore, in this embodiment, the addresses of the IP identifier lookup table and the information table are both fixed, and both device-side program data and host-side program data are written into them, wherein the host-side program data is the program data executed on the host side. In one implementation, a program mainly written in C / C++ uses the C / C++ standard library and OpenCL API functions on the host side to obtain the final host-side program data.

[0079] Step 02: The RDMA module sends the calculation result to the target receiving end based on the IP address.

[0080] In a specific implementation, the RDMA module can detect the memory, and when the memory is not empty, read the write address, the data length, and the IP address from the memory, read the calculation result from the memory according to the write address and the data length, and send the calculation result to the target receiving end based on the IP address.

[0081] Wherein, the target receiving end is a host end or a second device end; if the target receiving end is multiple second devices ends, then based on the communication function, the identification information of multiple second devices ends is written into the communication table, and the calculation result is sent to multiple second devices ends through the table parsing engine.

[0082] Furthermore, if the target receiving end is the second device end, the second device end receives the calculation result through its own RDMA module and stores the calculation result in its own memory. After the first device end sends the calculation result to the second device end based on the communication function, it can also send a transmission end information to the second device end. After receiving the transmission end information, the RDMA module of the second device end returns the starting address and length information of the calculation result in memory to the calculation engine of the second device end, so that the calculation engine of the second device end can perform subsequent operations based on the starting address and length information of the calculation result in memory.

[0083] Furthermore, during the execution of the device-side program data, if the first device reaches a target program instruction, it waits to receive data from the third device. After receiving the data from the third device, it continues with subsequent operations. The target program instruction is a program instruction compiled based on the communication function. It is understandable that multiple second devices also execute corresponding program instructions and wait to receive the calculation results from the first device.

[0084] It is understood that the embodiments of this application can provide a communication scheme for active interconnection between computing engines of any different architecture, eliminating the distinction between communication between nodes and communication within nodes, and eliminating the need for CPU to allocate communication tasks. It also has one-to-many communication capability. Whether it is between XPUs or between XPUs and the host, data only needs to be moved once during data transmission, which has high efficiency and performance.

[0085] For example, see Figure 2 As shown, Figure 2This is a schematic diagram of a specific distributed communication engine architecture provided in an embodiment of this application. Taking a fully software-programmable hardware FPGA device as an example, this architecture has two computing engines: a regular CPU on the host side and a GPU based on RISC-V processor extensions in the accelerator card (i.e., the device side). The accelerator card is an FPGA board, including an FPGA chip, an RDMA module integrated with the GPU on this chip, and a series of peripherals such as network ports and DDR. There is no master-slave relationship between the host and device sides. The host-side program is mainly written in C / C++, using the C / C++ standard library and OpenCL API functions on the host side. The device-side program mainly uses OpenCL to describe specific computing tasks and integrates a custom communication function library. When writing the code, the corresponding communication functions in this custom communication function library are added to the OpenCL code of the .cl file according to the communication requirements of the task, and compiled together with the .cl file to form the final device-side binary code. The upper-level CL compiler and the xPU chip jointly determine the addresses of the communication table and IP identifier lookup table for both parties and write them into the aforementioned binary code. In other words, the addresses of the communication tables and IP address lookup tables required for communication between the host and the device are pre-determined and written into the host program data and the device program data. Furthermore, custom IPs, including RDMA and table resolution engines, are written using HDL language, and combined with a custom communication function library implemented in C / C++ and some standard protocol libraries, to achieve arbitrary interconnection between any computing engines based on the public Ethernet protocol, without requiring the host to control the communication process.

[0086] Furthermore, the main working process of the aforementioned distributed communication engine architecture includes:

[0087] (1) Driver initialization phase: The RDMA module implements the functions of the RDMA NIC and automatically completes the following tasks: a) Upon power-up, each accelerator card is actually a DHCP client, that is, the RDMA module will send DHCP discovery information to find the DHCP server, that is, send specific broadcast information to the broadcast IP address 255.255.255.255; b) The host, as the DHCP server, will receive DHCP discovery messages from each accelerator card and assign IP address, device ID and other information to each accelerator card respectively; c) After receiving the information assigned to itself by the host, each accelerator card's RDMA module will reply with an acknowledgment message, which will also contain all the information assigned to itself in b), to declare to the host that the accelerator card will use this information for communication; d) After receiving the acknowledgment messages from each accelerator card, the host will send the information assigned to all accelerator cards and the host's IP and ID information in a custom format to each accelerator card, and also retain a copy on the host; e) After receiving the communication information containing all devices sent by the host in the previous step, each accelerator card's RDMA module will save it in the IP identifier lookup table.

[0088] (2) The host downloads relevant data to the accelerator card: The compiled host binary program is executed on the host side. A series of OpenCL API functions in the program will perform a series of tasks such as finding the device and initializing the device parameters. In addition, the program will also send the following three types of data to the accelerator card: the initial data required for calculation, some parameters (such as the starting address and length of the initial data in the accelerator card's DDR, etc., the address of these parameters in the accelerator card's DDR is fixed), and the compiled device binary program data on the disk (the starting address of this data in the accelerator card's DDR is also fixed and is independent of the specific task). The host side sends this data to the accelerator card through RDMA Ethernet.

[0089] (3) The two scenarios for communication between the accelerator card and the host are as follows:

[0090] a) Accelerator card reads data from the host: Usually, in the above (2), the host has already downloaded the relevant data to the accelerator card. The data required for GPU calculation in the accelerator card has been downloaded to the DDR of the accelerator card. The GPU does not need to read the data from the host.

[0091] b) Accelerator Card Writes Data to Host: After the GPU in the accelerator card completes the calculation, it writes the calculation result to the data area in the accelerator card's DDR, and saves the written address, length, and the ID of the peer (i.e., the target receiver) in the communication table. Then, the GPU notifies the table resolution engine, which retrieves the information from the communication table and marks the retrieved information to prevent duplicate retrieval. Based on the peer ID in the retrieved information, the table resolution engine queries the IP address corresponding to that ID in the IP identifier lookup table. After obtaining the IP address corresponding to the ID, the table resolution engine writes this information into a FIFO in the RDMA module. When the RDMA module detects that the FIFO is not empty, it continuously retrieves the information from it, reads the corresponding data from the accelerator card's DDR according to the accelerator card's DDR address and length in the information, and then sends the data to the peer according to the IP address. After the data transmission is complete, the accelerator card also sends a transmission completion message to the host.

[0092] (4) Reading and writing between accelerator cards is different from reading and writing between accelerator cards and the host. The former occurs in pairs, that is, when one end performs a write operation, the other end or multiple ends must perform the corresponding read operation. For example, if accelerator cards 2 and 3 need to read data from accelerator card 1, that is, after the GPUs in accelerator cards 2 and 3 have completed a certain stage of the task, they may need data from accelerator card 1. The fundamental reason for this is that the GPUs in both accelerator cards have executed the program instructions compiled from the read function in the custom communication function library. This program instruction indicates that the GPU needs to wait to receive data from Device ID=1 (that is, the device ID of the accelerator card is 1) before it can continue to execute subsequent tasks.

[0093] a) In accelerator card 1, after the GPU completes the calculation, it writes the calculation result into the data area of ​​the accelerator card 1 DDR and saves the written address, length, peer ID, etc. in a communication table with a fixed address in the DDR. Since there are two devices that need to be sent, two pieces of information with different peer IDs are written into the communication table. Then the GPU notifies the table resolution engine, which retrieves these two pieces of information from the communication table. Based on the two peer IDs (i.e., device IDs = 2 and 3) in the retrieved information, the table resolution engine queries the IP address corresponding to the ID in the IP identifier lookup table. After obtaining the IP addresses corresponding to the two IDs, the table resolution engine writes the two pieces of information and their IPs into a FIFO in the RDMA module. When the RDMA module detects that the FIFO is not empty, it continuously retrieves the information from it, reads the corresponding data from the accelerator card DDR according to the accelerator card DDR address and length in the information, and then sends the data to the peer devices (i.e., accelerator cards 2 and 3) according to the IP address. After the data transmission is completed, a data transmission end message is also sent to accelerator cards 2 and 3 respectively.

[0094] b) In accelerator cards 2 and 3, after the GPUs of each card execute the instructions generated by the read function, they will wait for the device ID given in the instructions to send data. When the RDMA modules of accelerator cards 2 and 3 receive the data sent by accelerator card 1, they will save it in their respective DDR. After receiving the end of transmission information sent by accelerator card 1, they will return the starting address and length of the received data in DDR to their respective GPUs. The GPUs can then perform subsequent operations based on the starting address and length of the data.

[0095] It should be noted that in other communication engine architectures, the following situation may occur: since accelerator card 1 and accelerator card 2 cannot communicate directly, accelerator card 1 will first send the intermediate results it calculates back to a certain address on the host side, and then the host side will notify accelerator card 2 to read the data at that address on the host side; since the accelerator cards in this communication architecture can communicate with each other, this situation will not exist.

[0096] In this way, by using a custom communication function library for upper-layer software written in a high-level language and a low-level xPU hardware IP written in HDL, taking a physical FPGA chip as an example, an efficient and universal communication engine between computing engines of different architectures is implemented. Depending on the specific type of the underlying xPU (in this example, a GPU based on RISC-V extensions), a custom communication function library is written in a standard high-level language, combined with an RDMA module and table parsing engine written in HDL. This communication engine further accelerates the connection between the computing engine and the communication engine by integrating the module responsible for RDMA network communication with the computing engine into a single chip. Furthermore, it enables active interconnection between computing engines of any different architectures within the same RDMA network, completely eliminating the need for CPU control in communication management and providing one-to-many communication capabilities. Whether between accelerator cards or between an accelerator card and the host, data only needs to be moved once during data transmission, effectively solving the problems of inefficient and slow communication between computing engines. The solution provided in this application is applicable to various xPU computing engines, such as GPUs, NPUs, IPUs, DPUs, and VPUs.

[0097] Furthermore, this application discloses a computing engine communication method, applied to the host side, including:

[0098] The host-side computing engine sends device-side program data to the device-side, so that the device-side computing engine on the device-side executes the device-side program data, performs calculations on the target data based on the calculation task function to obtain the calculation result, and sends the calculation result to the target receiving end based on the communication function.

[0099] The device-side program data is program data compiled from program code containing communication functions and computation task functions, and the communication functions are functions created based on communication requirements.

[0100] As can be seen, in this application, a communication function is created based on communication requirements, and the program code containing the communication function and the computation task function is compiled to obtain device-side program data. After the device receives the device-side program code sent by the host-side computing engine, the device-side computing engine completes the computation and communication tasks during the execution of the device-side program code and sends the computation results to the target receiving end. This realizes communication between the computing engine in the host and the computing engine in the device, as well as between the computing engine in the device and the computing engine in the target receiving end. There is no need for the host computing engine to allocate communication tasks, thus freeing up the computing power of the host computing engine. Furthermore, it is suitable for communication scenarios between different computing engines within the same node and between computing engines in different nodes.

[0101] See Figure 3 As shown in the figure, this application discloses a computing engine communication device, applied to a first device, comprising:

[0102] Communication engine 11 is used to acquire device-side program data sent by the host-side computing engine; wherein, the device-side program data is program data obtained by compiling program code containing communication functions and computing task functions, and the communication functions are functions created based on communication requirements;

[0103] The device-side computing engine 12 is used to execute the device-side program data, perform calculations on the target data based on the computing task function to obtain the calculation result, and send the calculation result to the target receiving end based on the communication function.

[0104] As can be seen, this application first obtains device-side program data sent by the host-side computing engine; wherein, the device-side program data is program data obtained by compiling program code containing communication functions and computing task functions, the communication functions are functions created based on communication requirements, then the device-side computing engine executes the device-side program data, calculates the target data based on the computing task functions to obtain the calculation results, and sends the calculation results to the target receiving end based on the communication functions, so as to realize communication between the device-side computing engine and the computing engine in the target receiving end. In other words, in this application, a communication function is created based on communication requirements, and the program code containing the communication function and the computation task function is compiled to obtain device-side program data. After the device receives the device-side program code sent by the host computing engine, the device-side computing engine completes the computation and communication tasks during the execution of the device-side program code and sends the computation results to the target receiving end. This realizes communication between the computing engine in the host and the computing engine in the device, as well as between the computing engine in the device and the computing engine in the target receiving end. There is no need for the host computing engine to allocate communication tasks, thus freeing up the computing power of the host computing engine. Furthermore, it is suitable for communication scenarios between different computing engines within the same node and between computing engines in different nodes.

[0105] Among them, the communication engine 11 is an RDMA module, which is integrated with the device-side computing engine on the same chip;

[0106] Accordingly, the device-side computing engine 12 is specifically used to send the calculation result to the target receiving end based on the communication function and through the RDMA module.

[0107] Furthermore, the RDMA module is specifically used to acquire target data, parameter information, and device-side program data sent by the host-side computing engine; wherein, the parameter information includes the starting address and length information of the target data to be written into the memory of the first device; and the target data is written into the memory based on the parameter information;

[0108] Accordingly, the device-side computing engine 12 is specifically used to execute the device-side program data, read the target data from the memory based on the parameter information, and perform calculations on the target data based on the computing task function to obtain the calculation result.

[0109] Furthermore, the device-side computing engine 12 writes the identification information of the target receiving end into the communication table based on the communication function and notifies the table parsing engine; the table parsing engine obtains the identification information of the target receiving end from the communication table and obtains the IP address corresponding to the identification information; the RDMA module is used to send the calculation result to the target receiving end based on the IP address.

[0110] Furthermore, the RDMA module is also used to write the calculation result into the data area in memory, and write the write address and data length into the communication table; correspondingly, the device-side calculation engine 12 is used to obtain the write address and data length from the communication table through the table parsing engine, and write the write address, data length, and IP address into the memory in the RDMA module. The RDMA module is used to detect the memory, and when the memory is not empty, read the write address, data length, and IP address from the memory, read the calculation result from the memory according to the write address and data length, and send the calculation result to the target receiving end based on the IP address.

[0111] Furthermore, obtaining the IP address corresponding to the identification information includes: obtaining the IP address corresponding to the identification information from an IP identification lookup table.

[0112] Correspondingly, the RDMA module is also used to send a discovery message to the host, so that the host can allocate IP addresses and identification information to each device based on the discovery messages of each device; obtain the IP addresses and identification information of each device and the IP address and identification information of the host sent by the host; and save the IP addresses and identification information of each device and the IP address and identification information of the host to the IP identification lookup table.

[0113] The RDMA module is further configured to, after sending a discovery message to the host, obtain the IP address and identification information allocated by the host to the first device, and reply with confirmation information to the host, so that after receiving the confirmation information from each device, the host can save the IP address and identification information of each device and send the IP address and identification information of each device, as well as the IP address and identification information of the host, to each device.

[0114] Furthermore, the target receiving end is either a host end or a second device end;

[0115] If the target receiving end is multiple second devices, the device-side computing engine writes the identification information of all the second devices into the communication table based on the communication function.

[0116] Furthermore, if the target receiving end is a second device, the second device receives the calculation result through its own RDMA module and stores the calculation result in its own memory.

[0117] Furthermore, the RDMA module is also used to send transmission end information to the second device after sending the calculation result to the second device based on the communication function, so that the RDMA module of the second device can return the starting address and length information of the calculation result in memory to the calculation engine of the second device after receiving the transmission end information.

[0118] Furthermore, during the execution of the device-side program data, if the device-side computing engine reaches the target program instruction, it waits to receive data from the third device and continues to execute subsequent operations after receiving the data from the third device.

[0119] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on its differences from other embodiments. Similar or identical parts between embodiments can be referred to interchangeably. For the apparatus disclosed in the embodiments, since it corresponds to the method disclosed in the embodiments, the description is relatively simple; relevant parts can be referred to in the method section.

[0120] The steps of the methods or algorithms described in conjunction with the embodiments disclosed herein can be implemented directly by hardware, a software module executed by a processor, or a combination of both. The software module can be located in random access memory (RAM), main memory, read-only memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, registers, hard disk, removable disk, CD-ROM, or any other form of storage medium known in the art.

[0121] The computing engine communication method and apparatus provided in this application have been described in detail above. Specific examples have been used to illustrate the principles and implementation methods of this application. The description of the above embodiments is only for the purpose of helping to understand the method and core ideas of this application. At the same time, for those skilled in the art, there will be changes in the specific implementation methods and application scope based on the ideas of this application. Therefore, the content of this specification should not be construed as a limitation of this application.

Claims

1. A computing engine communication method, characterized in that, Applied to the first device, including: Obtain device-side program data sent by the host-side computing engine; wherein, the device-side program data is program data obtained by compiling program code containing communication functions and computing task functions, and the communication functions are functions created based on communication requirements; The device-side computing engine executes the device-side program data, performs calculations on the target data based on the computing task function to obtain the calculation result, and sends the calculation result to the target receiving end based on the communication function, so as to realize the communication between the device-side computing engine and the computing engine in the target receiving end; The step of obtaining device-side program data sent by the host computing engine includes: obtaining device-side program data sent by the host computing engine through an RDMA module; wherein the RDMA module and the device-side computing engine are integrated in the same chip.

2. The computing engine communication method according to claim 1, characterized in that, Sending the calculation result to the target receiving end based on the communication function includes: sending the calculation result to the target receiving end based on the communication function and through the RDMA module.

3. The computing engine communication method according to claim 2, characterized in that, The step of obtaining device-side program data sent by the host-side computing engine through the RDMA module includes: The RDMA module acquires the target data, parameter information, and device program data sent by the host computing engine; wherein, the parameter information includes the starting address and length information of the target data written into the memory of the first device. The target data is written into the memory based on the parameter information; Accordingly, the step of using the device-side computing engine to execute the device-side program data and performing calculations on the target data based on the computing task function to obtain the calculation result includes: The device-side computing engine executes the device-side program data, reads the target data from the memory based on the parameter information, and performs calculations on the target data based on the computing task function to obtain the calculation result.

4. The computing engine communication method according to claim 2, characterized in that, The step of sending the calculation result to the target receiving end based on the communication function and through the RDMA module includes: Based on the communication function, the identification information of the target receiving end is written into the communication table, and the table parsing engine is notified; The target receiver's identification information is obtained from the communication table through the table parsing engine, and the corresponding IP address is obtained. The RDMA module sends the calculation result to the target receiving end based on the IP address.

5. The computing engine communication method according to claim 4, characterized in that, Also includes: The calculation result is written into the data area in memory, and the write address and data length are written into the communication table. The write address and data length are obtained from the communication table by the table parsing engine, and the write address, data length, and IP address are written into the memory of the RDMA module.

6. The computing engine communication method according to claim 5, characterized in that, The step of sending the calculation result to the target receiving end via the RDMA module based on the IP address includes: The RDMA module detects the memory, and when the memory is not empty, it reads the write address, the data length, and the IP address from the memory. Based on the write address and the data length, it reads the calculation result from the memory and sends the calculation result to the target receiving end based on the IP address.

7. The computing engine communication method according to claim 4, characterized in that, Obtaining the IP address corresponding to the identification information includes: Obtain the IP address corresponding to the identification information from the IP identifier lookup table.

8. The computing engine communication method according to claim 7, characterized in that, Also includes: The RDMA module sends a discovery message to the host, so that the host can assign an IP address and identification information to each device based on the discovery message. Obtain the IP addresses and identification information of each device sent by the host, and the IP address and identification information of the host. The IP addresses and identification information of each device, and the IP addresses and identification information of the host are saved to the IP identification lookup table.

9. The computing engine communication method according to claim 8, characterized in that, After sending the discovery message to the host via the RDMA module, the process further includes: The RDMA module obtains the IP address and identification information allocated by the host to the first device and replies with confirmation information to the host. After receiving the confirmation information from each device, the host saves the IP address and identification information of each device and sends the IP address and identification information of each device, as well as the IP address and identification information of the host, to each device.

10. The computing engine communication method according to claim 4, characterized in that, The target receiving end is either a host end or a second device end; If the target receiving end is multiple second devices, then the step of writing the identification information of the target receiving end into the communication table based on the communication function includes: Based on the communication function, the identification information of multiple second device terminals is written into the communication table.

11. The computing engine communication method according to claim 5, characterized in that, If the target receiving end is a second device, the second device receives the calculation result through its own RDMA module and stores the calculation result in its own memory.

12. The computing engine communication method according to claim 11, characterized in that, After sending the calculation result to the second device based on the communication function, the method further includes: Send a transmission end information to the second device so that the RDMA module of the second device can return the starting address and length information of the calculation result in memory to the calculation engine of the second device after receiving the transmission end information.

13. The computing engine communication method according to any one of claims 1 to 12, characterized in that, Also includes: During the execution of the device-side program data, if the target program instruction is executed, the system waits to receive data from the third device and continues to execute subsequent operations after receiving the data from the third device.

14. A computing engine communication method, characterized in that, Applied to the host side, including: The host-side computing engine sends device-side program data to the device-side, so that the device-side computing engine on the device-side executes the device-side program data, performs calculations on the target data based on the calculation task function to obtain the calculation result, and sends the calculation result to the target receiving end based on the communication function. The device-side program data is program data compiled from program code containing communication functions and computation task functions, wherein the communication functions are functions created based on communication requirements; the device obtains the device-side program data sent by the host-side computing engine through the RDMA module; the RDMA module and the device-side computing engine are integrated in the same chip.

15. A computing engine communication device, characterized in that, Applied to the first device, including: A communication engine is used to acquire device-side program data sent by the host-side computing engine; wherein, the device-side program data is program data obtained by compiling program code containing communication functions and computing task functions, and the communication functions are functions created based on communication requirements; The device-side computing engine is used to execute the device-side program data, perform calculations on the target data based on the computing task function to obtain the calculation result, and send the calculation result to the target receiving end based on the communication function; The step of obtaining device-side program data sent by the host computing engine includes: obtaining device-side program data sent by the host computing engine through an RDMA module; wherein the RDMA module and the device-side computing engine are integrated in the same chip.

Images