Remote in-memory computing interaction method and system based on RDMA (Remote Direct Memory Access) mechanism

Abstract

The invention relates to a remote in-memory computing interaction method and system based on an RDMA mechanism, and the method comprises the steps: transmitting a first RDMA write request to in-memory computing equipment by a host, writing task data into a computable memory space, packaging an address index number and a command code, putting the packaged address index number and command code into immediate data, and embedding the immediate data into the first RDMA write request in the process, and after receiving the request, the in-memory computing device computes the task data in the computable memory space according to a computing command indicated by a command code in the immediate data to obtain a computing result of the task data, replies the result and sends a second RDMA write request to the host, in the process, the address index number and the calculation completion state are packaged and put into immediate data to be embedded into a second RDMA write request, and the RDMA network card on the host side receives the request, writes the calculation result into the memory space of the host and sends a work request and the immediate data to the host. Therefore, an efficient read-write interaction mechanism of the remote in-memory calculation can be realized based on a universal RDMA mechanism, and the access performance of the remote in-memory calculation can be improved.

Description

Technical Field

[0001] The present disclosure relates to the technical fields of in-memory computing and remote direct memory access, and in particular to a remote in-memory computing interaction method and system based on the RDMA mechanism. Background Art

[0002] Remote Direct Memory Access (RDMA) is an innovative and efficient long-distance network data communication technology for modern networks. In the era of big data and large models, it has become a standard feature of efficient data systems. RDMA applies traditional DMA (Direct Memory Access) principles to data communication across remote networks, eliminating the need for CPU involvement in the entire communication process. Compared to the traditional TCP / IP mechanism, which requires a heavy CPU to run the protocol stack, RDMA achieves zero CPU involvement in the communication process. By moving the entire network stack from software to hardware execution, the hardware's processing speed advantage is fully realized. As a result, the RDMA mechanism achieves orders of magnitude improvements in both communication efficiency and CPU utilization.

[0003] The RDMA mechanism operates primarily through the host invoking the RDMA network interface to create a user-mode operation queue unit. Each RDMA operation queue unit consists of three queues: a submission queue (sq), a receive queue (rq), and a completion queue (cq). During operation, the initiator generates a work queue entry (wqe) and places it in the submission queue. The RDMA network card directly retrieves the work request from the submission queue and executes the work request. The RDMA mechanism operates in two modes: one is compatible with traditional message semantics, in which two users interact through messages, with the CPU process sending and the other receiving. The sending end sends a work request with the command opcode "rdma send" to the sending queue. The receiving end then places an "rdma receive" work request in its receiving queue, specifying the message storage location. Upon receiving the message, the network card notifies the receiving end, and the receiving CPU begins processing the message. The other is a direct memory semantic mechanism, where a process running on one CPU initiates a memory read / write request, while the other CPU can directly complete the remote memory read / write unilateral operation without participating. The operation process is that the sender puts a work request with the instruction rdma read / write to the sending queue. After receiving it, the RDMA network card of the sender will directly transmit data with the RDMA network card of the other end to complete the remote memory read and write, and the CPU of the other end does not need to participate at all. The biggest difference between the two mechanisms is that under direct memory semantic operations, the CPU of the other end does not need to participate in the data itself.

[0004] In the field of AI accelerators, in order to achieve more efficient resource utilization, large-scale data centers have begun to pool storage, computing power and other resources, and the current bottleneck of the von Neumann structure has become one of the main targets for breakthroughs. The bottleneck of the von Neumann structure is mainly that the computing processing logic in the host structure needs to continuously load data from the memory for calculation, and the memory read and write path has become one of the bottlenecks of computing. Existing accelerator hardware basically uses high-bandwidth memory for operation, which can alleviate the von Neumann bottleneck problem to a certain extent, and another development direction is to solve this problem from the memory itself. As a result, in-memory computing technology came into being, a technology that embeds computing units into storage units. In the field of AI accelerators, it is also called in-memory processing (PIM, processing-in-memory) accelerator structure. Such as Figure 1aThe traditional memory storage shown in FIG, such as dynamic random access memory (DRAM), is composed of a number of memory logic sub-arrays, which perform traditional data read and write operations with the processor (CPU or GPU) through external interfaces. Figure 1b The in-memory computing module shown includes a control unit, an in-memory computing processing unit, and a memory storage logic unit. The control unit sends a computing command to the in-memory computing processing unit, and the in-memory computing processing unit calculates the computing data in the memory storage logic unit. The control unit transmits data (including computing input and results) to the memory storage logic unit, and the memory storage logic unit stores the computing input data and computing results, so that the processor can transfer part of the computing to be executed to the in-memory computing module for execution.

[0005] Generally speaking, in-memory computing is primarily used to perform computations that require heavy memory reads and writes, such as large-scale matrix multiplication and addition in neural networks. If the processor were to perform these operations, it would require a massive amount of memory reads and calculations, creating a bottleneck in the memory read / write path. However, performing this logic in-memory computing can significantly improve performance.

[0006] Currently, the research, development, and testing of in-memory computing technologies are conducted within a single machine. However, with the advancement of network technology, the use of high-speed networks to share and pool computable memory as both a memory and computing resource has become a promising research direction for more efficient resource utilization. A prior art paper, "NetDAM," proposes a network technology for connecting computable memory to a network. The paper, titled "NetDAM: Network Direct Attached Memory with Programmable In-Memory Computing ISA," addresses two major issues with implementing remote in-memory computing over existing RDMA communication networks: efficiency, which increases latency due to traditional network interaction mechanisms; and inherent network issues, such as the increased latency caused by priority-based flow control (PFC) in the RDMA mechanism (e.g., deadlock and PFC storms), as well as the increased latency caused by the go-back-N retransmission method (retransmitting all packets after the Nth dropped packet). Therefore, NetDAM technology aims to achieve efficient network access for in-memory computing by completely redefining the network protocol.

[0007] However, defining a completely new set of network protocols would require significant system changes, requiring dedicated network protocol processing hardware. This would render a large amount of general-purpose RDMA hardware unusable, making data centers challenging in terms of both software and hardware development, deployment, and operations. Furthermore, with the recent technological advancements in RDMA itself, the aforementioned issues have been largely resolved. For example, end-to-end collaboration has largely resolved PFC storms and deadlocks, while selective retransmission reduces the latency jitter introduced by the go-back-N retransmission method. Therefore, the pressing issue is how to more efficiently apply existing general-purpose RDMA mechanisms to in-memory computing interactions and improve access performance. Summary of the Invention

[0008] In view of this, the present disclosure proposes a remote in-memory computing interaction method and system based on the RDMA mechanism, which can implement a set of efficient remote in-memory computing read and write interaction mechanisms based on the general standard RDMA mechanism. It can efficiently pool in-memory computing resources based on general RDMA network cards and network devices, which is beneficial to improving the access performance and read and write efficiency of remote in-memory computing devices, and further beneficial to improving the task execution efficiency of task processing devices.

[0009] According to a first aspect of the present disclosure, a remote in-memory computing interaction method based on the RDMA mechanism is provided, comprising: a host of a task processing device sends a first RDMA write request to a remote in-memory computing device through an RDMA network card, wherein the first RDMA write request carries task data to be calculated and a first immediate number, the first immediate number comprising an address index number and a command code, the command code being used to indicate a calculation command to be executed, the address index number being used to indicate first address information for storing the task data in the in-memory computing device, second address information for storing a calculation result of the task data in the in-memory computing device, and third address information for storing a calculation result of the task data in the memory of the host; in response to receiving the first RDMA write request, the in-memory computing device writes the task data into a first computable memory space corresponding to the first address information indicated by the address index number based on the address index number in the first immediate number, and performs computation on the first in-memory computing device according to the calculation command indicated by the command code in the first immediate number. The task data in the computable memory space is calculated to obtain a calculation result of the task data, and the calculation result is written into the second computable memory space corresponding to the second address information indicated by the address index number in the first immediate number; the in-memory computing device sends a second RDMA write request to the task processing device, and the second RDMA write request carries the calculation result in the second computable memory space and the second immediate number, and the second immediate number contains the address index number and the calculation completion status of the task data; the RDMA network card of the task processing device responds to receiving the second RDMA write request and, based on the address index number in the second immediate number, writes the calculation result into the host memory space in the memory of the host corresponding to the third address information indicated by the address index number, and sends a work request and the second immediate number to the host, so that the host completes the task to be executed according to the second immediate number and the calculation result in the host memory space in response to receiving the work request sent by the RDMA network card.

[0010] In a possible implementation of the first aspect, the task processing device and the in-memory computing device are each pre-installed with a memory pooling mapping list, the memory pooling mapping list including: multiple address index numbers and a set of memory pooling information corresponding to each address index number, the set of memory pooling information including: address information for storing task data in the in-memory computing device, address information for storing calculation results of task data in the in-memory computing device, and address information for storing calculation results of task data in the host's memory; wherein the address information includes the address and size of the memory space; the address index number includes: a list entry identifier of the memory pooling mapping list, and a queue identifier of a work queue on the host or a user identifier on the host; the work queue is used to cache work requests.

[0011] In a possible implementation of the first aspect, the host of the task processing device sends a first RDMA write request to a remote in-memory computing device through an RDMA network card, including: the host assigns an address index number to the task data to be calculated based on the memory pooling mapping list, and encodes the assigned address index number and the command code of the calculation command to be executed as the first immediate number; the host generates the first RDMA write request based on the first immediate number and the task data, and sends the first RDMA write request to the in-memory computing device through the RDMA network card.

[0012] In a possible implementation of the first aspect, the in-memory computing device includes: an RDMA network engine module, an in-memory computing controller and a plurality of computable memory modules, wherein the in-memory computing device, in response to receiving the first RDMA write request, writes the task data into a first computable memory space corresponding to the first address information indicated by the address index number based on the address index number in the first immediate number, and performs calculation on the task data in the first computable memory space according to the calculation command indicated by the command code in the first immediate number to obtain the calculation result of the task data, and writes the calculation result into a second computable memory space corresponding to the second address information indicated by the address index number in the first immediate number, including: the RDMA network engine module, in response to receiving the first RDMA write request, determines the first address information indicated by the address index number in the first immediate number based on the memory pooling mapping list. , and writes the task data into the first computable memory space in the computable memory module indicated by the first address information, and sends a work request and the first immediate value to the in-memory computing controller; in response to receiving the work request sent by the RDMA network engine module, the in-memory computing controller issues the computing command indicated by the command code to the computable memory module indicated by the first address information based on the command code in the first immediate value and the first address information indicated by the address index number; in response to receiving the computing command, the computable memory module performs the computing command corresponding to the computing command on the task data in the first computable memory space, obtains the computing result of the task data, and writes the computing result into the second computable memory space corresponding to the second address information indicated by the address index number, and sends a computing completion notification to the in-memory computing controller, wherein the computing completion notification includes the computing completion status of the task data.

[0013] In a possible implementation of the first aspect, the in-memory computing device sends a second RDMA write request to the task processing device, including: the in-memory computing controller, in response to receiving a task completion notification sent by the computable memory module, encodes the address index number and the calculation completion status in the first immediate number into a second immediate number, wherein the calculation completion status includes calculation success or calculation failure; the in-memory computing controller generates the second RDMA write request based on the calculation result in the second computable memory space and the second immediate number, and sends the second RDMA write request to the task processing device through the RDMA network engine module.

[0014] According to a second aspect of the present disclosure, a remote in-memory computing interaction method based on the RDMA mechanism is provided, which is applied to an in-memory computing device, the method comprising: receiving a first RDMA write request sent by a host of a task processing device, the first RDMA write request carrying task data to be calculated and a first immediate value, the first immediate value comprising an address index number and a command code, the command code being used to indicate the computing command to be executed, the address index number being used to indicate first address information for storing the task data in the in-memory computing device, second address information for storing the calculation result of the task data in the in-memory computing device, and third address information for storing the calculation result of the task data in the memory of the host; based on the address index number in the first immediate value, The task data is written into a first computable memory space corresponding to the first address information indicated by the address index number, and calculation is performed on the task data in the first computable memory space according to the calculation command indicated by the command code in the first immediate number to obtain a calculation result of the task data, and the calculation result is written into a second computable memory space corresponding to the second address information indicated by the address index number in the first immediate number; a second RDMA write request is sent to the task processing device, and the second RDMA write request carries the calculation result and the second immediate number, and the second immediate number contains the address index number and the calculation completion status of the task data, so that the host of the task processing device completes the task to be executed according to the calculation result and the second immediate number.

[0015] In a possible implementation of the second aspect, a memory pooling mapping list is pre-installed in the task processing device, and the memory pooling mapping list includes: multiple address index numbers and a set of memory pooling information corresponding to each address index number, and the set of memory pooling information includes: address information for storing task data in the in-memory computing device, address information for storing calculation results of task data in the in-memory computing device, and address information for storing calculation results of task data in the host's memory; wherein the address information includes the address and size of the memory space; the address index number includes: a list entry identifier of the memory pooling mapping list, and a queue identifier of a work queue on the host or a user identifier on the host; the work queue is used to cache work requests.

[0016] In a possible implementation of the second aspect, the in-memory computing device includes: an RDMA network engine module, an in-memory computing controller and a plurality of computable memory modules; wherein, based on the address index number in the first immediate number, the task data is written into a first computable memory space corresponding to the first address information indicated by the address index number, and according to the calculation command indicated by the command code in the immediate number, calculation is performed on the task data in the first computable memory space to obtain the calculation result of the task data, and the calculation result is written into the second computable memory space corresponding to the second address information indicated by the address index number in the first immediate number, including: the RDMA network engine module determines the first address information indicated by the address index number in the first immediate number based on the memory pooling mapping list, and writes the task data into the second computable memory space indicated by the first address information A first computable memory space in the computing memory module is calculated, and a work request and the first immediate value are sent to the in-memory computing controller; in response to receiving the work request sent by the RDMA network engine module, the in-memory computing controller issues the computing command indicated by the command code to the computable memory module indicated by the first address information based on the command code in the first immediate value and the first address information indicated by the address index number; in response to receiving the computing command, the computable memory module performs the computing corresponding to the computing command on the task data in the first computable memory space, obtains the computing result of the task data, writes the computing result into the second computable memory space corresponding to the second address information indicated by the address index number, and sends a computing completion notification to the in-memory computing controller, wherein the computing completion notification includes the computing completion status of the task data.

[0017] In a possible implementation of the second aspect, the sending of the second RDMA write request to the task processing device includes: the in-memory computing controller, in response to receiving a task completion notification sent by the computable memory module, encodes the address index number and the calculation completion status in the first immediate number into a second immediate number, wherein the calculation completion status includes calculation success or calculation failure; the in-memory computing controller generates the second RDMA write request based on the calculation result in the second computable memory space and the second immediate number, and sends the second RDMA write request to the task processing device through the RDMA network engine module.

[0018] According to a third aspect of the present disclosure, a remote in-memory computing interaction method based on the RDMA mechanism is provided, which is applied to a task processing device, wherein the task processing device includes a host and an RDMA network card, and the method includes: the host sends a first RDMA write request to the remote in-memory computing device through the RDMA network card, wherein the first RDMA write request carries task data to be calculated and a first immediate value, the first immediate value includes an address index number and a command code, the command code is used to indicate the calculation command to be executed, the address index number is used to indicate the first address information of the in-memory computing device for storing the task data, the second address information of the in-memory computing device for storing the calculation result of the task data, and the address index number of the host's memory for storing the task data. The RDMA network card receives the third address information of the calculation result of the task data, so that the in-memory computing device completes the in-memory calculation of the task data based on the first immediate number in the first RDMA write request; in response to receiving a second RDMA write request sent by the in-memory computing device, the RDMA network card writes the calculation result in the second RDMA write request into the host memory space corresponding to the third address information indicated by the address index number in the memory of the host, and sends a work request and the second immediate number to the host; in response to receiving the work request sent by the RDMA network card, the host completes the task to be executed according to the second immediate number and the calculation result in the host memory space.

[0019] In a possible implementation of the third aspect, a memory pooling mapping list is pre-installed in the task processing device, and the memory pooling mapping list includes: multiple address index numbers and a set of memory pooling information corresponding to each address index number, and the set of memory pooling information includes: address information for storing task data in the in-memory computing device, address information for storing calculation results of task data in the in-memory computing device, and address information for storing calculation results of task data in the host's memory; wherein the address information includes the address and size of the memory space; the address index number includes: a list entry identifier of the memory pooling mapping list, and a queue identifier of the work queue on the host or a user identifier on the host; the work queue is used to cache work requests.

[0020] In a possible implementation of the third aspect, the host sends a first RDMA write request to a remote in-memory computing device through the RDMA network card, including: the host assigns an address index number to the task data to be calculated based on the memory pooling mapping list, and encodes the assigned address index number and the command code of the calculation command to be executed as the first immediate number; the host generates the first RDMA write request based on the first immediate number and the task data, and sends the first RDMA write request to the in-memory computing device through the RDMA network card.

[0021] According to a fourth aspect of the present disclosure, a remote in-memory computing interaction system based on the RDMA mechanism includes: a task processing device and an in-memory computing device, the task processing device including a host and an RDMA network card; the in-memory computing device is used to implement the second aspect or one or several interaction methods among the multiple possible implementations of the second aspect, and the task processing device is used to implement the third aspect or one or several interaction methods among the multiple possible implementations of the third aspect.

[0022] According to various aspects of the present disclosure, by using the RDMA write request that supports immediate values provided by the RDMA mechanism to perform direct remote writing of memory semantics, and at the same time compressing the address index number and command code or calculation completion status into the immediate value and transmitting it to the other end, the traditional four network interactions are reduced to two interactions. Therefore, without modifying any existing network protocol, the standard RDMA mechanism can be used to implement interactive operations of remote in-memory computing, which is beneficial to improving the access performance and read and write efficiency of remote in-memory computing devices, and further beneficial to improving the task execution efficiency of task processing devices.

[0023] Further features and aspects of the present disclosure will become apparent from the following detailed description of exemplary embodiments with reference to the attached drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] The accompanying drawings, which are incorporated in and constitute a part of the specification, illustrate exemplary embodiments, features, and aspects of the disclosure and, together with the description, serve to explain the principles of the disclosure.

[0025] Figure 1a A schematic diagram showing the structure of a traditional memory storage device provided by the prior art is shown.

[0026] Figure 1b A schematic structural diagram of an in-memory computing module provided by the prior art is shown.

[0027] Figure 2 A schematic diagram of the architecture of a remote in-memory computing interaction system provided by an embodiment of the present disclosure is shown.

[0028] Figure 3The present invention provides a basic interactive process for implementing remote control in-memory computing through a standard RDMA mechanism, as provided in an embodiment of the present invention.

[0029] Figure 4 A flowchart of a remote in-memory computing interaction method based on the RDMA mechanism according to an embodiment of the present disclosure is shown.

[0030] Figure 5 A schematic diagram of a memory pooling mapping according to an embodiment of the present disclosure is shown.

[0031] Figure 6 A schematic diagram illustrating a remote in-memory computing interaction process according to an embodiment of the present disclosure is shown.

[0032] Figure 7 A schematic diagram illustrating a remote in-memory computing interaction process according to an embodiment of the present disclosure is shown. DETAILED DESCRIPTION

[0033] Various exemplary embodiments, features, and aspects of the present disclosure will be described in detail below with reference to the accompanying drawings. The same reference numerals in the accompanying drawings represent elements with the same or similar functions. Although various aspects of the embodiments are shown in the accompanying drawings, the drawings are not necessarily drawn to scale unless otherwise indicated.

[0034] The word “exemplary” is used exclusively herein to mean “serving as an example, example, or illustration.” Any embodiment described herein as “exemplary” is not necessarily to be construed as preferred or advantageous over other embodiments.

[0035] In addition, numerous specific details are provided in the following detailed description to better illustrate the present disclosure. Those skilled in the art will appreciate that the present disclosure can be practiced without certain specific details. In some instances, methods, means, components, and circuits well known to those skilled in the art are not described in detail in order to highlight the main points of the present disclosure.

[0036] It should be understood that the terms "first", "second", "third" prefixes, etc. used in this disclosure are only used to distinguish different objects, and cannot be understood as indicating or implying relative importance or implicitly indicating the number of technical features indicated, nor are they used to describe a specific order. Thus, features defined as "first", "second", "third" may explicitly or implicitly include one or more of the features. In the description of this disclosure, the meaning of "plurality" is two or more, unless otherwise clearly and specifically defined. The terms "include" and "comprising" used in this disclosure indicate the presence of the described features, wholes, steps, operations, elements and / or components, but do not exclude the presence or addition of one or more other features, wholes, steps, operations, elements, components and / or their collections.

[0037] To facilitate understanding of the remote in-memory computing interaction method based on the RDMA mechanism proposed in the embodiment of the present disclosure, the embodiment of the present disclosure first briefly introduces the existing in-memory computing read and write process.

[0038] In-memory computing is essentially embedding high-speed computing logic hardware (such as Figure 1b In-memory computing can achieve the purpose of in-memory computing. In-memory computing can implement large-scale data computing tasks through direct in-memory computing logic. Its basic operation process is as follows:

[0039] 1. Host (such as Figure 1b The processor in the memory writes the data into the input data address of the computing module;

[0040] 2. The host issues a calculation command;

[0041] 3. The in-memory computing module performs the calculation of the calculation command;

[0042] 4. After the in-memory calculation module completes the calculation, it notifies the host computer that the calculation is complete;

[0043] 5. The host reads the results from the in-memory computing module.

[0044] The above process can be divided into two main logical layers: the data plane, where the host reads and writes data, and the control plane, where the host issues computational commands and the in-memory computational unit notifies the host of completion status. On the data plane, the host writes data and reads the results. For example, the host can directly use MMIO (memory mapping I / O) with load or store instructions, or it can use DMA to write data to the in-memory computational module and then read the computational results into the host's memory. Because data is generally larger than a dozen bytes, MMIO instructions have a long instruction latency, so the data plane generally uses DMA.

[0045] On the control plane, in-memory computing typically implements complex, memory-intensive calculations, but the commands are typically simple, typically triggered by writing to a command register or, more simply, a doorbell register. Furthermore, in-memory computing modules can notify the host of computational completion status in a variety of ways, such as interrupts or simply setting a memory bit, which the host can then poll to query.

[0046] The RDMA mechanism can be used to share and pool in-memory computing resources through high-speed Ethernet. Under this mechanism, on the data plane, it can directly provide remote read and write operations of the computational memory. On the control plane, it can be an in-memory computing controller (such as Figure 1bThe control unit in the memory is used to manage in-memory computing, mainly issuing commands, managing the status of in-memory computing tasks, notifying the host, etc. For example, Figure 2 A schematic diagram of the architecture of a remote in-memory computing interaction system provided by an embodiment of the present disclosure is shown. Figure 2 As shown in the figure, the hosts of two task processing devices (host 1 and host 2) can share an in-memory computing device based on lossless Ethernet of ROCEv2 (Remote Direct Memory Access over Converged Ethernet version 2, a protocol for remote direct memory access (RDMA)), thereby achieving the purpose of resource sharing and pooling.

[0047] Among them, the task processing device can be understood as a device that performs computing tasks, for example, it can be an AI device that performs neural network training tasks and inference tasks. The host in the task processor can include AI accelerators, general-purpose processors (such as central processing unit CPU, image processing unit GPU) and memory storage and other electronic components; the in-memory computing device can be understood as a device with in-memory computing capabilities. The in-memory computing device can be any hardware form such as in-memory computing chip, device, unit, integrated circuit, etc. The in-memory computing device can be an independent device or a component in an electronic device (such as a server), and the embodiments of the present disclosure do not limit this.

[0048] like Figure 2 As shown, the in-memory computing device is mainly composed of two parts, namely the control part and the computable memory module part, wherein the control part mainly includes: RDMA network engine module, in-memory computing controller and high-speed memory interface module; RDMA network engine module can be understood as an in-chip RDMA network card connected to the external Ethernet, which is mainly used to connect to the external network and operate in-memory computing; the in-memory computing controller is mainly used to implement functions such as issuing computing commands, managing the status of in-memory computing tasks, and managing computable memory modules; the high-speed memory interface module is used to realize the connection between the computable memory module and each module in the control part, for example, the UCIe (Universal Chiplet Interconnect Express) in the chip or the PCIe (Peripheral Component Interconnect Express) on the board can be used to connect the computable memory module with the in-memory computing controller and the RDMA network engine module; the computable memory module part mainly includes multiple computable memory modules, and a single computable memory module includes a computing unit (such as Figure 1bIn-memory computing processing unit) and storage unit (such as Figure 1b In the memory storage logic unit), each computable memory module is connected to the control part through a high-speed connection (such as UCIe or PCIe). The computable memory module is mainly used to complete computing tasks and store input data and calculation results.

[0049] based on Figure 2 In the interactive system shown, if any host initiates an in-memory computing task to the in-memory computing device, the following can be performed: Figure 3 The following illustrates a basic interaction process for implementing remote control in-memory computing using a standard RDMA mechanism:

[0050] 31. The host uses the memory semantics unilateral operation rdma_write() to write data to the computable memory module. This step can be understood as the host sending an rdma_write write request to the RDMA network engine module.

[0051] 32. The RDMA network engine module in the in-memory computing device receives the write request and writes the data to be calculated in the write request to the computable memory space of the computable memory module at the corresponding address;

[0052] 33. The host sends a computing command to the in-memory computing device. This step can use the rdma_send() operation of the message semantic mechanism to send the computing command to the receiving queue rq of the in-memory computing controller;

[0053] 34. The in-memory computing controller receives the computing command and sends the computing command to the computable memory module, so that the computable memory module starts computing and simultaneously starts managing and tracking the computing status of the computable memory module;

[0054] 35. After the computational memory module completes the computation, it notifies the in-memory computation controller that the computation is complete;

[0055] 36. The in-memory calculation controller notifies the host of the completion of the calculation and sends the calculation completion status and the storage address and size of the calculation result. This step can be sent to the host's receive queue rq using the rdma_send() operation of the message semantic mechanism;

[0056] 37. After the host receives the calculation completion status and result address information sent by the in-memory calculation controller, it uses the rdma_read() operation of the memory semantic mechanism to read the calculation results in the in-memory calculation device into the host's own memory.

[0057] Figure 3In the interaction process shown, a total of four RDMA mechanism network interactions occurred, including two interactions of the message semantic mechanism as steps 33 and 36, and two interactions of the memory semantic mechanism as steps 31 and 37. Considering that in actual situations, there is a certain mapping relationship between the storage address of the calculation result and the storage address of the input data in the computable memory module, the size of the calculation result and the size of the input data are also known. For example, if the computable memory module performs matrix multiplication, the size of the calculation result can be deduced according to the operation rules of matrix multiplication, and the storage address of the calculation result also has a corresponding relationship with the storage address of the input data. Therefore, what really needs to interact in the interaction process is mainly the calculation command issued by the host, the memory address where the host stores the calculation result, and the status of the calculation result. It can be seen from the above Figure 3 In the interactive process shown, the interaction efficiency of remotely operating the in-memory computing device is low, and the access performance is not high.

[0058] In view of this, the embodiments of the present disclosure propose a remote in-memory computing interaction method based on the RDMA mechanism, which can compress control plane information (such as computing commands and computing completion status) through a memory pooling mechanism without changing the general network protocol and without adding any additional hardware functions. Without affecting the interaction between the control plane and the data plane, four network interactions are reduced to two interactions. Therefore, on the basis of standard RDMA mechanism network devices and without affecting the scalability of network devices, efficient remote in-memory computing interaction based on the RDMA mechanism is realized, which is beneficial to improving the access performance of remote in-memory computing devices and improving the task processing efficiency of task processing devices.

[0059] The following Figures 4 to 7 A remote in-memory computing interaction method based on the RDMA mechanism provided in this public embodiment is introduced in detail.

[0060] Figure 4 A flowchart of a remote in-memory computing interaction method based on the RDMA mechanism according to an embodiment of the present disclosure is shown. Figure 2 The interactive system shown, Figure 4 As shown, the method includes:

[0061] Step S41: The host of the task processing device sends a first RDMA write request to the remote in-memory computing device through the RDMA network card. The first RDMA write request carries the task data to be calculated and a first immediate value. The first immediate value includes an address index number and a command code. The command code is used to indicate the computing command to be executed. The address index number is used to indicate first address information for storing the task data in the in-memory computing device, second address information for storing the calculation result of the task data in the in-memory computing device, and third address information for storing the calculation result of the task data in the host's memory.

[0062] Step S42: In response to receiving the first RDMA write request, the in-memory computing device writes the task data into a first computable memory space corresponding to the first address information indicated by the address index number based on the address index number in the first immediate number, and performs a calculation on the task data in the first computable memory space according to the calculation command indicated by the command code in the first immediate number to obtain a calculation result of the task data, and writes the calculation result into a second computable memory space corresponding to the second address information indicated by the address index number in the first immediate number;

[0063] Step S43: The in-memory computing device sends a second RDMA write request to the task processing device. The second RDMA write request carries the calculation result in the second computable memory space and a second immediate value. The second immediate value includes an address index number and a calculation completion status of the task data.

[0064] In step S44, the RDMA network card of the task processing device responds to receiving the second RDMA write request, writes the calculation result into the host memory space corresponding to the third address information indicated by the address index number based on the address index number in the second immediate number, and sends a work request and the second immediate number to the host, so that the host completes the task to be executed according to the second immediate number and the calculation result in the host memory space in response to receiving the work request sent by the RDMA network card.

[0065] It is understandable that the message sent by the host to the in-memory computing controller, in addition to transmitting information such as commands and addresses, also serves to notify the in-memory computing controller to start triggering and managing tracking computing tasks. The message that the in-memory computing controller replies to the host, in addition to carrying result information, also serves to notify the host that the calculation is complete. Therefore, in order to save the purpose of two network message interactions on the control plane, in addition to transmitting control information (such as computing commands, etc.) to the other end, it is also necessary to notify the other end of the corresponding status information. Therefore, among the various operations of the standard RDMA mechanism, an efficient interaction method is provided, which is the command rdma_write_with_immediate() that can add an immediate value (immediate value) to the rdma_write() write operation of memory semantics. In the standard RDMA mechanism, the immediate value is a 4-Byte data, open to users, and stored in the work request wqe sent to the submission queue sq. Unlike sending a rdma_write() request alone, when the sender sends a rdma_write_with_immediate() request (that is, an RDMA request with an immediate value), the RDMA network card at the receiving end, in addition to executing rdma_write(), will also send a wqe to the receiving queue of the receiving end, and generate a completion queue entry cqe in the completion queue cq. cqe will contain the 4-byte immediate value sent by the sender, that is, a work request is sent to the receiving end and carries the immediate value. Therefore, in addition to the efficient operation of rdma_write() with memory semantics, the rdma_write_with_immediate() request can also play the role of notifying the other end, and can also use the immediate value to communicate 4 bytes of additional information. The first RDMA write request and the second RDMA write request of the embodiment of the present disclosure are both RDMA write requests that support immediate values.

[0066] In one possible implementation, the task processing device and the in-memory computing device are each pre-installed with a memory pooling mapping list, which includes: multiple address index numbers and a set of memory pooling information corresponding to each address index number, the set of memory pooling information including: address information for storing task data in the in-memory computing device, address information for storing calculation results of the task data in the in-memory computing device, and third address information for storing calculation results of the task data in the host's memory; wherein the address information includes the address and size of the memory space (i.e., the length of the memory space). Based on the security standards specified in the RDMA mechanism, the address information may also include a memory key (mem key) corresponding to the memory space to implement secure remote memory read and write interactive operations.

[0067] In practical applications, before performing read and write interactions of remote in-memory computing, a system initialization process can be performed first. The system initialization process mainly involves memory pooling and mapping and sending it to the other end to complete metadata synchronization. Specifically, the above-mentioned memory pool mapping list can be generated by the host of the task processing device and sent to the in-memory computing device. For example, before performing remote in-memory computing interactions, the host can first allocate the computing units and storage units of the computable memory module in the in-memory computing device, and pre-allocate the memory address in the host memory for storing the calculation results based on the memory required by the computing task (such as the size of the memory space required), map the address information for storing task data and calculation results in the in-memory computing device to the memory address for storing calculation results in the host memory, and generate a memory pool mapping list, which includes multiple list entry slots, each list entry containing: the storage address and size of the task data in the computable memory module in the in-memory computing device, the storage address and size of the calculation result in the computable memory module, and the memory address pre-allocated in the host memory for receiving and storing the calculation result. The host can then send the memory pool mapping list to the in-memory computing controller of the in-memory computing device and complete the memory registration to enable remote direct memory access between the two ends.

[0068] For example, Figure 5 A schematic diagram of a memory pool mapping is shown, such as Figure 5 As shown, the memory information table (also known as the memory pooling mapping table) set by the in-memory computing controller contains n list entries, each list entry includes: a list entry identifier (slot id, which can be used as an address index number here) and the corresponding computable memory input data address information (that is, the address information for storing task data in the in-memory computing device, including the address and length (that is, size)), computable memory result address information (that is, the address information for storing the calculation result of the task data in the in-memory computing device, including the address and length), and the host side result storage address information (that is, the third address information for storing the calculation result of the task data in the host memory, including the address and memory key (mem key)). For example, the computable memory input data address information corresponding to Slot id = 2 includes the address "x2" and the length "16384", the computable memory result address information includes the address "y2" and the length "4096", and the address information for storage on the host side includes the address "z2" and the memory key "k2".

[0069] In practical applications, the address index number may include: the list entry identifier (slotID) of the memory pooling mapping list, as well as the queue identifier (queue ID) of the work queue on the host or the user ID on the host. The work queue is used to cache work requests and typically includes a submission queue and a receive queue. This is because the slotID itself may not be globally unique, or different hosts may use the same slot ID. Therefore, the queueID or user ID can be used in conjunction with the slot ID as the address index number. This way, when two work requests with the same slot ID are sent to the in-memory computing device, the address index number can be mapped to different computable memory spaces due to different queue IDs or user IDs. Therefore, after the memory resources of the in-memory computing device are pooled, the in-memory computing device can use the slot ID combined with the queue ID or user ID to implement a memory pooling mapping list synchronized according to the system initialization process, converting the address index number into the corresponding address information, thereby completing the exchange of address information. A single address index number usually only requires one or two bytes and can be placed in a 4-byte immediate value, thereby realizing the transmission of address information.

[0070] It can be seen that a host can include multiple virtual machines, and different virtual machines can perform different tasks. Therefore, different virtual machines can correspond to different user identifiers and different queue identifiers. The in-memory computing tasks issued by different virtual machines can be identified by the user identifier or the queue identifier, so that the in-memory computing controller of the in-memory computing device can manage and track the in-memory computing tasks issued by different hosts or different virtual machines. Multiple virtual machines on the same host can respectively generate multiple first RDMA write requests and send them to the in-memory computing device, and different hosts can send multiple first RDMA write requests to the same in-memory computing device. The in-memory computing device can process different first RDMA write requests to perform multiple in-memory computing tasks in parallel, which also realizes the pooled sharing of in-memory computing resources of the in-memory computing device.

[0071] In summary, the embodiment of the present disclosure uses immediate numbers for information interaction notification on the basis of a highly scalable architecture that keeps the control plane and the data plane separated. It is understandable that a 4-byte immediate number is sufficient for information such as command codes, notifications or status of the control plane, but it is completely insufficient for address information. For example, an address of a modern host reaches 64 bits (8 bytes). Therefore, the embodiment of the present disclosure uses a memory pooling mechanism to complete the memory address information interaction through an address index number of one or two bytes. The task processing device and the in-memory computing device collaborate to perform memory pooling and map the memory on both sides to generate a memory pooling mapping list. The memory pooling information can be synchronized at both ends when the system is initialized. Thereafter, during the execution of the computing task, the memory address information interaction only needs to interact with the address index number. It should be understood that the number of memory pools can meet the number of calculations that can be performed in parallel for all requests in the work queue of the current operation, and the embodiment of the present disclosure does not impose any restrictions on this.

[0072] Based on the above memory pool mapping list, in step S41, the host of the task processing device sends a first RDMA write request to the remote in-memory computing device through the RDMA network card, including:

[0073] The host allocates an address index number to the task data to be calculated based on the memory pool mapping list, and encodes the allocated address index number and the command code of the calculation command to be executed as a first immediate value;

[0074] The host generates a first RDMA write request based on the first immediate value and the task data, and sends the first RDMA write request to the in-memory computing device through the RDMA network card.

[0075] The host can, for example, allocate available address index numbers to the task data based on the memory required for computing the task data and in combination with the memory pooling mapping list. This means allocating a set of memory pooling information that meets the computing requirements to the task data. The address index number and the command code to be executed can then be encoded into a 4-byte format as the first immediate value. The first immediate value and the task data are then attached to the rdma_write_with_immediate() write request to obtain the first RDMA write request and send it to the in-memory computing device. This process is equivalent to compressing the pooled address information into an address index number, placing the address index number and command code directly in the immediate values of different bits, and then completing the exchange of address information by attaching it to the first RDMA write request sent by the host to the in-memory computing device.

[0076] As above Figure 2The in-memory computing device shown includes: an RDMA network engine module, an in-memory computing controller, and multiple computable memory modules. Therefore, in step S42, the in-memory computing device responds to receiving a first RDMA write request, writes task data into a first computable memory space corresponding to the first address information indicated by the address index number based on the address index number in the first immediate number, performs calculations on the task data in the first computable memory space according to the calculation command indicated by the command code in the first immediate number, obtains the calculation result of the task data, and writes the calculation result into the second computable memory space corresponding to the second address information indicated by the address index number in the first immediate number, including:

[0077] In response to receiving the first RDMA write request, the RDMA network engine module determines, based on the memory pooling mapping list, first address information indicated by the address index number in the first immediate value, writes the task data into a first computable memory space in the computable memory module indicated by the first address information, and sends a work request and the first immediate value to the in-memory computing controller;

[0078] In response to receiving the work request sent by the RDMA network engine module, the in-memory computing controller issues a computing command indicated by the command code to the computable memory module indicated by the first address information based on the command code in the first immediate value and the first address information indicated by the address index number;

[0079] In response to receiving a calculation command, the computable memory module performs the calculation corresponding to the calculation command on the task data in the first computable memory space, obtains the calculation result of the task data, and writes the calculation result into the second computable memory space corresponding to the second address information indicated by the address index number, and sends a calculation completion notification to the in-memory calculation controller, where the calculation completion notification includes the calculation completion status of the task data, and the calculation completion status includes calculation success or calculation failure.

[0080] The above process can be understood as follows: after receiving the first RDMA write request sent by the host, the RDMA network engine module queries the memory pool mapping list, obtains the first address information indicated by the address index number in the first immediate value carried in the first RDMA write request, and writes the task data in the first RDMA write request into the first computable memory space in the computable memory module indicated by the first address information. Since the first RDMA write request carries an immediate value, the RDMA network engine module will simultaneously send a work request wqe to the receive queue rq of the in-memory computing controller, and generate a queue entry cqe to the completion queue cq, which contains the first An immediate number, wherein sending a work request is equivalent to notifying the in-memory computing controller; after receiving the first immediate number, the in-memory computing controller can obtain the address index number of this in-memory computing task, and then can obtain a corresponding set of memory pooling information (including first address information, second address information and third address information) by querying the memory pooling mapping list, and then can issue a computing command indicated by the command code to the computable memory module indicated by the first address information, and start to manage this in-memory computing task and track the task computing status. For example, the in-memory computing controller can use active polling or interruption to receive the computing completion notification sent by the computable memory module.

[0081] In one possible implementation, after the computable memory module notifies the in-memory computing controller that the computation is complete, the in-memory computing controller can proactively write the computation result to the host's memory through an RDMA write operation, thereby reducing one network interaction. Specifically, in step S43, the in-memory computing device sends a second RDMA write request to the task processing device, including:

[0082] The in-memory calculation controller, in response to receiving a task completion notification sent by the computable memory module, encodes the address index number and the calculation completion status in the first immediate value into a second immediate value;

[0083] The in-memory calculation controller generates a second RDMA write request according to the calculation result in the second computable memory space and the second immediate value, and sends the second RDMA write request to the task processing device through the RDMA network engine module.

[0084] The above process can be understood as follows: after the computable memory module completes the in-memory computing task, the in-memory computing controller queries the memory pool mapping list based on the address index number to obtain all the information returned to the host side (such as the calculation result and the third address information). Then, it can send a rdma_write_with_immediate() write request to the task processing device to write the calculation result to the memory space of the corresponding address of the host. By packaging the address index number and the calculation completion status in the second immediate number, the host side will receive a work request receive wqe and a completion queue entry cqe, and then can obtain the context of the task to be executed through the address index number in the second immediate number, and start subsequent processing based on the calculation completion status, such as continuing to execute the model training task.

[0085] Specifically, after receiving the second RDMA write request, the RDMA network card of the task processing device can query the memory pool mapping list to obtain the third address information indicated by the address index number in the second immediate number, and then write the calculation result in the second RDMA request into the host memory space corresponding to the third address information indicated by the address index number in the host's memory, and send a work request and the second immediate number to the host; in response to receiving the work request sent by the RDMA network card, the host completes the task to be executed according to the second immediate number and the calculation result in the host memory space.

[0086] By using the embodiment of the present disclosure, during result transmission, the host does not need to actively read the calculation results from the computable memory module. Instead, the in-memory calculation controller actively writes the calculation results to the host's memory. The calculation completion status returned by the in-memory calculation controller to the host only requires a small number of bits to complete, for example, using a bit to mark the success or failure of the calculation. Among them, in the case of in-memory calculation failure, the host can send an additional request to obtain further status. This part of the exception handling is not in the high-speed transmission process of the calculation task, so no additional optimization design is required, and a general exception handling method can be used. The in-memory calculation controller uses rdma_write_with_immediate() to actively write the calculation results to the host side memory, and at the same time packages the address index number and the calculation completion status into the second immediate number to complete the control plane information completion interaction, which can reduce one network interaction. The host will receive a work request wqe and a completion queue entry cqe, so that the host can obtain the second immediate number, and then retrieve the context of the task to be executed through the address index number to complete the task to be executed. In this way, remote in-memory calculation operations can be implemented with two interactions.

[0087] Figure 6A schematic diagram of a remote in-memory computing interaction process according to an embodiment of the present disclosure is shown, wherein result represents the memory space for storing the calculation result, and input represents the memory space for storing the task data. Figure 6 As shown, the host allocates an address index number that can be used (slot id is used here to represent the address index number), and the slot id (such as Figure 5 slot 2 in the memory) and the command code to be executed are encoded together into a 4-byte array as the first immediate value. The host sends a rdma_write_with_immediate() write request to the in-memory computing device. The immediate value carried in the request includes the address index number 2, and the input task data is written to the memory space corresponding to slot 2 of the computable memory module. Since the write request contains an immediate value, the RDMA network engine module will also send a receive wqe and a cqe to the in-memory computing controller. The in-memory computing controller will receive the first immediate value, obtain the slot id of this computing task, and query the memory pool mapping list to obtain the address information of the local and host ends receiving the computing results; the in-memory computing controller sends a computing command to the computable memory module, and starts to manage this task and track the task computing status. The in-memory computing controller can use active polling or interrupts to receive the task completion notification from the computable memory module. After the computational memory module completes the calculation, it writes the result to the memory space corresponding to slot 2 and notifies the in-memory computation controller of the calculation completion. The in-memory computation controller then retrieves the address information returned to the host based on the slot ID table lookup. It then sends a write request (rdma_write_with_immediate()) to the host. The request carries an immediate value containing the address index 2 and the computation completion status, writing the result to the memory space indicated by slot 2 in the host's memory. The host receives a wqe and a cqe. The cqe contains the second immediate value. The address index in the second immediate value is used to obtain the context for the task to be executed, and subsequent processing is performed based on the computation completion status.

[0088] The present disclosure also provides Figure 7 A schematic diagram of a remote in-memory computing interaction process is shown in FIG. Figure 7 As shown, the interaction process includes:

[0089] 71. The host of the task processing device sends a first RDMA write request to the in-memory computing device through the RDMA network card;

[0090] 72. The RDMA network engine module of the in-memory computing device receives the first RDMA write request, writes the task data into the first computable memory space of the computable memory module at the corresponding address, and simultaneously sends a work request and the first immediate value to the in-memory computing controller;

[0091] 73. The in-memory computing controller receives the work request and sends a computing command to the computable memory module;

[0092] 74. The computable memory module executes the calculation command, obtains the calculation result and writes it into the corresponding second computable memory space, and sends a calculation completion notification to the in-memory calculation controller;

[0093] 75. The in-memory computing controller receives the computing completion notification, generates a second RDMA write request, and sends the second RDMA write request to the task processing device;

[0094] 76. The RDMA network card of the task processing device receives the second RDMA write request, writes the calculation result into the host memory space of the host memory, and simultaneously sends the work request and the second immediate value to the host;

[0095] 77. The host receives the work request and the second immediate value, and performs subsequent processing based on the calculation result and the second immediate value to complete the task to be executed.

[0096] Compared to Figure 3 The basic interaction process shown involves four network interactions. By utilizing the remote memory computing interaction process of the embodiment of the present invention, information transmission on the data plane and the control plane can be completed simultaneously in one network interaction. The control process of the entire in-memory computing task only requires two network interactions, which greatly improves the interaction efficiency compared to the basic four network interactions.

[0097] According to the method of the embodiment of the present invention, by using the RDMA write request that supports immediate numbers provided by the RDMA mechanism to perform direct remote writing of memory semantics, and at the same time compressing the address index number and command code or calculation completion status into the immediate number and transmitting it to the other end, the traditional four network interactions are reduced to two interactions. Therefore, without modifying any existing network protocol, the standard RDMA mechanism can be used to implement interactive operations of remote in-memory computing, which is beneficial to improving the access performance and read and write efficiency of remote in-memory computing devices, and further beneficial to improving the task execution efficiency of task processing devices.

[0098] The contributions of the embodiments of the present disclosure are: first, by cooperating with the devices at both ends to pool and map the relevant memory in memory management, a memory pool mapping list is generated, and the memory information that needs to be interacted in the calculation process can be compressed into an address index number of one or two bytes; second, the two ends use the rdma_write_with_immediate() write request to perform direct remote writing of memory semantics, and at the same time compress the address index number and command code or calculation completion status into an immediate number as control plane information to be transmitted to the other end, so as to achieve the data plane and control plane information transmission function in one network interaction without affecting the separation of the data plane and the control plane; third, the process of replying the calculation result no longer requires the in-memory computing device to notify the host separately through the network, and the host then reads it. Instead, the in-memory computing device actively writes the calculation result to the host memory through the rdma_write_with_immediate() write request, and completes the notification and control plane information transmission through the immediate number, thereby reducing the two network interactions required to reply the calculation result to one interaction. It can be seen that by utilizing the interaction method of the embodiment of the present disclosure, it is possible to maintain an in-memory computing task on a network operation with memory semantics and keep the data plane and control plane separated, while optimizing the additional control plane interaction and reducing the efficiency of network interaction from four times to two times, thereby achieving an efficient remote in-memory computing interaction method based on the RDMA mechanism using existing RDMA network equipment without modifying any network protocol.

[0099] Based on the above Figure 4 The embodiment of the present disclosure provides a remote in-memory computing method applied to an in-memory computing device, including:

[0100] receiving a first RDMA write request sent by a host of a task processing device, wherein the first RDMA write request carries task data to be calculated and a first immediate value, wherein the first immediate value includes an address index number and a command code, wherein the command code is used to indicate a calculation command to be executed, and the address index number is used to indicate first address information in the in-memory computing device for storing the task data, second address information in the in-memory computing device for storing a calculation result of the task data, and third address information in the memory of the host for storing the calculation result of the task data;

[0101] Based on the address index number in the first immediate value, writing the task data into a first computable memory space corresponding to the first address information indicated by the address index number, and performing calculation on the task data in the first computable memory space according to the calculation command indicated by the command code in the first immediate value to obtain a calculation result of the task data, and writing the calculation result into a second computable memory space corresponding to the second address information indicated by the address index number in the first immediate value;

[0102] A second RDMA write request is sent to the task processing device, where the second RDMA write request carries the calculation result and a second immediate value, where the second immediate value includes the address index number and the calculation completion status of the task data, so that the host of the task processing device completes the task to be executed according to the calculation result and the second immediate value.

[0103] In one possible implementation, a memory pooling mapping list is pre-installed in the in-memory computing device, and the memory pooling mapping list includes: multiple address index numbers and a set of memory pooling information corresponding to each address index number, and the set of memory pooling information includes: address information for storing task data in the in-memory computing device, address information for storing calculation results of task data in the in-memory computing device, and address information for storing calculation results of task data in the host's memory; wherein the address information includes the address and size of the memory space; the address index number includes: a list entry identifier of the memory pooling mapping list, and a queue identifier of the work queue on the host or a user identifier on the host.

[0104] In one possible implementation, the in-memory computing device includes: an RDMA network engine module, an in-memory computing controller, and a plurality of computable memory modules; wherein, based on the address index number in the first immediate number, the task data is written into a first computable memory space corresponding to the first address information indicated by the address index number, and according to the calculation command indicated by the command code in the immediate number, the task data in the first computable memory space is calculated to obtain the calculation result of the task data, and the calculation result is written into a second computable memory space corresponding to the second address information indicated by the address index number in the first immediate number, including: the RDMA network engine module determines the first address information indicated by the address index number in the first immediate number based on the memory pooling mapping list, and writes the task data into the computable memory space indicated by the first address information a first computable memory space in a memory module, and sends a work request and the first immediate value to the in-memory computing controller; in response to receiving the work request sent by the RDMA network engine module, the in-memory computing controller issues a computing command indicated by the command code to the computable memory module indicated by the first address information based on the command code in the first immediate value and the first address information indicated by the address index number; in response to receiving the computing command, the computable memory module performs the computing command corresponding to the computing command on the task data in the first computable memory space, obtains the computing result of the task data, writes the computing result into the second computable memory space corresponding to the second address information indicated by the address index number, and sends a computing completion notification to the in-memory computing controller, wherein the computing completion notification includes the computing completion status of the task data.

[0105] In one possible implementation, the sending of the second RDMA write request to the task processing device includes: the in-memory computing controller, in response to receiving a task completion notification sent by the computable memory module, encoding the address index number and the calculation completion status in the first immediate number into a second immediate number, wherein the calculation completion status includes calculation success or calculation failure; the in-memory computing controller generates the second RDMA write request based on the calculation result in the second computable memory space and the second immediate number, and sends the second RDMA write request to the task processing device through the RDMA network engine module.

[0106] Furthermore, an embodiment of the present disclosure provides a remote in-memory computing method applied to a task processing device, wherein the task processing device includes a host and an RDMA network card. The method includes:

[0107] The host sends a first RDMA write request to a remote in-memory computing device through the RDMA network card, where the first RDMA write request carries task data to be calculated and a first immediate value, where the first immediate value includes an address index number and a command code, where the command code is used to indicate a calculation command to be executed, and the address index number is used to indicate first address information in the in-memory computing device for storing the task data, second address information in the in-memory computing device for storing a calculation result of the task data, and third address information in the host's memory for storing the calculation result of the task data, so that the in-memory computing device completes the in-memory calculation of the task data based on the first immediate value in the first RDMA write request;

[0108] In response to receiving the second RDMA write request sent by the in-memory computing device, the RDMA network card writes a calculation result in the second RDMA write request into a host memory space corresponding to third address information indicated by the address index number in the memory of the host based on the address index number in the second immediate value in the second RDMA write request, and sends a work request and the second immediate value to the host;

[0109] In response to receiving the work request sent by the RDMA network card, the host completes the task to be executed according to the second immediate value and the calculation result in the host memory space.

[0110] In one possible implementation, a memory pooling mapping list is pre-installed in the task processing device, and the memory pooling mapping list includes: multiple address index numbers and a set of memory pooling information corresponding to each address index number, and the set of memory pooling information includes: address information for storing task data in the in-memory computing device, address information for storing calculation results of task data in the in-memory computing device, and address information for storing calculation results of task data in the host's memory; wherein the address information includes the address and size of the memory space; the address index number includes: a list entry identifier of the memory pooling mapping list, and a queue identifier of a work queue on the host or a user identifier on the host; the work queue is used to cache work requests.

[0111] In one possible implementation, the host sends a first RDMA write request to a remote in-memory computing device through the RDMA network card, including: the host assigns an address index number to the task data to be calculated based on the memory pooling mapping list, and encodes the assigned address index number and the command code of the calculation command to be executed as the first immediate number; the host generates the first RDMA write request based on the first immediate number and the task data, and sends the first RDMA write request to the in-memory computing device through the RDMA network card.

[0112] By utilizing the interaction method of the embodiment of the present disclosure, it is possible to maintain an in-memory computing task on a network operation with memory semantics and keep the data plane and control plane separated, thereby optimizing the additional control plane interaction and reducing the efficiency of network interaction from four times to two times, thereby realizing an efficient remote in-memory computing interaction process based on the RDMA mechanism without any modification of the network protocol itself and using existing RDMA devices.

[0113] An embodiment of the present disclosure also provides a remote in-memory computing interaction system based on the RDMA mechanism, comprising: a task processing device and an in-memory computing device, the task processing device comprising a host and an RDMA network card; the in-memory computing device comprising: an RDMA network engine module, an in-memory computing controller and a plurality of computable memory modules; wherein the in-memory computing device is used to implement the method steps executed by the in-memory computing device in the above-mentioned interaction method, and the task processing device is used to implement the method steps executed by the task processing device in the above-mentioned interaction method.

[0114] Various aspects of the present disclosure are described herein with reference to flowcharts and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the present disclosure. It should be understood that each block of the flowcharts and / or block diagrams, and combinations of blocks in the flowcharts and / or block diagrams, can be implemented by computer-readable program instructions.

[0115] These computer-readable program instructions can be provided to an in-memory computing controller of an in-memory computing device and a processor in a host computer of a task processing device, thereby producing a machine such that when these instructions are executed, a device is generated to implement the functions / actions specified in one or more blocks in the flowchart and / or block diagram. These computer-readable program instructions can also be stored in a computer-readable storage medium, where these instructions cause a computer, a programmable data processing device, and / or other device to operate in a specific manner. Thus, the computer-readable medium storing the instructions comprises an article of manufacture that includes instructions for implementing various aspects of the functions / actions specified in one or more blocks in the flowchart and / or block diagram.

[0116] The flow charts and block diagrams in the accompanying drawings show the possible architecture, functions and operations of the systems, methods and computer program products according to multiple embodiments of the present disclosure. In this regard, each box in the flow chart or block diagram can represent a part of a module, program segment or instruction, and the part of the module, program segment or instruction contains one or more executable instructions for realizing the prescribed logical function. In some alternative implementations, the functions marked in the box can also occur in a sequence different from that marked in the accompanying drawings. For example, two consecutive boxes can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram and / or flow chart, and the combination of the boxes in the block diagram and / or flow chart can be implemented by a dedicated hardware-based system that performs the prescribed function or action, or can be implemented by a combination of dedicated hardware and computer instructions.

[0117] While various embodiments of the present disclosure have been described above, the foregoing description is intended to be illustrative, non-exhaustive, and not limited to the disclosed embodiments. Many modifications and variations will be apparent to those skilled in the art without departing from the scope and spirit of the described embodiments. The terminology used herein is selected to best explain the principles of the embodiments, their practical applications, or technological improvements in the marketplace, or to enable others skilled in the art to understand the embodiments disclosed herein.

Claims

1. A remote in-memory computing interaction method based on the RDMA mechanism, characterized in that: include: The host of the task processing device sends a first RDMA write request to a remote in-memory computing device through an RDMA network card. The first RDMA write request carries task data to be calculated and a first immediate value. The first immediate value includes an address index number and a command code. The command code is used to indicate a computing command to be executed. The address index number is used to indicate first address information in the in-memory computing device for storing the task data, second address information in the in-memory computing device for storing a calculation result of the task data, and third address information in the host's memory for storing the calculation result of the task data. In response to receiving the first RDMA write request, the in-memory computing device writes the task data into a first computable memory space corresponding to first address information indicated by the address index number based on the address index number in the first immediate number, performs calculations on the task data in the first computable memory space according to a calculation command indicated by the command code in the first immediate number, obtains a calculation result of the task data, and writes the calculation result into a second computable memory space corresponding to second address information indicated by the address index number in the first immediate number; The in-memory computing device sends a second RDMA write request to the task processing device, where the second RDMA write request carries a calculation result in the second computable memory space and a second immediate value, where the second immediate value includes the address index number and a calculation completion status of the task data; In response to receiving the second RDMA write request, the RDMA network card of the task processing device writes the calculation result into the host memory space corresponding to the third address information indicated by the address index number in the host memory based on the address index number in the second immediate number, and sends a work request and the second immediate number to the host, so that the host completes the task to be executed according to the second immediate number and the calculation result in the host memory space in response to receiving the work request sent by the RDMA network card.

2. The method according to claim 1, characterized in that The task processing device and the in-memory computing device are each pre-installed with a memory pooling mapping list, wherein the memory pooling mapping list includes: a plurality of address index numbers and a set of memory pooling information corresponding to each address index number, the set of memory pooling information including: address information for storing task data in the in-memory computing device, address information for storing calculation results of task data in the in-memory computing device, and address information for storing calculation results of task data in the memory of the host; wherein the address information includes the address and size of the memory space; The address index number includes: a list entry identifier of the memory pooling mapping list, and a queue identifier of a work queue on the host or a user identifier on the host; the work queue is used to cache work requests.

3. The method according to claim 2, characterized in that The host of the task processing device sends a first RDMA write request to the remote in-memory computing device through the RDMA network card, including: The host allocates an address index number to the task data to be calculated based on the memory pool mapping list, and encodes the allocated address index number and a command code of the calculation command to be executed as the first immediate value; The host generates the first RDMA write request based on the first immediate value and the task data, and sends the first RDMA write request to the in-memory computing device through the RDMA network card.

4. The method according to claim 2, characterized in that The in-memory computing device includes: an RDMA network engine module, an in-memory computing controller, and a plurality of computable memory modules, wherein, in response to receiving the first RDMA write request, the in-memory computing device writes the task data into a first computable memory space corresponding to the first address information indicated by the address index number based on the address index number in the first immediate number, performs calculation on the task data in the first computable memory space according to the calculation command indicated by the command code in the first immediate number, obtains the calculation result of the task data, and writes the calculation result into a second computable memory space corresponding to the second address information indicated by the address index number in the first immediate number, including: In response to receiving the first RDMA write request, the RDMA network engine module determines, based on the memory pooling mapping list, first address information indicated by the address index number in the first immediate value, writes the task data into a first computable memory space in the computable memory module indicated by the first address information, and sends a work request and the first immediate value to the in-memory computing controller; In response to receiving the work request sent by the RDMA network engine module, the in-memory computing controller issues a computing command indicated by the command code to the computable memory module indicated by the first address information based on the command code in the first immediate value and the first address information indicated by the address index number; In response to receiving the calculation command, the computable memory module performs the calculation corresponding to the calculation command on the task data in the first computable memory space, obtains the calculation result of the task data, writes the calculation result into the second computable memory space corresponding to the second address information indicated by the address index number, and sends a calculation completion notification to the in-memory calculation controller, where the calculation completion notification includes the calculation completion status of the task data.

5. The method according to claim 4, characterized in that The in-memory computing device sends a second RDMA write request to the task processing device, including: In response to receiving a task completion notification sent by the computable memory module, the in-memory calculation controller encodes the address index number and the calculation completion status in the first immediate value into a second immediate value, where the calculation completion status includes calculation success or calculation failure; The in-memory calculation controller generates the second RDMA write request according to the calculation result in the second computable memory space and the second immediate value, and sends the second RDMA write request to the task processing device through the RDMA network engine module.

6. A remote in-memory computing interaction method based on RDMA mechanism, characterized in that: Applied to an in-memory computing device, the method includes: receiving a first RDMA write request sent by a host of a task processing device, wherein the first RDMA write request carries task data to be calculated and a first immediate value, wherein the first immediate value includes an address index number and a command code, wherein the command code is used to indicate a calculation command to be executed, and the address index number is used to indicate first address information in the in-memory computing device for storing the task data, second address information in the in-memory computing device for storing a calculation result of the task data, and third address information in the memory of the host for storing the calculation result of the task data; Based on the address index number in the first immediate value, writing the task data into a first computable memory space corresponding to the first address information indicated by the address index number, and performing calculation on the task data in the first computable memory space according to the calculation command indicated by the command code in the first immediate value to obtain a calculation result of the task data, and writing the calculation result into a second computable memory space corresponding to the second address information indicated by the address index number in the first immediate value; A second RDMA write request is sent to the task processing device, where the second RDMA write request carries the calculation result and a second immediate value, where the second immediate value includes the address index number and the calculation completion status of the task data, so that the host of the task processing device completes the task to be executed according to the calculation result and the second immediate value.

7. The method according to claim 6, characterized in that A memory pooling mapping list is pre-installed in the in-memory computing device, and the memory pooling mapping list includes: a plurality of address index numbers and a set of memory pooling information corresponding to each address index number, the set of memory pooling information including: address information for storing task data in the in-memory computing device, address information for storing calculation results of the task data in the in-memory computing device, and address information for storing calculation results of the task data in the memory of the host; wherein the address information includes the address and size of the memory space; The address index number includes: a list entry identifier of the memory pooling mapping list, and a queue identifier of a work queue on the host or a user identifier on the host.

8. The method according to claim 7, characterized in that The in-memory computing device includes: an RDMA network engine module, an in-memory computing controller, and a plurality of computable memory modules; wherein, based on the address index number in the first immediate number, the task data is written into a first computable memory space corresponding to the first address information indicated by the address index number, and according to the calculation command indicated by the command code in the immediate number, the task data in the first computable memory space is calculated to obtain the calculation result of the task data, and the calculation result is written into a second computable memory space corresponding to the second address information indicated by the address index number in the first immediate number, including: The RDMA network engine module determines, based on the memory pooling mapping list, first address information indicated by the address index number in the first immediate value, writes the task data into a first computable memory space in the computable memory module indicated by the first address information, and sends a work request and the first immediate value to the in-memory computing controller; In response to receiving the work request sent by the RDMA network engine module, the in-memory computing controller issues a computing command indicated by the command code to the computable memory module indicated by the first address information based on the command code in the first immediate value and the first address information indicated by the address index number; In response to receiving the calculation command, the computable memory module performs the calculation corresponding to the calculation command on the task data in the first computable memory space, obtains the calculation result of the task data, writes the calculation result into the second computable memory space corresponding to the second address information indicated by the address index number, and sends a calculation completion notification to the in-memory calculation controller, where the calculation completion notification includes the calculation completion status of the task data.

9. The method according to claim 8, characterized in that The sending a second RDMA write request to the task processing device includes: In response to receiving a task completion notification sent by the computable memory module, the in-memory calculation controller encodes the address index number and the calculation completion status in the first immediate value into a second immediate value, where the calculation completion status includes calculation success or calculation failure; The in-memory calculation controller generates the second RDMA write request according to the calculation result in the second computable memory space and the second immediate value, and sends the second RDMA write request to the task processing device through the RDMA network engine module.

10. A remote in-memory computing interaction method based on RDMA mechanism, characterized in that: Applied to a task processing device, the task processing device includes a host and an RDMA network card, the method includes: The host sends a first RDMA write request to a remote in-memory computing device through the RDMA network card, where the first RDMA write request carries task data to be calculated and a first immediate value, where the first immediate value includes an address index number and a command code, where the command code is used to indicate a calculation command to be executed, and the address index number is used to indicate first address information in the in-memory computing device for storing the task data, second address information in the in-memory computing device for storing a calculation result of the task data, and third address information in the host's memory for storing the calculation result of the task data, so that the in-memory computing device completes the in-memory calculation of the task data based on the first immediate value in the first RDMA write request; In response to receiving the second RDMA write request sent by the in-memory computing device, the RDMA network card writes a calculation result in the second RDMA write request into a host memory space corresponding to third address information indicated by the address index number in the memory of the host based on the address index number in the second immediate value in the second RDMA write request, and sends a work request and the second immediate value to the host; In response to receiving the work request sent by the RDMA network card, the host completes the task to be executed according to the second immediate value and the calculation result in the host memory space.

11. The method according to claim 10, characterized in that The task processing device is pre-installed with a memory pooling mapping list, the memory pooling mapping list including: a plurality of address index numbers and a set of memory pooling information corresponding to each address index number, the set of memory pooling information including: address information for storing task data in the in-memory computing device, address information for storing calculation results of the task data in the in-memory computing device, and address information for storing calculation results of the task data in the memory of the host; wherein the address information includes the address and size of the memory space; The address index number includes: a list entry identifier of the memory pooling mapping list, and a queue identifier of a work queue on the host or a user identifier on the host; the work queue is used to cache work requests.

12. The method according to claim 11, characterized in that The host sends a first RDMA write request to a remote in-memory computing device through the RDMA network card, including: The host allocates an address index number to the task data to be calculated based on the memory pool mapping list, and encodes the allocated address index number and a command code of the calculation command to be executed as the first immediate value; The host generates the first RDMA write request based on the first immediate value and the task data, and sends the first RDMA write request to the in-memory computing device through the RDMA network card.

13. A remote in-memory computing interactive system based on RDMA mechanism, characterized in that: include: A task processing device and an in-memory computing device, wherein the task processing device includes a host and an RDMA network card; The in-memory computing device is used to implement the method described in any one of claims 6 to 9, and the task processing device is used to implement the method described in any one of claims 10 to 12.

Images