A memory object transfer method, node, system, computer program product and storage medium

Abstract

The embodiment of the invention provides a memory object transmission method, a node, a system, a computer program product and a storage medium. Semantics in a communication protocol between a processing unit and a network card device are expanded to optimize interaction logic between the network card device and the processing unit in a memory object transmission process, so that simplified interaction can be performed between the network card device and the processing unit according to new semantics such as a subscription request and a request response; the memory object is transmitted in a subscription mode. Therefore, the transmission problem of the memory object is converted into the state monitoring problem of the memory object, and the monitoring result can be transmitted between the processing unit and the network card equipment through the request response, so that the processing unit and the network card equipment do not need to carry out complex interaction according to traditional basic semantics such as receiving the received / sending the send and the like any more. Therefore, the transmission delay of the memory object can be effectively reduced.

Description

(19) State Intellectual Property Office (12) Invention Patent Application (10) Application Publication Number (43) Application Publication Date (21) Application Number 202411258955.7 (22) Application Date 2024.09.09 (71) Applicant Cloud Intelligent Asset Holdings (Singapore) Pte Ltd Address 51 Bras Basa Road, Singapore, Laizaanda Tower #03-06 (72) Inventors Fang Xing, Yang Hang, Fang Zhiyong (74) Patent Agency Beijing Taihe Jiusi Intellectual Property Agency Co., Ltd. 11610 Patent Attorney Deng Chunyan (51) Int.Cl. G06F 13 / 16 (2006.01) G06F 13 / 42 (2006.01) G06F 15 / 173 (2006.01) H04L 69 / 16 (2022.01) (54) Invention Title: A Method, Node, System, Computer Program Product, and Storage Medium for Transmitting Memory Objects (57) Abstract: This application provides a method, node, system, computer program product, and storage medium for transmitting memory objects. The semantics of the communication protocol between the processing unit and the network interface card (NIC) device are extended to optimize the interaction logic between the NIC device and the processing unit during memory object transmission. Based on this, the NIC device and the processing unit can perform simplified interaction according to new semantics such as subscription requests and request responses, realizing the transmission of memory objects through subscription. Thus, the problem of transmitting memory objects is transformed into a problem of monitoring the state of memory objects. The monitoring results can be transmitted between the processing unit and the NIC device through request responses. Therefore, the processing unit and the NIC device no longer need to perform complex interactions according to traditional basic semantics such as receive / send. Accordingly, the transmission latency of memory objects can be effectively reduced. Claims 3 pages, Description 16 pages, Drawings 6 pages, CN 121636392 A 2026.03.10 CN 1 21 63 63 92 A 1. A method for transmitting a memory object, characterized in that it is applicable to a first network interface card (NIC) device, the first NIC device being mounted on a first node, the first node also being mounted on a first processing unit, the method comprising: sending a subscription request for a target memory object to the first processing unit, the subscription request being used to request monitoring the ready state of the target memory object, the target memory object being any memory object to be transmitted; after receiving a request response from the first processing unit for the subscription request, determining a second node, the request response being issued by the first processing unit after monitoring the ready state of the target memory object as ready; transmitting the target memory object to the second node. 2. The method according to claim 1, characterized in that transmitting the target memory object to the second node comprises:1. The method according to claim 1, wherein sending a subscription request for the target memory object to the first processing unit includes: sending a shared subscription request to the first processing unit for prefetch requests issued by multiple nodes for the target memory object. 2. The method according to claim 3, wherein sending a shared subscription request to the first processing unit for prefetch requests issued by multiple nodes for the target memory object includes: upon receiving a prefetch request issued by any node for the target memory object, detecting whether a subscription request for the target memory object has already been sent to the first processing unit; if not, sending a subscription request for the target memory object to the first processing unit. 3. The method according to claim 4, further comprising: if it is detected that a subscription request for the target memory object has already been sent to the first processing unit, and the target memory object is not yet ready, adding the prefetch request issued by the second node to the wait list maintained for the target memory object. 6. The method according to claim 5, wherein determining the second node comprises: querying the issuing node corresponding to each prefetch request recorded in the waiting list as the second node, wherein the issuing node is the node that issued the prefetch request; wherein the number of the second nodes is one or more. 7. The method according to claim 5, wherein after receiving a prefetch request issued by any node for the target memory object, it further comprises: querying a ready list for recording ready memory objects; if the target memory object is not found in the ready list, then determining that the target memory object is not yet ready; wherein the ready list is maintained by the first processing unit or by the first network interface card device according to the received request response. 8. The method according to claim 5, wherein after receiving a prefetch request issued by any node for the target memory object, it further comprises: if it is determined that the target memory object is ready, then transmitting the target memory object to the second node. 9. The method according to claim 1, wherein sending an object subscription request for the target memory to the first processing unit comprises: after receiving a prefetch request for the target memory object from any node, sending a subscription request for the target memory object to the first processing unit for the node, wherein the prefetch request is used to indicate that the target memory object is obtained from the first node.10. The method according to claim 9, characterized in that, after receiving a prefetch request from any node for the target memory object, sending a subscription request for the target memory object to the first processing unit by the node includes: after receiving a prefetch request from any node for the target memory object, forwarding the prefetch request as the subscription request to the first processing unit. 11. The method according to claim 10, characterized in that, it further includes: receiving a response message from the first processing unit in response to the prefetch request as a response to the request, the response message containing a node identifier corresponding to the node that issued the prefetch request; determining the second node includes: determining the node pointed to by the node identifier contained in the received response message as the second node; wherein, the number of second nodes determined based on a single request response is one. 12. The method according to claim 3 or 9, characterized in that transmitting the target memory object to the second node comprises: transmitting the serialization information corresponding to the target memory object to the second node when the target memory object needs to be serialized before transmission; after determining that the serialization information has been transmitted to the second node, adding the attribute information of the serialized data corresponding to the target memory object and the identification information of the second node to a response message, the response message being used to characterize the completion of the prefetch request response issued by the second node; sending the response message to the second node to trigger the second node to obtain the serialized data and perform deserialization operation on the serialized data based on the serialization information to restore the target memory object. 13. The method according to claim 3 or 9, characterized in that, transmitting the target memory object to the second node includes: transmitting the target memory object to the second node when the target memory object does not need to be serialized and then transmitted; the method further includes: after determining that the target memory object has been transmitted to the second node, sending a response message to the second node, the response message being used to indicate that the prefetch request issued by the second node has been completed. 14. A memory object transmission method, characterized in that, it is applicable to a second network interface card device, the second network interface card device being mounted on a second node, the second node being further mounted with a second processing unit, the method including: receiving a prefetch request issued by the second processing unit, the prefetch request being used to indicate that a target memory object is obtained from a first node; forwarding the prefetch request to the first node to trigger the first node to return the target memory object after listening to the ready state of the target memory object as ready;The received target memory object is written into the memory reserved space set by the second processing unit for the prefetch request. 15. The method according to claim 14, wherein writing the received target memory object into the memory reserved space set by the second processing unit for the prefetch request includes: receiving a response message returned by the first node for the prefetch request and serialization information corresponding to the target memory object; writing the serialization information into the memory reserved space set by the second processing unit for the prefetch request; forwarding the response message to the second processing unit to trigger the second processing unit to continue requesting to obtain the serialization data corresponding to the target memory object according to the attribute information of the serialization data contained in the response message. 16. The method according to claim 15, further comprising: after forwarding the response message to the second processing unit, receiving a read request initiated by the second processing unit for serialized data corresponding to the target memory object; reading back the serialized data from the first node according to the read request; after determining that the serialized data has been read, sending a read response to the read request to the second processing unit to trigger the second processing unit to perform deserialization processing on the serialized data according to the serialization information to restore the target memory object. 17. A memory object transmission system, comprising a first node and a second node, wherein the first node is equipped with a first network device and a first processing unit, and the second node is equipped with a second network device and a second processing unit; the first network device is used to execute the memory object transmission method according to any one of claims 1-13, and the second network device is used to execute the memory object transmission method according to any one of claims 14-16, to transmit a target memory object between the first node and the second node. 18. A node, characterized in that it comprises a processing unit and a network interface card (NIC) device; said NIC device is configured to interact with the processing unit according to the memory object transfer method according to any one of claims 1-16 to perform memory object transfer. 19. A computer-readable storage medium storing a computer program, characterized in that when the computer program is executed by one or more processors, it causes the one or more processors to perform the memory object transfer method according to any one of claims 1-16. 20. A computer program product, characterized in that it comprises a computer program, which, when executed by one or more processors, causes the one or more processors to perform the memory object transfer method according to any one of claims 1-16. Claims 3 / 3 Page 4 CN 121636392 A Memory Object Transfer Method, Node, System, Computer Program Product and Storage Medium Technical Field

[0001] This application relates to the field of cloud computing technology, and in particular to a method, node, system, computer program product, and storage medium for transferring memory objects. Background Art

[0002] Inter-process communication frequently occurs in distributed computing and other technical scenarios. Since different processes run in different address spaces, and different processes cannot directly access each other's address spaces, it is often necessary to transfer memory objects between different processes.

[0003] Among them, the network card device installed on the computing node where the process is located needs to participate in the transfer of memory objects. Moreover, on the computing node where the process is located, the number of interactions required between the CPU and the network card device to realize the transfer of memory objects is considerable, which brings too much disturbance to the CPU, affects CPU performance, and leads to excessively high latency in the transfer of memory objects. Summary of the Invention

[0004] Several aspects of this application provide a method, node, system, computer program product, and storage medium for transferring memory objects, so as to reduce the latency in the transfer of memory objects.

[0005] This application embodiment provides a memory object transmission method, applicable to a first network interface card (NIC) device, the first NIC device being mounted on a first node, and the first node also being mounted on a first processing unit. The method includes:

[0006] sending a subscription request for a target memory object to the first processing unit, the subscription request being used to request monitoring the ready state of the target memory object, the target memory object being any memory object to be transmitted;

[0007] after receiving a request response from the first processing unit for the subscription request, determining a second node, the request response being issued by the first processing unit after monitoring the ready state of the target memory object as ready;

[0008] transmitting the target memory object to the second node.

[0009] This application embodiment also provides a memory object transmission method, applicable to a second network interface card (NIC) device, the second NIC device being mounted on a second node, and the second node also being mounted on a second processing unit. The method includes:

[0010] receiving a prefetch request issued by the second processing unit, the prefetch request being used to indicate obtaining a target memory object from a first node;

[0011] forwarding the prefetch request to the first node to trigger the first node to return the target memory object after detecting that the target memory object is ready;

[0012] writing the received target memory object into the memory reserved space set by the second processing unit for the prefetch request.

[0013] This application embodiment also provides a memory object transmission system, including a first node and a second node, the first node being mounted on a first network device and a first processing unit, and the second node being mounted on a second network device and a second processing unit.Two processing units; the first network device and the second network device are used to execute the aforementioned memory object transfer method (page 1 / 16, CN 121636392 A) to transfer a target memory object between the first node and the second node.

[0014] This application embodiment also provides a node, including a processing unit and a network interface card (NIC) device;

[0015] The NIC device is used to interact with the processing unit according to the aforementioned memory object transfer method to perform memory object transfer.

[0016] This application embodiment also provides a computer-readable storage medium storing a computer program, which, when executed by one or more processors, causes the one or more processors to execute the aforementioned memory object transfer method.

[0017] This application embodiment also provides a computer program product, including a computer program, which, when executed by one or more processors, causes the one or more processors to execute the aforementioned memory object transfer method.

[0018] In this embodiment, the semantics of the communication protocol between the processing unit and the network interface card (NIC) device are extended to optimize the interaction logic between the NIC device and the processing unit during memory object transmission. Based on this, the NIC device and the processing unit can perform simplified interaction according to new semantics such as subscription requests and request responses, realizing the transmission of memory objects through subscription. In this way, the problem of memory object transmission is transformed into the problem of listening to the state of memory objects. The processing unit and the NIC device can transmit the listening results through request responses. Therefore, the processing unit and the NIC device no longer need to perform complex interactions according to traditional basic semantics such as receive / send. Accordingly, on the one hand, this can effectively reduce the number of interactions between the network card device and the processing unit during the transmission of memory objects, thereby reducing the number of interrupt signals received by the processing unit, that is, reducing the disturbance caused to the processing unit, and thus completing the transmission of memory objects more efficiently and effectively reducing the transmission latency of memory objects; on the other hand, based on the behavioral rules indicated by these new semantics, the control plane work such as determining the second node that needs to be responded to during the transmission of memory objects can be offloaded to the network card device, which can effectively reduce the processing pressure of the processing unit, thereby further reducing the transmission latency of memory objects. Brief Description of the Drawings

[0019] The accompanying drawings described herein are provided to further understand the present application and constitute a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation of the present application. In the drawings:

[0020] FIG1 provides an existing memory object transmission scheme;

[0021] FIG2 is a schematic flowchart of a memory object transmission method provided by an exemplary embodiment of the present application;

[0022] FIG3a is an example of the interaction between the network device and the processing unit on the first node provided by an exemplary embodiment of the present application.Figure 3b is a logical schematic diagram of the task processing mechanism on the first processing unit provided in an exemplary embodiment of this application; Figures 4 and 5 are logical schematic diagrams of two optional implementation methods provided in an exemplary embodiment of this application; Figure 6 is a logical schematic diagram of a preferred transmission method provided in an exemplary embodiment of this application; Figure 7 is a flowchart of a memory object transmission method provided in another exemplary embodiment of this application; Figure 8 is a logical schematic diagram of the interaction between the network device and the processing unit on the second node provided in another exemplary embodiment of this application; Figure 9 is a logical schematic diagram of the interaction in the case where serialization is required before transmission provided in another exemplary embodiment of this application; Figure 10 is a structural schematic diagram of a memory transmission system provided in yet another exemplary embodiment of this application; Figure 11 is a structural schematic diagram of a node provided in yet another exemplary embodiment of this application. Specification 2 / 16 pages 6 CN 121636392 A Detailed Implementation

[0031] To make the purpose, technical solution and advantages of this application clearer, the technical solution of this application will be clearly and completely described below in conjunction with specific embodiments of this application and corresponding drawings. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments in this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0032] Before starting to describe the technical solutions provided by each embodiment of this application in detail, the following is a simple explanation of several technical concepts involved in this application.

[0033] Distributed computing can be understood as a computing mode executed on a distributed system. This computing mode divides a computing task into multiple sub-tasks; then, these sub-tasks are assigned to multiple computing nodes for processing; finally, these computing results are combined to obtain the final result, thereby improving computing efficiency and the ability to solve complex problems.

[0034] Inter-process communication can be understood as the process of transmitting or exchanging data between different processes. In distributed computing and other technological scenarios, inter-process communication (IPC) is frequently required. The processes communicating may be distributed across different computing nodes or on the same computing node. Therefore, IPC may be cross-node or non-cross-node. Regardless of the type of IPC, if the processes do not share memory but run in different address spaces, IPC may involve the transfer of memory objects.

[0035] A memory object can be understood as a data structure stored in memory. A memory object can be...Any type of memory content, including but not limited to variables, arrays, structures, class instances, and buffers. In high-performance computing and distributed systems, memory objects typically refer to data that needs to be shared or transmitted between computing nodes.

[0036] During their research, the inventors discovered that the transmission of memory objects mainly relies on the software in the CPU (operating system and network card device driver) and the network card device. The network card device and the CPU interact according to a communication protocol, which refers to the rules and conventions that the two parties must follow to complete communication. The communication protocol may include elements such as semantics, syntax, and timing. Semantics is used to define the interaction method between the two parties, including the format of requests and responses, specific operations, and the purpose of data transmission. Currently, the communication protocol between the network card device and the CPU usually uses traditional basic semantics such as receive / send. For communication tasks such as memory object transmission, these basic semantics need to be combined in multiple rounds to form the complete semantics of memory object transmission. Therefore, the network interface card (NIC) device and the CPU need to interact according to this complex combination semantics in order to complete the transfer of memory objects. This leads to the phenomenon mentioned in the background art: the number of interactions between the NIC device and the CPU during the transfer of memory objects is considerable.

[0037] Figure 1 provides an existing memory object transfer scheme. Referring to Figure 1, the two processes are distributed on different computing nodes. For ease of distinction and description, in Figure 1, the node where the process providing the memory object is located is marked as node 1; the node where the process that needs to obtain the memory object is located is marked as node 2; the NIC device installed on node 1 is marked as NIC device 1, and the NIC device installed on node 2 is marked as NIC device 2.

[0038] Referring to Figure 1, taking the interaction between the CPU and the NIC device according to the two basic semantics of receive and send as an example, the CPUs on both nodes need to first send a receive request to the associated NIC device to prepare for receiving. Subsequently, the CPU on node 2 can send a send request to network interface card (NIC) device 2. This send request carries node 2's request to retrieve memory objects. NIC device 2 forwards the send request to NIC device 1, which then passes it to the CPU of node 1. The CPU of node 1 needs to send a completion notification (send ACK) in response to this send request. This send ACK is forwarded by NIC device 1 and NIC device 2 before reaching the CPU of node 2. Afterward, the CPU of node 2 continues to send a completion notification (receive ACK) in response to this send ACK.The ACK continues to be forwarded by network interface card (NIC) device 2 and NIC device 1, and is then transmitted to the CPU of node 1. Thus, after multiple rounds of interaction, node 2's request to acquire the memory object is finally transmitted to the CPU of node 1, completing the transmission phase of the acquisition request.

[0039] After learning of this request, the CPU of node 1 waits until the memory object required by node 2 is ready, and then sends a send request to NIC device 1. Referring to Figure 1, the CPUs and NIC devices on both nodes need to go through several rounds of requests and completion notifications based on receive and send semantics before the memory object can be transmitted to node 2.

[0040] Referring to Figure 1, it can be clearly understood that during the memory object return phase, the CPUs and NIC devices on both nodes need to perform complex interactions according to these basic semantics, and the number of interactions may be as many as dozens. During the research process, the inventors discovered that the interaction between the network card device and the CPU also relies on the interrupt mechanism: after receiving a network signal (such as a request or response sent by the peer network card device in Figure 1), the network card device will actively send an interrupt signal to the CPU. After receiving the interrupt signal, the CPU needs to immediately stop the ongoing calculation in order to analyze and process the interrupt signal. The interrupt mechanism can ensure effective communication between the network card device and the CPU, and ensure that the computing node can respond to and process network events in a timely manner when communicating on the network. Referring to Figure 1, during the transfer of memory objects, due to the large number of interactions between the network card device and the CPU, the number of times the network card device sends interrupt signals to the CPU is also considerable, causing the CPU to frequently process interrupt signals. This brings too much disturbance to the CPU and affects the CPU's processing performance. This performance impact leads to a long transfer latency of memory objects.

[0041] In order to improve the transfer latency of memory objects, after a series of studies, the inventors proposed a new memory object transfer scheme in this application. In the memory object transfer scheme provided in this application, the semantics in the aforementioned communication protocol are extended, and a new semantics for memory object transfer is designed. Therefore, this application can modify the program code within the network card device and modify the network card device driver. After these modifications, the network card device can send fewer requests to the processing unit and proactively undertake some control work according to the redesigned execution logic in the program code; the processing unit can then call some new functions provided in the network card device driver of this application (such as functions for issuing subscription requests) when using the network card device driver, thereby enabling the processing unit to understand the requests sent by the network card device according to the technical concept of this embodiment.

[0042] Considering that memory object transmission usually involves two ends: the requesting end and the responding end, the requesting end is...The process requesting the memory object is located at one end, and the responding end is located at the other end, which is the process providing the memory object. For ease of distinction and description, in various embodiments of this application, the responding end involved in the memory object transmission process is described as the first node, and the requesting end is described as the second node. It is worth noting that for a computing node, it can sometimes act as the requesting end and sometimes as the responding end. In addition, as mentioned above, inter-process communication may be cross-node or non-cross-node. Therefore, in the case of cross-node communication, the first node and the second node in this application can be used to refer to two different computing nodes, while in the case of non-cross-node communication, the first node and the second node in this application can be used to refer to the same computing node.

[0043] The network card device in various embodiments of this application can be any device with network communication and computing capabilities, such as some cloud infrastructure processing units (CIPU), smart network cards (Smart NIC), and network interface cards (NIC) chips or modules that integrate network card modules.

[0044] The processing unit in each embodiment of this application can be a device on a computing node that has computing capabilities and needs to manage memory objects, such as a central processing unit (CPU), a data processing unit (DPU), and a graphics processing unit (GPU) chip or a module as described on page 4 / 16 of the specification, CN 121636392 A.

[0045] The technical solutions provided by each embodiment of this application will be described in detail below with reference to the accompanying drawings.

[0046] Figure 2 is a schematic flowchart of a memory object transfer method provided by an exemplary embodiment of this application. This method can be implemented by a first network card device mounted on the aforementioned first node (i.e., the responding end), and a first processing unit is also mounted on the first node. Referring to Figure 2, the method may include:

[0047] Step 200: Sending a subscription request for a target memory object to a first processing unit, the subscription request being used to request monitoring the ready state of the target memory object, the target memory object being any memory object that needs to be transmitted;

[0048] Step 201: After receiving a request response from the first processing unit for the subscription request, determining a second node, the request response being issued by the first processing unit after monitoring that the ready state of the target memory object is ready;

[0049] Step 202: Transmitting the target memory object to the second node.

[0050] As mentioned above, in this embodiment, the semantics in the communication protocol between the network card device and the processor are...The extended semantics include: subscription request and subscription response. The subscription request is used to request to listen to the ready state of a memory object, and the response indicates that the memory object is ready.

[0051] In this embodiment, for ease of description, the technical solution will be explained using a target memory object as an example. It should be understood that the target memory object can be any memory object that needs to be transmitted.

[0052] Figure 3a is a schematic diagram of the interaction logic between a network device and a processing unit on a first node provided in an exemplary embodiment of this application. Referring to Figures 2 and 3a, in step 200, the first network card device can send a subscription request for the target memory object to the first processing unit. In this embodiment, the original communication channel between the first network card device and the first processing unit is not modified. The original communication channel between them can be a bus (Peripheral Component Interconnect Express, PCIE) or other channels, which are not limited here, nor are further examples provided. In step 200, the first network card device can send the subscription request to the first processing unit through the original communication channel between it and the first processing unit.

[0053] Referring to FIG3a, after translation by the modified network card device driver in this embodiment, the subscription request can be transformed into a work task that the first processing unit can understand and process. The first processing unit can start a worker thread for the work task. The object that the worker thread is concerned with is the target memory object. Since the memory object needs to be transmitted in the memory object transmission scenario, in this embodiment, the ready event that the worker thread is concerned with can be set as a readable event. It is worth noting that in this embodiment, the main function of the transformed work task is to add the target memory object to the event listening range of the first processing unit. In practical applications, it is not even necessary to describe the relevant information of the requesting end in the work task. It is sufficient to ensure that after the target memory object is detected to be readable, the worker thread corresponding to the work task can call back the specified function in the network card device driver in this embodiment.

[0054] FIG3b is a logical schematic diagram of the work task processing mechanism on the first processing unit provided by an exemplary embodiment of this application. Referring to FIG3b, optionally, for the first processing unit, the worker thread corresponding to the subscription request can be added to a queue pair (QP). A queue pair can contain two queues: one queue records worker threads awaiting processing, and the other queue records the completion status of events on the objects that these worker threads are interested in. Furthermore, the first processing unit typically has an event listening mechanism, such as select, poll, or epoll. According to the event listening mechanism, the first...The first processing unit can monitor whether a ready event has occurred on the object it is monitoring. If a ready event is detected on a monitored object, the first processing unit can wake up the worker thread that is monitoring that object.

[0055] Following the event monitoring mechanism illustrated in Figure 3b, in this embodiment, for the first processing unit, the object monitored by the worker thread corresponding to the subscription request in QP is the target memory object; the ready event monitored on the target object is the readable event. Based on this, the first processing unit can monitor whether a readable event has occurred on the target memory object; after monitoring whether a readable event has occurred on the target memory object, the first processing unit can wake up the worker thread corresponding to the subscription request.

[0056] It should be understood that the task processing mechanism in the first processing unit shown in Figure 3b is only exemplary. In actual applications, the task processing mechanism in the first processing unit is usually designed based on the principles of multiple aspects such as the network protocol (TCP / IP protocol or RDMA protocol, etc.), memory semantics, or event listening mechanism. This embodiment does not limit, interfere with, or provide further examples in this regard.

[0057] It can be seen that in this embodiment, after the subscription request is sent to the first processing unit in step 201, the first processing unit can be triggered to listen to the ready state of the target memory object. That is, the first processing unit is triggered to listen to whether the target memory object has been written into the memory space managed by the first processing unit.

[0058] Regardless of the event listening mechanism adopted by the first processing unit, after the first processing unit listens that the target memory object is ready (i.e., readable), the worker thread corresponding to the subscription request will be awakened and start working. The worker thread will then make a function call in the network card device driver (i.e., the callback mentioned above). Based on the design of the network interface card (NIC) device driver in this embodiment, the working process can call a specified function in the NIC device driver, and through this specified function, a request response to the subscription request can be sent to the first NIC device.

[0059] In this way, the first NIC device can receive the request response sent by the first processing unit for the subscription request. It should be understood that, as analyzed above, the request response is sent by the first processing unit only after it has detected that the target memory object is ready. Therefore, in this embodiment, when the first NIC device receives the request response to the subscription request, it can indicate that the target memory object is ready.

[0060] Based on the design of the program code in the NIC device in this embodiment, processing logic for the request response has been added to the first NIC device. Referring again to Figures 2 and 3, in step 201, after receiving the request response to the subscription request, the first NIC device can determine the second node that needs to be responded to, that is, determine which / which nodes the target memory object needs to be transmitted to.

[0061] Here, comparing the traditional memory object transmission scheme in Figure 1, it can be seen that in the traditional scheme, the first processing unit needs to initiate the send operation to control the timing and destination of memory object transmission. However, in this embodiment, this traditional concept is changed, and the work of this control plane is offloaded to the first network card device. The first network card device is responsible for interfacing with or controlling the network's demand for acquiring the target memory object. This can effectively reduce the processing pressure of the first processing unit. The first processing unit can be responsible for simple event listening work. Therefore, the number of interactions caused by the control plane work can be eliminated.

[0062] Continuing to refer to Figures 2 and 3a, in step 202, after the second node is determined, the target memory object can be transmitted to the second node.

[0063] In this embodiment, the network protocol used by the first network card device to transmit the target memory object to the second node is not modified. The original network protocol used by the first network card device can be used, such as TCP / IP protocol or RDMA protocol. In the case of using the RDMA protocol, the first network card device can cooperate with the network card device on the second node to write relevant data into the memory of the second node without the need for the processing unit on the second node to participate, thus achieving more efficient transmission. Here, the network protocol used by the first network card device when transmitting the target memory object is not limited, not interfered with, or further examples are given.

[0064] Thus, based on the memory object transmission scheme provided in this embodiment, a subscription-based memory object transmission mechanism can be realized, supporting the initiation of a subscription request before the memory object is ready. Once the memory object is ready, the transmission of the memory object can be automatically started, which can effectively reduce the transmission latency of the memory object.

[0065] In summary, in this embodiment, the semantics of the communication protocol between the processing unit and the network card device have been extended. The interaction logic between the network card device and the processing unit during memory object transmission is optimized according to CN 121636392 A on page 6 / 16 of the specification. Based on this, the network card device and the processing unit can perform simplified interaction according to the new semantics such as subscription requests and request responses, realizing the transmission of memory objects through subscription. In this way, the problem of transferring memory objects is transformed into a problem of monitoring the status of memory objects. The processing unit and the network interface card (NIC) can exchange monitoring results through request and response. Therefore, the processing unit and the NIC no longer need to perform complex interactions according to traditional basic semantics such as receive / send. Accordingly, on the one hand, this can effectively reduce the number of interactions between the NIC and the processing unit during the transfer of memory objects, thereby reducing the number of interrupt signals received by the processing unit, that is, reducing the disturbance to the processing unit, and thus completing the transfer of memory objects more efficiently, effectively reducing the transfer latency of memory objects; on the other hand, based on the behavioral rules indicated by these new semantics, the internal...The control plane work, such as determining the second node that needs to respond, involved in the transmission of memory objects is offloaded to the network card device. This can effectively reduce the processing pressure of the processing unit, thereby further reducing the transmission latency of memory objects.

[0066] In the above or below embodiments, various implementation methods can be used to implement the aforementioned step 200. Two optional implementation methods are provided below as examples. Figures 4 and 5 are logical schematic diagrams of two optional implementation methods provided by an exemplary embodiment of this application.

[0067] Referring to Figure 4, in the first optional implementation method:

[0068] After receiving a prefetch request issued by any node for the target memory object, the first network card device can send a subscription request for the target memory object to the first processing unit for the node. The prefetch request is used to indicate that the target memory object is obtained from the first node.

[0069] The prefetch request is also a new semantic extended in this embodiment. The first network card device can understand the prefetch request. Here, the specific logic of how the node issues the prefetch request is not limited, as long as the prefetch request issued by the node conforms to the new semantic designed in this embodiment. In this embodiment, the parameters carried in the prefetch request may include, but are not limited to:

[0070] 1. Local information and peer information

[0071] Peer information, used by the local network card device to route the prefetch request to the peer (i.e., the first node in this embodiment);

[0072] Local information, used by the peer network card device to route when returning the target memory object;

[0073] Key value (KEY), used to indicate the target memory object to be obtained;

[0074] Access permission characterization (RKEY): used to indicate the access permission of the memory object;

[0075] 2. Information on local reserved space:

[0076] Local address;

[0077] Local address length;

[0078] Local address access permission (lkey).

[0079] It should be understood that the above parameters are only exemplary, and other parameters may be carried in the prefetch request. No further examples are given here. In practical applications, the parameters carried in the prefetch request can be flexibly designed according to the requirements.

[0080] Thus, in this optional implementation, for each prefetch request received by the first network interface card (NIC) device, a dedicated subscription request will be sent to the first processing unit. That is, there is a one-to-one relationship between the prefetch request and the subscription request. For example, the first NIC device can directly forward the received prefetch request as a subscription request to the first processing unit. It is worth emphasizing that, based on the design of the NIC device driver in this embodiment, although the first NIC device directly forwards the received prefetch request to the first processing unit, the main function of the resulting task after translation by the NIC device driver is still to add the target memory object to the event listening range of the first processing unit. Therefore, for page 7 / 16 of the specification, 11 CNFor the first processing unit (121636392 A), it can still monitor the ready status of the target memory object in a timely manner. The node identification information carried in the prefetch request (such as the local information in the previous example) can be retained in the work task converted from the subscription request, so that the first processing unit can distinguish the node pointed to by different subscription requests.

[0081] As mentioned above, the first network interface card (NIC) device can receive the request response issued by the first processing unit for the subscription request. Referring to Figure 4, in this optional implementation: the first NIC device sends a subscription request to the first processing unit for each prefetch request; therefore, the first NIC device can receive the request responses returned by the first processing unit for different subscription requests. Based on the design of the NIC device driver in this embodiment, in this optional implementation, the request response issued by the first processing unit can also carry node identification information to distinguish the pointed-to node. In this way, the request response can point to a specific node.

[0082] For example, when the first network interface card (NIC) device directly forwards the received prefetch request to the first processing unit, the first NIC device can receive a response message sent by the first processing unit in response to the prefetch request as a request response. As mentioned above, the local information carried in the prefetch request can be retained in the work task converted for the prefetch request. Based on this, the first processing unit can also carry the corresponding local information (e.g., node identifier, etc.) in the response message sent in response to the prefetch request to indicate the node pointed to by different response messages.

[0083] Accordingly, in this optional implementation, referring to FIG4, after receiving a prefetch request, the first NIC device sends a subscription request to the first processing unit and can receive a request response returned by the first processing unit in response to the prefetch request. That is, for a prefetch request, the first NIC device and the first processing unit can complete one memory object transfer by having one interaction (i.e., sending a subscription request and receiving a request response). Comparing Figure 1, it is clearly perceptible that the number of interactions between the first network interface card (NIC) device and the first processing unit is significantly reduced during a single memory object transfer.

[0084] Referring to Figure 5, in the second optional implementation:

[0085] The first NIC device can send a shared subscription request to the first processing unit for prefetch requests issued by multiple nodes for the target memory object.

[0086] The difference between the second optional implementation and the aforementioned first optional implementation is that it is no longer necessary to send a dedicated subscription request to the first processing unit for each prefetch request, but rather to send a shared subscription request to the first processing unit for multiple prefetch requests for the same memory object. That is, there is a many-to-one relationship between prefetch requests and subscription requests.

[0087] For example, if there are 5 nodes that have sent prefetch requests for the target memory object to the first NIC device,According to the first optional implementation method mentioned above, the first network interface card (NIC) device can send 5 subscription requests to the first processing unit for these 5 prefetch requests; while according to the second optional implementation method, the first NIC device only needs to send 1 subscription request to the first processing unit for these 5 prefetch requests.

[0088] It can be seen that in the second optional implementation method, the prefetch requests for the same memory object are aggregated. Compared with the first optional implementation method, the number of subscription requests sent by the first NIC device to the first processing unit can be further reduced. Correspondingly, the number of request responses that the first processing unit needs to return to the first NIC device is also reduced. Therefore, this can further reduce the number of interactions between the two.

[0089] In the second optional implementation method, the timing of sending the subscription request is not limited. In an exemplary design:

[0090] After receiving a prefetch request from any node for a target memory object, the first network interface card (NIC) device can detect whether a subscription request for the target memory object has already been sent to the first processing unit; if not, it sends the subscription request for the target memory object to the first processing unit. Correspondingly, if it has already been sent, it no longer needs to send a subscription request.

[0091] Since the subscription requests are all issued by the first NIC device, it can detect without difficulty whether a subscription request has been sent for a certain memory object. The engineering implementation method supporting detection is not limited here; for example, a record table can be maintained, and detection can be performed by looking up the table, etc., but no further examples are given here.

[0092] It should be understood that the above-mentioned sending timing is only exemplary and is not limited to this in the second optional implementation. For example, in some design schemes, the received prefetch requests can be counted for the target memory object. After the number of prefetch requests reaches a preset threshold, a subscription request for the target memory object can be sent to the first processing unit. No further examples of sending timing are given here.

[0093] Considering that in this second optional implementation, the prefetch requests received by the first network card device may not be processed in a timely manner as in the first optional implementation, referring to FIG5, the second optional implementation also proposes:

[0094] After receiving a prefetch request sent by any node for the target memory object, if it is detected that a subscription request for the target memory object has been sent to the first processing unit and the target memory object is not yet ready, the prefetch request sent by the node is added to the wait list maintained for the target memory object.

[0095] In practical applications, the first network card device can maintain and store the wait list. This wait list can be in key-value format, where the key refers to a memory object, and the value records the pending items.Requests are retrieved. Additionally, separate wait lists can be maintained for different memory objects. Alternatively, multiple memory objects can be recorded in a single wait list, which is not limited here.

[0096] Thus, the wait list records the subscription relationship between nodes and memory objects, allowing the first network interface card (NIC) device to centrally manage multiple prefetch requests associated with the same memory object based on the wait list.

[0097] Here, during the maintenance of the wait list, it is also mentioned that an operation to determine whether the target memory object is ready has been added to the first NIC device. To support this operation, it is proposed that a ready list for recording ready memory objects can be maintained by the first processing unit or the first NIC device. If maintained by the first processing unit, based on the design of the NIC device driver in this embodiment, the first processing unit adds the target memory object to the ready list when it issues a request response to the subscription request. If maintained by the first NIC device, based on the design of the program code within the NIC device in this embodiment, the first NIC device can add the target memory object to the ready list after receiving a request response from the first processing unit for the subscription request.

[0098] Based on this, the first network interface card (NIC) device can query the ready list; if the target memory object is not found in the ready list, it is determined that the target memory object is not yet ready; if it is found, it is determined that the target memory object is ready. Wherein, if the ready list is maintained by the first processing unit, the first processing unit can store the ready list in the memory space it manages. Of course, it can also be stored in other storage locations. The first NIC device can query the ready list by initiating a memory access request to the first processing unit, etc., which will not be further explained here.

[0099] In this second implementation, there is also a situation where, after receiving a prefetch request from any node for the target memory object, the first NIC device detects that a subscription request has already been sent for the target memory object, and finds that the target memory request is ready. In this situation, the first network interface card (NIC) device can directly transmit the target memory object to the node without adding the prefetch request issued by the node to the aforementioned waiting list. This reduces the response latency to the prefetch request in such cases, which in turn reduces the transmission latency of the corresponding memory object.

[0100] It is understood that the implementation details in the above two optional implementation methods are merely exemplary and are not limited to the implementation details described on page 9 / 16 of the specification, CN 121636392 A. For example, in the first optional implementation method, the first NIC device can construct a subscription request for each prefetch request instead of directly using the prefetch request as a subscription request, etc., and no further examples will be given here. In addition, the above two optional implementation methods are merely exemplary, and other implementation methods can also be used in this embodiment to implement the above.Step 200 described above will not be further illustrated here.

[0101] In summary, in this embodiment, the requirement to acquire a memory object can be conveyed through a prefetch request, and the first network interface card (NIC) device can process the prefetch request individually or aggregate it. One interaction between the first NIC device and the first processing unit can resolve one prefetch request, or even multiple prefetch requests can be resolved in one interaction. Therefore, the number of interactions between the first NIC device and the first processing unit can be effectively reduced, and the transmission latency of the memory object can be reduced.

[0102] In the above or following embodiments, as mentioned above, based on the design of the program code in the NIC device in this embodiment, processing logic for request response has been added to the first NIC device: after receiving the request response, the first NIC device can determine the second node that needs to be responded to.

[0103] In this embodiment, the implementation method for determining the second node can be adaptively designed according to the implementation method used to send the subscription request for the target memory object to the first processing unit. The following describes the process of determining the second node in the first NIC device by following the two implementation methods provided for step 200 in the aforementioned embodiments.

[0104] Following the first optional implementation of step 200 in the foregoing embodiments, the first network interface card (NIC) device can determine the node pointed to by the received request response as the second node; wherein, the number of second nodes determined based on a single request response is one. As mentioned above, the first processing unit carries the aforementioned local information in the request response to point to the corresponding node. Based on this, the first NIC device can parse the local information carried in the request response to determine the node pointed to by the request response.

[0105] Following the second optional implementation of step 200 in the foregoing embodiments, since the subscription request does not point to a specific node, the request response issued by the first processing unit for the subscription request will not point to a specific node. After receiving the request response, the first NIC device can query the issuing node corresponding to each prefetch request recorded in the waiting list corresponding to the target memory object as the second node. Since there may be multiple prefetch requests recorded in the waiting list corresponding to a single memory object, the second node determined by the first NIC device after receiving a request response may be multiple. After identifying multiple second nodes, the first network interface card (NIC) device can centrally respond to the prefetch requests for the target memory object issued by the multiple second nodes. This echoes the earlier statement that the first NIC device can centrally manage multiple prefetch requests.

[0106] In summary, this embodiment provides optional implementation methods for the process of the first NIC device sending a subscription request to the first processing unit. The number of subscription requests sent in these optional implementation methods may be different, and the timing of initiating the subscription request may be different. However, all of them can ensure that the first processing unit is promptly triggered to access the target memory object by sending the subscription request.The object performs state monitoring. In this embodiment, optional implementation methods are also provided for the first network card device to determine the operation of the second node. Based on these optional implementation methods, after receiving the request response from the first processing unit, the first network card device can independently, accurately and comprehensively determine which / which nodes the memory object corresponding to the request response should be transmitted to, without relying on the first processing unit for control, and without needing to send more interactions to the first processing unit for this purpose, thereby effectively ensuring the accuracy and timeliness of memory object transmission.

[0107] In the above or the following embodiments, the first network card device can adopt a variety of implementation methods to transmit the target memory object to the second node. Figure 6 is a logical schematic diagram of a preferred transmission method provided by an exemplary embodiment of this application. Referring to Figure 6, in this preferred transmission method:

[0108] The first network card device can read the target memory object from the memory space managed by the first processing unit; if the number of nodes is multiple, the read target memory object can be transmitted to multiple second nodes respectively. In practical applications, the first network interface card (NIC) device can perform a copy operation on the read target memory object to obtain multiple data copies; and distribute the multiple data copies to multiple second nodes.

[0109] As mentioned in the previous embodiments, after receiving a request response, the first NIC device may determine one or more second nodes. If there are multiple second nodes, the first NIC device can centrally respond to the prefetch requests issued by these second nodes for the target memory object. In this preferred transmission method, the first NIC device can perform a memory access operation to read the target memory object from the memory space managed by the first processing unit. Based on this, the first NIC device can perform a copy operation to obtain multiple data copies corresponding to the target memory object and distribute them to each second node. It can be seen that the first NIC device does not need to perform a memory access operation for each second node separately, which can effectively save the input / output bandwidth (I / O bandwidth) consumed by the first NIC device.

[0110] There is a case where the first NIC device determines one second node after receiving a request response. In this case, after receiving a request response corresponding to the target memory object, the first network interface card (NIC) device records the identified second node, but temporarily does not transmit the target memory object to the second node; the second NIC device can start the operation of transmitting the target memory object after the number of second nodes recorded for the target memory object accumulates to a specified number, or the accumulation time reaches a specified time. After such accumulation, the first NIC device can generate target memory objects to multiple second nodes according to the preferred transmission method provided in this embodiment.

[0111] The inventors discovered during the research process that in the memory space managed by the first processing unit, memory objects mayIt may be necessary to serialize before transmission, or it may be possible to transmit directly. Serialization is the process of converting an object into transmittable byte information. Deserialization is the process of restoring byte information back into an object. In this embodiment, the first network interface card device provides transmission support schemes for these two situations.

[0112] For the case where serialization is required before transmission

[0113] The first network interface card device can transmit the serialized information corresponding to the target memory object to the second node;

[0114] After determining that the serialized information has been transmitted to the second node, a response message for the prefetch request is sent to the second node. The response message carries the attribute information of the serialized data corresponding to the target memory object;

[0115] The response message is sent to the second node to trigger the second node to obtain the serialized data and restore the target memory object through deserialization.

[0116] For cases where serialization is not required before transmission

[0117] the first network interface card (NIC) device can transmit the target memory object itself to the second node;

[0118] after determining that the target memory object has been transmitted to the second node, the NIC device sends a response message for the prefetch request to the second node;

[0119] the response message is sent to the second node.

[0120] Referring to Figures 4 and 5, in either case, the first NIC device can send a response message for each prefetch request to indicate that the prefetch request response is complete. In this embodiment, the response message can be generated in various ways. Referring to the first optional implementation method provided for step 200 in the previous embodiment, for the prefetch request sent by the second node for the target memory object, the first NIC device has received a request response from the first processing device that can point to a specific node. In this regard, the first NIC device can forward the request response as a response message to the second node. Referring to the second optional implementation method provided for step 200 in the aforementioned embodiment, the request response returned by the first processing device does not indicate the node. In this regard, the first network card device can construct response messages for different prefetch requests issued by the second node for the target memory object and send them to the second node. The method of generating the response message is not limited to the specification page 11 / 16 15 CN 121636392 A, nor are further examples provided.

[0121] In addition, in this embodiment, the parameters carried in the response message may include, but are not limited to:

[0122] 1. Whether the target memory object is directly copied:

[0123] Yes, it means that the target memory object does not need to be serialized before transmission;

[0124] No, it means that the target memory object needs to be serialized before transmission;

[0125] 2. A custom parameter return value, which can be used to return to the upper-level caller who needs to obtain the target memory object, such as a process, and can be used for callback purposes.

[0126] It should be understood that the above parameters are exemplified, and other parameters may also be carried in the response message, which will not be further exemplified here. In practical applications, the parameters carried in the response message can be flexibly designed according to the requirements.

[0127] In summary, in this embodiment, multiple prefetch requests issued by multiple second nodes for the same memory object can be centrally responded to without performing multiple memory access operations for multiple prefetch requests. Instead, a single memory access operation is performed for multiple prefetch requests, and multiple copies of the data are automatically copied and distributed to multiple second nodes. This mechanism of solving multiple prefetch requests through a single memory access operation can effectively reduce the I / O bandwidth occupied by the first network card device, thereby avoiding the transmission latency of memory objects due to insufficient I / O bandwidth. Moreover, the first network card device can also send a response message to the second node to indicate that the prefetch request has been responded to. By carrying relevant parameters in the response message, it can indicate whether the target memory object is transmitted after serialization, thereby triggering the second node to continue to obtain the serialized data of the target memory object as needed. This enables the first network card device to support the transmission of various target memory objects.

[0128] FIG7 is a flowchart illustrating a memory object transfer method provided in another exemplary embodiment of this application. This method can be implemented by a second network interface card (NIC) device mounted on the aforementioned second node (i.e., the requesting end), and the second node is also equipped with a second processing unit. Referring to FIG7, the method may include:

[0129] Step 700: Receiving a prefetch request issued by the second processing unit, the prefetch request indicating the acquisition of a target memory object from the first node;

[0130] Step 701: Forwarding the prefetch request to the first node to trigger the first node to return the target memory object after detecting that it is ready;

[0131] Step 702: Writing the received target memory object into the memory reserved space set by the second processing unit for the subscription request.

[0132] FIG8 is a schematic diagram illustrating the interaction logic between the network device and the processing unit on the second node provided in another exemplary embodiment of this application.

[0133] Referring to FIGS. 7 and 8, when any process on the second node needs to acquire a target memory object, the second processing unit may issue a prefetch request for the target memory object. Here, the prefetch request conforms to the new semantics extended for the communication protocol between the network interface card device and the processing unit in this embodiment. Regarding the parameters that can be carried in the prefetch request, please refer to the relevant descriptions above, which will not be repeated here.

[0134] In this embodiment, the memory object acquisition requirement can be carried through the prefetch request. Comparing with Figure 1, it can be seen that in this embodiment, it is no longer necessary to perform multiple rounds of interaction through send semantics to realize the transmission of the memory object acquisition requirement; instead, the transmission of the memory object acquisition requirement can be realized through a single prefetch request. This can effectively reduce the number of interactions required between the second network interface card device and the second processing unit in the memory object acquisition requirement transmission process.

[0135] Referring again to Figures 7 and 8, in step 701, the second network interface card (NIC) device can forward the prefetch request issued by the second processing unit to the first node. Specifically, the second NIC device can forward the prefetch request to the first NIC device mounted on the first node. After the prefetch request is passed to the first NIC device, the interaction logic between the first NIC device and the first processing unit can be referred to page 12 / 16 of the specification of the first node in the aforementioned embodiment, which will not be repeated here. Referring to the aforementioned embodiment, after the prefetch request is passed to the first NIC device, the first node can be triggered to return the target memory object after listening to the target memory object being ready.

[0136] It can be seen that in this embodiment, for the second NIC device, after sending the prefetch request to the first node, there is no need for further interaction with the second processing unit and the first node, and the first node can automatically return the target memory object.

[0137] Referring to FIG7, in step 702, the first network interface card device can write the received target memory object into the memory reserved space set by the second processing unit for the subscription request.

[0138] In this embodiment, when the second network interface card device initiates the prefetch request, it sets up memory reserved space for the prefetch request to store the data returned by the first node for the prefetch request. As mentioned above, the target memory object may need to be serialized before transmission, or it may not need to be serialized before transmission. Therefore, in this embodiment, the second network interface card device can provide transmission support schemes for these two types of requests.

[0139] For the case where it does not need to be serialized before transmission

[0140] The second network interface card device can receive the response message and the target memory object returned by the first node for the prefetch request;

[0141] It will write the target memory object into the memory reserved space set by the second processing unit for the corresponding subscription request.

[0142] In this case, after the writing is completed, the transmission process for the target memory object is completed.

[0143] For cases where serialization is required before transmission

[0144] the second network interface card (NIC) device can receive the response message returned by the first node in response to the prefetch request and the serialization information corresponding to the target memory object;

[0145] the serialization information corresponding to the target memory object received from the first node is written into the memory reserved space set by the second processing unit for the corresponding subscription request;

[0146] the response message is forwarded to the second processing unit to trigger the second processing unit to continue requesting to obtain the serialized data corresponding to the target memory object.

[0147] As described above regarding the first node, under this request, the response message received from the first node will contain parameters such as "whether the target memory object is directly copied". Based on the design of the NIC device driver in this embodiment, the second NIC device can receive the response message returned by the first node in response to the prefetch request and the serialization information corresponding to the target memory object.Upon receiving such a response message, the second processing unit can determine that the target memory object is serialized before transmission through the parameters in the response message. Accordingly, based on the design of the network card device driver in this embodiment, the second processing unit will continue to make function calls to continue requesting the serialized data corresponding to the target memory object.

[0148] Figure 9 is a schematic diagram of the interaction logic provided by another exemplary embodiment of this application in the case where serialization is required before transmission. Referring to Figure 9, in this exemplary scheme:

[0149] After forwarding the response message to the second processing unit, the second network card device can receive a read request initiated by the second processing unit for the serialized data corresponding to the target memory object;

[0150] According to the read request, the serialized data is read back from the first node;

[0151] After determining that the serialized data has been read, a read response for the read request is sent to the second processing unit to trigger the second processing unit to perform deserialization processing on the serialized data according to the serialization information to restore the target memory object.

[0152] Wherein, the necessary information required to be included in the read request can be obtained from the aforementioned serialization information. These necessary information may include, but are not limited to, the address and length of the serialized data.

[0153] In this embodiment, the reading method used by the second network card device when executing the read request is not limited. Preferably, the second network card device can execute the read request according to the Remote Direct Memory Access (RDMA) protocol. Correspondingly, when the second processing unit issues a read request, it can initiate an RDMA request according to the RDMA protocol, as described on pages 13 / 16 of the specification in this embodiment, CN 121636392 A.

[0154] Referring to FIG9, the second network card device can read back the serialized data corresponding to the target memory object from the first node according to the read request, and after reading, send a read response to the second processing unit to indicate that the read request response is completed. Based on the design of the network card device driver in this embodiment, the read response may contain parameters used to characterize the need for deserialization operation. After parsing such parameters from the read response, the second processing unit can perform deserialization operation by calling the corresponding function in the network card device driver to obtain the target memory object.

[0155] Accordingly, in this embodiment, when a memory object needs to be obtained, during the request transmission phase, the network interface card (NIC) device and the processing unit on the local node no longer need to perform multiple rounds of interaction according to the traditional basic semantics such as send and receive. Instead, the request for obtaining the memory object is carried through the extended prefetch request semantics in this embodiment. In this way, the transmission of this request is achieved through a single unidirectional transmission chain, effectively reducing the number of interactions between the NIC device and the processing unit during this phase. During the memory object return phase, there is also no interaction between the NIC device and the processing unit.Instead of using traditional basic semantics like send and receive for control plane interaction, the local network card device can automatically perform data plane transmission with the peer network card device based on the prefetch request triggered by the peer node in the previous stage. In the control plane, the local network card only needs to forward the response message for the prefetch request to the processing unit in a timely manner so that the processing unit can know the execution status of the prefetch request in a timely manner. In this stage, the number of interactions between the network card device and the processing unit is further reduced.

[0156] It is worth noting that the technical details and conceptual explanations of the above embodiments of the memory object transmission method on the second node side can be referred to the relevant descriptions of the above embodiments on the first node side as needed. To save space, they will not be repeated here, but this should not cause any loss of the scope of protection of this application.

[0157] In addition, some processes described in the above embodiments and figures include multiple operations that appear in a specific order. However, it should be clearly understood that these operations may not be executed in the order they appear in this document or may be executed in parallel. The sequence numbers of the operations, such as 201, 202, etc., are only used to distinguish different operations, and the sequence numbers themselves do not represent any execution order. In addition, these processes may include more or fewer operations, and these operations may be executed sequentially or in parallel. It should be noted that the descriptions such as "first" and "second" in this document are used to distinguish different nodes, network card devices, and processing units, etc., and do not represent the order of priority, nor are they limited to "first" and "second" being different types.

[0158] Figure 10 is a schematic diagram of the structure of a memory transfer system provided in another exemplary embodiment of this application. Referring to Figure 10, the system may include a first node and a second node. The first node is equipped with a first network device and a first processing unit, and the second node is equipped with a second network device and a second processing unit. The first network device and the second network device can execute the memory object transfer method described in the foregoing method embodiments to transfer a target memory object between the first node and the second node.

[0159] As mentioned above, the transfer of memory objects typically involves two ends: a requesting end and a responding end. The requesting end is the end that requests to obtain the memory object, and the responding end is the end that provides the memory object. For ease of distinction and description, in this embodiment, the responding end involved in the transfer of memory objects is described as the first node, and the requesting end is described as the second node. It is worth noting that a computing node can sometimes act as the requesting end and sometimes as the responding end. In addition, as mentioned above, inter-process communication may be cross-node or non-cross-node. Therefore, in the case of cross-node communication, the first node and the second node in this application can be used to refer to two different computing nodes, while in the case of non-cross-node communication, the first node and the second node in this application can be used to refer to the same computing node.

[0160] In practical applications, the network interface card (NIC) device driver and the program code within the NIC device can be modified in this embodiment. These modifications can cover the processing logic required by both the requesting and responding ends. Thus, regardless of whether the node to which the NIC device belongs (pages 14 / 16, CN 121636392 A) acts as a requesting end or a responding end, it can interact with the processing unit according to the memory transfer scheme provided in this embodiment.

[0161] Referring to the relevant descriptions in the foregoing method embodiments, after executing the memory object transfer scheme provided in this embodiment on the first node side, the number of interactions between the NIC device and the processing unit on the first node side can be effectively reduced; after executing the memory object transfer scheme provided in this embodiment on the second node side, the number of interactions between the NIC device and the processing unit on the second node side can be effectively reduced. Therefore, if both the first and second node sides execute the memory object transfer scheme provided in this embodiment, the number of interactions between the NIC device and the processing unit involved in the entire memory object transfer process can be significantly reduced, thereby effectively reducing the memory object transfer latency in the memory transfer system. In addition, the control pressure on the processing units on both sides can be reduced, which can further reduce the transmission latency of memory objects.

[0162] It is worth noting that the above-mentioned technical details regarding the memory object transmission system can be referred to the relevant descriptions in the aforementioned method embodiments. To save space, they will not be elaborated here, but this should not cause any loss of the scope of protection of this application.

[0163] FIG11 is a schematic diagram of the structure of a node provided in another exemplary embodiment of this application. As shown in FIG11, the node may include: a processing unit 11 and a network card device 12;

[0164] The network card device 12 is used to execute the aforementioned memory object transmission method to interact with the processing unit 11 and realize memory object transmission.

[0165] It should be understood that the node shown in FIG10 can be used as the first node mentioned in the aforementioned embodiments, or as the second node mentioned in the aforementioned embodiments.

[0166] Further, as shown in FIG11, the node also includes: a memory 13 and a power supply component 14 and other components. FIG11 only schematically shows some components, and does not mean that the node only includes the components shown in FIG11.

[0167] It is worth noting that the technical details regarding the various embodiments of the nodes described above can be referred to the relevant descriptions in the foregoing method embodiments. To save space, these details will not be repeated here, but this should not cause any loss of the scope of protection of this application.

[0168] Accordingly, this application also provides a computer-readable storage medium storing a computer program, which, when executed, can implement the steps in the above method embodiments.

[0169] Accordingly, this application also provides a computer program product, in which the computer program, when executed, can implement the steps in the above method embodiments.

[0170] The memory in Figure 11 above is used to store computer programs and can be configured to store various other data to support operation on the computing platform. Examples of such data include instructions for any application or method operating on the computing platform, contact data, phone book data, messages, pictures, videos, etc. The memory can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic storage, flash memory, disk, or optical disk.

[0171] The network card component in Figure 11 above is configured to facilitate wired or wireless communication between the device and other devices. The device containing the network card component can access wireless networks based on communication standards, such as WiFi, 2G, 3G, 4G / LTE, 5G, and other mobile communication networks, or combinations thereof. In an exemplary embodiment, the network card component receives broadcast signals or broadcast-related information from an external broadcast management system via a broadcast channel. In one exemplary embodiment, the network card component further includes a near field communication (NFC) module to facilitate short-range communication. For example, the NFC module may be implemented based on radio frequency identification (RFID) technology, Infrared Data Association (IrDA) technology, ultra-wideband (UWB) technology, Bluetooth (BT) technology, and other technologies.

[0172] The power supply component in FIG11 above provides power to various components of the device in which the power supply component is located. The power supply component may include a power management system, one or more power supplies, and other components associated with generating, managing, and distributing power to the device in which the power supply component is located.

[0173] Those skilled in the art will understand that embodiments of this application may be provided as methods, systems, or computer program products. Therefore, this application may take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Moreover, this application may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0174] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this application. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce...A machine is created such that instructions, which execute by a processor of a computer or other programmable data processing apparatus, produce means for implementing the functions specified in one or more flowcharts and / or one or more block diagrams.

[0175] These computer program instructions may also be stored in a computer-readable storage medium capable of directing a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means that implement the functions specified in one or more flowcharts and / or one or more block diagrams.

[0176] These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer-implemented process, such that the instructions, which execute on the computer or other programmable apparatus, provide steps for implementing the functions specified in one or more flowcharts and / or one or more block diagrams.

[0177] It should also be noted that the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0178] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use, and processing of related data must comply with the relevant laws, regulations, and standards of the relevant countries and regions, and corresponding operation entry points are provided for users to choose to authorize or refuse.

[0179] The above descriptions are merely embodiments of this application and are not intended to limit this application. Various modifications and variations can be made to this application by those skilled in the art. Any modifications, equivalent substitutions, or improvements made within the spirit and principles of this application shall be included within the scope of protection of this application. Specification 16 / 16 pages 20 CN 121636392 A Figure 1 Figure 2 Specification Drawings 1 / 6 pages 21 CN 121636392 A Figure 3a Figure 3b Specification Drawings 2 / 6 pages 22 CN 121636392 A Figure 4 Figure 5 Specification Drawings 3 / 6 pages 23 CN 121636392 A Figure 6 Figure 7 Specification DrawingsPage 4 / 6, 24 CN 121636392 A, FIGS. 8 and 9, Drawings of the Specification Page 5 / 6, 25 CN 121636392 A, FIGS. 10 and 11, Drawings of the Specification Page 6 / 6, 26 CN 121636392 A, Abstract Embodiments of the present application provide a memory object transmission method, node, system, computer program product and storage medium. The semantics in the communication protocol between the processing unit and the network card device are extended to optimize the interaction logic between the network card device and the processing unit in the memory object transmission process. Based on this, the network card device and the processing unit can perform simplified interaction according to new semantics such as subscription request and request response, and implement transmission of the memory object in the form of subscription. In this way, the transmission problem of the memory object is converted into the state monitoring problem of the memory object, and the processing unit and the network card device can deliver the monitoring result through request response, sothat the processing unit and the network card device no longer need to perform complex interaction according to traditional basic semantics such as receive and send. Accordingly, the transmission delay of the memory object can be effectively reduced.

Claims

1. A method of transferring in-memory objects, the method comprising: The method is suitable for a first network card device, the first network card device is installed on a first node, and a first processing unit is also installed on the first node, and the method comprises the following steps: sending a subscription request for a target memory object to the first processing unit, the subscription request being used for requesting to listen to a ready state of the target memory object, the target memory object being any memory object needing to be transmitted; after receiving a request response sent by the first processing unit for the subscription request, determining a second node, the request response being sent by the first processing unit after listening to the ready state of the target memory object being ready; transmitting the target memory object to the second node.

2. The method of claim 1, wherein, The step of transmitting the target memory object to the second node comprises the following steps: reading the target memory object from a memory space managed by the first processing unit; if the number of the second nodes is multiple, transmitting the read target memory object to the multiple second nodes respectively.

3. The method of claim 1, wherein, The step of sending a subscription request for the target memory object to the first processing unit comprises the following steps: sending a common subscription request to the first processing unit for prefetch requests sent by multiple nodes for the target memory object.

4. The method of claim 3, wherein, The step of sending a common subscription request to the first processing unit for prefetch requests sent by multiple nodes for the target memory object comprises the following steps: after receiving a prefetch request sent by any node for the target memory object, detecting whether a subscription request for the target memory object has been sent to the first processing unit; if not, sending a subscription request for the target memory object to the first processing unit.

5. The method of claim 4, wherein, The method further comprises the following steps: if it is detected that a subscription request for the target memory object has been sent to the first processing unit and the target memory object is not ready, adding the prefetch request sent by the second node to a waiting list maintained for the target memory object.

6. The method of claim 5, wherein, The step of determining a second node comprises the following steps: inquiring the recorded sending nodes corresponding to each prefetch request in the waiting list as the second node, the sending node being a node sending a prefetch request; wherein the number of the second nodes is one or more.

7. The method of claim 5, wherein, After receiving a prefetch request sent by any node for the target memory object, the method further comprises the following steps: inquiring a ready list used for recording ready memory objects; if the target memory object is not inquired in the ready list, determining that the target memory object is not ready; wherein the ready list is maintained by the first processing unit or by the first network card device according to the received request response.

8. The method of claim 5, wherein, After receiving a prefetch request sent by any node for the target memory object, the method further comprises the following steps: if it is determined that the target memory object is ready, transmitting the target memory object to the second node.

9. The method of claim 1, wherein, The step of sending a subscription request for the target memory object to the first processing unit comprises the following steps: In response to receiving a prefetch request from any node for the target memory object, sending a subscription request for the target memory object to the first processing unit on behalf of the node, the prefetch request indicating to obtain the target memory object from the first node.

10. The method of claim 9, wherein, In response to receiving a prefetch request from any node for the target memory object, sending a subscription request for the target memory object to the first processing unit on behalf of the node, including: In response to receiving a prefetch request from any node for the target memory object, forwarding the prefetch request as the subscription request to the first processing unit.

11. The method of claim 10, wherein, Further comprising: Receiving a response message from the first processing unit for the prefetch request as the request response, the response message containing a node identifier corresponding to the node that sent the prefetch request; The determination of the second node includes: Determining the node pointed to by the node identifier contained in the received response message as the second node; Wherein, the number of second nodes determined according to a single request response is one.

12. The method of claim 3 or 9, wherein, Transmitting the target memory object to the second node includes: In the case that the target memory object needs to be serialized before transmission, transmitting serialization information corresponding to the target memory object to the second node; After determining that the serialization information has been transmitted to the second node, adding attribute information of serialization data corresponding to the target memory object and identification information of the second node to a response message, the response message indicating that the response to the prefetch request sent by the second node is complete; Sending the response message to the second node to trigger the second node to obtain the serialization data through the response message and to perform deserialization operation on the serialization data based on the serialization information to restore the target memory object.

13. The method of claim 3 or 9, wherein, Transmitting the target memory object to the second node includes: In the case that the target memory object does not need to be serialized before transmission, transmitting the target memory object to the second node; The method further includes: After determining that the target memory object has been transmitted to the second node, sending a response message to the second node, the response message indicating that the response to the prefetch request sent by the second node is complete.

14. A method of transferring memory objects, the method comprising: The method is suitable for a second network card device, the second network card device being installed on a second node, the second node further being installed with a second processing unit, the method including: Receiving a prefetch request sent by the second processing unit, the prefetch request indicating to obtain a target memory object from a first node; Forwarding the prefetch request to the first node to trigger the first node to return the target memory object after listening to a ready state of the target memory object being ready; Writing the received target memory object into a memory reserved space set by the second processing unit for the prefetch request.

15. The method of claim 14, wherein, Writing the received target memory object into a memory reserved space set by the second processing unit for the prefetch request includes: receiving a response message returned by the first node for the pre-fetch request and serialized information corresponding to the target memory object; writing the serialized information into a memory reserved space set by the second processing unit for the pre-fetch request; forwarding the response message to the second processing unit to trigger the second processing unit to continue to request the serialized data corresponding to the target memory object according to attribute information of the serialized data contained in the response message.

16. The method of claim 15, wherein, Further comprising: after forwarding the response message to the second processing unit, receiving a read request initiated by the second processing unit for the serialized data corresponding to the target memory object; reading the serialized data from the first node according to the read request; after determining that the serialized data reading is completed, sending a read response for the read request to the second processing unit to trigger the second processing unit to deserialize the serialized data according to the serialized information to restore the target memory object.

17. A memory object transfer system, comprising: comprising a first node and a second node, the first node being equipped with a first network device and a first processing unit, and the second node being equipped with a second network device and a second processing unit; the first network device is used to execute the memory object transmission method in any one of claims 1-13, and the second network device is used to execute the memory object transmission method in any one of claims 14-16 to transmit a target memory object between the first node and the second node.

18. A node, characterized by comprising a processing unit and a network card device; the network card device is used to interact with the processing unit to perform memory object transmission according to the memory object transmission method in any one of claims 1-16.

19. A computer-readable storage medium storing a computer program, characterized in that, when the computer program is executed by one or more processors, the one or more processors are caused to execute the memory object transmission method in any one of claims 1-16.

20. A computer program product, characterised in that, comprising a computer program, when the computer program is executed by one or more processors, the one or more processors are caused to execute the memory object transmission method in any one of claims 1-16.

Images