Apparatus and method for remote direct memory access

Abstract

The requesting device of the present invention is capable of transmitting multiple prompt messages simultaneously, and enables the responding device to optimize various aspects through memory prefetching, cache warming, and multiple atomic operations. These operations are used to enable the responding device to begin prefetching memory regions in advance (if necessary), or to prepare to receive incoming data packets in an efficient manner.

Description

TECHNICAL FIELD

[0001] The present invention relates generally to data transfer; more specifically, to devices and methods for remote direct memory access. BACKGROUND

[0002] In recent years, the high-performance computing (HPC) field has leveraged high-performance and low-latency networks such as InfiniBand on top of existing IP infrastructure and custom lossless technologies. These interconnects provide advanced network functionality such as remote direct memory access (RDMA) to achieve high throughput and low latency as well as low CPU occupancy. RDMA is now used in high-performance computing (HPC), data centers, Web 2.0, big data, cloud, storage, finance services and insurance (FSI), healthcare, and federal domains. However, in networks consisting of multiple computing devices, high-speed data communication increases concerns about data reliability and latency.

[0003] Generally, RDMA involves direct memory access from the memory of one computing device to the memory of another computing device. Notably, RDMA technology does not require the operating system (OS) of either the traditional sending device and the traditional receiving device to enable network-based high-throughput, low-latency data transfer and is particularly useful in deployments where performance is critical. For example, in an exemplary implementation scenario of a traditional RDMA system, such as a RDMA read or write operation, a network adapter that supports RDMA is used to transfer data (or packets) directly from a source memory (of a traditional requesting device) to a target memory (of a traditional responding device). Specifically, it does not involve the operating system of either the central processing unit (CPU) (or processor) or either of the interacting devices (i.e., the traditional sending device and the traditional receiving device).

[0004] Conventional responder devices fetch queue pair (QP) context from host memory into local cache, issue a direct memory access (DMA) request to target memory, read memory in an atomic operation, perform the required operation, and write back memory upon receiving a data packet from a requester device. As such, the above operations take a significant amount of processing time, increasing latency during transmission. Moreover, performing these operations at wire speed with modern RDMA network interface controllers (RNICs) (25 Gbps - 400 Gbps) becomes extremely challenging. As such, fetching information from host memory in conventional systems takes time and often requires several iterations, which also results in increased latency and decreased performance.

[0005] In an example scenario, a conventional non-pinned RDMA technique can be used to reduce cost. In conventional non-pinned RDMA techniques, a conventional responder device requires a virtual address (e.g., target virtual address) of a conventional requester device to send data. If the virtual address of the conventional requester device triggers a fault (e.g., a page fault), the data transmission is paused or discarded and can result in data retransmission, which further increases transaction completion latency. Moreover, RDMA techniques limit the number of messages (or data packets) that can be processed at any given time, which depends on the responder (e.g., conventional responder device). As such, during high frequency data transmission, for very large messages or many small messages, can result in RDMA processing errors and faults.

[0006] Accordingly, in light of the above discussion, there is a need to overcome the above-mentioned drawbacks associated with conventional RDMA techniques for data communication. SUMMARY

[0007] The inventors have realized that, unlike a requester (e.g., conventional requester device) that has knowledge (QP context, operation information) about a data packet at the time of data packet generation, a responder (e.g., conventional responder device) only receives the knowledge (QP context), target memory address, and required operation upon receiving the data packet from the network, which results in the above-discussed increased latency and decreased performance. However, by having the responder also have this knowledge at the time of data packet generation or at least ahead of time, latency can be reduced and performance can be improved.

[0008] Accordingly, the present invention seeks to provide a requesting device, a responding device, a method and a system for remote direct memory access (RDMA). The present invention seeks to address the existing problems of inefficient and unreliable data communication by conventional devices, methods and conventional remote direct memory access systems. It is an object of the present invention to provide a solution that at least partially overcomes the problems encountered in the prior art and which provides an improved device, method and system for efficient (e.g. reduced latency) and reliable data communication for RDMA operations. It is an object of the present invention to achieve this by the solution provided in the independent claims appended hereto. Advantageous implementations of the present invention are further defined by the dependent claims.

[0009] It is an object of the present invention to achieve this by the solution provided in the independent claims appended hereto. Advantageous implementations of the present invention are further defined by the dependent claims.

[0010] In one aspect, the present invention provides a requesting device. The requesting device comprises a memory, a controller and a communication interface. The memory is configured to store a plurality of work elements in one or more request queues. Each work element is indicative of a requester, a responder and an operation. The controller is configured to retrieve one or more of the work elements from the memory. The controller is further configured to generate a hint message comprising an indication of at least one operation of a corresponding at least one of the retrieved one or more work elements. The controller is further configured to send the hint message to a responding device via the communication interface. The responding device corresponds to the responder of the at least one of the retrieved one or more work elements. The controller is further configured to send a request to the responding device via the communication interface in relation to the operation of the at least one of the retrieved one or more work elements, the request being indicative of the operation.

[0011] The requesting device of the present invention is capable of transmitting multiple hint messages simultaneously and enables the responding device to optimize various aspects through memory prefetching, cache warm-up and multiple atomic operations into hardware. These operations are used to enable the responding device to start prefetching memory regions in advance (if necessary) or to prepare to receive incoming data packets in an efficient manner. Furthermore, sending multiple hints in relation to multiple transactions (or operations) simultaneously enables the responding device to start preparing memory for each transaction (or operation) and has a higher chance of being serviced without a page fault. Furthermore, in the case of atomic operations, the responder can start reading the atomic data packet into a preferred local memory (e.g. RNIC cache) before the data packet arrives, which enables the responding device to perform the atomic operation and send an acknowledgement message with very low latency.

[0012] In an implementation, the work element further includes an object list including one or more objects to perform the operation, and wherein the prompt message further includes an indication of at least one of the one or more objects.

[0013] The one or more objects included in the object list are data transfer objects (DTOs). The one or more objects are used to reduce the number of calls during communication between processes typically conducted via a remote interface (e.g., a web service), where each call is a high-cost operation. The one or more objects (DTOs) support aggregation of data traditionally transmitted through several calls, with a single call providing the service.

[0014] In another implementation, one or more of the retrieved one or more work elements are for a first responder, and one or more of the retrieved one or more work elements are for a second responder. The controller is further configured to generate the prompt message and send the prompt message to the responding device by generating a first prompt message and sending the first prompt message to a first responding device corresponding to the first responder, and generating a second prompt message and sending the second prompt message to a second responding device corresponding to the second responder.

[0015] Sending multiple prompt messages (i.e., first prompt message, second prompt message) for multiple transactions with different responders (i.e., first responder, second responder) at the same time allows the respective responders to start preparing memory for all transactions, thereby having a higher chance of being serviced without a page fault.

[0016] In another implementation, one or more of the retrieved one or more work elements are associated with a first requestor, and one or more of the retrieved one or more work elements are associated with a second requestor.

[0017] In an implementation, the controller is further configured to receive an acknowledgement message from the responding device prior to sending the request associated with the operation.

[0018] The acknowledgement message received from the responding device allows the requesting device to adapt or configure transmission of data packets accordingly, thereby facilitating trouble-free operation. Typically, the acknowledgement message means that the first message is successfully sent to the receiving device. Thus, the controller sends the request associated with the operation upon receiving the acknowledgement message, irrespective of any external factors. Thus, the acknowledgement message shortens the total execution time of data transmission and reduces latency. Reducing latency indicates higher speed of data communication.

[0019] In another implementation, the controller is further to retrieve a batch of work elements and change a size of the batch during the operation.

[0020] The controller can change the size of the batch of work elements and enable the requester (or requesting device) to perform a lookahead on the requests queued in the send queue up to a preconfigured depth, so that information about the data packets can be collected before the data is actually received and the memory region can be better prepared, resulting in faster data communication and reduced latency.

[0021] In one implementation, the hint message further includes an indication of a memory region related to the operation.

[0022] The indication of the memory region in the hint message enables the responding device to allocate the specified memory region before the data packets are received, to speed up the data communication.

[0023] In another implementation, the indication of the memory region and the operation includes an address of the memory region on which the operation is to be performed.

[0024] The address of the indication of the memory region on which the operation is to be performed enables the responder (or responding device) to better prepare the memory region for efficient performance of the operation.

[0025] In another implementation, the indication of the memory region and the operation includes a prefetch operation to prefetch the indicated memory region.

[0026] The prefetch operation to prefetch the indicated memory region enables the responder (or responding device) to prefetch the memory range to reduce or eliminate page fault interrupts when the request arrives.

[0027] In another implementation, the indication of the memory region and the operation includes a cache warm-up operation.

[0028] The cache warm-up operation to prepare (or warm up) the indicated memory region enables the responder (or responding device) to prefetch the memory range to bring the corresponding QP context into the cache memory to reduce latency when the request arrives.

[0029] In another implementation, the indication of the memory region and the operation includes an atomic operation.

[0030] The atomic operation to prepare the indicated memory region enables the responder (or responding device) to read the atomic target into the RNIC to speed up the computation when the request arrives.

[0031] In one implementation, the hint message is a dedicated hint command message.

[0032] The dedicated hint message supports configuring the hint message to adapt based on the requirements of the responder, the requester, or the data communication to improve the overall efficiency of the system.

[0033] In another implementation, the hint message includes the indication of the operation as an additional payload of a command message of another command.

[0034] In one implementation, the requesting device is arranged for RDMA.

[0035] Arranging the requesting device for RDMA enables the requesting device to send and receive data to and from remote buffers (zero copy), access the memory of a remote server without consuming CPU cycles on the remote server (low CPU involvement), with improved latency and throughput (kernel bypass), with convergence, i.e. storage and computation supported by a single fabric.

[0036] In another aspect, the present application provides a requesting device. The requesting device includes a memory and a communication interface, and a software module. The software module includes a work element storage module for storing a plurality of work elements in one or more request queues in the memory, each work element indicating a requester, a responder, and an operation. The software module further includes a work element retrieval module for retrieving at least one of the work elements from the memory. The software module further includes a hint message generation module for generating a hint message, the hint message including an indication of at least one operation of a corresponding at least one of the retrieved one or more work elements. The software module further includes a hint message transmission module for transmitting the hint message to a responding device through the communication interface, the responding device corresponding to the responder of the at least one of the retrieved one or more work elements. The software module further includes a request transmission module for transmitting a request related to the operation of the at least one of the retrieved one or more work elements to the responding device through the communication interface, the request indicating the operation.

[0037] The software module is executed to enable the requesting device to store and retrieve a plurality of work elements in one or more request queues for proper ordering and implementation of transmission of work elements as required. In addition, the software module is further executed to generate and transmit a hint message including an indication of at least one operation. Beneficially, the hint message enables the responding device to prepare a memory region for an incoming work element for faster and more efficient data communication. In addition, the software module is executed to transmit a request related to the operation to the responding device to enable the requesting device to initiate transmission of a data packet, thereby reducing page fault interruption and improving data communication reliability.

[0038] In yet another aspect, the present application provides a method for a requesting device. The method comprises storing a plurality of work elements in one or more request queues in the memory, each work element indicating a requester, a responder, and an operation. The method further comprises retrieving at least one of the work elements from the memory. The method further comprises generating a prompt message, the prompt message including an indication of at least one operation of a corresponding at least one of the retrieved one or more work elements. The method further comprises sending the prompt message to a responding device through the communication interface, the responding device corresponding to the responder of the at least one of the retrieved one or more work elements. The method further comprises sending a request related to the operation of at least one of the retrieved one or more work elements to the responding device through the communication interface, the request indicating the operation.

[0039] The method of the present aspect realizes all the advantages and effects of the requesting device of the present application.

[0040] In another aspect, the present application provides a computer readable medium carrying computer instructions, which, when loaded into and executed by a controller of a requesting device, enable the requesting device to realize the method.

[0041] The computer readable medium (in particular, the non-transitory computer readable medium) carrying computer instructions realizes all the advantages and effects of the sending device or method.

[0042] In one aspect, the present application provides a responding device. The responding device comprises a memory, a controller, and a communication interface. The controller is configured to receive a prompt message from a requesting device through the communication interface, the prompt message including an indication of at least one operation. The controller is further configured to determine a preemption action based on the indication of the at least one operation. The controller is further configured to perform the preemption action. The controller is further configured to receive a request related to the at least one operation from the requesting device through the communication interface, the request indicating at least one of the at least one operation. The controller is further configured to perform at least one of the at least one operation.

[0043] Since the pre-emptive action is associated with the at least one operation, the controller of the responding device determines the pre-emptive action based on the indication of the at least one operation in the hint message and responds accordingly to prepare the associated memory region for faster data communication. In one example, the pre-emptive action comprises a cache warm-up operation to populate data into cache memory for faster data communication. Advantageously, these pre-emptive actions performed by the responding device substantially reduce the latency in servicing the RDMA message itself when the RDMA message arrives at the responding device and makes data communication more efficient.

[0044] In one implementation, the operation is associated with a memory region, and the controller is further configured to determine the pre-emptive action based on the memory region and the operation, and to perform the pre-emptive action by determining whether the memory region needs to be loaded and, if so, loading the memory region into the memory.

[0045] By determining whether the memory region needs to be loaded before performing the pre-emptive action, the total time taken for data communication is reduced by eliminating the possibility of wasted effort if the memory does not need to be populated. Furthermore, this determination also significantly reduces the chance of a page fault due to an unpopulated memory region.

[0046] In another implementation, the controller is further configured to send an acknowledgement message to the requesting device, the acknowledgement message indicating whether the pre-emptive action is successful.

[0047] The acknowledgement message received from the controller of the responding device enables the requesting device to adapt or configure the transmission of data packets accordingly, thereby facilitating glitch-free operation. Typically, the acknowledgement message means that the first message is successfully sent to the receiving device. Thus, the controller sends the request related to the operation upon receiving the acknowledgement message, irrespective of any external factors. Thus, the acknowledgement message shortens the total execution time of data transmission and reduces latency. Reducing latency indicates faster data communication.

[0048] In another aspect, the present application provides a method for a responding device. The method comprises receiving a hint message from a requesting device, the hint message comprising an indication of at least one operation. The method further comprises determining a pre-emptive action based on the indication of the at least one operation. The method further comprises performing the pre-emptive action. The method further comprises receiving a request from the requesting device related to the at least one operation, the request indicating at least one of the at least one operation. The method further comprises performing at least one of the at least one operation.

[0049] The method of the present aspect achieves all the advantages and effects of the receiving device of the present application.

[0050] In an implementation form, the application provides a computer readable medium carrying computer instructions which, when loaded into and executed by a controller of a responding device, enable the responding device to implement the method.

[0051] The computer readable medium carrying computer instructions implements all advantages and effects of the receiving device or method.

[0052] In yet another aspect, the application provides a responding device. The responding device comprises a memory, a communication interface and software modules. The software modules comprise a prompt message receiving module for receiving a prompt message from a requesting device, the prompt message comprising an indication of at least one operation. The software modules further comprise a pre-emption action determining module for determining a pre-emption action based on the indication of the at least one operation. The software modules further comprise a pre-emption action performing module for performing the pre-emption action. The software modules further comprise a request receiving module for receiving a request from the requesting device relating to the at least one operation, the request indicating at least one of the at least one operation. The software modules further comprise an operation performing module for performing at least one of the at least one operation.

[0053] The software modules are executed to determine a pre-emption action based on an indication of at least one operation. Advantageously, determining based on the requirements of the requesting device (in the prompt message) enables the responding device to determine the most appropriate option to reduce the likelihood of a page fault interrupt and can better prepare the memory region for more efficient data communication.

[0054] It is to be understood that all the above implementation forms can be combined.

[0055] It is noted that all devices, elements, circuits, units and modules described in the present application can be implemented in software or hardware elements or any type of combination thereof. All steps which are performed by the various entities described in the present application as well as the functionalities described to be performed by the various entities are intended to mean that the respective entity is adapted to or configured to perform the respective steps and functionalities. Although in the following description of illustrative examples, specific functionalities or steps to be performed by external entities are not reflected in the description of specific detailed elements of the entities which perform these specific steps or functionalities, it should be clear for the skilled person that these methods and functionalities can be implemented in respective software or hardware elements, or any type of combination thereof. It is understood that features of the present application are easily combined with each other and are not limited to the combinations as described, without departing from the scope of the present application, which is defined by the appended claims.

[0056] Additional aspects, advantages, features and objects of the application will become apparent to those skilled in the art from the detailed description of illustrative implementations, which follows, in conjunction with the accompanying drawings. BRIEF DESCRIPTION OF DRAWINGS

[0057] The foregoing summary, as well as the following detailed description of illustrative embodiments, can be better understood when read in conjunction with the appended drawings. For the purpose of illustrating the application, exemplary constructions of the present application are shown in the drawings. However, the present application is not limited to the specific methods and tools disclosed herein. In addition, one skilled in the art will understand that the drawings are not drawn to scale. Where possible, the same elements have been given the same numerals throughout the various figures.

[0058] Embodiments of the present application will now be described, by way of example only, with reference to the following drawings:

[0059] Figure 1 is a block diagram of a requesting device that can be used to perform a remote direct memory access (RDMA) operation, provided by embodiments of the present application;

[0060] Figure 2 is a flowchart of a method for a requesting device to perform a RDMA operation, provided by embodiments of the present application;

[0061] Figure 3 is a block diagram of a responding device that can be used to perform a RDMA operation, provided by embodiments of the present application;

[0062] Figure 4 is a flowchart of a method for a responding device to perform a RDMA operation, provided by embodiments of the present application;

[0063] Figure 5 is a block diagram of a RDMA system having a requesting device and a responding device, provided by embodiments of the present application;

[0064] Figure 6 is a flowchart of a method for a system including a responding device and a requesting device to perform a RDMA operation, provided by embodiments of the present application;

[0065] Figures 7 to 9 is a schematic diagram of an exemplary scenario describing an implementation of a RDMA system, in accordance with various embodiments of the present application.

[0066] In the drawings, underlined numerals are used to designate items in the drawings by which the underlined numerals are indicated or items adjacent to which the underlined numerals are indicated. Non-underlined numerals are associated with items identified by lines that associate the non-underlined numerals with the items. When a numeral is non-underlined and has an associated arrow, the non-underlined numeral is used to identify the general item to which the arrow points. DETAILED DESCRIPTION

[0067] The following detailed description illustrates embodiments of the present invention and ways in which these embodiments can be implemented. While some modes of implementing the invention have been disclosed, those skilled in the art will recognize that other embodiments for implementing or practicing the invention may also exist.

[0068] Now for reference Figure 1 The diagram illustrates a requesting device 100 provided by an embodiment of the present invention, which can be used to perform remote direct memory access (RDMA) operations. Figure 1 Various exemplary components of the requesting device 100 are shown. As illustrated, the requesting device 100 includes memory 102 for storing a plurality of working elements 108. The requesting device 100 also includes a controller 104 and a communication interface 106.

[0069] The requesting device 100 is configured for RDMA. Typically, the requesting device 100 is configured and operable to perform RDMA operations together with the responding device. Figure 1 Not shown in the text, see below for reference. Figure 3 (This will be explained in more detail in this document). The requesting device 100 can directly access the memory of the responding device without involving the operating systems (OS) of both the requesting and responding devices. It should be noted that data packets are transmitted directly without being copied between the communication channel layers (e.g., network layer, application layer, transport layer, and link layer) of the requesting and responding devices. Each communication channel layer performs specific functions such as transmission and encryption. The combination of these communication channel layers enables communication across different interacting devices (e.g., the requesting device 100 and the responding device).

[0070] Data packets are derived from one or more work elements. Work elements include information associated with packet sequence numbers (PSNs) to enforce correct packet ordering. Work elements also include source and destination queue pair (QP) numbers, which distinguish applications and their valid memory regions. In one embodiment, a memory region has a memory key (R-key) that is part of the data packet, associating the memory region with the requester and vice versa. Work elements also include information associated with the operation type (e.g., RDMA read, RDMA write, send, or atomic), and parameters of the operation, such as message length, destination memory address, operation type, and operand data.

[0071] The responding device is configured to reassemble incoming data packets based on headers including at least a destination queue pair number and a data packet sequence number, e.g., a base transport header (BTH). In one example, the BTH header includes at least a target memory address, a length of the RDMA operation, and optionally a remote key that can be used to identify the target memory address and to provide access. In the case of an atomic operation, the BTH header includes at least an opcode / operation code, a target memory address, a data packet to be exchanged or compared, and a remote key for access. Obviously, the requesting device 100 will include a queue pair (QP), i.e., a send queue (SQ) and a receive queue (RQ). In addition, the SQ of the QP will include a work element carrying responder details.

[0072] According to one embodiment, the requesting device 100 includes appropriate logic, circuitry, interfaces, and / or code configured for high speed data communication. The requesting device 100 can have direct access to the memory of the requesting device 100 for performing any RDMA operation, e.g., RDMA read, RDMA write, send, or atomic operation.

[0073] The requesting device 100 can potentially be used as a sending device (or sender). Examples of the requesting device 100 can include, but are not limited to, a communication channel adapter, a server, a computing device in a computer cluster, e.g., a massively parallel computer cluster, a communication device including a portable or non-portable electronic device, a wireless modem, a supercomputer, or other RDMA-based device. Similarly, the responding device includes appropriate logic, circuitry, interfaces, and / or code configured for high speed data communication. In addition, the responding device can potentially be used as a receiving device (or receiver) during operation. Examples of the responding device can include, but are not limited to, a communication channel adapter, a server, a computing device in a computer cluster, e.g., a massively parallel computer cluster, a communication device including a portable or non-portable electronic device, a wireless modem, a supercomputer, or other RDMA-based device. According to the present application, the responding device prepares its memory 102, e.g., a local memory buffer, shortly after receiving an event packet, e.g., a "hint" packet, from the requesting device 100, in order to reduce response time, as will be explained in more detail below.

[0074] Memory 102 of request device 100 is configured to store a plurality of work elements 108 in one or more request queues. Each work element (of the plurality of work elements 108) indicates a requestor (e.g., request device 100), a responder (e.g., a response device or a sink), and an operation. Memory 102 includes appropriate logic, circuitry, and / or interfaces to store instructions executable to control operation of request device 100. For example, memory 102 can store an operating system and / or other program products to operate request device 100. Memory 102 can store data (communicated in the form of work elements or data packets) for processing at request device 100. Examples of implementation of memory 102 can include, but are not limited to, electrically erasable programmable read-only memory (EEPROM), random access memory (RAM), read only memory (ROM), hard disk drive (HDD), flash memory, secure digital (SD) card, solid-state drive (SSD), and / or CPU cache memory. Computer readable storage media for providing non-transitory memory can include, but are not limited to, electronic, magnetic, optical, electromagnetic, semiconductor, or any suitable combination of the foregoing.

[0075] In one embodiment, each of the plurality of work elements 108 further includes an object list including one or more objects to perform the operation. The object list refers to a linear data structure in which each element is a separate object. An object refers to a data carrier or container between processes configured to accommodate storage, retrieval, serialization, and deserialization of data packets. It is noted that these objects do not include any inherent business logic, and are mainly used for data transfer and encapsulating one or more parameters during the operation. The one or more objects are used to perform the operation. Further, the one or more objects included in the object list are data transfer objects (DTOs). The one or more objects (DTOs) are used to reduce the number of calls during communication between processes, which is typically facilitated by a remote interface (e.g., a web service), where each call is a high-cost operation. The one or more objects (DTOs) support aggregating data traditionally transferred through several calls, with a single call providing the service.

[0076] The communication interface 106 of the requesting device 100 includes a medium through which the requesting device 100 communicates with a responding device, such as a communication channel. Examples of the communication interface 106 include, but are not limited to, a communication channel in a computer cluster, a local area network (LAN) communication channel, a cellular communication channel, a wireless sensor communication channel (WSN), a cloud communication channel, a vehicle-to-communication network (V2N) communication channel, a metropolitan area network (MAN) communication channel, and / or the Internet. Further, the requesting device 100 in the communication channel environment is configured to connect to the responding device according to various communication channel protocols that support RDMA communication. Examples of such communication channel protocols, communication standards, and technologies can include, but are not limited to, InfiniBand (IB), RDMA over converged Ethernet (RoCE), internet wide area RDMA protocol (iWARP), or modifications and variations thereof.

[0077] According to one embodiment, the communication interface 106 is an arrangement of interconnected programmable and / or non-programmable components that function to facilitate data communication between one or more electronic devices. For example, the communication interface 106 refers to a network interface card (NIC) that functions to process a send queue (SQ), read work queue elements (WQEs), and generate data packets for transmission to another computing device. The communication interface 106 can support communication protocols for all or a portion of one or more peer-to-peer networks, hybrid peer-to-peer networks, local area networks (LANs), radio access networks (RANs), metropolitan area networks (MANs), wide area networks (WANs), public networks such as the Internet, private networks, cellular networks, and any other one or more communication systems in one or more locations. Further, the communication interface 106 supports wired or wireless communication that can be performed over any number of known protocols, including, but not limited to, Internet protocol (IP), wireless access protocol (WAP), frame relay, or asynchronous transfer mode (ATM). Moreover, the communication interface 106 can also employ and support any other suitable protocols using voice, video, data, or combinations thereof.

[0078] In the present disclosure, the controller 104 is an RDMA controller that includes suitable logic, circuitry, and / or interfaces for generating a hint message that includes an indication of a type of operation. The controller 104 is a computing element that functions to process instructions to drive the requesting device 100. Examples of the controller 104 include, but are not limited to, a network interface controller, a microprocessor, a microcontroller, a complex instruction set computing (CISC) microprocessor, a reduced instruction set (RISC) microprocessor, or a very long instruction word (VLIW) microprocessor. Optionally, one or more controllers similar to the controller 104, interactive devices (e.g., the requesting device 100, the responding device), and elements can be arranged in various architectures to respond and process instructions to drive the requesting device 100.

[0079] In operation, the controller 104 is configured to retrieve one or more work elements of the plurality of work elements 108 from the memory 102 of the requesting device 100. The one or more retrieved work elements are stored in one or more request queues (or send queues). A request queue refers to an abstract data structure configured to store a plurality of work elements in a specified order.

[0080] In one embodiment, the controller 104 is configured to retrieve a batch of work elements and change the size of the batch during operation. The controller 104 is configured to process the plurality of work elements 108 in the batch by separating the plurality of work elements 108 in the batch having one or more work elements therein while retrieving these work elements from the memory 102. The controller 104 is configured to change the size of the batch of work elements by performing a lookahead on requests queued in one or more response queues (corresponding to one or more request queues of the requesting device 100) until a preconfigured depth is supported to collect information about incoming data packets before actually retrieving the work elements 108. The batch of work elements can be in the order of 4, 8, 32, 64, etc., and the upcoming batch of work elements is known to the responding device through a hint message. The controller 104 can perform a lookahead on a single QP request queue, and / or the requesting device 100 can perform parallel lookaheads in multiple QP request queues for the same responding device (or responder) as the host (using a different special QP to send hints). Further, the requesting device 100 can also handle parallel QPs for different responders.

[0081] The controller 104 is configured to generate a hint message including an indication of at least one operation of a corresponding at least one work element of the one or more retrieved work elements. The hint message is generated by the controller 104 to initiate one or more RDMA operations with the responding device.

[0082] The controller 104 is configured to generate a hint message based on each of the one or more work elements, wherein each work element indicates a requester (e.g., the requesting device 100), a responder (i.e., the responding device), and an operation (or operation type). The hint message indicates the operation (or operation type), such as an RDMA read operation, an RDMA write operation, a send operation, or an atomic operation. For example, the hint message is sent in the form of one or more data packets. The hint message also includes a source and a destination queue pair (QP) number. For example, each QP number has a context on the requesting device 100 and the responding device.

[0083] In another embodiment, the work element further includes an object list including one or more objects to perform the operation, and wherein the hint message further includes an indication of at least one of the one or more objects. The one or more objects included in the object list are data transfer objects (DTOs). The one or more objects are used to reduce the number of calls during communication between processes that are typically conducted via a remote interface (e.g., web service), where each call is a high cost operation. The one or more objects (DTOs) support aggregation of data that is conventionally transferred through several calls by a single call to service. The one or more objects included in the object list are used to perform the operation (e.g., RDMA read operation, RDMA write operation), and the hint message includes an indication of at least one of the one or more objects related to at least one of the operations.

[0084] In one embodiment, the hint message further includes an indication of a memory region related to the operation. The memory region refers to a target memory region to perform the operation (e.g., RDMA read operation, RDMA write operation, RDMA atomic operation). For example, the hint message includes an indication of a memory region of the responding device to perform the operation.

[0085] In another embodiment, the hint message includes an indication of a memory region, and the operation further includes an address of the memory region to perform the operation. The address of the memory region includes a target memory address of the operation. In one example, if the first operation is an RDMA read operation, and is associated with a memory region. The RDMA read operation is performed by reading a particular value or data at the memory region (or memory address). Further, if the second operation is an RDMA write operation, the operation is performed by writing a particular value or data at the memory region (or memory address). The memory region of the operation can or can not be the same. In one example, the operations can not be performed on the same memory region, or, can be subsequently performed on the same memory region.

[0086] According to one embodiment, the hint message further includes various parameters such as message length, operation type, and operand data. The operand data refers to data associated with the one or more objects in the object list to perform the operation. The message length includes a length of the RDMA read operation or the RDMA write operation. The send operation has no or needs a target memory address. Further, the atomic operation further includes the operation type and the operand data.

[0087] The operation type can be a tagged operation (e.g., RDMA read, RDMA write, or atomic operation) or an untagged operation type (e.g., RDMA Send). Here, for a tagged operation, the hint message includes RDMA target information including a target memory address and an operation length (e.g., length of RDMA write or RDMA read operation) determined from at least one work element of the one or more work elements 108 retrieved. Further, the target memory address and operation length are placed in the header of the hint message. On the other hand, for an untagged operation (e.g., RDMA Send operation), the hint message does not include RDMA target information. However, in this case, the RDMA target information is not available, the operation targets one or more request queues at the responding device, and where local RDMA target information is specified. In one embodiment, the RDMA target information is not available, the hint message includes a count indicating how many untagged messages to expect so that the responding device can prepare enough buffer space to receive the incoming messages.

[0088] In another embodiment, the hint message including the memory region and the indication of the operation includes a hint operation for pre-fetching the indication of the memory region. For example, such a hint message including the pre-fetch operation can be considered a "pre-fetch hint." The pre-fetch operation is a technique to speed up the fetch operation by starting the fetch operation before the data packet is retrieved. The pre-fetch operation uses a pre-fetch hint sent before the data packet is sent. The pre-fetch operation enables the responding device to start pre-fetching the memory region (or memory range) associated with the operation as needed, or start preparing to receive the incoming data packet, while using other techniques, such as but not limited to scratchpad buffer, memory allocation, etc. In one example, the pre-fetch hint can include a tuple list including at least one of a remote key, a target memory address, a length for a tagged operation, and a sequence ID, and a total number of incoming messages for an untagged operation. The responding device uses the hint message including the pre-fetch hint to reduce or eliminate page faults. A page fault refers to an exception raised by computer hardware when a running program accesses a memory page that is not currently mapped by the responding device into the target memory address space associated with the operation (or process). Further, sending the hint message including one or more pre-fetch hints for multiple transactions (or data transfers) at the same time enables the responding device to start preparing the memory region according to each of the one or more pre-fetch hints. The pre-fetch operation has several benefits, including reducing the number of page faults. In one example, even if a first transaction has to be paused or stopped due to inefficient mapping or mapping (paging-in memory) at insufficient rate by the responding device, there is a higher chance that subsequent transactions (e.g., second transaction, third transaction) are serviced without page faults.

[0089] In another embodiment, the hint message includes an indication of a memory region, and the operation includes a cache warm-up operation. For example, such a hint message including a cache warm-up operation can be considered a "cache warm-up hint." In one example, upon sending a hint message including at least a queue pair list and a target memory region associated with an incoming data packet (or work element 108), the responding device can extract a queue pair (QP) context into a predefined memory region (or cache memory), or perform a cache warm-up operation. A cache warm-up operation refers to populating a cache memory to improve the performance of execution of subsequent transactions. In one example, a first transaction after populating the cache will go through an empty cache or a cold cache and experience "cache misses" as the cache memory will retrieve the data packet from the memory 102 of the requesting device 100 to perform the warm-up cache operation for the first transaction. Each subsequent transaction can be served from the cache memory after warm-up to eventually reduce latency and improve performance of execution upon receiving an incoming data packet.

[0090] Additionally, the responding device can issue an on-demand paging (ODP) request (similar to a "prefetch hint" mechanism) to a specific memory region that is an ODP memory region, to significantly reduce the latency associated with servicing the RDMA message itself when received by the responding device. ODP is a technique to simplify the memory registration of incoming data packets. Here, there is no longer a need to determine the underlying physical target address space and track mapping validity. Instead, the hint message including an indication of a memory region and an operation includes requesting the latest translation from the responder (or responding device) when the memory region does not exist, and the responding device invalidates translations that are no longer valid due to the non-existence of the memory region or mapping changes.

[0091] In another embodiment, the indication of a memory region and an operation includes an atomic operation. An atomic operation is an operation that is executed as a single unit of work (i.e., a single work element), during which a processor can read a target address and write to that target address in the same operation. This will prevent any other processor or I / O device from writing or reading the memory until the operation is complete. In other words, an atomic operation (e.g., RDMA get and add operations, RDMA compare and swap operations) is not subject to interference during the operation. The hint message includes a list of atomic operations for the responder to read the targets of the atomic operations to improve compute speed upon receiving a request for an RDMA access.

[0092] According to one embodiment, the prompt message is a dedicated prompt command message (Opcode). For example, the prompt message can be dedicated or configured to accommodate implementation requirements of the RDMA system or the particular responding device. For example, the controller 104 of the requesting device 100 configures each prompt message (also referred to herein as a command message) to be sent in one or more data packets with a header (e.g., BTH) and an optional additional payload. It is noted that the header provides basic information (e.g., a start indicator, a length of the data packet, and optionally a command code identifying the command to be processed by the responding device) while the payload provides any data associated with the command message. It is noted that the amount of data associated with the command message can vary for different commands, and certain commands can provide a variable length payload. In some embodiments, the commands can be defined such that any particular command message is valid in a single direction during the data communication. Optionally, the payload includes only the command of the command message and the associated data, and the controller 104 can generate additional information related to the data communication or the interacting computing device (e.g., the requesting device 100, the responding device) before forwarding the data packet (or command message) to the responding device.

[0093] In one embodiment, the prompt message includes an indication of an operation (e.g., an RDMA read, an RDMA write operation) as an additional payload of another command message (a first command message). The requesting device 100 indicates the operation as an additional payload of the command message. Here, the command message includes a command (e.g., an instruction of the operation) and a payload. The payload refers to any parameter or data associated with each command message. Further, the payload includes a token related to the physical capabilities, processing capabilities, and other additional information required by the responding device. The controller 104 can determine whether the command in the command message is a data transfer command or a data retrieval command. In one example, the controller 104 determines that the data transfer command can transmit a message corresponding to the payload of the data transfer command to the responder (or responding device). Further, the command message including the data transfer command instructs the controller 104 of the requesting device 100 to extract the command message (including any associated data) and forward the command message (and, where applicable, the associated data) to the responding device.

[0094] In one example, the prompt message can be associated with multiple QPs. In this case, the controller 104 has a look-up depth of work elements in the request queue. Accordingly, the controller 104 generates the prompt message based on the multiple QPs. The following is an exemplary table describing data associated with multiple work elements in the multiple QPs and the corresponding generated prompt message.

[0095] As shown in Table 1, there are 4 work elements associated with the send queue of a queue pair. For example, column 1 shows the number of work elements, column 2 shows the QP associated with the work elements, column 3 shows the indication of at least one operation associated with the work elements, and column 4 shows the parameters or attributes associated with the work elements. In other words, Table 1 shows the data associated with a work element of the one or more work elements retrieved from memory 102, based on which a hint message is generated.

[0096] Table 1

[0097] Number QP Type Parameter 1 14 Read R KEY = 30, VA = 0x1000, length = 0x8000 2 17 Write R KEY = 40, VA = 0x100000, length = 0x3000 3 14 Read R KEY = 30, VA = 0x50000, length = 0x1000 4 23 Atomic R KEY = 50, VA = 0x60000, OP = Extract & Add

[0098] Referring now to Table 2, an exemplary hint message (or information carried by the hint packet) corresponding to Table 1, i.e., based on the work elements of Table 1, is shown. As shown, the hint message includes the following information, i.e., row 1 includes the detailed information of the queue pair number, and rows 2 and 3 include the memory details and operation indication (or type) required to complete the RDMA operation based on the work elements.

[0099] Table 2

[0100]

[0101] In another example, the hint message can be generated based on a single QP. Further, the single QP can be associated with multiple work elements. Referring now to Table 3, the data associated with a work element of the one or more work elements retrieved from memory 102, based on which a hint message is generated, is shown. As shown, column 1 of (Table 3) shows the number of work elements, column 2 shows the indication of at least one operation associated with the work elements, and column 3 shows the parameters or attributes associated with the work elements. In this example, the hint message is also associated with an operation, e.g., a prefetch operation for prefetching the memory region indicated. As shown, there are 6 work elements, and based on these work elements, a memory prefetch hint (e.g., when using RDMA on-demand-paging (ODP) method) is to be generated. Controller 104 knows the work element index in the SQ, e.g., QP #4. Thereafter, controller 104 generates the hint message (or hint packet) on the same QP (QP 4) that the data path is using and sends it to the responding device.

[0102] Table 3

[0103]

[0104] Referring now to Table 4, exemplary hint messages (i.e., information carried by the hint packets) corresponding to Table 3, i.e., based on the work elements of Table 3, are shown. As shown, the first and second rows of Table 4 include memory details and operation indication (or type) required to complete the RDMA operation based on the work elements.

[0105] Table 4

[0106]

[0107] The controller 104 is configured to send the hint messages to the responding devices through the communication interface 106, the responding devices corresponding to the responding party of at least one of the retrieved one or more work elements. As mentioned above, each work element indicates the responding party, and thus the responding device corresponds to the designated responding party of at least one of the retrieved one or more work elements. In other words, based on the responding party details carried by the retrieved one or more work elements, the controller 104 sends such work elements to the corresponding or designated responding party. For example, the responding party details can include the target memory address of the responding party or responding device, for supporting the sending of the work elements from the controller 104 accordingly.

[0108] In one embodiment, one or more of the retrieved one or more work elements are for a first responding party, and one or more of the retrieved one or more work elements are for a second responding party. Further, the controller 104 is further configured to generate the hint messages and send the hint messages to the corresponding responding devices by generating a first hint message and sending the first hint message to a first responding device corresponding to the first responding party, and generating a second hint message and sending the second hint message to a second responding device corresponding to the second responding party. Here, each of the first and second hint messages is sent to their respective responding devices (i.e., the first and second responding devices) before the actual sending of the one or more work elements to their respective responding parties based on the responding party details. In other words, a single requesting device (e.g., the requesting device 100) is declared to perform RDMA communication with multiple responding parties, i.e., by separating the retrieved work elements based on their responding party details.

[0109] In another embodiment, one or more of the retrieved one or more work elements are associated with a first requester (e.g., a requesting device 100), and one or more of the retrieved one or more work elements are associated with a second requester (e.g., another requesting device 100). In other words, multiple requesting devices (e.g., two or more requesting devices 100) are declared to RDMA communicate with a single or common responder. As described above, each work element indicates a requester, and thus, one or more of the retrieved one or more work elements can be separated based on the indication of the requester, which in turn depends on the number of requesters. Accordingly, the responding device receives a hint message from each of the first and second requesting devices before actually sending one or more work elements from the first and second requesting devices to the common responder. Similarly, multiple requesting devices, e.g., two or more requesting devices 100, are declared to RDMA communicate with multiple responders.

[0110] According to various embodiments of the present application, the hint message can be implemented (e.g., generated and sent) in various ways. In one example, the hint message can be implemented by a new header added to an existing data packet to transport the hint (i.e., the indication of the operation) instead of a new BTH opcode. Alternatively, the hint message can be implemented, which can be carried as an additional payload in an existing data packet. Further, an existing opcode can be used with a special flag (e.g., a '0-sized read / write) instead of a BTH opcode. Further, the hint message can be implemented by a new BTH opcode.

[0111] In one embodiment, the hint message can be explicitly activated at the application layer (e.g., a new InfiniBand Verb). The hint message can be negotiated between the requesting device 100 and the responder (i.e., the responder "requests" the use of the operation indication and its type according to its local state to the requester). For example, if the responder does not use non-pinned memory, then a prefetch hint can not be needed.

[0112] In another embodiment, the hint message generation can be controlled by the RNIC FW (RDMA Network Interface Card Firmware).

[0113] Optionally, the requesting device 100 can be used to send a hint message to all QPs using a special QP to the same responder. Alternatively, the requesting device 100 can use an unreliable datagram (UD) QP to multicast the hint message to all QPs, even if they are on different hosts on the responder. Other QP types include reliable connected (RC) and unreliable connected (UC). The UD QP can be limited to QPs in a given user application (for security and isolation considerations), or it can be a global per-device QP managed by a single trusted entity (e.g., a physical function).

[0114] According to one embodiment, the processing of the hint at the responder can be divided between: RDMA network interface card hardware (RNIC HW), RDMA network interface card firmware (RNIC FW), system memory management unit / input / output memory management unit (MMU / IOMMU) HW, device driver, OS kernel, or virtual machine hypervisor, such as a virtual machine / virtual machine manager (VM / VMM).

[0115] According to one embodiment, the controller 104 is further configured to send, through the communication interface 106, a request to the responding device related to the operation of at least one of the retrieved one or more work elements, the request indicating the operation. The request is sent to the responding device to indicate the operation to be performed and to solicit permission from the responding device to perform the operation. In other words, the request is a remote direct memory access (RDMA) access request from the requesting device 100 to the responding device for performing an RDMA operation related to (or associated with) the operation of at least one of the retrieved one or more work elements. It is to be understood that the controller 104 is configured to initially send a hint message to the responding device and then send the request. Thus, the responding device initially updates the possible operations that the responding device needs to perform based on the “hint message”, which is the indication of the operation of at least one of the retrieved one or more work elements. This allows the responding device to order the corresponding or complementary operations required at the responding device end. Thereafter, the controller 104 sends the “request” to seek permission from the responding device to perform the possible requested operation based on the work element.

[0116] In another embodiment, the request relates to a storage operation. The requesting device 100 sends a request related to a storage operation of one or more work elements 108 stored in one or more request queues. In other words, the requesting device 100 seeks permission from the responding device 100 to further enable the responding device to perform at least one operation by sending to store one or more work elements 108 in the responding device (or cache memory) by storing the one or more work elements 108 in the responding device (or cache memory).

[0117] In yet another embodiment, the request relates to an artificial intelligence operation. The request relates to artificial intelligence operations, including but not limited to machine learning applications based on pre-trained and customized deep neural networks, which require complex computations, fast and efficient data transfer. In one example, the request is for intelligent offload through RDMA operations and appropriate hardware and in-network computing capabilities to significantly improve neural network training performance and overall machine learning applications. In another example, InfiniBand and Ethernet (e.g. RoCE) network schemes can be used for complete, high-performance artificial intelligence infrastructure.

[0118] In another embodiment, the requesting device 100 is arranged for high-performance computing (HPC). In other words, the requesting device 100 is configured to execute software applications and operations using HPC techniques. The requesting device 100 provides users with HPC functionality that is dynamically allocated among various computing devices (e.g., responding devices) with I / O performance substantially similar to processing performance. The requesting device 100 can be a standalone computing environment or any other suitable environment. The requesting device 100 can be arranged for any HPC computing environment that includes high computing nodes and supports performing operations on work elements or data packets. As used herein, the term "dynamic" generally indicates that certain processing is determined, at least in part, at runtime based on one or more variables. As used herein, the term "automatically" generally indicates that appropriate processing is substantially performed by at least a portion of the requesting device 100. It should be appreciated that "automatically" further contemplates any suitable device having the requesting device 100 without departing from the scope of the present application.

[0119] In another embodiment, the request sent by the requesting device 100 is related to a remote procedure call (RPC), i.e., when the requesting device 100 causes an operation or procedure (or subroutine) to be executed in a different address space (or target memory address) without the need for explicit coding of the details of the remote interaction. This is a form of client-server interaction (the requester is the client and the performer or responder is the server), typically implemented through a request-response messaging system. It should be noted that the RPC model implies a level of location transparency, i.e., the calling procedure can be different or the same, enabling a distinction between local or remote calls, which are typically orders of magnitude slower and less reliable than local calls, making said distinction important. RPC is a form of inter-process communication (IPC) in which different operations are related to different address spaces. In one example, the operations require different virtual address spaces if they exist on the same host (e.g., the requesting device 100), even if the physical address space is the same; in another example, the physical address space is different if they exist on different hosts or host machines.

[0120] In one embodiment, the controller 104 is further configured to receive an acknowledgement message from the responding device prior to sending the request related to the operation. The responding device provides the acknowledgement message as a response (or reply) to the hint message to the requesting device 100. Typically, the responding device is configured to provide the acknowledgement message (or reply message) indicating that the hint message has been processed and the requesting device 100 is ready and capable of performing the requested operation indicated by the hint message. It is to be noted that the reply message can be an acknowledgement (ACK) message indicating that the hint message has been successfully processed or a negative acknowledgement (NAK) message indicating a rejection related to the processing of the hint message or indicating some error or incapability of the responding device.

[0121] Optionally, the requesting device 100 is configured to wait for a predefined amount of time in accordance with a protocol between the requesting device 100 and the responding device. The predefined amount of time depends on the hint message, the type of the hint message, the capability of the responding device, etc. Programming the predefined amount (or time interval) between the transmission of various work elements enables reliable data communication and reduces latency, which in turn speeds up the data communication of the RDMA operation. The requesting device 100 of the present application further comprises a software module 110. The software module 110 can be stored in the memory 102 of the requesting device 100. The one or more software modules 110 include a work element storage module 112, a work element retrieval module 114, a hint message generation module 116, a hint message transmission module 118, and a request transmission module 120. The work element storage module 112 is configured to store the plurality of work elements 108 in one or more request queues in the memory 102. The work element retrieval module 114 is configured to retrieve at least one of the work elements from the memory 102. The hint message generation module 116 is configured to generate a hint message including an indication of at least one operation of a corresponding at least one of the one or more retrieved work elements. The hint message transmission module 118 is configured to send the hint message to a responding device corresponding to the at least one of the one or more retrieved work elements via the communication interface 106. The request transmission module 120 is configured to send a request related to the operation of the at least one of the one or more retrieved work elements to the responding device via the communication interface 106, the request indicating the operation.

[0122] Reference is now made to Figure 2 , which shows a flowchart of a method 200 for performing an RDMA operation by the requesting device 100 according to an embodiment of the present application. Figure 2 It is to be understood in conjunction with Figure 1 . For example, the method 200 can be performed by the controller 104 of the requesting device 100 in conjunction with Figure 1shown and explained. As shown, the method 200 for a requesting device (e.g., the requesting device 100) includes steps 202-210.

[0123] In step 202, the method 200 includes storing a plurality of work elements in one or more request queues in a memory (e.g., the memory 102) of the requesting device 100. Each work element indicates a requestor (e.g., the requesting device 100), a responder (or responding device), and an operation. The plurality of work elements are stored by a controller (e.g., the controller 104) in the memory 102 of the requesting device 100 for further implementation. Figure 1

[0124] In step 204, the method 200 includes retrieving at least one of the work elements from the memory 102. The at least one of the plurality of work elements 108 is retrieved by the controller 104 of the requesting device 100.

[0125] In step 206, the method 200 includes generating a hint message including an indication of at least one operation of a corresponding at least one of the one or more retrieved work elements. The hint message is generated by the controller 104 of the requesting device 100 to initiate a remote direct memory access operation with another computing device (i.e., the responding device). The hint message indicates the operation (or operation type) associated with the at least one work element, such as a read operation, a write operation, a send operation, or an atomic operation.

[0126] In step 208, the method 200 includes sending the hint message to the responding device through the communication interface 106. The responding device corresponds to the responder of the at least one of the one or more retrieved work elements. The hint message is sent by the controller 104 of the requesting device 100 to the responding device through the communication interface 106. The responding device performs the operation according to the hint message.

[0127] In step 210, the method 200 includes sending a request related to the operation of the at least one of the one or more retrieved work elements to the responding device through the communication interface 106, the request indicating the operation. The request is sent by the controller 104 of the requesting device 100 to the responding device through the communication interface 106.

[0128] Steps 202 and 210 are merely illustrative, and other alternatives can be provided where one or more steps are added, one or more steps are deleted, or one or more steps are provided in an order different from that described herein without departing from the scope of the claims herein.

[0129] ​The present application also provides a computer program product including a non-transitory computer-readable storage medium storing computer program code executable by a processor to perform the method 200. Generally, the method 200 is used by a requesting device (e.g., in conjunction with the Figure 1 The requesting device 100 shown and explained is in communication with a responding device to perform RDMA operations based on a hint message. Examples of implementation of the non-transitory computer-readable storage medium include, but are not limited to, electrically erasable programmable read-only memory (EEPROM), random access memory (RAM), read only memory (ROM), hard disk drive (HDD), flash memory, secure digital (SD) card, solid-state drive (SSD), computer-readable storage medium, and / or CPU cache memory. The computer-readable storage medium used to provide the non-transitory memory can include, but is not limited to, electronic, magnetic, optical, electromagnetic, semiconductor, or any suitable combination of the foregoing.

[0130] Reference is now made to Figure 3 , which shows a block diagram of a responding device 300 that can be used to perform RDMA operations in accordance with embodiments of the present application. The responding device 300 is to be understood in conjunction with the requesting device 100 of Figure 1 Generally, the responding device 300 is in communication with the requesting device 100 to perform RDMA operations based on a hint message.

[0131] As shown, the responding device 300 includes a memory 302, a controller 304, and a communication interface 306. The memory 302 and the communication interface 306 are similar to the memory 102 and the communication interface 106 of the requesting device 100 of Figure 1 , and therefore, for the sake of brevity, are not explained.

[0132] The controller 304 includes appropriate logic, circuitry, and / or interfaces for receiving a hint message from a requesting device (e.g., the requesting device 100) through the communication interface 306, the hint message indicating at least one operation. The controller 304 is a computing element for processing instructions to drive the responding device 300. Examples of the controller 304 include, but are not limited to, a network interface controller, a microprocessor, a microcontroller, a complex instruction set computing (CISC) microprocessor, a reduced instruction set (RISC) microprocessor, or a very long instruction word (VLIW) microprocessor. Optionally, one or more controllers, processing devices, and elements similar to the controller 304 can be arranged in various architectures for responding and processing instructions to drive the responding device 300.

[0133] In operation, the controller 304 is configured to receive a hint message from the requesting device 100 through the communication interface 306. The hint message includes an indication of at least one operation. The hint message includes an indication of at least one operation, such as a read operation, a write operation, a send operation, or an atomic operation. The hint message is sent in one or more data packets. The data packets of the hint message include a packet sequence number (PSN) to enforce a correct order of the hint message. The hint message also includes a source and destination queue pair (QP) number. The hint message also includes a memory address for read, write, and atomic operations. The first message also includes an operation type and operand data.

[0134] The controller 304 is also configured to determine a preemption action based on the indication of the at least one operation. The hint message includes an indication of at least one operation, based on which the controller 304 is configured to determine a preemption action (or a preparation action) to be followed by the responding device 300. The preemption action is an operation performed by the responding device 300 to prepare a memory region (e.g., a local memory buffer) before receiving a request for a remote direct memory access operation. The preemption action includes at least one of a prefetch operation, a cache warm-up operation, an atomic operation. The preemption action corresponds to the hint message (sent by the requesting device 100) received by the responding device 300. For example, the hint message can be one of a prefetch hint message, a cache warm-up hint message, an atomic hint message, described above in connection with FIG. 2. Figure 1The request device 100 is explained herein. For example, when using a non- fixed remote direct memory access system, the request device 100 sends a pre-fetch hint message to the response device 300 as a hint message. The hint message (or pre-fetch hint) gives the response device 300 time to prepare its memory region (i.e., local memory buffer) before the actual request for the remote direct memory access operation to the memory region (i.e., local memory buffer) arrives.

[0135] The controller 304 is further configured to perform a pre-emptive action comprising at least one of a pre-fetch operation, a cache warm-up operation, an atomic operation. The controller 104 is configured to perform the pre-emptive action to prepare the memory region (e.g., local memory buffer) based on the hint message prior to receiving the request for the remote direct memory access operation.

[0136] The controller 304 is further configured to receive a request from the request device 100 related to the at least one operation, the request indicating at least one of the at least one operation. Typically, the request is a remote direct memory access (RDMA) access request from the request device 100 to the response device 300 for performing the RDMA operation. The request is sent by the request device 100 to the response device 300 for seeking permission from the response device 300 to perform the operation based on the hint message.

[0137] The controller 304 is further configured to perform at least one of the at least one operation. The at least one of the at least one operation is selected from operations including, but not limited to, a RDMA read operation, a RDMA write operation, a send operation, or an atomic operation. Typically, the at least one of the at least one operation is based on the indication of one or more operations of the hint message.

[0138] In one embodiment, the operation is related to a memory region, and wherein the controller 304 is further configured to determine the pre-emptive action based on the memory region and the operation. The hint message includes an indication of the at least one operation and a memory region related to the at least one operation to determine the pre-emptive action to be performed prior to performing the operation. The controller 304 determines the pre-emptive action based on the memory region and the operation, the pre-emptive action comprising at least one of a pre-fetch operation, a cache warm-up operation, an atomic operation, to prepare the memory region (e.g., local memory buffer) prior to receiving the request for the remote direct memory access operation. Further, the controller 304 is configured to perform the pre-emptive action by determining whether the memory region needs to be loaded into memory, and if so, loading the memory region into memory. The controller 304 determines whether the memory region related to the operation needs to be loaded (or populated), and if there is a requirement to perform the pre-emptive action, loads the memory region.

[0139] In another embodiment, the controller 304 is further configured to send an acknowledgement message to the requesting device 100. The acknowledgement message indicates whether the pre-emption action was successful. The responding device 300 is configured to provide an acknowledgement message (or a reply message) indicating whether the pre-emption action was successful. Thus, whether the responding device 300 is ready to receive the incoming data packet from the requesting device 100 (i.e., allow the successful transmission of the data packet from the requesting device 100 to the responding device 300). The responding device 300 provides the acknowledgement message to the requesting device 100 as a response (or reply) to the prompt message indicating the status of the pre-emption action. It is noted that the acknowledgement message can be an acknowledgement (ACK) message indicating that the pre-emption action was successfully performed and the memory region is ready, or a negative acknowledgement (NAK) message indicating that the pre-emption action was not successfully performed and the memory region is not ready to receive the incoming data packet.

[0140] In operation (or at implementation level), the responding device 300 is configured to perform a plurality of actions upon receiving the prompt message. Generally, the actions include, but are not limited to, determining the relevant QP number (from the data packet header) and ensuring the QP context is in the RNIC cache, determining the relevant DMA target and issuing a DMA request to the memory 302, and performing an atomic operation. Further, based on the present application, the responding device 300 upon receiving the prompt message performs the following actions: stores the requesting context of the QP into the cache of the responding device 300, immediately sends a memory page request for each range with appropriate attributes (without waiting for the requesting data packet to arrive from the requesting device 100), fetches the atomic operation raw data from the memory 302, and optionally sends a reply (or acknowledgement) that the prompt is processed. The acknowledgement message received from the responding device enables the requesting device 100 to adapt or configure the transmission of the data packet accordingly, thereby facilitating a glitch-free operation. Generally, the acknowledgement message means that the first message was successfully sent to the receiving device. Thus, the controller 304 upon receiving the acknowledgement message sends the request related to the operation irrespective of any external factors. Thus, the reply message shortens the total execution time of the data transmission and reduces the latency. Reducing the latency indicates a higher speed of data communication.

[0141] The responding device 300 of the present application further includes a software module 310. The software module 310 can be stored in the memory 302 of the responding device 300. The software module 310 includes a prompt message receiving module 312, a pre-emption action determining module 314, a pre-emption action performing module 316, a request receiving module 318, and an operation performing module 320.

[0142] The prompt message receiving module 312 is configured to receive a prompt message from the requesting device 100, the prompt message comprising an indication of at least one operation. The prompt message receiving module receives the prompt message from the requesting device 100 to further determine a pre-emptive action with information from the prompt message. The pre-emptive action determining module 314 is configured to determine a pre-emptive action based on the indication of at least one operation. Upon receiving the prompt message, the pre-emptive action determining module determines the pre-emptive action based on the information provided by the prompt message. The pre-emptive action performing module 316 is configured to perform the pre-emptive action and prepare a memory region associated with the at least one operation by means of the pre-emptive action comprising but not limited to a pre-fetch operation, a cache warm-up, etc. The request receiving module 318 is configured to receive a request from the requesting device 100 related to the at least one operation, the request indicating at least one of the at least one operation. Upon determining the pre-emptive action and preparing the memory region associated with the at least one operation, the request receiving module receives the request from the requesting device 100 related to the at least one operation, in other words, the request accesses the associated memory region to perform the at least one operation. The operation performing module 320 is configured to perform at least one of the at least one operation. Upon receiving the request from the requesting device 100, the operation performing module is configured to perform at least one of the at least one operation on the associated memory region.

[0143] Reference will now be made to Figure 4 , which shows a flowchart of a method 400 for a responding device to perform a RDMA operation, provided by embodiments of the present application. Figure 4 Should be understood in conjunction with Figures 1 to 3 For example, the method 400 can be performed by the controller 304 of the responding device 300, which is shown and explained in conjunction with Figure 3 Furthermore, the responding device 300 communicates with the requesting device 100, which is shown and explained in conjunction with Figure 1 and Figure 2 As shown, the method 400 for the responding device comprises steps 402-410.

[0144] In step 402, the method 400 comprises receiving a prompt message, the prompt message comprising an indication of at least one operation. The prompt message indicates the operation (or operation type), such as a read operation, a write operation, a send operation or an atomic operation.

[0145] In step 404, the method 400 comprises determining a pre-emptive action based on the indication of at least one operation. Generally, the prompt message comprises an indication of at least one operation, based on which the controller 304 is configured to determine a pre-emptive action to be followed by the responding device 300. The pre-emptive action is an operation performed by the responding device 300 to prepare a memory region (e.g. a local memory buffer) prior to receiving a request of a remote direct memory access operation. The pre-emptive action comprises at least one of a pre-fetch operation, a cache warm-up operation, an atomic operation.

[0146] In step 406, the method 400 includes performing a pre-emptive action using the controller 304 of the responding device 300. Upon determining the pre-emptive action, the controller is configured to perform the pre-emptive action to prepare the memory region (e.g. local memory buffer) prior to receiving a request for a remote direct memory access operation.

[0147] In step 408, the method 400 includes receiving a request from the requesting device 100 related to at least one operation, the request indicating at least one of the at least one operation. The request is sent by the requesting device 100 to the responding device 300 via the communication interface 306 to indicate the at least one operation to be performed and to seek permission from the responding device 300 to perform at least one of the at least one operation. In one example, the request is a remote direct memory access (RDMA) access request from the requesting device 100 to the responding device 300 for performing one or more RDMA operations.

[0148] In step 410, the method 400 includes performing at least one of the at least one operation. At least one of the at least one operation is selected from a plurality of operations including but not limited to a RDMA read operation, a RDMA write operation, a send operation or an atomic operation and is performed by the controller of the responding device 300.

[0149] Steps 402 and 410 are merely illustrative and other alternatives can be provided where one or more steps are added, one or more steps are removed, or one or more steps are provided in a different sequence without departing from the scope of the claims herein.

[0150] The present application also provides a computer program product comprising a non-transitory computer readable storage medium storing computer program code executable by a processor to perform the method 400. Generally, the method 400 is configured to be performed by a responding device (e.g. in conjunction with the controller 304 of the responding device 300) to perform a pre-emptive action prior to receiving a request for a remote direct memory access operation. Figure 3The illustrated and explained responding device 300) communicates with the requesting device 100 to perform RDMA operations based on the hint message. Implementation examples of the non-transitory computer-readable storage medium include, but are not limited to, electrically erasable programmable read-only memory (EEPROM), random access memory (RAM), read only memory (ROM), hard disk drive (HDD), flash memory, secure digital (SD) card, solid-state drive (SSD), computer-readable storage medium, and / or CPU cache memory. The computer-readable storage medium for providing the non-transitory memory can include, but is not limited to, an electronic storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any suitable combination of the foregoing.

[0151] Reference is now made to Figure 5 , which shows a block diagram of an RDMA system 500 having a requesting device 100 and a responding device 300, as provided by embodiments of the present application. Figure 5 Should be understood in conjunction with Figures 1 to 4 . In particular, the RDMA system 500 includes a requesting device, such as the requesting device 100 (shown and explained in conjunction with Figure 1 and Figure 2 ), and a responding device, such as the responding device 300 (shown and explained in conjunction with Figure 3 and Figure 4 ). As shown, the system 500 includes the requesting device 100 and the responding device 300 communicably coupled to the requesting device 100 through a communication network 502.

[0152] In one embodiment, the communication network 502 can be a computer cluster, a local area communication channel (LAN), a cellular communication channel, a wireless sensor communication channel (WSN), a cloud communication channel, a vehicle-to-communication channel (V2N) communication channel, a metropolitan area communication channel (MAN), and / or the Internet. Further, the communication network 502 should be compliant with various communication channel protocols that support RDMA communications. Examples of such communication channel protocols, communication standards, and technologies can include, but are not limited to, InfiniBand (IB), RDMA over converged Ethernet (RoCE), internet wide area RDMA protocol (iWARP), or modifications and variations thereof.

[0153] In the RDMA system 500, the requesting device 100 can directly access the memory of the responding device 300 without involving the operating system (OS) of the requesting device 100 and the responding device 300. Further, the requesting device 100 communicates (exchanges data) with the responding device 300 based on a hint message, which can include at least one of a memory prefetch hint, a cache warm-up hint, and an atomic operation hint. The RDMA system 500 does not support sharing of the requesting device's 100 knowledge about the RDMA target with the responding device 300. Generally, in the RDMA system 500, multiple hints are sent from the requesting device 100 at a time to enable the responding device 300 to optimize various aspects, such as: memory prefetch, cache warm-up, offloading multiple atomic operations to hardware, and the like. Further, in the remote direct memory access (RDMA) system 500, data is transferred directly without copying between the communication channel layer (or network layer) of the communication interfaces 106, 306 in the requesting device 100 and the responding device 300. For example, in the remote direct memory access system 500, data is transferred in the form of one or more packets. The packets include information related to a packet sequence number (PSN) to enforce proper packet ordering. The packets also include source and destination queue pair (QP) numbers, which distinguish the application and the memory region of the application. In one embodiment, the memory region has a memory key (R-key) as part of the packet, which associates the memory region with the application, and vice versa. The packets also include information related to the message type (e.g., RDMA read, RDMA write, send, or atomic), and parameters of the message, such as message length, target memory address, operation type, and operand data.

[0154] Reference is now made to Figure 6 , which shows a flowchart of a method 600 for performing RDMA operations by a system including a responding device and a requesting device, according to embodiments of the application. Figure 6 It should be appreciated that Figure 5 , Figure 5 relates to an RDMA system 500 having a responding device and a requesting device. Further, Figure 5 It should be appreciated that Figures 1 to 4 elements. Specifically, the RDMA system 500 includes a requesting device, such as the requesting device 100 (shown and explained in connection with Figure 1 and Figure 2 ), and a responding device, such as the requesting device 300 (shown and explained in connection with Figure 3 and Figure 4 ). The method 600 includes steps 602-614.

[0155] In step 602, the method 600 includes requesting the device 100 to store a plurality of work elements in one or more request queues in the memory 102. Each work element indicates a requester, a responder, and an operation.

[0156] In step 604, the method 600 includes requesting the device 100 to retrieve at least one of the work elements. The at least one of the work elements is retrieved by the controller 104 from the memory 102 of the requesting device 100 from the plurality of work elements.

[0157] In step 606, the method 600 includes generating a hint message including an indication of at least one operation of a corresponding at least one of the one or more retrieved work elements. The hint message is generated by the controller 104 of the requesting device 100 to initiate a remote direct memory access operation with the responding device 300. The hint message indicates the operation (or operation type) associated with the at least one work element, such as a read operation, a write operation, a send operation, or an atomic operation.

[0158] In step 608, the method 600 includes requesting the device 100 to send the hint message to the responding device 300 through the communication interface 306. The responding device 300 corresponds to the responder of the at least one of the one or more retrieved work elements. The hint message is sent by the controller 104 of the requesting device 100 to the responding device 300 through the communication interface 106.

[0159] In step 610, the method 600 includes the responding device receiving the hint message from the requesting device 100 including the indication of the at least one operation. The controller 304 of the responding device 300 is used to receive the hint message. The hint message includes the indication of the at least one operation, such as a read operation, a write operation, a send operation, or an atomic operation. The hint message is sent in the form of one or more data packets. The data packets of the hint message include a packet sequence number (PSN) to enforce correct order of the hint message. The hint message also includes a source and destination queue pair (QP) number. The hint message also includes a memory address for read, write, and atomic operations. The hint message also includes operand data related to one or more objects to perform the operation.

[0160] In step 612, the method 600 includes determining, by the responding device 300, a preemption action based on the indication of the at least one operation and performing the preemption action. The controller 304 of the responding device 300 is configured to determine the preemption action based on the indication of the at least one operation of the hint message. The preemption action is an operation performed by the responding device 300 to prepare the memory region (e.g., local memory buffer) prior to receiving the request for the remote direct memory access operation. The preemption action includes at least one of a prefetch operation, a cache warm-up operation, and an atomic operation.

[0161] In step 614, the method 600 includes sending, by the requesting device 100, a request to the responding device 300 related to the operation. The request indicates the operation and the memory region.

[0162] In step 616, the method 600 includes receiving, by the responding device 300, the request from the requesting device 100 related to the at least one operation, the request indicating at least one of the at least one operation and performing the operation. The request is sent by the requesting device 100 to the responding device 300 to indicate the at least one operation to be performed and to seek permission from the responding device 300 to perform the operation. The request also includes performing the operation.

[0163] Reference is now made to Figures 7 to 9 , which shows a schematic diagram of an exemplary scenario describing implementations of the RDMA systems 700, 800, and 900, in accordance with various embodiments of the present application. The RDMA systems 700, 800, 900 are similar to the RDMA system 500 shown and explained in connection with Figure 5 . Furthermore, Figures 7 to 9 the RDMA systems 700, 800, 900 should also be understood in connection with the description of the requesting device 100 and the responding device 300. Figures 1 to 4 As shown, the RDMA system 700 includes a requesting device, such as the requesting device 100 shown and explained in connection with Figure 1 and Figure 2 . The RDMA system 700 also includes two responding devices, such as the responding device 300 shown and explained in connection with Figure 3 and Figure 4 . Similarly, each of the RDMA systems 800, 900 includes a single requesting device (e.g., the requesting device 100) and a pair of responding devices (e.g., the responding device 300).

[0164] In one embodiment, as Figure 7As shown, in operation, the requesting device 100 communicates with multiple responding devices (e.g., responding device 300) via multiple messages. As shown, the requesting device 100 sends a first message 702 (e.g., QP 11) and a second message 704 (e.g., QP 21) to one of the two responding devices 300 (e.g., a first responding device). Similarly, the requesting device 100 sends a third message 706 (e.g., QP 12) and a fourth message 708 (e.g., QP 22) to another of the two responding devices 300 (e.g., a second responding device). In general, each of the first message 702, the second message 704, the third message 706, and the fourth message 708 is associated with one or more queue pairs, which at the element level include work elements that make up an RDMA packet. In this embodiment, the requesting device 100 is configured and operable to send the first message 702, the second message 704, the third message 706, and the fourth message 708 in a manner such that each message carries both data and a hint. In other words, the RDMA system 700 describes an exemplary scenario in which multiple QPs are sent by the requesting device 100 to multiple responding devices 300, and the hint for each QP is sent on the same QP.

[0165] In another embodiment, as Figure 8 shown, in operation, the requesting device 100 communicates with the responding device 300 via multiple messages. As shown, the requesting device 100 sends a first message 802 (e.g., QP 21), a second message 804 (e.g., QP 21), and a special message 806 (e.g., QP 101) to one of the two responding devices 300 (e.g., a first responding device). Similarly, the requesting device 100 sends a third message 812 (e.g., QP 12), a fourth message 814 (e.g., QP 22), and a special message 816 (e.g., QP 102) to another of the two responding devices 300 (e.g., a second responding device). In this embodiment, the requesting device 100 is configured and operable to send the first message 802, the second message 804, the third message 812, and the fourth message 814 in a manner such that each message carries only data. In addition, the requesting device 100 is configured and operable to send the special messages 806, 816 carrying aggregated hints. For example, the special message 806 carries aggregated hints for the first message 802 and the second message 804, and similarly, the special message 816 carries aggregated hints for the third message 812 and the fourth message 814. In other words, the RDMA system 800 describes an exemplary scenario in which a special management QP is used for each pair of requesting device and responding device to aggregate hints from all QPs between each pair of requesting device and responding device.

[0166] In yet another embodiment, asFigure 9 As shown, in operation, the requesting device 100 communicates with the responding devices 300 through a plurality of messages. As shown, the requesting device 100 sends a first message 902 (e.g., QP 11), a second message 904 (e.g., QP 21), and a special multicast message 910 (e.g., QP 200) to one of the two responding devices 300 (e.g., the first responding device). Similarly, the requesting device 100 sends a third message 912 (e.g., QP 12), a fourth message 914 (e.g., QP 22), and the same special multicast message 910 (e.g., QP 200) to the other of the two responding devices 300 (e.g., the second responding device). In this embodiment, the requesting device 100 is configured and operable to send the first message 902, the second message 904, the third message 912, and the fourth message 914 in a manner such that each message carries data only. Further, the requesting device 100 is configured and operable to send the special multicast message 910 carrying all the hints. For example, the special multicast message 910 carries the hints of the first message 902, the second message 904, the third message 912, and the fourth message 914. In other words, the RDMA system 900 describes an exemplary scenario in which a special "multicast" UD QP is used to aggregate hints from all QPs and send them to all responding devices 300. Further, each of the responding devices 300 infers which hints are relevant to it based on its active QPs with the requesting device 100.

[0167] Modifications can be made to the embodiments of the application described above without departing from the scope of the application as defined in the appended claims. Where, in the foregoing, descriptions of embodiments of the present application have used terminology such as "comprising", "adjoining", "having", "is / are", or the like, such terminology is used in the sense of allowing for items, components or elements not explicitly described or claimed to also be present. Reference to singular is also to be interpreted as a reference to plural. The word "exemplary" is used herein to mean "serving as an example, instance, or illustration." Any implementation described as "exemplary" is not necessarily to be construed as preferred or advantageous over other implementations. The word "may" is used herein to mean "can" in some embodiments and "shall" in other embodiments. The use herein of the word "optionally" means "in some embodiments providing and in other embodiments not providing." It is to be understood that certain features of the application that are described in the context of separate embodiments can also be provided in combination with each other. Conversely, various features of the application that are described in the context of a single embodiment can also be provided separately or in any appropriate sub-combination.

Claims

1. A requesting device (100), characterized in that, The requesting device (100) includes a memory (102), a controller (104), and a communication interface (106). The memory (102) is used to store multiple work elements (108) in one or more request queues, each work element indicating a requester, a responder, and an operation. The controller (104) is used to: Retrieve one or more working elements from the memory (102); Generate a prompt message, the prompt message including an indication of at least one operation corresponding to at least one of the retrieved work elements; The prompt message is sent to the response device (300) via the communication interface (106), the response device (300) corresponding to the responder of at least one of the retrieved work elements; The communication interface (106) sends a request to the response device (300) relating to the operation of at least one of the retrieved work elements, the request indicating the operation.

2. The requesting device (100) according to claim 1, characterized in that, The working element also includes a list of objects, which includes one or more objects to perform the operation, and wherein the prompt message also includes an indication of at least one of the one or more objects.

3. The requesting device (100) according to claim 1 or 2, characterized in that, One or more of the retrieved working elements are used by the first responder. One or more of the retrieved working elements are used for the second responder, and the controller (104) is further configured to generate the prompt message and send the prompt message to the responding device in the following manner: Generate a first prompt message and send the first prompt message to the first response device (300) corresponding to the first responder; A second prompt message is generated and sent to the second response device (300) corresponding to the second responder.

4. The requesting device (100) according to claim 1 or 2, characterized in that, One or more of the retrieved working elements are related to the first requester. One or more of the retrieved working elements are related to the second requester.

5. The requesting device (100) according to claim 1 or 2, characterized in that, The controller (104) is also configured to receive an acknowledgment message from the response device (300) before sending a request related to the operation.

6. The requesting device (100) according to claim 1 or 2, characterized in that, The controller (104) is also used to retrieve batch work elements and change the size of the batch during operation.

7. The requesting device (100) according to claim 1, characterized in that, The notification message also includes an indication of the memory region associated with the operation.

8. The requesting device (100) according to claim 7, characterized in that, The memory region and the indication of the operation include the address of the memory region to be performed.

9. The requesting device (100) according to claim 7 or 8, characterized in that, The memory region and the indication of the operation include a prefetch operation for prefetching the memory region of the indication.

10. The requesting device (100) according to claim 7 or 8, characterized in that, The memory region and the indication of the operation include a cache preheating operation.

11. The requesting device (100) according to claim 7 or 8, characterized in that, The memory region and the indication of the operation include atomic operations.

12. The requesting device (100) according to claim 1, 2, 7 or 8, characterized in that, The prompt message is a dedicated prompt command message.

13. The requesting device (100) according to claim 1, 2, 7 or 8, characterized in that, The prompt message includes the indication of the operation as an additional payload of a command message for another command.

14. The requesting device (100) according to claim 1, 2, 7 or 8, characterized in that, The requesting device is configured for remote direct memory access.

15. A requesting device (100), characterized in that, The requesting device (100) includes a memory (102) and a communication interface (106), as well as a software module (110), the software module (110) including: The work element storage module (112) is used to store multiple work elements in one or more request queues in the memory, each work element indicating a requester, a responder, and an operation; A working element retrieval module (114) is used to retrieve at least one working element from the memory among the working elements; The prompt message generation module (116) is used to generate a prompt message, the prompt message including an indication of at least one operation of at least one working element corresponding to one or more of the retrieved working elements; A prompt message transmission module (118) is used to send the prompt message to a response device (300) through the communication interface (106), the response device (300) corresponding to the responder of at least one of the retrieved work elements; The request transmission module (120) is configured to send a request related to the operation of at least one of the retrieved work elements to the response device (300) via the communication interface (106), the request indicating the operation.

16. A method (200) for requesting a device (100), characterized in that, The method (200) includes: Multiple work elements (108) are stored in one or more request queues in memory (102), each work element indicating the requester, the responder, and the operation; Retrieve at least one working element from the working elements from the memory (102); Generate a prompt message, the prompt message including an indication of at least one operation corresponding to at least one of the retrieved work elements; The prompt message is sent to the response device (300) via the communication interface (106), the response device (300) corresponding to the responder of at least one of the retrieved work elements; The communication interface (106) sends a request to the response device (300) relating to the operation of at least one of the retrieved work elements, the request indicating the operation.

17. A computer-readable medium, characterized in that, When computer instructions are loaded into and executed by the controller (104) of the requesting device (100), the requesting device (100) is enabled to perform the method (200) according to claim 16.

18. A response device (300), characterized in that, The response device (300) includes a memory (302), a controller (304), and a communication interface (306), wherein the controller (304) is used for: The system receives a prompt message from the requesting device (100) via the communication interface (306), the prompt message including an instruction for at least one operation; The preemption action is determined based on the instruction of the at least one operation; Execute the preemption action; The request related to the at least one operation is received from the requesting device (100) through the communication interface (306), the request indicating at least one of the at least one operations; Perform at least one of the at least one operations.

19. The response device (300) according to claim 18, characterized in that, The operation is related to a memory region, wherein the controller (304) is further configured to determine the preemption action based on the memory region and the operation, and to execute the preemption action by determining whether the memory region needs to be loaded, and if so, to load the memory region into the memory (302).

20. The response device (300) according to claim 18 or 19, characterized in that, The controller (304) is also configured to send a confirmation message to the requesting device (100), the confirmation message indicating whether the preemption action was successful.

21. A method (400) for responding to a device (300), characterized in that, The method (400) includes: Receive a prompt message from the requesting device (100), the prompt message including an indication of at least one operation; The preemption action is determined based on the instruction of the at least one operation; Execute the preemption action; The request is received from the requesting device (100) in relation to the at least one operation, the request indicating at least one of the at least one operations; Perform at least one of the at least one operations.

22. A computer-readable medium, characterized in that, When computer instructions are loaded into and executed by the controller (304) of the response device (300), the response device (300) is enabled to perform the method according to claim 21.

23. A response device (300), characterized in that, The response device (300) includes a memory (302), a communication interface (306), and a software module (310), the software module (310) including: A prompt message receiving module (312) is configured to receive a prompt message from a requesting device (100), the prompt message including an indication of at least one operation; A preemption action determination module (314) is used to determine a preemption action based on the indication of the at least one operation; The preemption action execution module (316) is used to execute the preemption action; A request receiving module (318) is configured to receive from the requesting device (100) a request related to the at least one operation, the request indicating at least one of the at least one operations; Operation execution module (320) is used to execute at least one of the at least one operations.

Images