Reducing transactional aborts in a remote direct memory access system

Abstract

To reduce RDMA request drops in a remote direct memory access (RDMA) system, a requesting device sends a message including a prefetch operation to a responding device. The prefetch operation indicates a memory region to be loaded by the responding device into a memory of the responding device before a new RDMA request or RDMA command is received.

Description

Technical Field

[0001] This invention generally relates to the field of data communication and remote direct memory access (RDMA) systems; more specifically, it relates to requesting devices, responding devices, and methods for reducing transaction dropping in remote direct memory access systems. Background Technology

[0002] High-speed data communication between multiple computing devices in a network has raised concerns about the reliability and efficiency of data communication. Traditionally, remote direct memory access (RDMA) technology (or protocols) are used, in which hardware devices such as traditional requesting devices (or clients) directly access the memory of traditional responding devices (or servers). Traditional RDMA technology requires fixing (meaning it always exists in physical random access memory (RAM)) virtual memory (i.e., a portion of the traditional responding device's memory) for any RDMA transaction (e.g., local and remote RDMA transactions) when the traditional requesting device attempts a direct memory access (DMA) request (e.g., an RDMA read request or an RDMA write request). Therefore, in traditional RDMA technology, general methods such as memory fixing are used to ensure that the traditional requesting device, which directly accesses memory (e.g., physical memory), does not have to deal with the mapping of pages that can be changed or completely swapped out by other entities (e.g., a typical operating system). Memory pinning is a technique typically used by operating systems to prevent the memory subsystem from moving physical pages from one location to another, potentially altering the translation from virtual address (VA) to physical address (PA). Memory pinning also prevents memory pages from the memory subsystem from moving from random access memory (RAM) to backup memory or the swap space (a process known as "swapout"). For example, in one scenario, pinned memory (i.e., virtual memory or the memory subsystem that cannot be swapped out or reclaimed by a typical operating system) is used by traditional requesting devices for direct memory access (DMA) because most traditional hardware devices cannot handle page faults when attempting to access the memory subsystem. Therefore, pinning memory for DMA is often standard practice in traditional data communication applications. Pinned memory constitutes a significant portion of total memory and is very expensive (estimated at 40% of the total cost of a traditional response device (or server)). However, memory pinning negatively impacts memory utilization because it cannot be swapped out and limits the operating system's ability to over-provision memory, which in turn reduces the total memory available to other applications. Total memory corresponds to physical memory and virtual memory. Furthermore, fixed memory can only be used by a single guest operating system at a time, thus reducing the total memory available to other virtual machines and different processes.Furthermore, common hypervisors (such as kernel-based virtual machines, KVM) pin the entire memory of a virtual machine (VM) for direct memory access (DMA), thus limiting the total memory available to the rest of the computer or system (this process is called static pinning). Therefore, memory pinning (or static memory pinning) increases server costs and memory requirements, leading data centers to allocate significant amounts of RAM to each server to account for pinned memory, even if most of this pinned memory is unused at any given point in time—a situation that is undesirable.

[0003] Currently, some attempts have been made to reduce server costs and memory requirements by using traditional non-fixed RDMA technology. Traditional non-fixed RDMA technology offers greater flexibility in memory management than fixed RDMA technology, allowing memory (i.e., physical memory content or application virtual memory) to be paged out and swapped back into memory on demand as RDMA transaction requests arrive. However, traditional non-fixed RDMA technology suffers from high transaction drop rates, resulting in high and unreliable transaction completion latency in a given RDMA system. Traditional non-fixed RDMA technology requires a mechanism to service page faults whenever a hardware device attempts to obtain a translation of a virtual address (VA) that is not currently in RAM. Traditional non-fixed RDMA technology can be used with traditional requesting devices (or traditional requesters) and traditional responding devices (or traditional responders). For example, in a traditional requesting device, the virtual address is known when preparing a work queue element (WQE). Therefore, conventional request devices handle the translation of the physical address (VA) to the physical address (PA) (and page loading if necessary) before the request packet (e.g., an RDMA request packet) is generated. In conventional response devices, the virtual address is only known when the request packet (i.e., an RDMA request packet) is received over the network. The virtual address is obtained from the request packet header (e.g., in the case of a marked operation) or from the conventional response device's receive queue (e.g., in the case of an unmarked operation). After obtaining the virtual address, the conventional response device typically begins processing the required translation request. If a page fault occurs while processing the translation request due to an unmapped or swapped-out page, the conventional response device's software (or device driver) handles the call or allocation request. Typically, the time required to servicing a page fault (e.g., up to hundreds of microseconds) is much longer than the time required to send or receive data pages to or from the network at line speed (e.g., 25 Gbps to 400 Gbps). This means that in traditional non-fixed RDMA systems, RDMA transaction requests are paused or dropped at least part of the time, which increases transaction completion latency and / or the risk of connection drops due to various errors. Therefore, traditional request devices, traditional response devices, methods, and traditional remote direct memory access systems suffer from inefficient and unreliable data communication technical problems.

[0004] Therefore, based on the above discussion, it is necessary to overcome the aforementioned drawbacks associated with conventional request devices, conventional response devices, and conventional methods of conventional remote direct memory access (RDMA) technology used for data communication. Summary of the Invention

[0005] This invention aims to provide a requesting device, a responding device, and a method for reducing transaction drop in remote direct memory access (RDMA) systems. The invention aims to provide a solution to the existing problems of inefficient and unreliable data communication using conventional requesting devices, conventional responding devices, methods, and conventional RDMA systems. The object of this invention is to provide a solution that at least partially overcomes the problems encountered in the prior art and provides improved devices, methods, and improved RDMA systems to achieve efficient (e.g., reduced transaction drop) and reliable data communication.

[0006] In one aspect, the present invention provides a requesting device. The requesting device includes a memory, a controller, and a communication interface. The controller is configured to send a message including a prefetch operation to a response device via the communication interface, the prefetch operation indicating that the response device will load a memory region of its memory. The controller is further configured to send a request to the response device via the communication interface, the request relating to request data and the memory region. The controller is also configured to receive a response message from the response device via the communication interface.

[0007] The requesting device of this invention can prefetch required memory pages that may not currently be allocated or swapped out, thereby reducing latency (or full transaction latency). Transmitting a message including a prefetch operation (or prefetch hint) before a request (e.g., an RDMA read request or an RDMA write request) enables the responding device to prefetch the required memory pages in advance (as needed) and prepare to accept the request (i.e., the RDMA read request or RDMA write request) using other methods (e.g., a register buffer, memory allocation, etc.). Furthermore, if the responding device does not have a physical memory buffer to serve the request (i.e., an RDMA read request or RDMA write request), transmitting a message including a prefetch operation (or prefetch hint) before the request reduces the likelihood of the responding device discarding or pausing the request (i.e., the RDMA read request or RDMA write request). The disclosed requesting device and responding device share information about the RDMA request (i.e., the RDMA read request or RDMA write request) to reduce the likelihood of the RDMA request (i.e., the RDMA read request or RDMA write request) being paused or discarded, thus improving the reliability of data communication. The request (i.e., an RDMA read request or an RDMA write request) is associated with a memory page prefetched using a prefetch operation, thus reducing latency. Furthermore, a response message from the responding device confirms whether the requesting device successfully executed the request (i.e., the RDMA read request or RDMA write request), further improving the reliability and efficiency of data communication.

[0008] In one implementation, the request is a write command that carries the request data for writing the request data into the memory area, and the response message is an acknowledgment of the write command.

[0009] In one example, the request relates to writing requested data to a desired memory region (e.g., a memory address) using a write command. The requesting device receives a response message that provides confirmation of whether the write command was successfully executed.

[0010] In another implementation, the request is a read command carrying a memory address for reading the requested data from the memory address in the memory region, and the response message is a message carrying the read requested data.

[0011] In one example, the request might relate to reading requested data from a memory address in a memory region using a read command. The requesting device receives a response message that provides the requested data read upon successful execution of the read command.

[0012] In another implementation, the size of the memory region is larger than the requested data. The controller is also configured to send multiple requests related to the memory region to the response device via the communication interface.

[0013] RDMA transactions (such as RDMA read or RDMA write transactions) involve exchanging very long messages (or data), such as up to 2 gigabytes (GB), between a requesting device (or initiator) and a responding device (or target). To include the exchange of such long messages (e.g., approximately 2 GB each), the controller sends multiple requests associated with the memory regions of the responding devices. Therefore, in this case, the total memory region is larger than the RDMA transaction, allowing multiple requests (or messages) to be served simultaneously between a client (or a single requesting device) and multiple servers (or multiple responding devices), or between a server (or a single responding device) and multiple clients (or multiple requesting devices). When the RDMA transaction exceeds the total physical memory size and memory is not fixed, memory can be swapped in and / or swapped out as needed to enable processing of large RDMA transactions within a virtual memory context larger than the RDMA transaction itself.

[0014] In another implementation, the controller is also configured to receive a confirmation message for the prefetch operation before sending the request.

[0015] The requesting device's controller receives an acknowledgment message for the prefetch operation before sending the request, indicating whether the prefetch operation was successfully executed. Successful execution of the prefetch operation reduces the likelihood of RDMA transactions (i.e., RDMA read or RDMA write transactions) being paused and the retransmission wait time at the responding device. For example, in one scenario where the required memory page is swapped out, sending an acknowledgment message for the prefetch operation before sending the request reduces latency compared to sending a negative acknowledgment (e.g., receiver not ready (RNR)) or simply discarding the transaction (e.g., an RDMA read or RDMA write transaction) on the responding device and waiting for the cycle time objective (RTO, i.e., the maximum amount of time to re-access data after an unplanned transaction is discarded). In another scenario where the required memory page is not swapped out, the maximum latency is one round trip time (RTT).

[0016] In another implementation, the controller is also configured to send the request (e.g., an RDMA read transaction or an RDMA write transaction) after a waiting period following the sending of the prefetch operation.

[0017] The waiting time elapsed from sending a prefetch operation allows the responding device to efficiently execute requests (such as RDMA read requests or RDMA write requests). Furthermore, the waiting time reduces the likelihood of requests being dropped on the responding device, thus making data communication more reliable.

[0018] In another implementation, the message including the prefetch operation is a dedicated prefetch operation message.

[0019] Dedicated prefetch operation messages provide prefetch prompts to responding devices to proactively prefetch currently unallocated or swapped-out memory pages (or memory addresses), thereby reducing latency.

[0020] In another implementation, the message including the prefetch operation includes the prefetch operation in the additional payload of a request message for another request.

[0021] The prefetch operation in the additional payload of another request message enables the responding device to prefetch the required memory pages in advance and be ready to efficiently accept requests (i.e., RDMA read requests or RDMA write requests).

[0022] In another implementation, the requesting device is configured for RDMA.

[0023] The requesting device has the ability to execute efficiently in RDMA.

[0024] In another aspect, the present invention provides a method for requesting a device. The requesting device includes a memory, a controller, and a communication interface. The method includes sending a message including a prefetch operation to a responding device via the communication interface, the prefetch operation indicating that the responding device will load a memory region of its memory. The method further includes sending a request to the responding device via the communication interface, the request relating to request data and the memory region. The method further includes receiving a response message from the responding device via the communication interface.

[0025] The method described in this respect achieves all the advantages and effects of the requested device of the present invention.

[0026] In one implementation, a computer-readable medium carrying computer instructions is provided, which, when loaded into and executed by a controller of a requesting device, enable the requesting device to implement the method.

[0027] A computer-readable medium carrying computer instructions (specifically, a non-transitory computer-readable medium) realizes all the advantages and effects of the requested device or method.

[0028] In another aspect, the present invention provides a request device. The request device includes a memory, a communication interface, and a software module. The software module includes a prefetch operation message sender module, which sends a message including a prefetch operation to a response device via the communication interface, the prefetch operation indicating that the response device will load the data into a memory region of the response device's memory. The software module also includes a request sender module for sending a request to the response device via the communication interface, the request relating to request data and a memory region. The software module further includes a response message receiver module for receiving a response message from the response device via the communication interface.

[0029] The software modules associated with the prefetch operation message sender module, upon execution, cause the requesting device to send a message including a prefetch operation to the responding device. The prefetch operation involves prefetching a required memory region into the responding device's memory (e.g., the CPU's cache memory or internal memory), thereby enabling fast data communication. The software modules associated with the request sender module, upon execution, cause the requesting device to send a request (e.g., a read request or a write request) to the responding device. This request is associated with a required memory region, which is prefetched into the responding device's memory, thus reducing latency (or lowering response time). The execution of the software modules associated with the response message receiver module confirms whether the requesting device has successfully executed the request (i.e., the read request or write request), thereby improving the reliability of data communication.

[0030] In another aspect, the present invention provides a response device. The response device includes a memory, a controller, and a communication interface. The controller is configured to receive, via the communication interface, a message including a prefetch operation, the prefetch operation indicating a memory region to be loaded into the memory by the response device. The controller is further configured to load the indicated memory region into the memory. The controller is also configured to receive, via the communication interface, a request from the requesting device, the request relating to request data and a memory region. The controller is further configured to execute the request on the request data in the memory region. The controller is also configured to send a response message to the requesting device via the communication interface.

[0031] The response device of this invention uses a message including a prefetch operation from a requesting device to prefetch the required memory. The prefetched memory region is loaded into the response device's memory (e.g., the CPU's cache memory or internal memory), thereby enabling faster data communication. Furthermore, the response device receives requests associated with requested data (e.g., read requests or write requests) and the memory region (i.e., the prefetched memory region), thus reducing latency. The controller of the response device sends a response message to the requesting device to provide confirmation of whether the request associated with the requested data and memory region was successfully executed, thus making data communication more reliable.

[0032] In one implementation, the controller is further configured to determine whether the indicated memory region is stored in the memory before loading the indicated memory region into the memory, and if the memory region is not stored in the memory, then load the indicated memory region into the memory.

[0033] The controller of the responding device determines the availability of the indicated memory region in memory, or otherwise loads the indicated memory region into memory, in order to reduce the likelihood and latency of pausing RDMA transactions on the responding device. This further makes data communication more reliable and faster.

[0034] In another implementation, the controller is further configured to send an acknowledgment message to the requesting device, the acknowledgment message indicating whether the controller can load the indicated memory region into the memory.

[0035] The transmission of an acknowledgment message provides the requesting device with an indication of whether the specified memory region has been loaded into memory. This acknowledgment in the implementation allows RDMA transactions to be executed successfully and reduces latency.

[0036] On the other hand, the present invention provides a method for a response device. The response device includes a memory, a controller, and a communication interface. The method includes receiving, via the communication interface, a message including a prefetch operation from a requesting device, the prefetch operation indicating a memory region to be loaded into the memory by the response device. The method further includes loading the indicated memory region into the memory. The method further includes receiving, via the communication interface, a request from the requesting device, the request relating to request data and a memory region. The method further includes executing the request on the request data in the memory region. The method further includes sending a response message to the requesting device via the communication interface.

[0037] The method described in this respect achieves all the advantages and effects of the response device of the present invention.

[0038] In one implementation, a computer-readable medium carrying computer instructions is provided, which, when loaded into and executed by a controller of a response device, enable the response device to implement the method.

[0039] A computer-readable medium carrying computer instructions (specifically, a non-transitory computer-readable medium) realizes all the advantages and effects of a responsive device or method.

[0040] In another aspect, the present invention provides a response device. The response device includes a memory, a communication interface, and a software module. The software module includes a prefetch operation receiving module, which receives a message including a prefetch operation from a requesting device via the communication interface, the prefetch operation indicating that a memory region to be loaded by the response device into the memory. The software module also includes a memory loading module for loading the indicated memory region into the memory. The software module further includes a request receiving module for receiving a request from the requesting device via the communication interface, the request relating to request data and a memory region. The software module further includes an execution module for executing the request on the request data in the memory region. The software module further includes a sending module for sending a response message to the requesting device via the communication interface.

[0041] The execution of the prefetch operation receiving module enables the responding device to prefetch the indicated memory region, resulting in faster data communication. The execution of the memory loading module for loading the indicated memory region into the responding device's memory reduces latency. The execution of software modules associated with the request receiving module, which receives requests related to requested data and memory regions (e.g., the indicated memory region), and enables faster request processing, further facilitates faster request processing. Furthermore, the sending module, when executed, causes the responding device to send a response message that confirms to the requesting device whether the request was successfully executed, thus improving the reliability of data communication.

[0042] In another aspect, the present invention provides a system. The system includes a requesting device and a responding device.

[0043] The system of the present invention improves the reliability of data communication by reducing transaction loss (e.g., RDMA read transactions or RDMA write transactions) and reducing latency, thereby accelerating data communication.

[0044] On the other hand, the present invention provides a method for a system. The system includes a responding device and a requesting device. The method includes the requesting device sending a message including a prefetch operation to a responding device, the prefetch operation indicating a memory region to be loaded into the memory of the responding device. The method also includes the responding device receiving the message including the prefetch operation and loading the indicated memory region into memory. The method further includes the requesting device sending a request to the responding device, the request relating to requested data and the memory region. The method also includes the responding device receiving the request from the requesting device, executing the request on the requested data in the memory region, and sending a response message to the requesting device. The method further includes the requesting device receiving a response message from the responding device.

[0045] The method in this aspect achieves all the advantages and effects of the system of the present invention.

[0046] It should be noted that all devices, elements, circuits, units, modules, and apparatuses described in this application can be implemented in software or hardware elements or any combination thereof. All steps performed by the various entities described in this application, and the functions described as being performed by the various entities, are intended to indicate that the corresponding entities are used to perform the corresponding steps and functions. Although the specific functions or steps performed by external entities are not reflected in the detailed descriptions of the specific elements of the entities performing the specific steps or functions in the following description of specific embodiments, it will be apparent to those skilled in the art that these methods and functions can be implemented by corresponding hardware or software elements or any combination thereof. It is understood that the features of the present invention are readily combined in various combinations.

[0047] Additional aspects, advantages, features and objects of the present invention will become apparent from the accompanying drawings and the following detailed description of implementations. Attached Figure Description

[0048] The above-described invention and the following detailed description of illustrative embodiments can be better understood when read in conjunction with the accompanying drawings. Exemplary structures of the invention are shown in the drawings to illustrate the invention. However, the invention is not limited to the specific methods and tools disclosed herein. Furthermore, those skilled in the art will understand that the drawings are not drawn to scale. Where possible, the same elements are represented by the same numbers.

[0049] The embodiments of the present invention will now be described with reference to the following figures, which are merely examples.

[0050] Figure 1 This is a network environment diagram of an exemplary remote direct memory access (RDMA) system with requesting and responding devices provided by an embodiment of the present invention.

[0051] Figure 2A This is a block diagram of various exemplary components of a requesting device provided in embodiments of the present invention;

[0052] Figure 2B This is a block diagram of various exemplary components of the response device provided in embodiments of the present invention;

[0053] Figure 3 This is a flowchart of a method for requesting a device provided in an embodiment of the present invention;

[0054] Figure 4 This is a flowchart of a method for responding to a device provided in an embodiment of the present invention;

[0055] Figure 5 This is a flowchart of a method for a system including a requesting device and a responding device provided by an embodiment of the present invention;

[0056] Figure 6 This is an illustration of an exemplary implementation scenario of a remote direct memory access (RDMA) system with prefetch hints provided by embodiments of the present invention.

[0057] In the accompanying diagrams, underlined numbers indicate the item in which the underlined number appears or the item adjacent to the underlined number. Ununderlined numbers relate to the item identified by the line that associates the ununderlined number with the item. When a number is ununderlined and has an associated arrow, the ununderlined number is used to identify the general item that the arrow points to. Detailed Implementation

[0058] The following detailed description illustrates embodiments of the present invention and ways in which these embodiments can be implemented. While some modes of implementing the invention have been disclosed, those skilled in the art will recognize that other embodiments for implementing or practicing the invention may also exist.

[0059] Figure 1 This is a network environment diagram of an exemplary remote direct memory access (RDMA) system with requesting and responding devices provided in an embodiment of the present invention. (See reference) Figure 1A network diagram of an RDMA system 100, including requesting device 102 and responding device 104, is shown. Network 106 is also shown.

[0060] In RDMA system 100, requesting device 102 can directly access the memory subsystem (e.g., virtual memory or a portion of total memory (or physical memory)) of responding device 104 without involving the operating systems (OS) of requesting device 102 and responding device 104. In conventional RDMA systems, the memory subsystem is shared between the user application and the hardware device (e.g., a conventional RDMA network interface card (RNIC)). The memory subsystem accessible by a conventional RNIC requires memory pinning; therefore, the user device (required by the user application) must register it under a process called a "registered memory region" using the InfiniBand (IB) verb to access this memory subsystem. Since the user device does not previously know which part of the memory subsystem (or registered memory) the conventional RDMA network interface card will access at any given time, this memory subsystem (or registered memory) typically remains registered even when not in use. The registration mechanism of the memory subsystem varies from process to process; therefore, a portion of the memory subsystem (or registered memory) cannot be shared between different processes. Registered memory (or pinned memory) leads to high memory requirements to support connectivity in traditional RDMA systems. Various schemes have been proposed to efficiently utilize the memory subsystem, such as traditional non-pinned direct memory access (NP DMA) schemes. These include input / output (I / O) bounce buffers, dynamic memory pinning, delayed memory pinning, and on-demand paging (ODP). Input / output (I / O) bounce buffers are intermediate buffers used as the destination for direct memory access (DMA) operations. However, they require additional memory buffer allocation on traditional response devices, continuous management of the buffer pool, and incur latency and bus overhead due to the need for additional copying, making them less preferred. Dynamic and delayed memory pinning require on-demand pinning and unpinning, resulting in complex logic that is difficult to generalize and optimize, and therefore, they are less commonly used. On-demand paging (e.g., implemented by Mellanox RNIC) enables non-fixed memory prefetching on local endpoints, requires prefetching to be explicitly activated by the application layer, and is independent of hard page faults, including those in storage switching devices. RDMA system 100 largely solves the aforementioned problems by using a message that includes a prefetch hint to prefetch non-fixed memory. The requesting device 102 sends a prefetch hint message to the responding device 104.The prefetch cue message enables the responding device 104 to begin prefetching the required memory pages (or a portion of the memory subsystem) in advance and prepare to receive one or more data packets. The transmission of the prefetch cue message in the RDMA system 100 reduces the likelihood of pausing or dropping one or more data packets. In this way, the RDMA system 100 achieves a more reliable and efficient data communication system with lower latency than conventional RDMA systems. Furthermore, compared to conventional RDMA systems and conventional non-fixed direct memory access schemes, the RDMA system 100 has lower complexity and is easier to deploy.

[0061] The requesting device 102 includes appropriate logic, circuitry, interfaces, and / or code for processing the send queue (SQ), reading work queue elements (WQE), and generating one or more data packets for transmission to the responding device 104. For example, in an RDMA system 100, data is transmitted in the form of one or more data packets (e.g., RDMA packets). These one or more data packets (i.e., RDMA packets) include information associated with a packet sequence number (PSN) to enforce correct packet ordering. In one example, the one or more data packets (i.e., RDMA packets) also include source and destination queue pair (QP) numbers that distinguish the application and its valid memory regions. Each QP also has a context on both the requesting device 102 and the responding device 104. In one embodiment, a memory region has a storage key (R-key) that is part of one or more data packets, associating the memory region with the requester and vice versa. One or more data packets (i.e., RDMA packets) also include information related to the message type (e.g., RDMA READ, RDMA WRITE, SEND, or ATOMIC) and various parameters of the message (e.g., message length, destination memory address, operation type, and operand data). The message length includes the length of the RDMA READ message or RDMAWRITE message. The memory address includes the destination memory address for the RDMA READ message, RDMA WRITE message, and ATOMIC message. The SEND operation has no destination memory address. The operation type and operand data only exist for atomic message types. Furthermore, the requesting device 102 sends a prefetch cue message to the responding device 104 to proactively prefetch the required memory pages (or portions of the memory subsystem) that are currently unallocated or swapped out. Prefetching the required memory pages (or portions of the memory subsystem) reduces the likelihood of pausing or discarding one or more data packets (i.e., RDMA packets), thereby improving the reliability of data communication and reducing latency. The requesting device 102 may also be referred to as the requesting RDMA network interface card in the RDMA system 100. In one example, the requesting device 102 may be used for high-performance computing (HPC). Examples of the requesting device 102 may include, but are not limited to, network adapters, servers, computing devices in computer clusters (e.g., massively parallel computer clusters), communication devices including portable or non-portable electronic devices, telematics control units (TCUs) in vehicles, drones, wireless modems, supercomputers, or other RDMA-based devices. For example, in Figure 2AVarious exemplary components of the requesting device 102 are described in detail.

[0062] The responding device 104 includes appropriate logic, circuitry, interfaces, and / or code for processing incoming data packets (i.e., RDMA data packets), performing operations on the incoming data packets, and optionally, returning information to the requesting device 102. For example, in the case of a marking operation (e.g., RDMA READ, RDMA WRITE, or ATOMIC), the responding device 104 obtains the direct memory access (DMA) target (e.g., virtual address (VA) + message length) from the packet header (e.g., RDMA extended transport header (RETH) or atomic extended transport header (Atomic ETH)) of one or more data packets (e.g., the first data packet) of an RDMA message transmitted on the line. Therefore, in another case where there is no DMA target (i.e., virtual address (VA) + message length) in the header of an unmarked operation (e.g., SEND), the responding device 104 determines the DMA target (i.e., virtual address (VA) + message length) from the work queue element (WQE) of the receive queue (RQ). Furthermore, the responding device 104 receives a prefetch cue message from the requesting device 102 and (as needed) prefetches the required memory pages in advance, and is prepared to accept RDMA requests (e.g., RDMA read requests or RDMA write requests) using other methods (e.g., register buffers, memory allocation, etc.). The prefetch message at the responding device 104 reduces the likelihood of dropping or pausing RDMA requests (i.e., RDMA read requests or RDMA write requests), thus supporting more reliable and efficient data communication. The responding device 104 can also be referred to as the responding RDMA network interface card in the RDMA system 100. Examples of response device 104 may include, but are not limited to, network adapters, servers, computing devices in computer clusters (e.g., massively parallel computer clusters), communication devices including portable or non-portable electronic devices, telematics control units (TCUs) in vehicles, drones, wireless modems, supercomputers, or other RDMA-based devices. For example, in Figure 2B Various exemplary components of the response device 104 are described in detail.

[0063] Network 106 includes a medium (e.g., a communication channel) through which requesting device 102 may communicate with responding device 104. Examples of network 106 include, but are not limited to, computer networks in a computer cluster, local area networks (LANs), cellular networks, wireless sensor networks (WSNs), cloud networks, vehicle-to-network (V2N) networks, metropolitan area networks (MANs), and / or the Internet. Requesting device 102 in the network environment is used to connect to responding device 104 according to various network protocols supporting RDMA. Examples of such network protocols, communication standards, and technologies may include, but are not limited to, InfiniBand (IB), RDMA over converged Ethernet (RoCE), the Internet Wide Area RDMA protocol (iWARP), or modifications and variations thereof.

[0064] Figure 2A This is a block diagram of various exemplary components of a requesting device provided in embodiments of the present invention. Figure 2A Combination Figure 1 The components are described. (Refer to...) Figure 2A The diagram shows a block diagram 200A of a requesting device 102. The requesting device 102 includes a memory 202, a controller 204, and a communication interface 206. In one implementation, the requesting device 102 further includes one or more software modules, such as software module 208.

[0065] Memory 202 includes appropriate logic, circuitry, and / or interfaces for storing executable instructions to control requesting device 102. Memory 202 may store data (transmitted in the form of data packets) for processing on requesting device 102. Examples of implementations of memory 202 may include, but are not limited to, random access memory (RAM), electrically erasable programmable read-only memory (EEPROM), read-only memory (ROM), hard disk drive (HDD), flash memory, secure digital (SD) cards, solid-state drives (SSDs), and / or CPU cache memory. Memory 202 may store an operating system and / or other program products to operate requesting device 102. Computer-readable storage media used to provide non-transitory memory may include, but are not limited to, electronic storage devices, magnetic storage devices, optical storage devices, electromagnetic storage devices, semiconductor storage devices, or any suitable combination thereof.

[0066] Controller 204 includes appropriate logic, circuitry, and / or interfaces for sending a message including a prefetch operation to response device 104 via communication interface 206, the prefetch operation indicating that the response device 104 will load the data into its memory region. Controller 204 is a computing element for processing instructions that drive requesting device 102. Examples of controller 204 include, but are not limited to, network interface controllers, microprocessors, microcontrollers, complex instruction set computing (CISC) microprocessors, reduced instruction set computing (RISC) microprocessors, or very long instruction word (VLIW) microprocessors.

[0067] Communication interface 206 is an arrangement of interconnected programmable and / or non-programmable components used to facilitate data communication between one or more electronic devices. For example, a network interface card (NIC) is arranged in communication interface 206 to handle send queue (SQ) WQEs, read work queue elements (WQEs), and generate data packets to send to responding device 104. Furthermore, the NIC arranged in communication interface 206 can handle receive queue (WQEs) and incoming write requests (e.g., requesting device 102 can also act as responding device 104). Communication interface 206 can support communication protocols for one or more of the following networks: peer-to-peer networks, hybrid peer-to-peer networks, local area networks (LANs), radio access networks (RANs), metropolitan area networks (MANs), wide area networks (WANs), all or part of public networks (e.g., a global computer network known as the Internet), private networks, cellular networks, and any other communication systems in one or more locations. Furthermore, communication interface 206 supports wired or wireless communications that can be performed via any number of known protocols, including but not limited to Internet Protocol (IP), Wireless Access Protocol (WAP), Frame Relay, or Asynchronous Transfer Mode (ATM). Additionally, any other suitable protocols using voice, video, data, or combinations thereof may also be adopted and supported by communication interface 206.

[0068] In one exemplary implementation, software module 208 includes a prefetch operation message sender module 208a, a request sender module 208b, and a response message receiver module 208c. In one implementation, software module 208 (including software modules 208a to 208c) may be implemented as separate circuitry within the requesting device 102. Alternatively, in another implementation, software module 208 is implemented as circuitry performing various operations of software modules 208a to 208c.

[0069] In operation, controller 204 is used to send a message including a prefetch operation to a response device (e.g., response device 104) via communication interface 206. This prefetch operation indicates a memory region to be loaded into the memory of response device 104. The message including the prefetch operation enables response device 104 to prefetch the indicated memory region in advance and load it into its memory, thereby reducing latency. In one case, the memory of response device 104 may be high-level memory, such as CPU cache memory or internal memory, thereby accelerating (…). Figure 1 Data communication in the RDMA system 100. At the requesting device 102, a message including a prefetch operation (or a prefetch prompt message) can be automatically generated along the transmission data path (i.e., the path along which different data packets are transmitted from the requesting device 102 to the responding device 104). Therefore, information about the indicated memory region (or target memory range) at the responding device 104 is available along the transmission data path. Optionally, the generation of the message including the prefetch operation can be controlled by the RDMA network interface card (RNIC) firmware (FW). In another alternative approach, the prefetch operation at responding device 104 (after receiving a message including the prefetch operation) can be delegated from the RDMA network interface card (RNIC) hardware (HW) to various other devices, such as RDMA network interface card (RNIC) firmware (FW) or system memory management unit (MMU), or input / output (I / O) MMU hardware, device drivers, operating system (OS) kernels, or hypervisors (e.g., virtual machine monitors (VMMs)). The prefetch operation at responding device 104 can be distributed among any of the aforementioned devices.

[0070] The controller 204 is also used to send a request to the responding device 104 via the communication interface 206, the request relating to requested data and a memory region. For example, a request relating to requested data and a memory region may also be referred to as an RDMA message, which may be written to or read from a memory region indicated by the responding device 104. RDMA messages are sent from the requesting device 102 to the responding device 104 in the form of various data packets, such as RDMA data packets. RDMA data packets include information relating to the RDMA message type (such as RDMA READ, RDMA WRITE, SEND, or ATOMIC) and various parameters of the RDMA message (such as message length, destination memory address, operation type, and operand data).

[0071] Controller 204 is also configured to receive response messages from response device 104 via communication interface 206. The response message from response device 104 acknowledges to controller 204 of requesting device 102 whether the request related to the requested data and memory region was successfully executed. Furthermore, the response message sends the requested data to requesting device 102. For example, in the case of a read request, this includes reading the requested data from the indicated memory region. In this case, the response message sends the requested data to requesting device 102.

[0072] The prefetch operation provides requesting device 102 with the ability to prefetch required memory pages at responding device 104, which may not currently be allocated or swapped out, to reduce latency (or overall transaction latency). Transmitting a message including the prefetch operation (or prefetch hint) before a request (e.g., an RDMA read request or an RDMA write request) enables responding device 104 to prefetch required memory pages as needed and prepare to accept the request (i.e., an RDMA read request or RDMA write request) using other methods (e.g., a register buffer, memory allocation, etc.). Furthermore, if there is no physical memory buffer at responding device 104 to serve the request (i.e., an RDMA read request or RDMA write request), transmitting a message including the prefetch operation (or prefetch hint) before the request reduces the likelihood that the request (i.e., an RDMA read request or RDMA write request) will be dropped or paused at responding device 104. Since the requesting device 102 and the responding device 104 share information about the RDMA request (i.e., the RDMA read request or the RDMA write request), the possibility of the RDMA request (i.e., the RDMA read request or the RDMA write request) being paused or dropped is reduced. Therefore, compared with the traditional RDMA system with more dropped transactions, more transaction delays and unreliable data, the reliability of data communication is significantly improved.

[0073] For example, in a traditional RDMA system with a non-fixed memory method, prefetch cue messages (or prefetch cue packets) are not used. The traditional requesting device sends a request (e.g., a write request, such as WRITE(VA 0x1000)) to the traditional responding device without any prefetch cue message. The traditional responding device finds that the indicated memory address (i.e., VA 0x1000) has been page-fetched, and therefore sends a negative acknowledgment message (e.g., receiver not ready (RNR)) to the traditional requesting device. Upon receiving the negative acknowledgment message (i.e., receiver not ready (RNR)), the traditional requesting device waits for the RNR to time out and retransmits the original request (i.e., the write request, such as WRITE(VA 0x1000)) to the traditional responding device. During the interval (i.e., from sending a negative acknowledgment message (i.e., receiver not ready (RNR)) to receiving the original request again), the conventional response device starts page-loading the memory address (i.e., VA 0x1000) and prepares the memory address to accept requests (i.e., requests retransmitted by the conventional requesting device). In this way, conventional RDMA systems suffer from high transaction drop, high transaction latency, and data unreliability. However, in RDMA system 100, the requesting device 102 sends a prefetch cue message to the response device 104, enabling the response device 104 to prefetch the indicated memory region in advance. This implementation reduces transaction drop at the response device 104, thereby further reducing transaction latency and improving data reliability and efficiency.

[0074] According to an embodiment, controller 204 is also configured to receive an acknowledgment message for a prefetch operation before sending a request. The acknowledgment message for the prefetch operation (e.g., the ACKREQ bit) provides controller 204 of requesting device 102 with information on whether the prefetch operation (i.e., loading the indicated memory region into the memory of responding device 104) was successfully completed. The acknowledgment message (i.e., the ACKREQ bit) can be a regular InfiniBand acknowledgement (IB ACK) message or a new special data packet or header, or any other means of signaling (or notifying) requesting device 102 of the execution of the prefetch operation. Receiving the acknowledgment message (i.e., the ACKREQ bit) at requesting device 102 before sending a request (e.g., an RDMA message) reduces the likelihood of pausing or discarding the request at responding device 104 and also results in lower latency. Furthermore, the acknowledgment message (i.e., the ACKREQ bit) reduces the likelihood of receiving a negative acknowledgment message (e.g., receiver-not-ready (RNR)) at requesting device 102. The acknowledgment message (i.e., the ACKREQ bit) of the prefetch operation acts like a fence around the request, relating to the requested data and memory region. The fence means that the request is only sent after the controller 204 of the requesting device 102 receives the acknowledgment message. For example, in one implementation, the fence can be implicit or dynamic. In another implementation, the fence can be configurable or negotiated between the requesting device 102 and the responding device 104.

[0075] According to an embodiment, the message including a prefetch operation is a dedicated prefetch operation message. A message including a prefetch operation can also be referred to as a prefetch cue message or a prefetch cue packet. For example, in one case, the prefetch cue message (or prefetch cue packet) is sent as a dedicated prefetch operation message (or special packet). In this case, the requesting device 102 can request acknowledgment of the prefetch cue message (i.e., an acknowledgment message (i.e., the ACKREQ bit)) and wait for acknowledgment. While processing WQE, the requesting device 102, on the communication interface 206, creates a packet with a new base transport header (BTH) opcode, which is referred to as a prefetch cue message (or prefetch cue packet). For tagged requests (e.g., RDMA READ, RDMA WRITE, or ATOMIC), the prefetch cue message (or prefetch cue packet) includes the target direct memory access (DMA), which includes various parameters such as R_KEY, virtual address (VA), and length at responding device 104 (i.e., R_KEY + VA + length). For untagged requests (e.g., SEND), the prefetch cue message (or prefetch cue packet) includes a SEND sequence ID (and optionally, length), which determines the receive queue element (RQE) index of responding device 104 (or the responder), which further provides the DMA target at responding device 104. The prefetch cue message (or prefetch cue packet) is sent to responding device 104 via the line prior to requests related to requested data and memory regions (e.g., RDMA messages).

[0076] According to an embodiment, the message including the prefetch operation includes the prefetch operation in the additional payload of a request message for another request. For example, a prefetch cue message (or prefetch cue packet) may be sent as an additional payload in another request (or an existing data packet). Alternatively, instead of a new base transport header (BTH) opcode (OPCODE), a new header may be added to the other request (or existing data packet) to send the prefetch cue message (or prefetch cue packet). Alternatively, instead of a new opcode (OPCODE), an existing opcode may also be used with a special flag (e.g., a READ / WRITE flag of size 0). In another alternative, the prefetch cue message (or prefetch cue packet) may be explicitly activated at the application layer (e.g., a new InfiniBand (IB) verb).

[0077] According to an embodiment, controller 204 is also configured to send a request after a waiting period from the time the prefetch operation is sent. For example, in one case, receiving an acknowledgment of the prefetch operation (i.e., an acknowledgment message) requires a waiting period from the time the prefetch hint message (or prefetch hint packet) is sent. The waiting period reduces the likelihood of the request being dropped on response device 104, thus making data communication more reliable.

[0078] According to one embodiment, the request is a write command carrying request data for writing the request data to a memory region, and the response message is an acknowledgment of the write command. For example, in one case, the request is a write command carrying request data for writing the request data to a memory region indicated by the memory of the responding device 104. In this case, the response message from the responding device 104 provides the requesting device 102 with acknowledgment (or information) that the write command was successfully executed. Optionally, the response message may provide the requesting device 102 with a negative acknowledgment (NAK) to notify that the write command was not successfully executed.

[0079] Optionally, in one exemplary implementation, the request (i.e., the request sent by controller 204 after a waiting period since the prefetch operation) is associated with a remote procedure call (RPC). In an RPC, requesting device 102 (or client) sends a procedure or function to responding device 104 (or server) via a network (e.g., network 106). The RPC causes the procedure or function to be executed at a different memory address in responding device 104 (i.e., a different memory address than that of requesting device 102). Response device 104 (or server) executes the procedure or function and sends a response message (i.e., a reply with the procedure or function parameters) to requesting device 102. While responding device 104 is processing the procedure or function, requesting device 102 may be blocked and will only resume when responding device 104 has completed the execution of the procedure or function.

[0080] Alternatively, in another exemplary implementation, the request relates to artificial intelligence (AI) operations. Typically, AI operations refer to the design of an intelligent machine (more specifically, an intelligent computer program) capable of sensing input from its environment, learning from that input, and providing relevant and flexible behavior similar to human behavior based on the same input. AI operations require high-performance computing or supercomputing. The requesting device 102 knows in advance which memory address the responding device 104 needs to load when performing an AI operation. Furthermore, the requesting device 102 is able to adapt to different network conditions by using AI operations.

[0081] According to one embodiment, the request is a read command carrying a memory address for reading requested data from that memory address in a memory region, and the response message is a message carrying the read requested data. For example, in one case, the request is a read command carrying a memory address for reading requested data. The memory address is located in a memory region indicated by the memory of the responding device 104. In this case, the response message from the responding device 104 provides the read requested data to the requesting device 102.

[0082] Optionally, the request may be related to a storage operation. For example, in an artificial intelligence operation or a remote procedure call, the storage operation may require additional memory (i.e., memory 202 in addition to the requesting device 102). Therefore, the requesting device 102 sends a request related to the storage operation to the responding device 104 in order to obtain access to the additional memory (which may be part of the responding device 104's memory). In this way, the storage operation can be performed more efficiently in the RDMA system 100.

[0083] According to one embodiment, the size of the memory region is larger than the requested data, and the controller 204 is also configured to send multiple requests related to the memory region to the responding device 104 via the communication interface 206. Figure 1 The RDMA system 100 can exchange very long messages (or data packets), such as up to 2 gigabytes (GB), between a requesting device 102 (or initiator) and a responding device 104 (or target). To include such long message exchanges (i.e., up to 2 GB), the controller 204 is used to send multiple requests associated with a memory region of the responding device 104. Therefore, in this case, the total memory region is larger than the RDMA transaction, making it possible to serve multiple requests (or multiple messages) simultaneously between a client and multiple servers or between a server and multiple clients. When the RDMA transaction is larger than the total physical memory size and memory is not fixed, memory can be swapped in and / or swapped out as needed to enable processing of large RDMA transactions within a virtual memory context larger than the RDMA transaction itself.

[0084] According to an embodiment, requesting device 102 is configured for RDMA. Requesting device 102 (or the requester) processes the send queue (SQ) WQE, reads work queue elements (WQE), and generates data packets for transmission to responding device 104 in the RDMA system 100. Furthermore, requesting device 102 can also process the receive queue (RQ) WQE and incoming write requests (e.g., requesting device 102 can also function as responding device 104).

[0085] In an exemplary aspect, requesting device 102 includes memory 202, communication interface 206, and software module 208. Software module 208, when executed (e.g., by controller 204), causes requesting device 102 to perform various operations, as described below in the example. Software module 208 includes a prefetch operation message sender module 208a for sending a message including a prefetch operation to responding device 104 via communication interface 206, the prefetch operation indicating a memory region to be loaded by responding device 104 into its memory. Software module 208 also includes a request sender module 208b for sending a request to responding device 104 via communication interface 206, the request relating to request data and a memory region. Software module 208 also includes a response message receiver module 208c for receiving a response message from responding device 104 via communication interface 206.

[0086] In an exemplary aspect, the prefetch operation message sender module 208a, upon execution, causes the requesting device 102 to send a message including a prefetch operation to the responding device 104. This message enables the responding device 104 to prepare the indicated memory region and load the indicated memory region into its memory, thereby providing faster data communication. Optionally, an acknowledgment message is sent to the requesting device 102, providing information on whether the prefetch operation was successfully completed. The request sender module 208b, upon execution, causes the requesting device 102 to send a request related to the requested data and the storage region (i.e., the indicated memory region) to the responding device 104. The request related to the requested data may be an RDMA READ request, an RDMA WRITE request, a SEND request, or an ATOMIC request. The response message receiver module 208c, upon execution, causes the requesting device 102 to receive a response message from the responding device 104 to acknowledge that the request related to the requested data and the memory region was successfully executed, thereby making data communication more reliable. The software module 208 is executed by the controller 204 of the requesting device 102.

[0087] Therefore, by sending a prefetch cue message to the responding device 104, the requesting device 102 can achieve a more reliable, efficient, and low-latency data communication system (i.e., RDMA system 100).

[0088] Figure 2B This is a block diagram of various exemplary components of a response device provided in embodiments of the present invention. Figure 2B Combination Figure 1 and Figure 2A The components are described. (Refer to...) Figure 2B , showed ( Figure 1The block diagram 200B shows the response device 104. The response device 104 includes a memory 210, a controller 212, and a communication interface 214. In one implementation, the response device 104 also includes one or more software modules, such as software module 216.

[0089] Memory 210 includes appropriate logic, circuitry, and / or interfaces for storing executable instructions to control response device 104. Memory 210 may store data (transmitted in the form of data packets) for processing on response device 104. Examples of implementations of memory 210 may include, but are not limited to, random access memory (RAM), electrically erasable programmable read-only memory (EEPROM), read-only memory (ROM), hard disk drive (HDD), flash memory, secure digital (SD) cards, solid-state drives (SSDs), and / or CPU cache memory. Memory 210 may store an operating system and / or other program products to operate response device 104. Computer-readable storage media used to provide non-transitory memory may include, but are not limited to, electronic storage devices, magnetic storage devices, optical storage devices, electromagnetic storage devices, semiconductor storage devices, or any suitable combination thereof.

[0090] Controller 212 includes appropriate logic, circuitry, and / or interfaces for receiving messages including a prefetch operation from a requesting device (e.g., requesting device 102) via communication interface 214. This prefetch operation indicates that the response device 104 will load the message into a memory region of memory 210. Controller 212 is a computing element for processing instructions that drive the response device 104. Examples of controller 212 include, but are not limited to, network interface controllers, microprocessors, microcontrollers, complex instruction set computing (CISC) microprocessors, reduced instruction set computing (RISC) microprocessors, or very long instruction word (VLIW) microprocessors.

[0091] Communication interface 214 is an arrangement of interconnected programmable and / or non-programmable components used to facilitate data communication between one or more electronic devices. For example, a network interface card (NIC) is arranged in communication interface 214 to process incoming messages (or data packets) and perform various operations on those messages. Furthermore, communication interface 214 supports communication over various networks, such as peer-to-peer networks, hybrid peer-to-peer networks, local area networks (LANs), radio access networks (RANs), metropolitan area networks (MANs), wide area networks (WANs), all or part of public networks (such as a global computer network known as the Internet), private networks, cellular networks, and any other communication system in one or more locations. Additionally, communication interface 214 supports wired or wireless communication that can be performed via any number of known protocols, including but not limited to Internet Protocol (IP), Wireless Access Protocol (WAP), Frame Relay, or Asynchronous Transfer Mode (ATM).

[0092] In one exemplary implementation, software module 216 includes a prefetch operation receiving module 216a, a memory loading module 216b, a request receiving module 216c, an execution module 216d, and a sending module 216e. In one implementation, software module 216 (including software modules 216a to 216e) may be implemented as separate circuitry within the responding device 104. Alternatively, in another implementation, software module 216 may be implemented as circuitry performing various operations of software modules 216a to 216e.

[0093] In operation, controller 212 receives a message including a prefetch operation from a requesting device (e.g., requesting device 102) via communication interface 214. This prefetch operation indicates a memory region to be loaded into memory 210 by responding device 104. Controller 212 of responding device 104 receives the message including the prefetch operation, prefetches the indicated memory region, and loads the indicated memory region into memory 210 of responding device 104. The message including the prefetch operation reduces the likelihood of RDMA requests (e.g., RDMA write requests or RDMA read requests) being dropped or paused at responding device 104, thus supporting more reliable and efficient data communication.

[0094] The controller 212 is also configured to load the indicated memory region into the memory 210. The controller 212 loads the indicated memory region into the memory 210 of the response device 104. For example, in one case, the memory 210 of the response device 104 may be part of advanced memory, such as the CPU's cache memory or internal memory (e.g., random access memory, RAM) to enable low-latency or faster data communication. Those skilled in the art will understand that the memory 210 may also be part of the host where the central processing unit (CPU) resides (e.g., the response device 104 acts as the host).

[0095] Controller 212 is also configured to receive requests from requesting device 102 via communication interface 214, the requests relating to requested data and a memory region. Controller 212 receives requests relating to requested data and a storage region (i.e., the indicated storage region). Requests relating to a memory region (i.e., the indicated memory region) enable faster transactions. For example, in one case, the request could be a read request carrying a memory address for reading the requested data. The memory address is located within the indicated memory region. In another case, the request could be a write request for writing the requested data to the indicated memory region. Therefore, the requests may differ for different application scenarios.

[0096] The controller 212 is also used to execute requests for requested data in the memory region. The controller 212 executes requests (i.e., read requests or write requests) for requested data in the memory region indicated by the memory 210 of the response device 104.

[0097] Controller 212 is also configured to send a response message to requesting device 102 via communication interface 214. Controller 212 sends the response message to requesting device 102 to provide information about the successful execution of the request. The response message can initiate another request from requesting device 102 to responding device 104.

[0098] According to one embodiment, the controller 212 is further configured to determine whether the indicated memory region is stored in the memory 210 before loading the indicated memory region into the memory 210, and if the memory region is not stored in the memory 210, then the indicated memory region is loaded into the memory 210. For example, in one case, if the memory region has not been swapped out and is already stored in the memory 210 of the response device 104, then the loading step can be omitted. Therefore, the controller 212 determines whether the memory region is stored in the memory 210 before loading the indicated memory region into the memory 210. In another case, if the memory region has been swapped out and is not stored in the memory 210, then the controller 212 loads the indicated memory region into the memory 210 of the response device 104. In yet another case, the indicated memory region is loaded into a small portion of the memory 210 of the response device 104 for a short duration (i.e., the time for processing the request), also referred to as partial memory loading.

[0099] According to an embodiment, controller 212 is further configured to send an acknowledgment message to requesting device 102, the acknowledgment message indicating whether controller 212 can load the indicated memory region into memory 210. The acknowledgment message from controller 212 provides requesting device 102 with information that the indicated memory region is ready to accept requests related to the requested data and memory region (e.g., RDMA read requests or RDMA write requests). The acknowledgment message reduces transaction drop (or request drop) at responding device 104, thereby enabling reliable and efficient data communication.

[0100] In an exemplary aspect, the response device 104 includes a memory 210, a communication interface 214, and a software module 216. When executed (e.g., by a controller 212), the software module 216 causes the response device 104 to perform various operations, as described below in the example. The software module 216 includes a prefetch operation receiving module 216a for receiving a message including a prefetch operation from a requesting device (e.g., requesting device 102) via the communication interface 214, the prefetch operation indicating a memory region to be loaded by the response device 104 into the memory 210. The software module 216 also includes a memory loading module 216b for loading the indicated memory region into the memory 210. The software module 216 also includes a request receiving module 216c for receiving a request from the requesting device 102 via the communication interface 214, the request relating to request data and a memory region. The software module 216 also includes an execution module 216d for executing the request on the request data in the memory region. Software module 216 also includes a sending module 216e, used to send a response message to the requesting device 102 via communication interface 214.

[0101] In an exemplary aspect, the prefetch operation receiving module 216a, upon execution, causes the responding device 104 to receive a message including a prefetch operation. The responding device 104 prefetches the indicated memory region in advance, thereby achieving lower latency. The memory loading module 216b, upon execution, causes the responding device 104 to load the indicated memory region into the memory 210 of the responding device 104. The request receiving module 216c, upon execution, causes the responding device 104 to receive a request related to requested data and the memory region (i.e., the indicated memory region). The request related to the requested data may be an RDMA READ request, an RDMA WRITE request, a SEND request, or an ATOMIC request. The execution module 216d, upon execution, causes the responding device 104 to execute the request for the requested data in the indicated memory region. The sending module 216e, upon execution, causes the responding device 104 to send a response message to the requesting device 102 to provide information about the successful execution of the request. The software module 216 is executed by the controller 212 of the responding device 104.

[0102] Therefore, the response device 104 reduces request drop (or transaction drop) and retransmission time by prefetching the indicated memory region (or the required memory region) in advance. The response device 104 performs all preparations before accepting the request (i.e., the RDMA request) and processes and executes the request more reliably and efficiently.

[0103] Figure 3 This is a flowchart of a method for requesting a device provided by an embodiment of the present invention. Figure 3 Combination Figure 1 , Figure 2A and Figure 2B The components are described. (Refer to...) Figure 3 This illustrates a method 300 for reducing RDMA transaction drop (or RDMA request drop) in a remote direct memory access system (e.g., RDMA system 100). Method 300 is executed by the controller 204 of the requesting device 102, which has been demonstrated, for example, in... Figure 1 and Figure 2A The details are described below. Method 300 includes steps 302 through 306.

[0104] In step 302, method 300 includes sending a message including a prefetch operation to a response device (e.g., response device 104) via communication interface 206. This prefetch operation indicates a memory region to be loaded into the memory of response device 104. The message including the prefetch operation (or prefetch hint message) enables response device 104 to prefetch the indicated memory region into memory (e.g., memory 210) in advance, thereby further enabling faster and more reliable execution of RDMA transactions (or RDMA requests, such as RDMA read requests or RDMA write requests). Prefetching the indicated memory region at response device 104 reduces the likelihood of RDMA transactions (or RDMA requests) being dropped at response device 104. The controller 204 of requesting device 102 is used to send the message including the prefetch operation (or prefetch hint message) to response device 104 via communication interface 206.

[0105] In step 304, method 300 further includes sending a request to response device 104 via communication interface 206, the request relating to requested data and a memory region. The request relating to requested data and a memory region (i.e., the indicated memory region) corresponds to an RDMA transaction (or RDMA request) executed by controller 212 of response device 104. This request can be an RDMA read request, an RDMA write request, a SEND request, or an ATOMIC request. Controller 204 of requesting device 102 is used to send the request to response device 104 via communication interface 206.

[0106] In step 306, method 300 further includes receiving a response message from response device 104 via communication interface 206. The response message from response device 104 notifies requesting device 102 whether a request related to requested data and a memory region (i.e., an RDMA request) was successfully executed at response device 104. The controller 204 of requesting device 102 is used to receive the response message from response device 104 via communication interface 206.

[0107] Steps 302 to 306 are merely illustrative and other alternatives may be provided, in which one or more steps are added, one or more steps are deleted, or one or more steps are provided in a different order.

[0108] According to an embodiment, a computer-readable medium carrying computer instructions is provided, which, when loaded into and executed by a controller (e.g., controller 204) of a requesting device (e.g., requesting device 102), enable the requesting device 102 to implement method 300. The computer-readable medium carrying the computer instructions provides non-transitory memory, which may include, but is not limited to, electronic storage devices, magnetic storage devices, optical storage devices, electromagnetic storage devices, semiconductor storage devices, or any suitable combination thereof.

[0109] Figure 4 This is a flowchart of a method for a response device provided in an embodiment of the present invention. Figure 4 Combination Figure 1 , Figure 2A , Figure 2B and Figure 3 The components are described. (Refer to...) Figure 4 This illustrates a method 400 for reducing RDMA transaction drop (or RDMA request drop) in a remote direct memory access system (e.g., RDMA system 100). Method 400 is executed by controller 212 of responding device 104, which has been demonstrated, for example, in... Figure 1 and Figure 2B The method is described in detail below. Method 400 includes steps 402 through 410.

[0110] In step 402, method 400 includes receiving a message including a prefetch operation from a requesting device (e.g., requesting device 102) via communication interface 214. This prefetch operation indicates a memory region to be loaded into memory 210 by responding device 104. Responding device 104 prefetches the indicated memory region according to the message including the prefetch operation. Controller 212 of responding device 104 loads the indicated memory region into memory 210 of responding device 104. By prefetching the indicated memory region before receiving an RDMA request, responding device 104 reduces the likelihood of the RDMA request being discarded, thus improving (…). Figure 1 The RDMA system 100 improves data communication reliability and efficiency. The controller 212 of the response device 104 receives messages including prefetch operations from the requesting device 102 via the communication interface 214.

[0111] In step 404, method 400 further includes loading the indicated memory region into memory 210. The controller 212 of the response device 104 is used to load the indicated memory region into memory 210 of the response device 104. Memory 210 of the response device 104 may be advanced memory, such as CPU cache memory or internal memory (e.g., random access memory (RAM)). Advanced memory enables faster transactions (i.e., RDMA transactions) with very low latency.

[0112] In step 406, method 400 further includes receiving a request from requesting device 102 via communication interface 214, the request relating to requested data and a memory region. Controller 212 of responding device 104 is configured to receive the request relating to requested data and a memory region (i.e., the indicated memory region) from requesting device 102 via communication interface 214.

[0113] In step 408, method 400 further includes executing a request for requested data in a memory region. The controller 212 of the response device 104 is used to execute the request for requested data in the memory region. The memory region corresponds to an indicated memory region loaded into the memory 210 of the response device 104.

[0114] In step 410, method 400 further includes sending a response message to requesting device 102 via communication interface 214. The controller 212 of responding device 104 is used to send the response message to requesting device 102 via communication interface 214. The response message confirms to requesting device 102 that the request was successfully executed at responding device 104.

[0115] Steps 402 to 410 are merely illustrative and other alternatives may be provided, in which one or more steps are added, one or more steps are deleted, or one or more steps are provided in a different order.

[0116] According to an embodiment, a computer-readable medium carrying computer instructions is provided, which, when loaded into and executed by a controller 212 of a response device 104, enable the response device 104 to implement method 400. The computer-readable medium carrying computer instructions provides non-transitory memory, which may include, but is not limited to, electronic storage devices, magnetic storage devices, optical storage devices, electromagnetic storage devices, semiconductor storage devices, or any suitable combination thereof.

[0117] Figure 5 This is a flowchart of a method for a system including a requesting device and a responding device provided by an embodiment of the present invention. Figure 5 Combination Figure 1 , Figure 2A , Figure 2B , Figure 3 and Figure 4 The components are described. (Refer to...) Figure 5 This illustrates a method 500 for a remote direct memory access system (e.g., RDMA system 100), the system including a requesting device (e.g., requesting device 102) and a responding device (e.g., responding device 104). Method 500 is executed by the RDMA system 100, which has, for example... Figure 1 The method is described in detail below. Method 500 includes steps 502 through 510.

[0118] In step 502, method 500 includes requesting device 102 sending a message including a prefetch operation to responding device 104, the prefetch operation indicating a memory region to be loaded by responding device 104 into the memory of responding device 104 (e.g., memory 210). The message including the prefetch operation includes the memory region prefetched at responding device 104, which further accelerates data communication in RDMA system 100.

[0119] In step 504, method 500 further includes responding device 104 receiving a message including a prefetch operation and loading the indicated memory region into memory 210. A controller 212 of responding device 104 is configured to load the indicated memory region into memory 210 of responding device 104.

[0120] In step 506, method 500 further includes the requesting device 102 sending a request to the responding device 104, the request relating to requested data and a memory region. The controller 204 of the requesting device 102 is used to send the request to the responding device 104 via communication interface 206. The request (or RDMA transaction) relating to the requested data and memory region may be reading requested data from the memory region, writing requested data to the memory region, storing requested data in the memory region, or many similar actions.

[0121] In step 508, method 500 further includes a responding device 104 receiving a request from the requesting device 102, executing the request on the requested data in the memory region, and sending a response message to the requesting device 102. The controller 212 of the responding device 104 is used to receive the request (i.e., an RDMA transaction) from the requesting device 102. The controller 212 of the responding device 104 is also used to execute the request on the requested data in the memory region. The controller 212 of the responding device 104 is also used to send a response message to the requesting device 102.

[0122] In step 510, method 500 further includes requesting device 102 receiving a response message from responding device 104. A controller 204 of requesting device 102 is used to receive the response message from responding device 104. The response message provides requesting device 102 with information regarding whether the request related to the requested data and memory region was successfully executed.

[0123] Steps 502 to 510 are merely illustrative and other alternatives may be provided, in which one or more steps are added, one or more steps are deleted, or one or more steps are provided in a different order.

[0124] Figure 6 This is an illustration of an exemplary implementation scenario of a remote direct memory access (RDMA) system with prefetch hints provided by embodiments of the present invention. Figure 6Combination Figure 1 , Figure 2A , Figure 2B , Figure 3 , Figure 4 and Figure 5 The components are described. (Refer to...) Figure 6 An exemplary scenario of an RDMA system 600 including a requesting device 602 and a responding device 604 is illustrated. The requesting device 602 sends a prefetch cue message 606 and a request 608 to the responding device 604. The responding device 604 sends an acknowledgment message 610 to the requesting device 602 as a response (or acknowledgment) to the request 608. Memory page loading latency 612 and total transaction latency 614 are also illustrated.

[0125] In the RDMA system 600, the requesting device 602 and the responding device 604 correspond to Figure 1 The requesting device 102 and the responding device 104.

[0126] In operation, requesting device 602 sends a prefetch cue message 606 to responding device 604 as a special data packet or as an additional payload (or header) to an existing data packet. The prefetch cue message 606 includes a prefetch operation that prefetches a memory address (e.g., a virtual address (VA 0x1000)) at responding device 604. Upon receiving the prefetch cue message 606, responding device 604 begins prefetching the memory address (i.e., VA 0x1000) and loads the memory address (i.e., VA 0x1000) into its memory (e.g., memory 210). At this point, the memory address (i.e., VA 0x1000) of responding device 604 is ready to accept any requests related to data and the memory address (i.e., VA 0x1000). The time taken by responding device 604 from receiving the prefetch cue message 606 to loading the memory address (i.e., VA 0x1000) into memory 210 is referred to as the memory page load latency 612. Optionally, in one case, the responding device 604 may send a response message to the requesting device 602, providing information about the successful execution of the prefetch cue message 606. In another case, the requesting device 602 may request a response message from the responding device 604 to know about the execution of the prefetch cue message 606. The transmission of the response message for the prefetch cue message 606 may be implicit and dynamic, or it may be configurable and negotiated between the requesting device 602 and the responding device 604. The requesting device 602 also sends a request 608 (e.g., as a write request for WRITE (VA 0x1000)) to the responding device 604. The responding device 604 receives the request 608 (i.e., WRITE (VA 0x1000)) and writes the requested data at the memory address (i.e., VA 0x1000). After executing the request 608 (i.e., WRITE (VA 0x1000)), the responding device 604 sends an acknowledgment message 610 to the requesting device 602. The requesting device 602 receives an acknowledgment message 610, which provides information about the successful execution of request 608 at the responding device 604. After receiving the acknowledgment message 610, the requesting device 602 can begin transmitting another request. The total time spent by the requesting device 602 from sending the prefetch cue message 606 to receiving the acknowledgment message 610 is called the total transaction delay 614. In the RDMA system 600, the total transaction delay 614 (or total completion time) is lower because, compared to a conventional RDMA system that does not use prefetch cue messages, the responding device 604 neither needs to discard request 608 nor wait for retransmission. For example, in... Figure 1 and Figure 2A The limitations of traditional RDMA systems have been described in detail in the literature.

[0127] In an exemplary aspect, the present invention provides a system (e.g., RDMA system 100 or 600). The system (i.e., RDMA system 100 or 600) includes a requesting device (e.g., requesting device 102 or 602) and a responding device (e.g., responding device 104 or 604). In one implementation, the system (i.e., RDMA system 100 or 600) may further include a requesting device 102 comprising a memory 202, a communication interface 206, and a software module 208, and a responding device 104 comprising a memory 210, a communication interface 214, and a software module 216. For example, in Figure 1 , Figure 2A and Figure 2B The various operations of the requesting device 102 and the responding device 104 and their components have been described in detail.

[0128] Modifications to the embodiments of the invention described above may be made without departing from the scope of the invention. Terms such as “comprising,” “integrating,” “having,” and “is / are” used to describe and claim the invention are intended to be interpreted in a non-exclusive manner, allowing for the presence of items, components, or elements not explicitly described. Singular references should also be interpreted as relating to the plural. The word “exemplary” as used herein means “as an example, instance, or illustration.” Any embodiment described as “exemplary” is not necessarily to be construed as more preferred or advantageous than other embodiments, and / or as excluding combinations of features from other embodiments. The word “optionally” as used herein means “provided in some embodiments but not in others.” It should be understood that certain features of the invention described in the context of a single embodiment for clarity may also be provided in combination in a single embodiment. Conversely, various features of the invention described in the context of a single embodiment for brevity may also be provided individually or in any suitable combination or as embodiments of any other described aspect of the invention.

Claims

1. A requesting device (102, 602), characterized in that, Includes a memory (202), a controller (204), and a communication interface (206), wherein the controller (204) is used for: The communication interface (206) sends a message (606) including a prefetch operation to the response device (104, 604), the prefetch operation indicating that the response device (104, 604) will load the data into the memory area of ​​the memory (210) of the response device (104, 604). The response device (104, 604) receives an acknowledgment message for the prefetch operation, which indicates whether the prefetch operation was successfully executed. In response to the successful execution of the prefetch operation, a request (608) is sent to the response device (104, 604) through the communication interface (206), the request (608) being related to the request data and the memory region; Receive response message (610) from the response device (104, 604) through the communication interface (206). The request (608) is a read command that carries a memory address for reading the requested data from the memory address in the memory region, and the response message (610) is a message carrying the read requested data.

2. The requesting device (102, 602) according to claim 1, characterized in that, The memory region is larger than the requested data, and the controller (204) is also configured to send multiple requests related to the memory region to the response device (104, 604) via the communication interface (206).

3. The requesting device (102, 602) according to claim 1, characterized in that, The controller (204) is also configured to send the request (608) after a waiting period from the time the prefetch operation is sent.

4. The requesting device (102, 602) according to claim 1, characterized in that, The message (606) including the prefetch operation is a dedicated prefetch operation message.

5. The requesting device (102, 602) according to claim 1, characterized in that, The message (606) including the prefetch operation includes the prefetch operation in the additional payload of a request message for another request.

6. The requesting device (102, 602) according to claim 1, characterized in that, The requesting device (102, 602) is configured for remote direct memory access (RDMA).

7. A method for requesting a device (102, 602), characterized in that, The requesting device (102, 602) includes a memory (202), a controller (204), and a communication interface (206), and the method includes: The communication interface (206) sends a message (606) including a prefetch operation to the response device (104, 604), the prefetch operation indicating that the response device (104, 604) will load the data into the memory area of ​​the memory (210) of the response device (104, 604). The controller (204) receives a confirmation message for the prefetch operation from the response device (104, 604), the confirmation message indicating whether the prefetch operation was successfully executed; In response to the successful execution of the prefetch operation, a request (608) is sent to the response device (104, 604) through the communication interface (206), the request (608) being related to the request data and the memory region; Receive response message (610) from the response device (104, 604) through the communication interface (206). The request (608) is a read command that carries a memory address for reading the requested data from the memory address in the memory region, and the response message (610) is a message carrying the read requested data.

8. A computer-readable medium carrying computer instructions, characterized in that, When the computer instructions are loaded into and executed by the controller (204) of the requesting device (102, 602), the requesting device (102, 602) is enabled to perform the method according to claim 7.

9. A response device (104, 604), characterized in that, The response device (104, 604) includes a memory (210), a controller (212), and a communication interface (214), the controller (212) being used for: The communication interface (214) receives a message (606) from the requesting device (102, 602) including a prefetch operation, the prefetch operation indicating that the response device (104, 604) will load the message into a memory region of the memory (210); Load the indicated memory region into the memory (210); A confirmation message is sent to the requesting device (102, 602), the confirmation message indicating whether the controller (212) can load the indicated memory region into the memory; If the prefetch operation is successful, a request (608) is received from the requesting device (102, 602) through the communication interface (214), the request (608) being related to the request data and the memory region; Execute the request (608) on the requested data in the memory region. A response message (610) is sent to the requesting device (102, 602) through the communication interface (214). The request (608) is a read command that carries a memory address for reading the requested data from the memory address in the memory region, and the response message (610) is a message carrying the read requested data.

10. The response device (104, 604) according to claim 9, characterized in that, The controller (212) is further configured to determine whether the indicated memory region is stored in the memory (210) before loading the indicated memory region into the memory (210), and if the memory region is not stored in the memory (210), then load the indicated memory region into the memory (210).

11. The response device (104, 604) according to claim 9 or 10, characterized in that, The controller (212) is also configured to send an acknowledgment message to the requesting device (102, 602), the acknowledgment message indicating whether the controller (212) can load the indicated memory region into the memory (210).

12. A method for responding to a device (104, 604), characterized in that, The response device (104, 604) includes a memory (210), a controller (212), and a communication interface (214), and the method includes: The communication interface (214) receives a message (606) from the requesting device (102, 602) including a prefetch operation, the prefetch operation indicating that the response device (104, 604) will load the message into a memory region of the memory (210); Load the indicated memory region into the memory (210); The controller (212) sends an acknowledgment message to the requesting device (102, 602), the acknowledgment message indicating whether the controller (212) can load the indicated memory region into the memory; If the prefetch operation is successful, a request (608) is received from the requesting device (102, 602) through the communication interface (214), the request (608) being related to the request data and the memory region; Execute the request (608) on the requested data in the memory region. A response message (610) is sent to the requesting device (102, 602) through the communication interface (214). The request (608) is a read command that carries a memory address for reading the requested data from the memory address in the memory region, and the response message (610) is a message carrying the read requested data.

13. A computer-readable medium carrying computer instructions, characterized in that, When the computer instructions are loaded into and executed by the controller (212) of the response device (104, 604), the response device (104, 604) is enabled to perform the method according to claim 12.

Images