Apparatus, method and system for reducing latency in a remote direct memory access system

Abstract

To reduce latency in an RDMA system, a local timer is used and messages being sent are distributed in time so that necessary previous operations are completed before a new operation or command is sent. This is particularly useful for an RDMA system when sending prefetch commands.

Description

Technical Field

[0001] This invention generally relates to the field of data communication and remote direct memory access (RDMA) systems; more specifically, it relates to transmitting devices, receiving devices, systems, and methods for reducing latency in remote direct memory access systems. Background Technology

[0002] High-speed data communication between multiple computing devices in a network has raised concerns about reliable data communication and latency. Traditionally, multiple computing devices employ conventional Remote Direct Memory Access (RDMA) technology to improve data communication speed. Conventional RDMA technology allows a computing device (e.g., a conventional transmitting device) to directly access the memory of another computing device (e.g., a conventional receiving device), without involving the operating systems (OS) of either device. Conventional RDMA technology relies on synchronization events to maintain a specific execution order of data operations. For example, in an exemplary implementation of a conventional RDMA system, imperfect synchronization between the conventional transmitting and receiving devices can lead to unreliable data communication. That is, if the conventional transmitting device (i.e., the RDMA transmitter) initiates the transmission of data (e.g., data packets) with the conventional receiving device (i.e., the RDMA receiver) before the conventional receiving device performs a previous data operation, the transmission of the data (or data packets) is suspended or dropped. Furthermore, asynchronous legacy sending and receiving devices can lead to incomplete data (or data packets) transmission and retransmissions, which in turn increase latency (e.g., tail latency) and reduce throughput. Additionally, flow control cannot be applied to a single connection scheme; it can adversely affect other connections sharing the same traffic class (or traffic network). Flow control is not fine-grained on an event-by-event basis (e.g., page faults, cache misses), resulting in a significant increase in latency.

[0003] In another exemplary scenario, conventional non-fixed RDMA technology can be used to accelerate data communication. In conventional non-fixed RDMA, the conventional responding device requires the virtual address of the conventional requesting device (e.g., the target virtual address) to send data. If the virtual address of the conventional requesting device triggers a fault (e.g., a page fault), data transmission is paused or dropped, potentially leading to data retransmission, which further increases transaction completion latency.

[0004] In another exemplary scenario of traditional RDMA, when packets arrive at a traditional receiving device, additional information may be required, such as queue pair (QP) context, memory protection table (MPT), and memory translation table (MTT). Traditional transmitting and receiving devices have a limited number of network interface controllers (e.g., network interface cards). Therefore, if multiple connections are established using network interface controllers, information may be evicted from several connections to host memory due to the limited size of the network interface controller's buffer. Consequently, in traditional systems, retrieving information from host memory is frequent and time-consuming, often requiring several iterations, leading to increased latency and performance degradation.

[0005] In certain scenarios, traditional RDMA frameworks can use fencing to enforce the sequential execution of data operations. For example, when an RDMA request has a "fence flag," the request is delayed by the traditional RDMA transmitting device until the traditional RDMA receiving device sends a signal back to the traditional RDMA transmitting device indicating that the previous operation has been performed. In these cases, the total execution time of the RDMA request increases significantly due to the presence of fencing, resulting in high latency. Fence flags (or fences) can also overload the traditional RDMA receiving device when storing too many work queue elements (WQEs). Therefore, traditional devices, methods, and traditional remote direct memory access systems suffer from inefficient and unreliable data communication technical problems.

[0006] Therefore, based on the above discussion, it is necessary to overcome the aforementioned drawbacks associated with conventional transmitting devices, conventional receiving devices, and conventional methods of conventional remote direct memory access (RDMA) technology used for data communication. Summary of the Invention

[0007] This invention aims to provide transmitting devices, receiving devices, methods, and systems for reducing latency in remote direct memory access (RDMA) systems. It also aims to provide a solution for addressing existing problems of inefficient and unreliable data communication using conventional devices, methods, and conventional RDMA systems. The object of this invention is to provide a solution that at least partially overcomes the problems encountered in the prior art and to provide improved devices, methods, and systems for efficient (e.g., reduced latency) and reliable data communication. This object is achieved by the solutions provided in the appended independent claims. Advantageous implementations of the invention are further defined in the dependent claims.

[0008] In one aspect, the present invention provides a transmitting device. The transmitting device includes a memory, a controller, and a communication interface. The memory stores a local pacing timer. The controller generates a first message including an indication of a first operation type. The controller is also configured to transmit the first message to a receiving device via the communication interface. The controller further generates a second message including an indication of a second operation type. The controller is also configured to determine whether the second operation type is associated with the first operation type; if associated, the controller further determines whether the local pacing timer has exceeded the timer duration since the transmission of the first message, wherein the local pacing timer is a timer for the first operation type; if exceeded, the controller further transmits the second message to the receiving device via the communication interface.

[0009] The transmitting device of the present invention can determine whether a second operation type related to remote direct memory access (RDMA) is associated with a first operation type. In this case, the transmitting device can send a second message to a receiving device without waiting for an indication message from the receiving device indicating that the first operation type has been completed. The transmitting device uses a local pacing timer to pace the transmission of multiple messages (or data packets), including the first and second messages. The local pacing timer measures the timer duration since the transmission of the first message. After the local pacing timer exceeds the timer duration, the controller sends the second message to the receiving device without waiting for the indication message. Therefore, by using the local pacing timer, the transmitting device significantly reduces the total execution time of the first and second messages, thereby reducing latency. Furthermore, the local pacing timer can maintain the correct transmission order of the first and second messages, thus enabling reliable data communication and improving overall throughput and efficiency.

[0010] In one implementation, the controller is further configured to determine whether the second operation type is not associated with the first operation type. If not, the controller is further configured to send a second message to the receiving device via a communication interface, regardless of whether the local pacing timer has exceeded the timer duration since the first message was sent.

[0011] If the second operation type is not associated with the first operation type, the controller of the transmitting device sends a second message to the receiving device. Furthermore, the controller does not wait for the local pacing timer to exceed its duration, which in turn reduces latency (or improves fast data transactions) while maintaining the correct transmission order of the first and second messages.

[0012] In another implementation, the controller is also used to generate a third message. The third message includes an indication of a third operation type, which is associated with the first operation type. The controller is also used to determine whether the local pacing timer has exceeded the timer duration since the second message was sent; if so, the controller is further used to send the third message to the receiving device via the communication interface.

[0013] The third operation type is associated with the first operation type. For example, the third operation type begins execution while the second operation type has already been executed. The controller of the transmitting device uses a local pacing timer to pace the transmission of the third message sequentially after the transmission of the second message. The local pacing timer measures the timer duration since the transmission of the second message. After the local pacing timer exceeds its duration, the controller sends the third message to the receiving device without waiting for an indication message. Therefore, the use of the local pacing timer significantly reduces the total execution time of the first, second, and third messages, thereby reducing latency. Furthermore, the local pacing timer maintains the correct transmission order of the first, second, and third messages, enabling reliable data transmission and improving throughput and efficiency.

[0014] In another implementation, the third operation type is the second operation type. This allows multiple responses to be sent and received simultaneously using the same timer. In one example, if the second operation type is an RDMA read operation, the third operation type may also be an RDMA read operation. In this case, the response time of the receiving device is significantly reduced.

[0015] In another implementation, the first operation type is the second operation type. This supports sending multiple operations of the same type using the same local timer. In one example, if the first operation type is an RDMA write operation, the second operation type can also be an RDMA write operation. In this case, the response time of the receiving device is significantly reduced.

[0016] In another implementation, the controller is also used to receive an acknowledgment message from the receiving device and, in response to the acknowledgment message, send a second message to the receiving device through the communication interface, regardless of whether the local pacing timer has exceeded its duration.

[0017] In some cases, an acknowledgment message signifies that the first message was successfully sent to the receiving device. Therefore, upon receiving the acknowledgment message, the controller sends a second message to the receiving device regardless of whether the local pacing timer has exceeded its duration. Thus, the acknowledgment message reduces the total execution time of the first and second messages, thereby lowering latency. Reduced latency indicates faster data communication.

[0018] In another implementation, the controller is also used to adjust the timer duration.

[0019] The local pacing timer is dynamic and can be configured via hardware, firmware, or software to account for different network conditions, such as the number of active connections, network congestion, and memory policies on RDMA devices (e.g., responders). For example, sending multiple messages over a communication interface can increase the duration. Therefore, the controller adjusts the timer duration based on the timer length to maintain the correct message sending order and reduce latency while increasing throughput.

[0020] In another implementation, the controller is also used to adjust the timer duration based on one or more of the following: the number of active connections established through the communication interface, information about the operating status of the receiver (i.e., the receiving device), the number of negative acknowledgments (e.g., NAK) messages received, and indications received from the receiver (i.e., the receiving device).

[0021] The aforementioned varying network conditions are the valid reasons for the timer duration changes. Therefore, the controller is adjusted according to these network conditions so that the local pacing timer can determine the correct duration for sending the second message to the receiving device.

[0022] In another implementation, the timer duration is associated with the operation type, wherein the first timer duration is associated with one operation type and the second timer duration is associated with another operation type.

[0023] The durations of the first and second timers may differ because different operation types require different amounts of time. Therefore, a local pacing timer can be used to account for different operation types to maintain the correct transmission order of different messages by the transmitting device.

[0024] In another implementation, the first operation type is associated with a memory region, and the second operation type is associated with the same memory region.

[0025] Associating the first and second operation types with the same memory region speeds up data communication.

[0026] In another implementation, the controller is also used to determine, in addition to determining whether the local pacing timer has exceeded the timer duration since the first message was sent, whether the traffic congestion timer has exceeded the congestion timer duration since the first message was sent. If it has, the controller sends the second message to the receiving device through the communication interface.

[0027] Network congestion can occur when multiple messages are sent simultaneously during multiple remote direct memory access operations on a communication interface. A flow congestion timer is used to determine the duration of the congestion timer since the first message was sent, in order to successfully send the second message in the event of network congestion. Therefore, the controller provides reliable data transmission by using a flow congestion timer.

[0028] In another implementation, the controller is also used to determine, in addition to determining whether the local pacing timer has exceeded the timer duration since the first message was sent, whether the local pacing timer has exceeded the traffic congestion timer duration since the first message was sent. If it has exceeded the timer duration, the controller sends the second message to the receiving device through the communication interface.

[0029] The controller can control the duration of the local pacing timer and the traffic congestion timer since the first message was sent. When the local pacing timer exceeds the duration of both the local pacing timer and the congestion timer since the first message was sent, the controller sends a second message to the receiving device. Therefore, in the event of network congestion, the controller can successfully send the second message by using the local pacing timer and the traffic congestion timer.

[0030] In another implementation, the transmitting device acts as a requesting device in a remote direct memory access system. The requesting device uses a local pacing timer to determine the precise duration for transmitting the second message in the RDMA system, thereby ensuring reliable transmission of the second message.

[0031] In another implementation, a first operation type indicates a preparatory action to be performed by the receiving device, and a second operation type indicates the operation to perform the preparatory action. The preparatory action allows the receiving device to prepare its memory (e.g., a local memory buffer) before an actual remote direct memory access (RDMA) request arrives at the receiving device. Performing the preparatory action reduces the execution time of the second operation type, thereby reducing overall transaction latency and making data communication faster.

[0032] In another implementation, the transmitting device serves as a response device in a remote direct memory access system. The transmitting device has the capability to function as both a requesting device and a response device as needed.

[0033] In another implementation, the first operation type indicates a read operation, the first message indicates a read result, the second operation type indicates a read operation, and the second message indicates a subsequent read result. Therefore, the second operation type is executed consecutively with the first operation type, and the second message provides a subsequent read result based on the read result provided by the first message.

[0034] In another aspect, the present invention provides a transmitting device. The transmitting device includes a memory, a communication interface, and a software module. The software module includes a first message generator software module for generating a first message. The first message includes an indication of a first operation type. The software module also includes a first message sender software module for sending the first message to a receiving device via the communication interface. The software module further includes a second message generator software module for generating a second message. The second message includes an indication of a second operation type. The software module also includes an operation type association determination software module for determining whether the second operation type is associated with the first operation type; if associated, a duration determination software module is executed to determine whether a local pacing timer has exceeded the timer duration since the first message was sent. The local pacing timer is a first operation type timer; in this case, the second message sender software module is executed to send the second message to the receiving device via the communication interface.

[0035] The software module is executed to determine whether a second operation type is associated with a first operation type in a remote direct memory access operation. In this case, the software module can send a second message to the receiving device without waiting for an indication message from the receiving device indicating that the first operation type has completed. A duration determination software module is used to pace the transmission of the first and second messages. A local pacing timer is used to measure the timer duration since the first message was sent. After the local pacing timer exceeds its duration, the second message transmitter software module sends the second message to the receiving device without waiting for the indication message. Therefore, when executed, the duration determination software significantly reduces the total execution time of the first and second messages, thereby accelerating data communication. Furthermore, when executed, the duration determination software is also able to maintain the correct transmission order of the first and second messages, thereby improving overall throughput and data communication reliability.

[0036] In another aspect, the present invention provides a method for a transmitting device. The method includes generating a first message, the first message including an indication of a first operation type. The method further includes sending the first message to a receiving device via a communication interface. The method further includes generating a second message, the second message including an indication of a second operation type. The method further includes determining whether the second operation type is associated with the first operation type; if associated, determining whether a local pacing timer has exceeded a timer duration since the first message was sent, wherein the local pacing timer is a timer for the first operation type; if exceeded, sending the second message to the receiving device via the communication interface.

[0037] The method in this aspect achieves all the advantages and effects of the transmitting device of the present invention.

[0038] In one implementation, a computer-readable medium carrying computer instructions is provided, which, when loaded into and executed by a controller of the transmitting device, enable the transmitting device to implement the method.

[0039] A computer-readable medium carrying computer instructions (specifically, a non-transitory computer-readable medium) achieves all the advantages and effects of a transmitting device or method.

[0040] In another aspect, the present invention provides a receiving device including a communication interface and a controller, wherein the controller is configured to receive a first message from a transmitting device via the communication interface, the first message indicating a first operation. The controller is also configured to perform the first operation. The controller is further configured to receive a second message from the transmitting device via the communication interface, the second message indicating a second operation, wherein the second operation is associated with the first operation. The controller is also configured to determine whether the first operation is being performed, and if so, generate an indication message including an indication of this result, and send the indication message to the transmitting device.

[0041] Because the second operation is associated with the first operation, the receiving device's controller determines whether the first operation is being executed before starting to execute the second message. This is to ensure that the receiving device's controller executes the first and second messages in the correct order. The receiving device's controller sends an indication message to the sending device to confirm that the first operation was executed successfully. This reduces buffer overflows on the receiving device and increases overall throughput.

[0042] In another aspect, the present invention provides a receiving device. The receiving device includes a memory, a communication interface, and a software module. The software module includes a first message receiving software module for receiving a first message from a transmitting device via the communication interface. The first message indicates a first operation. The software module also includes an operation execution software module for performing the first operation. The software module includes a second message receiving software module for receiving a second message from the transmitting device via the communication interface. The second message indicates a second operation, wherein the second operation is associated with the first operation. The software module also includes an operation execution determining software module for determining whether the first operation is being performed, and if so, executing: an indication message generating software module for generating an indication message including an indication of this result; and an indication message sending software module for sending the indication message to the transmitting device.

[0043] The operation execution determination software module determines whether the first operation is being executed before starting to execute the second message. This ensures that the receiving device's controller executes the first and second messages in the correct order. The receiving device's indication message sender software module sends an indication message to the sending device to confirm that the first operation has been executed. This reduces buffer overflows on the receiving device and increases overall throughput.

[0044] In another aspect, the present invention provides a method for a receiving device. The method includes receiving a first message from a transmitting device via a communication interface, the first message indicating a first operation. The method further includes performing the first operation. The method also includes receiving a second message from the transmitting device via the communication interface, the second message indicating a second operation, wherein the second operation is associated with the first operation. The method further includes determining whether the first operation is being performed, and if so, generating an indication message including an indication of this result, and sending the indication message to the transmitting device. The method of this aspect achieves all the advantages and effects of the receiving device of the present invention.

[0045] In one implementation, a computer-readable medium carrying computer instructions is provided, which, when loaded into and executed by a controller of a receiving device, enable the receiving device to implement the method. The computer-readable medium carrying the computer instructions achieves all the advantages and effects of the receiving device or method.

[0046] In another implementation, the present invention provides a system. This system includes a transmitting device and a receiving device according to the preceding claims. The system of the present invention provides improved data communication reliability and low latency, which in turn accelerates the speed of data communication.

[0047] It should be noted that all devices, elements, circuits, units, modules, and components described in this application can be implemented by software or hardware elements or any combination thereof. All steps performed by the various entities described in this application and the functions described as being performed by the various entities are intended to indicate that the respective entities are used to perform the corresponding steps and functions. Although the specific functions or steps performed by external entities are not reflected in the detailed description of the specific elements of the entities performing the specific steps or functions in the following detailed description of specific embodiments, those skilled in the art will understand that these methods and functions can be implemented by corresponding hardware or software elements or any combination thereof. It should be understood that various combinations of the features of the invention can be made without departing from the scope of the invention as defined in the appended claims.

[0048] Additional aspects, advantages, features and objects of the invention will become apparent from the accompanying drawings and the detailed description of illustrative implementations as explained in conjunction with the following appended claims. Attached Figure Description

[0049] A better understanding of the above overview and the following detailed description of illustrative embodiments can be obtained by reading the accompanying drawings. Exemplary structures of the invention are shown in the drawings to illustrate the invention. However, the invention is not limited to the specific methods and tools disclosed herein. Furthermore, those skilled in the art will understand that the drawings are not drawn to scale. Where possible, the same elements are represented by the same numbers.

[0050] The following figures will now be used as examples to describe embodiments of the present invention.

[0051] Figure 1 This is a network environment of an exemplary remote direct memory access (RDMA) system having a transmitting device and a receiving device according to an embodiment of the present invention.

[0052] Figure 2A This is a block diagram of various exemplary components of a transmitting device according to an embodiment of the present invention.

[0053] Figure 2B This is a block diagram of various exemplary components of a receiving device according to an embodiment of the present invention.

[0054] Figure 3 This is a flowchart of a method for a transmitting device according to an embodiment of the present invention.

[0055] Figure 4 This is a flowchart of a method for a receiving device according to an embodiment of the present invention.

[0056] Figure 5This is an illustration of an exemplary scenario of remote direct memory access (RDMA) operation with a prefetch flow pacemaker according to an embodiment of the present invention.

[0057] Figure 6 This is an illustration of an exemplary scenario of remote direct memory access (RDMA) operation with a prefetch flow pacemaker according to another embodiment of the present invention.

[0058] Figure 7 This is an illustration of an exemplary scenario of implementing a remote direct memory access (RDMA) system according to an embodiment of the present invention.

[0059] In the accompanying diagram, underlined numbers indicate the item to which the underlined number is located or the item adjacent to the underlined number. Ununderlined numbers refer to the items identified by the line connecting the ununderlined number to the item. When a number is ununderlined and has an associated arrow, the ununderlined number identifies the general item that the arrow points to. Detailed Implementation

[0060] The following detailed description illustrates embodiments of the present invention and ways in which these embodiments can be implemented. Although some modes of implementing the invention have been disclosed, those skilled in the art will recognize that other embodiments for implementing or practicing the invention may also exist.

[0061] Figure 1 This is a network environment diagram of an exemplary remote direct memory access (RDMA) system having a transmitting device and a receiving device according to an embodiment of the present invention. (Reference) Figure 1 A network diagram of an RDMA system 100, including transmitting device 102 and receiving device 104, is shown. Network 106 is also shown.

[0062] In the remote direct memory access (RDMA) system 100, the transmitting device 102 can directly access the memory of the receiving device 104 without involving the operating systems (OS) of the transmitting and receiving devices 102 and 104. In the RDMA system 100, data is transmitted directly without copying between the network layers of the transmitting and receiving devices 102 and 104. For example, in the RDMA system 100, data is transmitted in the form of one or more data packets. The data packets include information associated with a packet sequence number (PSN) to enforce correct packet ordering. In one example, the data packets also include a source queue pair (QP) number and a destination QP number, which distinguish the application and the valid memory region of the application. In one embodiment, the memory region has a memory key (R-key) as part of the data packet, which associates the memory region with the applicant and vice versa. The data packets also include information related to the message type (e.g., RDMA read, RDMA write, send, or atomic), as well as message parameters such as message length, destination memory address, operation type, and operand data. In RDMA system 100, pacing of any number of consecutive data packets is achieved by using a timer to control (or program) the time intervals between the transmissions of various data packets. Programming the time intervals between the transmissions of various data packets enables reliable data communication and reduced latency, which in turn speeds up data communication on conventional RDMA systems without time interval programming.

[0063] Transmitting device 102 includes appropriate logic, circuitry, interfaces, and / or code for a high-speed data communication system, such as Remote Direct Memory Access system 100. Transmitting device 102 can directly access the memory of receiving device 104 to perform any RDMA operation, such as RDMA read, RDMA write, transmit, or atomic operations. Transmitting device 102 uses locally running timers (e.g., in...). Figure 2AThe locally running timer (described in detail below) is used to pace the transmission of different data packets. The locally running timer is dynamic and can be configured via hardware, software, or firmware to account for various network conditions, such as network congestion and memory policies. The locally running timer can be started after a data packet is sent, which can be marked as an event packet, such as a "hint" packet or a "fence" packet. The locally running timer is used independently for different data packets, regardless of event packets. Using a locally running timer to send different data packets reduces the likelihood of packets being dropped or paused at the receiving device 104, thus enabling reliable data communication. Furthermore, using a locally running timer at the transmitting device 102 avoids the penalty of waiting for a full round-trip time when receiving acknowledgment messages for event packets such as "hint" or "fence" packets, thus achieving low latency. Fine-tuning (or programming) of the locally running timer reduces processing time at the receiving device 104. The locally running timer can be detected by observing wired traffic and detecting special data packets (such as event packets). In this way, by using a locally running timer, the transmitting device 102 enables the RDMA system 100 to have improved data communication reliability and low latency. The low latency of the RDMA system 100 translates to improved data communication speed, meaning that the RDMA system 100 is significantly faster than conventional RDMA systems. The transmitting device 102 may function as a requesting device (or requester) in the RDMA system 100. Examples of the transmitting device 102 may include, but are not limited to, network adapters, servers, computing devices in computer clusters (e.g., massively parallel computer clusters), communication devices including portable or non-portable electronic devices, telematics control units (TCUs) in vehicles, drones, wireless modems, supercomputers, or other RDMA-based devices. For example, in Figure 2A Various exemplary components of the transmitting device 102 are described in detail.

[0064] Receiving device 104 includes appropriate logic, circuitry, interfaces, and / or code for high-speed data communication systems, such as remote direct memory access (RDMA) system 100. Receiving device 104 prepares its memory (e.g., a local memory buffer) shortly after receiving an event packet (e.g., a "prompt" packet or a "fence" packet) from transmitting device 102 to reduce response time. Receiving device 104 does not need to send an acknowledgment message of successful reception of the event packet to transmitting device 102 using a locally running timer. This, in turn, reduces the overall transaction time of the RDMA operation and speeds up data communication. Receiving device 104 can potentially be used as a responding device (or responder) in RDMA system 100. Examples of receiving device 104 may include, but are not limited to, network adapters, servers, computing devices in computer clusters (e.g., massively parallel computer clusters), communication devices including portable or non-portable electronic devices, telematics control units (TCUs) in vehicles, drones, wireless modems, supercomputers, or other RDMA-based devices. For example, in Figure 2B The various exemplary components of the receiving device 104 are explained in detail.

[0065] Network 106 includes a medium (e.g., a communication channel) through which transmitting device 102 may communicate with receiving device 104. Examples of network 106 include, but are not limited to, computer networks in a computer cluster, local area networks (LANs), cellular networks, wireless sensor networks (WSNs), cloud networks, vehicle-to-network (V2N) networks, metropolitan area networks (MANs), and / or the Internet. Transmitting device 102 in the network environment is used to connect to receiving device 104 according to various network protocols supporting RDMA. Examples of these network protocols, communication standards, and technologies may include, but are not limited to, InfiniBand (IB), RDMA over converged Ethernet (RoCE), the Internet Wide Area RDMA protocol (iWARP), or modifications and variations thereof.

[0066] Figure 2A This is a block diagram of various exemplary components of a transmitting device according to an embodiment of the present invention. (Reference) Figure 1The diagram illustrates a transmitting device 102. The transmitting device 102 includes a memory 202, which also includes a local pacing timer 204. The transmitting device 102 also includes a controller 206 and a communication interface 208. In one implementation, the transmitting device 102 further includes one or more software modules, such as software module 210.

[0067] Memory 202 includes appropriate logic, circuitry, and / or interfaces for storing executable instructions to control transmitting device 102. Memory 202 may store data (transmitted in the form of data packets) for processing at transmitting device 102. Examples of implementations of memory 202 may include, but are not limited to, electrically erasable programmable read-only memory (EEPROM), random access memory (RAM), read-only memory (ROM), hard disk drive (HDD), flash memory, secure digital (SD) cards, solid-state drives (SSDs), and / or CPU cache memory. Memory 202 may store an operating system and / or other program products to operate transmitting device 102. Computer-readable storage media used to provide non-transitory memory may include, but are not limited to, electronic storage devices, magnetic storage devices, optical storage devices, electromagnetic storage devices, semiconductor storage devices, or any suitable combination thereof.

[0068] The local pacing timer 204 is a dedicated type of clock used to measure time intervals during the transmission of different data packets. The local pacing timer 204 counts down from a specified time interval to generate a time delay. Remote direct memory access (RDMA) operations are typically time-sensitive, and the transmitted data packets rely on specific timing events to reduce latency and increase throughput. The local pacing timer 204 is used to pace multiple data packets in a RDMA system. Additionally, the local pacing timer 204 is used to maintain the correct transmission order of multiple data packets based on the time intervals between them. In one implementation, the local pacing timer 204 may differ from a globally available timer. The local pacing timer 204 is dynamic and configurable based on different network conditions. The local pacing timer 204 is also used in situations where a "hint" packet is not used before the transmission of a data packet. The local pacing timer can also be referred to as a prefetch flow pacer.

[0069] Controller 206 includes appropriate logic, circuitry, and / or interfaces for generating a first message, which includes an indication of a first operation type. Controller 206 is a computing element for processing instructions that drive the transmitting device 102. Examples of controller 206 include, but are not limited to, network interface controllers, microprocessors, microcontrollers, complex instruction set computing (CISC) microprocessors, reduced instruction set computing (RISC) microprocessors, or very long instruction word (VLIW) microprocessors.

[0070] Communication interface 208 is an arrangement of interconnected programmable and / or non-programmable components used to facilitate data communication between one or more electronic devices. For example, communication interface 208 refers to a network interface card (NIC) for processing send queues (SQ), reading work queue elements (WQE), and generating data packets for transmission to another computing device. Communication interface 208 may support one or more of the following communication protocols: peer-to-peer networks, hybrid peer-to-peer networks, local area networks (LANs), radio access networks (RANs), metropolitan area networks (MANSs), wide area networks (WANs), all or part of public networks (such as the global computer network known as the Internet), private networks, cellular networks, and any other one or more communication systems at one or more locations. Additionally, communication interface 208 supports wired or wireless communications that can be performed via any number of known protocols, including but not limited to Internet Protocol (IP), Wireless Access Protocol (WAP), Frame Relay, or Asynchronous Transfer Mode (ATM). In addition, the communication interface 208 can also adopt and support any other suitable protocols using voice, video, data, or a combination thereof.

[0071] In an exemplary implementation, software module 210 includes one or more message generator software modules (e.g., a first message generator software module 210a and a second message generator software module 210b), one or more message sender software modules (e.g., a first message sender software module 210c and a second message sender software module 210d), an operation type association determination software module 210e, and a duration determination software module 210f. In one implementation, software module 210 (including software modules 210a to 210f) may be implemented as separate circuitry within the transmitting device 102. Alternatively, in another implementation, software module 210 is implemented as circuitry performing various operations of software modules 210a to 210f.

[0072] In operation, controller 206 generates a first message that includes an indication of a first operation type. The first message, generated by controller 206, initiates a remote direct memory access operation with another computing device (i.e., receiving device 104). The first message indicates the first operation type, such as an RDMA read operation, an RDMA write operation, a send operation, or an atomic operation. For example, the first message is sent in the form of one or more data packets. Controller 206 of sending device 102 is also used to process the send queue (SQ), read work queue elements (WQE), generate data packets, and send them to receiving device 104.

[0073] In one example, the first message (or data packet) may include a packet sequence number (PSN) to enforce the correct order of the first message and subsequent messages (or data packets). The first message also includes a source queue pair (QP) number and a destination QP number. Each queue pair (QP) number has a context in both the sending device 102 and another computing device (i.e., the receiving device 104). The first message also includes various parameters such as message length, destination memory address, operation type, and operand data. The message length includes the length of an RDMA read operation, RDMA write operation, or send operation. The memory address includes the destination memory address for RDMA read operations, RDMA write operations, and atomic operations. Send operations do not have a destination memory address. The atomic operation type includes operation type and operand data.

[0074] The controller 206 is also configured to send a first message to the receiving device 104 via the communication interface 108. The first message is sent to the receiving device 104 to process a first operation type. For example, if the first operation type is a remote direct memory access (RDMA) read operation, the receiving device 104 performs an RDMA read operation according to the first message.

[0075] The controller 206 is also configured to generate a second message, which includes an indication of a second operation type. In the sequence of the first message, the second message is generated by the controller 206 to initiate a second remote direct memory access (RDMA) operation with the receiving device 104. The second message indicates the second operation type, which can be an RDMA read operation, an RDMA write operation, a transmit operation, or an atomic operation. For example, the second message may also be transmitted in the form of one or more data packets. The data packets of the second message include various parameters (e.g., message type, message length, or memory address) with different values ​​relative to the data packets of the first message.

[0076] The controller 206 is also configured to determine whether the second operation type is associated with the first operation type. If associated, it determines whether the local pacing timer 204 has exceeded the timer duration since the first message was sent, where the local pacing timer 204 is the timer for the first operation type. If it has exceeded the timer duration, it sends the second message to the receiving device 104 via the communication interface 208. Typically, the association between the second operation type and the first operation type is represented by fencing. In the case of fencing, if the second operation type is fenced by the first operation type, the second operation type is only executed after the first operation type has been executed. In this case, a conventional receiving device sends a response message (or multiple response messages) about the first operation type to a conventional transmitting device. The response message indicates that the first operation type should be executed. After receiving the response message, the conventional transmitting device sends the second message (including the second operation type) to the conventional receiving device; this complete cycle consumes a significant amount of time. This, in turn, increases latency and reduces the speed of data communication. However, in this invention, this problem is solved by using the local pacing timer 204. In the case of fencing, the second operation type is fenced by the first operation type, and the local pacing timer 204 is used by the transmitting device 102. A local pacing timer 204 is used to introduce a predefined time delay between the transmission of the second message (including the second operation type) and the execution of the first operation type. After the predefined time delay expires, the transmitting device 102 sends the second message (including the second operation type) to the receiving device 104, regardless of whether the first operation type has been fully executed. In this way, the local pacing timer 204 also reduces the total response time of the transmitting device 102 and the receiving device 104, which in turn reduces latency, thereby making data communication faster.

[0077] According to one embodiment, controller 206 is further configured to determine, in addition to determining whether local pacing timer 204 has exceeded the timer duration since the first message was sent, whether traffic congestion timer has exceeded the congestion timer duration since the first message was sent. If so, a second message is sent to receiving device 104 via communication interface 108. Typically, multiple messages are sent at once to initiate multiple remote direct memory access operations via communication interface 208. If communication interface 208 carries more messages than it can handle, this can lead to network congestion. Network congestion can cause queuing delays, message loss, or new connection blocking, which in turn can increase total tail latency and reduce throughput. For example, a second message might be sent when local pacing timer 204 exceeds the timer duration since the first message was sent, regardless of network congestion that could cause the second message to fail to send and result in message loss. Therefore, when network congestion occurs, it is necessary to control the sending of multiple messages. In the event of network congestion, the occurrence of multiple messages is controlled by using traffic congestion timers or other congestion control mechanisms. A traffic congestion timer is a special type of clock used to control the duration of a congestion timer. The congestion timer duration represents the time required for controller 206 to successfully send a message under network congestion conditions. The traffic congestion timer can be started when sending device 102 sends a first message and stopped when an acknowledgment of receipt of the first message arrives at sending device 102. Controller 206 is used to send a second message when both the local pacing timer 204 and the traffic congestion timer exceed their respective congestion timer durations since the first message was sent, in order to successfully send the second message under network congestion conditions.

[0078] According to one embodiment, the controller 206 is further configured to determine, in addition to determining whether the local pacing timer 204 has exceeded the timer duration since the first message was sent, whether the local pacing timer 204 has exceeded the congestion timer duration since the first message was sent. If it has, the controller 206 sends a second message to the receiving device 104 via the communication interface 208. The local pacing timer 204 can control the timer duration and the congestion timer duration since the first message was sent. When the local pacing timer 204 exceeds both the timer duration and the congestion timer duration since the first message was sent, the controller 206 sends a second message to the receiving device 104 via the communication interface 208.

[0079] According to one embodiment, controller 206 is further configured to determine whether the second operation type is not associated with the first operation type. If not, a second message is sent to receiving device 104 via communication interface 208, regardless of whether the local pacing timer 204 has exceeded the timer duration since the first message was sent. If the second operation type (e.g., a read or write operation) is not associated with (or not fenced) with the first operation type, the second message is sent to receiving device 104 via communication interface 208 regardless of whether the local pacing timer 204 has expired. Therefore, it is not necessary to determine whether the local pacing timer 204 has exceeded the timer duration since the first message was sent.

[0080] According to one embodiment, controller 206 is further configured to receive an acknowledgment message from receiving device 104, and in response to the acknowledgment message, send a second message to receiving device 104 via communication interface 208, regardless of whether the local pacing timer 204 has exceeded its timer duration. An acknowledgment message is a signal transmitted between communication devices to indicate successful message transmission. When controller 206 of transmitting device 102 receives an acknowledgment message from receiving device 104 after sending the first message, it means that receiving device 104 has successfully received the first message. Therefore, after receiving the acknowledgment message, controller 206 sends a second message to receiving device 104 regardless of whether the local pacing timer 204 has exceeded its timer duration.

[0081] According to one embodiment, the first operation type is the second operation type. For example, if the first operation type is a read operation, the second operation type can also be a read operation. In this case, the response time of the transmitting device 102 and the receiving device 104 is significantly reduced.

[0082] According to one embodiment, a first operation type indicates a read operation, a first message indicates a read result, a second operation type indicates a read operation, and a second message indicates a subsequent read result. In the first message, the sending device 102 sends a request for a read operation to be performed by the receiving device 104. Similarly, in the second message, after the read operation in the first message, the sending device 102 sends a request for another read operation to be performed by the receiving device 104. The first and second messages display subsequent read results related to each other.

[0083] According to one embodiment, a first operation type is associated with a memory region, and a second operation type is associated with the same memory region. For example, in one case, the first operation type is a read operation and is associated with a memory region of the receiving device 104 (e.g., a memory address or local memory buffer of the receiving device, etc.). The second operation type is a write operation and is performed by writing a specific value to the same memory region read by the first operation type (i.e., a memory address or local memory buffer of the receiving device 104, etc.). Then, in this case, the second operation type is associated with the same memory region as the first operation type.

[0084] According to one embodiment, controller 206 is further configured to generate a third message including an indication of a third operation type, wherein the third operation type is associated with the first operation type. Controller 206 is also configured to determine whether the local pacing timer 204 has exceeded the timer duration since the second message was sent; if so, to send the third message to receiving device 104 via communication interface 208. The third message is generated by controller 206 to initiate a third remote direct memory access operation with receiving device 104. The third message indicates the third operation type, which may be a read operation, write operation, send operation, or atomic operation. The third message is sent in the form of one or more data packets. The third operation type is fenced to force the first operation type, second operation type, and third operation type to be sent in a specific order with a predetermined time delay. For example, the third operation type is executed after a predetermined delay from the start of execution of the second operation type. Transmitting device 102 uses the local pacing timer 204 to control the predetermined time delay between the transmission of the third message and the second message. The predetermined time delay is the amount of time required to maintain the correct execution order of the third operation type and the second operation type. In one example, the local pacing timer 204 may be started by the controller 206 of the transmitting device 102 after sending a second message to the receiving device 104. In another example, the local pacing timer 204 may be started by the controller 206 of the transmitting device 102 after sending a first message to the receiving device 104. The local pacing timer 204 expires when the timer delay exceeds a predetermined timer duration since the sending of the second message. After the local pacing timer 204 expires, the controller 206 sends a third message to the receiving device 104 via the communication interface 208. Therefore, the local pacing timer 204 significantly reduces the total transmission time of the third and second messages and reduces latency.

[0085] In one example, the third message packet may include a packet sequence number (PSN) to enforce the correct order of the third messages. The third message may also include the source queue pair (QP) number and the destination QP number. The third message may also include memory addresses for read, write, and atomic operations. The third message may also include operation type and operand data.

[0086] According to one embodiment, the third operation type is the second operation type. For example, if the second operation type is a read operation, then under the same circumstances, the third operation type can also be a read operation.

[0087] According to one embodiment, the timer duration is associated with an operation type, wherein a first timer duration is associated with one operation type and a second timer duration is associated with another operation type. The first timer duration is the duration required to maintain the correct transmission order of the first and second messages on the communication interface 208 since the first message was sent. The second timer duration is the duration required to maintain the correct transmission order of the third and second messages on the communication interface 208 since the second message was sent. The first and second timer durations may vary depending on network conditions.

[0088] According to one embodiment, controller 206 is also used to adjust the timer duration. The duration of the first timer and the duration of the second timer can vary according to changes in the communication interface 208. Furthermore, the local pacing timer 204 is dynamic and configurable to account for different durations under varying network conditions. Therefore, controller 206 adjusts the timer duration to maintain the correct transmission order of the first, second, and third messages.

[0089] According to one embodiment, controller 206 is also configured to adjust the timer duration based on one or more of the following: the number of active connections established through communication interface 208, information regarding the operational status of the receiver (i.e., receiving device 104), the number of received negative acknowledgment (NAK) messages, and indications received from the receiver (i.e., receiving device 104). Multiple messages are sent simultaneously to initiate multiple remote direct memory access operations in communication interface 208. If communication interface 208 carries more messages than it can handle, this can lead to network congestion. For example, a second message might be sent when local pacing timer 204 exceeds the timer duration since the first message was sent, without considering network congestion that could cause the second message to fail to be sent and result in message loss. Therefore, controller 206 adjusts the timer duration based on network congestion.

[0090] Furthermore, the controller 206 adjusts according to the operating state of the receiver (i.e., receiving device 104). For example, the controller 206 adjusts to a timer duration determined by various conditions, such as the time delay of message memory reallocation, software and hardware failures of the receiver, or the rate at which the receiver (i.e., receiving device 104) executes messages.

[0091] A negative acknowledgment message (e.g., a negative acknowledgment (NAK)) is sent to transmitting device 102 to indicate rejection of a previously received message or to indicate some error in the receiver. The negative acknowledgment message (NAK) may increase the timer duration between the transmission of the first message and the transmission of the second message. Transmitting device 102 may need time to receive the indication from receiving device 104, which may further increase the timer duration. Therefore, controller 206 adjusts according to the change in timer duration to successfully transmit the second message.

[0092] According to one embodiment, the transmitting device 102 functions as a requesting device in a remote direct memory access system. The transmitting device 102 (or the requesting device) generates a message to be sent to the receiving device 104 in the RDMA system 100. According to another embodiment, the transmitting device functions as a responding device in the RDMA system 100.

[0093] According to one embodiment, a first operation type indicates a preparatory action to be performed by the receiving device 104, and a second operation type indicates an operation to perform the preparatory action. For example, the first operation type may include the execution of an event packet (e.g., a cue packet or a fence packet). Upon receiving an event packet, the receiving device 104 prepares its memory (e.g., a local memory buffer) for performing the second operation type (e.g., an actual RDMA write or read request). The preparation of memory by the receiving device 104 indicates the preparatory action to be performed by the receiving device 104, which in turn reduces the overall response time and speeds up data communication.

[0094] In one exemplary aspect, the transmitting device 102 includes a memory 202, a communication interface 208, and a software module 210. The software module 210 includes a first message generator software module 210a for generating a first message, the first message including an indication of a first operation type. The software module 210 also includes a first message sender software module 210c for sending the first message to the receiving device 104 via the communication interface 208. The software module 210 also includes a second message generator software module 210d for generating a second message, the second message including an indication of a second operation type. The software module 210 further includes an operation type association determination software module 210e for determining whether the second operation type is associated with the first operation type; if associated, then executing a duration determination software module 210f to determine whether a local pacing timer 204 has exceeded the timer duration since the first message was sent, wherein the local pacing timer 204 is a timer for the first operation type; if exceeded, then executing the second message sender software module 210d to send the second message to the receiving device 104 via the communication interface 208.

[0095] When the second operation type is associated with the first operation type, the second message transmitter software module 210d sends (or causes to send) the second message to the receiving device 104 without waiting for an indication message from the receiving device 104 indicating that the first operation type has been completed. The duration determination software module 210f is used to pace the transmission of the first and second messages. A local pacing timer 204 is used to measure the timer duration since the first message was sent. After the local pacing timer 204 exceeds the timer duration, the second message transmitter software module 210d sends the second message to the receiving device 104 without waiting for the indication message. Therefore, when executed, the duration determination software module 210f significantly reduces the total execution time of the first and second messages, thereby accelerating data communication. Furthermore, when executed, the duration determination software module 210f is also able to maintain the correct transmission order of the first and second messages, thereby improving overall throughput and data communication reliability. Software module 210 is executed by the controller 206 of the transmitting device 102.

[0096] Figure 2B This is a block diagram of various exemplary components of a receiving device according to embodiments of the present invention. (In conjunction with...) Figure 1 and Figure 2A Component description Figure 2B . refer to Figure 2B It shows ( Figure 1 The block diagram 200B shows the receiving device 104. The receiving device 104 includes a communication interface 212, a controller 214, and a memory 216. In one implementation, the receiving device 104 also includes one or more software modules, such as software module 218.

[0097] Communication interface 212 is an arrangement of interconnected programmable and / or non-programmable components used to facilitate data communication between one or more electronic devices. For example, a network interface card (NIC) is arranged in communication interface 212 to process incoming messages and perform various operations on those messages. Furthermore, communication interface 212 supports communication over various networks, such as peer-to-peer networks, hybrid peer-to-peer networks, local area networks (LANs), radio access networks (RANs), metropolitan area networks (MANSs), wide area networks (WANs), all or part of public networks (such as the global computer network known as the Internet), private networks, cellular networks, and any other one or more communication systems at one or more locations. Additionally, communication interface 212 supports wired or wireless communication that can be performed via any number of known protocols, including but not limited to Internet Protocol (IP), Wireless Access Protocol (WAP), Frame Relay, or Asynchronous Transfer Mode (ATM).

[0098] Controller 214 includes appropriate logic, circuitry, and / or interfaces for receiving a first message from a transmitting device (e.g., transmitting device 102) via communication interface 212, the first message indicating a first operation. Controller 214 is a computing element for processing instructions that drive receiving device 104. Examples of controller 214 include, but are not limited to, network interface controllers, microprocessors, microcontrollers, complex instruction set computing (CISC) microprocessors, reduced instruction set computing (RISC) microprocessors, or very long instruction word (VLIW) microprocessors.

[0099] In operation, controller 214 is used to receive a first message from transmitting device 102 via communication interface 212. The first message indicates a first operation. The first message indicates a first operation, such as a read operation, write operation, send operation, or atomic operation. The first message is sent in the form of one or more data packets. In one example, the data packets of the first message may include a packet sequence number (PSN) to enforce the correct order of the first messages. The first message also includes a source queue pair (QP) number and a destination QP number. The first message may also include memory addresses for read operations, write operations, and atomic operations. The first message also includes operation type and operand data. Controller 214 of receiving device 104 is also used to process incoming data packets from transmitting device 102, perform operations accordingly, and return information to transmitting device 102 when necessary.

[0100] The controller 214 is also used to perform a first operation. The controller 214 of the receiving device 104 processes the first message and performs the operation according to the first operation indicated by the first message. For example, the first message indicates a read operation as the first operation, and then the controller 214 of the receiving device 104 performs the read operation accordingly.

[0101] Controller 214 is also configured to receive a second message from transmitting device 102 via communication interface 212. The second message indicates a second operation, which is associated with the first operation. The second message indicates the type of the second operation, such as a read operation, write operation, send operation, or atomic operation. The second message is sent in the form of one or more data packets. The data packets of the second message include a packet sequence number (PSN) to enforce the correct order of the second messages. The second message also includes a source queue pair (QP) number and a destination QP number. The second message includes memory addresses for read, write, and atomic operations. The second message includes operation type and operand data. The second operation is associated with the first operation. For example, the second operation type is executed after a predefined time delay from the start of execution of the first operation type. Therefore, controller 214 receives the second message after a predefined time delay since the first message was sent has elapsed. This duration is maintained by a local pacing timer 204 of transmitting device 102.

[0102] Controller 214 is also configured to determine whether a first operation is being performed, and if so, generate an indication message including an indication of the result, and send the indication message to sending device 102. Since the second operation is associated with the first operation, controller 214 determines whether the first operation is being performed before starting to execute the second operation. This is to ensure that controller 214 sends the first and second messages in the correct order. If controller 214 has already started executing the first message, controller 214 of receiving device 104 generates an indication message indicating the result of the first operation performed by controller 214 of receiving device 104. Controller 214 sends the indication message to sending device 102 to confirm that the first operation was successfully executed. The indication message may include information based on the result of the first operation. Therefore, controller 214 can start executing the second message after the first operation is completed. This reduces buffer overflows on receiving device 104 and increases overall throughput.

[0103] In another embodiment, the second operation may not be associated with the first operation. Then, the controller 214 can receive the second message regardless of whether the first operation has been performed.

[0104] In one exemplary aspect, receiving device 104 includes memory 216, communication interface 212, and software module 218. Software module 218 includes a first message receiving software module 218a for receiving a first message from transmitting device 102 via communication interface 212, the first message indicating a first operation. Software module 218 also includes an operation execution software module 218b for performing the first operation. Software module 218 further includes a second message receiving software module 218c for receiving a second message from transmitting device 102 via communication interface 212, the second message indicating a second operation, wherein the second operation is associated with the first operation. Software module 218 also includes an operation execution determining software module 218d for determining whether the first operation is being performed, and if so, executing: an indication message generation software module 218e for generating an indication message including an indication of this result; and an indication message sender software module 218f for sending the indication message to transmitting device 102.

[0105] The operation execution determination software module 218d determines whether the first operation is being executed before starting the execution of the second message. This ensures that the controller 214 of the receiving device 104 executes the first and second messages in the correct order. The indication message sender software module 218f of the receiving device 104 sends an indication message to the sending device 102 to confirm that the first operation has been executed. This reduces buffer overflows on the receiving device 104 and increases overall throughput. Software module 218 is executed by the controller 214 of the receiving device 104.

[0106] Figure 3This is a flowchart of a method for a transmitting device according to an embodiment of the present invention. Figure 1 , Figure 2A and Figure 2B Component description Figure 3 . refer to Figure 3 This illustrates a method 300 for reducing latency in a remote direct memory access system (e.g., RDMA system 100). Method 300 is executed by a controller 206 of transmitting device 102, for example in… Figure 1 and Figure 2A The method 300 has been described in detail. It includes steps 302 through 312.

[0107] In step 302, method 300 includes generating a first message. The first message includes an indication of a first operation type. The first message is generated by the controller 206 of the transmitting device 102 to initiate a remote direct memory access operation with another computing device. The first message indicates a first operation type, such as a read operation, a write operation, a send operation, or an atomic operation.

[0108] In step 304, method 300 further includes sending a first message to the receiving device via communication interface 208. The first message is sent by the controller 206 of the sending device 102 to the receiving device 104 via communication interface 208. The receiving device 104 performs a first operation type based on the first message. For example, if the first operation type is a read operation, the receiving device 104 performs a read operation on the first message.

[0109] In step 306, method 300 further includes generating a second message. The second message includes an indication of a second operation type. The second message is generated by the controller 206 of the transmitting device 102 to initiate a remote direct memory access operation with the receiving device 104. The second message indicates a second operation type, such as a read operation, a write operation, a transmit operation, or an atomic operation.

[0110] In step 308, method 300 further includes determining whether a second operation type is associated with a first operation type. The controller 206 of transmitting device 102 determines whether the second operation type is associated with the first operation type. For example, in one case, the second operation type can be fenced to force the transmission of a second message and a first message in a specific order. Fence of the second operation type may cause transmitting device 102 to delay the transmission of the second message. Therefore, the second operation type is executed after a predefined time delay from the start of execution of the first operation type. Fence of the second operation type indicates association with the first operation type; therefore, a local pacing timer 204 is needed to introduce a predefined time delay to execute the first and second operation types in a specific order while consuming less time. In another case, if the second operation type is not fenced (or not associated) with the first operation type, the second operation type is executed regardless of whether the first operation type is executed. In this case, the local pacing timer 204 is not needed.

[0111] In step 310, method 300 further includes determining whether a local pacing timer 204 has exceeded a timer duration since the transmission of the first message, wherein the local pacing timer is a first operation type timer. In this scenario, if a second operation type is associated with the first operation type, the transmitting device 102 uses the local pacing timer 204 to control the time delay between the transmission of the first message and the transmission of the second message. The timer duration is the amount of time required to maintain the correct execution order of the first and second operation types. The local pacing timer 204 can be started and expired according to the determined timer duration. For example, the local pacing timer 204 can be started by the controller 206 of the transmitting device 102 after the first message is transmitted to the receiving device 104. The local pacing timer 204 expires when the time delay exceeds a predetermined timer duration since the transmission of the first message. As discussed above, before transmitting the second message to the receiving device 104 via the communication interface 108, it can also be determined whether a traffic congestion timer has exceeded a congestion timer duration since the transmission of the first message.

[0112] In step 312, method 300 further includes sending a second message to receiving device 104 via communication interface 208. After the local pacing timer 204 expires, controller 206 sends the second message to receiving device 104 via communication interface 208. Therefore, the local pacing timer 204 significantly reduces the total transmission time of the first and second messages and reduces latency, which in turn speeds up data communication.

[0113] Steps 302 and 312 are merely illustrative, and other alternatives may be provided, in which one or more steps are added, one or more steps are deleted, or one or more steps are provided in a different order, without departing from the scope of the claims herein.

[0114] According to one embodiment, a computer-readable medium carrying computer instructions is provided that, when loaded into and executed by a controller 206 of a transmitting device 102, enable the transmitting device 102 to implement method 200. The computer-readable medium carrying the computer instructions provides non-transitory memory, which may include, but is not limited to, electronic storage devices, magnetic storage devices, optical storage devices, electromagnetic storage devices, semiconductor storage devices, or any suitable combination thereof.

[0115] Figure 4 This is a flowchart of a method for a receiving device according to an embodiment of the present invention. Figure 1 and Figure 3 Component description Figure 4 . refer to Figure 4 Method 400 is illustrated. Method 400 is executed by the controller 214 of the receiving device 104, for example in... Figure 1 and Figure 2B The method 400 has been described in detail. The method 402 includes steps 402 to 410.

[0116] In step 402, method 400 includes receiving a first message from transmitting device 102 via communication interface 212. The first message indicates a first operation. The first message indicates the first operation, such as a read operation, a write operation, a send operation, or an atomic operation.

[0117] In step 404, method 400 further includes performing a first operation. The first operation is performed by controller 214 of receiving device 104. For example, if the first operation is a read operation, the read operation is performed by controller 214.

[0118] In step 406, method 400 further includes receiving a second message from transmitting device 102 via communication interface 212. The second message indicates a second operation, wherein the second operation is associated with the first operation. Controller 214 is used to receive the second message. The second message indicates a second operation type, such as a read operation, write operation, send operation, or atomic operation. The second operation is associated with the first operation. For example, the second message is fenced by the first message, and the second operation type is executed after a predetermined delay from the start of execution of the first operation type. Therefore, controller 214 receives the second message after a timer duration since the first message was sent has elapsed. This timer duration is maintained by a local pacing timer 204 of transmitting device 102.

[0119] In step 408, method 400 further includes determining whether a first operation is being performed. Since the second operation is associated with the first operation, controller 214 determines whether the first operation was successfully performed before starting to execute the second message. This is to ensure that controller 214 of receiving device 104 executes the first and second messages in the correct order.

[0120] Method 400 further includes step 410: if a first operation is being performed, an indication message including the result is generated and sent to sending device 102. If controller 214 has already started executing the first message, controller 214 of receiving device 104 generates an indication message indicating the result of the first operation. Controller 214 sends the indication message to sending device 102 to confirm that the first operation is being performed. Therefore, controller 214 can start executing the second message before the first operation is completed. This reduces buffer overflows on receiving device 104 and increases overall throughput.

[0121] Steps 402 and 410 are merely illustrative, and other alternatives may be provided, in which one or more steps are added, one or more steps are deleted, or one or more steps are provided in a different order, without departing from the scope of the claims herein.

[0122] In one embodiment, receiving device 104 is used to implement a pacing timer for sending a response or other message to transmitting device 102. Effectively, in this implementation, receiving device 104 functions as both a receiving device and a transmitting device 102.

[0123] According to one embodiment, a computer-readable medium carrying computer instructions is provided that, when loaded into and executed by a controller 214 of a receiving device 104, enable the receiving device 104 to implement method 400. The computer-readable medium carrying the computer instructions provides non-transitory memory, which may include, but is not limited to, electronic storage devices, magnetic storage devices, optical storage devices, electromagnetic storage devices, semiconductor storage devices, or any suitable combination thereof.

[0124] Figure 5 This is an illustration of an exemplary scenario of remote direct memory access operation with a prefetch flow pacemaker according to an embodiment of the present invention. (In conjunction with...) Figure 1 , Figure 2A , Figure 2B , Figure 3 and Figure 4 Component description Figure 5 . refer to Figure 5An exemplary scenario 500 is illustrated, including a transmitting device 502 and a receiving device 504. The transmitting device 502 is configured to send a first message 502A, a second message 502B, and a third message 502C to the receiving device 504 for remote direct memory access (RDMA) operation. The receiving device 504 provides responses to the transmitting device 502, for example, providing indication messages 504A, 504B, and 504C in response to the first message 502A, providing 506A and 506B in response to the second message 502B, and providing 510 in response to the third message 502C.

[0125] In the exemplary scenario 500, the transmitting device 502 and the receiving device 504 correspond to Figure 1 The transmitting device 102 and the receiving device 104. In the exemplary scenario 500, the first message 502A is a first read operation type, and the second message 502B is a second read operation type. The third message 502C is a write operation type. The first read operation type (indicated by the first message 502A) is not associated with the second read operation type (indicated by the second message 502B). Therefore, regardless of whether the first message 502A is completed, the second message 502B can be sent after the first message 502A is sent to the receiving device 504.

[0126] When the first read operation type is executed (indicated by the first message 502A), in response, the controller of the receiving device 504 (e.g., controller 214, not shown here for simplicity) continuously generates the first indication message 504A, the second indication message 504B, and the third indication message 504C to indicate that the first read operation type has been successfully executed.

[0127] When the first read operation is being performed and the first indication message 504A is sent by the controller of the sending device 502 (e.g., Figure 2A When the controller 206 (not shown here for simplicity) receives the data, a second message 502B is sent. When a second read operation (indicated by the second message 502B) is being performed, in response, the controller of the receiving device 504 (e.g., the controller of the receiving device 504) sends a second message. Figure 2B The controller 214 generates a first indication message 506A and a second indication message 506B to indicate that the second read operation type was successfully executed.

[0128] The write operation type (indicated by the third message 502C) is associated with the second read operation type (indicated by the second message 502B). Therefore, the third message 502C is sent after a timer duration 508 (controlled by a local pacing timer such as local pacing timer 204) since the second message 502B was sent has elapsed. The controller of the transmitting device 502 (e.g., controller 206) determines whether the local pacing timer (e.g., local pacing timer 204, not shown here for simplicity) has exceeded the timer duration 508 since the second message 502B was sent. If the local pacing timer (e.g., local pacing timer 204) has exceeded the timer duration 508, the controller of the transmitting device 502 (e.g., controller 206) sends the third message 502C to the receiving device 504. The write operation type (indicated by the third message 502C) is determined by the controller of the receiving device 504 (e.g., controller 206) after the second read operation type is executed. Figure 2B The controller 214 of the receiving device 504 is activated. After executing the write operation type (indicated by the third message 502C), the controller of the receiving device 504 (e.g., controller 214) sends an acknowledgment message 510 to the transmitting device 502. Therefore, the local pacing timer (e.g., local pacing timer 204) supports sending the third message 502C before executing the second message 502B, which significantly reduces the total execution time of the first, second, and third operations, lowers latency, and speeds up data communication.

[0129] Figure 6 This is an illustration of an exemplary scenario of remote direct memory access operation with a prefetch flow pacemaker according to an embodiment of the present invention. (In conjunction with...) Figure 1 , Figure 2A , Figure 2B , Figure 3 , Figure 4 and Figure 5 Component description Figure 6 . refer to Figure 6 The illustration shows an exemplary scenario 600 including a transmitting device 602 and a receiving device 604. The transmitting device 602 sends a first message 606 and a second message 608 to the receiving device 604. The first message 606 includes an indication of a first operation type, and the second message 608 includes an indication of a second operation type. The receiving device 604 provides an acknowledgment message 610 to the transmitting device 602 as a response (or reply) to the first message 606 and the second message 608.

[0130] In the exemplary scenario 600, the transmitting device 602 and the receiving device 604 correspond to Figure 1The transmitting device 102 and the receiving device 104. In exemplary scenario 600, a first message 606 includes an indication of a first operation type, which pertains to a preparatory action to be followed by the receiving device 604. The preparatory action is an operation performed by the receiving device 604 to prepare memory (e.g., a local memory buffer) before receiving a request for a remote direct memory access operation. For example, when using a non-fixed remote direct memory access system, the transmitting device 602 sends a prefetch cue message to the receiving device 604 as the first message 606. The first message 606 (or prefetch cue) gives the receiving device 604 time to prepare its memory (i.e., its local memory buffer) before the actual request for a remote direct memory access operation for memory (i.e., the local memory buffer) arrives. For example, if a second message 608 includes an indication of a write operation type, the receiving device 604 prepares virtual memory address 0x1000 after receiving the first message 606 (or prefetch cue) to perform a write operation in a shorter time. The second message 608 is sent to the receiving device 604 after a duration 610 from the transmission of the first message 606 to ensure successful execution of the preparation action. The duration 610 between the transmission of the first message 606 and the transmission of the second message 608 is determined by a local pacing timer (e.g., Figure 1 The local pacing timer (e.g., local pacing timer 204) is controlled. For example, the local pacing timer (e.g., local pacing timer 204) starts when the first message 606 is sent and ends when the second message 608 is sent. The duration 610 is determined based on conditions such as the number of messages queued for transmission on the communication interface (e.g., communication interface 208), the average processing time of the preparation action, etc. For example, the duration 610 is determined based on the delay of the receiving device 604 in preparing to load the memory page at virtual memory address 0x1000 after receiving the first message 606 (or prefetch cues). The receiving device 604 may send an indication message 612 to the sending device 602 to confirm the successful execution of the second operation type. Therefore, the local pacing timer (e.g., local pacing timer 204) reduces the total transmission time of the first message 606 and the second message 608 and reduces the latency by eliminating the complete round-trip time of the confirmation message 612 for the preparation action.

[0131] Figure 7 This is an illustration of an exemplary scenario of a remote direct memory access system according to an embodiment of the present invention. (In conjunction with...) Figure 1 , Figure 2A , Figure 2B and Figures 3 to 6 Component description Figure 7 . refer to Figure 7An exemplary scenario of a remote direct memory access (RDMA) system 700 is shown, including a transmitting device 702 (i.e., a requesting device) and two responding devices referred to as a first receiving device 704A and a second receiving device 704B.

[0132] Transmitting device 702 corresponds to ( Figure 1 The transmitting device 102. Similarly, each of the first receiving device 704A and the second receiving device 704B corresponds to ( Figure 3 The receiving device 704A is also referred to as the requesting device or requester. The first receiving device 704A can also be referred to as the first responding device (e.g., responder #1). Similarly, the second receiving device 704B can also be referred to as the second responding device (e.g., responder #2).

[0133] In operation, in exemplary scenario 700, transmitting device 702 is used to send multiple messages to first receiving device 704A and second receiving device 704B respectively to initiate multiple remote direct memory access operations. Transmitting device 702 is used to send a first message 706A (e.g., QP 11) and a second message 706B (e.g., QP 21) to first receiving device 704A. The second message 706B is sent after the first message 706A. The first message 706A (i.e., QP 11) indicates a read operation type. The first message 706A includes parameters for performing the read operation type, such as a remote key of 30, a virtual memory address of 0x1000, and a first message length of 0x8000. Furthermore, the second message 706B (i.e., QP 21) indicates a write operation type. The second message 706B includes parameters for performing the write operation type, such as a remote key of 40, a virtual memory address of 0x100000, and a second message length of 0x1040.

[0134] The transmitting device 702 is also used to send a third message 708A (i.e., QP 12) and a fourth message 708B (e.g., QP 22) to the second receiving device 704B. The fourth message 708B will be sent after the third message 708A. The third message 708A (i.e., QP 12) indicates a read operation type. The third message 708A (i.e., QP 12) includes parameters for performing the read operation type, such as a remote key of 30, a virtual memory address of 0x50000, and a third message length of 0x1000. Furthermore, the fourth message 708B (i.e., QP 22) indicates an atomic operation type. The fourth message 708B (i.e., QP 22) also includes parameters for performing the atomic operation type, such as a remote key of 50 and a virtual memory address of 0x60000.

[0135] Transmitting device 702 generates a first prefetch prompt message 710A (e.g., QP 101) instructing the first receiving device 704A to perform a preparation action. Furthermore, transmitting device 702 generates a second prefetch prompt message 710B (i.e., QP 102) instructing the second receiving device 704B to perform a preparation action. The first prefetch prompt message 710A (i.e., QP 101) and the second prefetch prompt message 710B (i.e., QP 102) can be sent by transmitting device 702 in a special management message. For example, user datagram (UD) messages can be sent to both the first receiving device 704A and the second receiving device 704B for the first prefetch prompt message 710A and the second prefetch prompt message 710B.

[0136] The first prefetch cue message 710A (i.e., QP 101) instructs the first receiving device 704A to perform preparation actions by using different parameters, such as the remote key, virtual memory address, and message length, for the first message 706A (i.e., QP 11) and the second message 706B (i.e., QP 21). Similarly, the second prefetch cue message 710B (i.e., QP 102) instructs the second receiving device 704B to perform preparation actions by using different parameters, such as the remote key, virtual memory address, message length, and operand data (for atomic type operations), for the third message 708A (i.e., QP 12) and the fourth message 708B (i.e., QP 22).

[0137] After receiving the first prefetch cue message 710A (QP 101), the first receiving device 704A requests that the context of the first message 706A (QP 11) and the second message 706B (QP 21) be brought into its buffer. Similarly, after receiving the second prefetch cue message 710B (QP 102), the second receiving device 704B requests that the context of the third message 708A (QP 12) and the fourth message 708B (QP 22) be brought into its buffer. Furthermore, both the first and second receiving devices 704A and 704B immediately send requests to their respective memory pages to retrieve each range of ranges with appropriate attributes (without waiting for the request packet to arrive). The second receiving device 704B also retrieves the original atomic operation data from its memory. The first and second receiving devices 704A and 704B may also send indication messages to the sending device 702 to acknowledge that the prefetch cues have been processed.

[0138] Furthermore, the transmitting device 702 includes a local pacing timer (e.g., local pacing timer 204, not shown here for simplicity). The local pacing timer controls the duration between the transmission of prefetch cues 710A and 710B and the first message 706A, the second message 706B, the third message 708A, and the fourth message 708B. For example, the transmitting device 702 starts the local pacing timer after sending the first prefetch cue message 710A and the second prefetch cue message 710B to the first receiving device 704A and the second receiving device 704B, respectively. This duration is determined based on conditions such as the number of messages queued for transmission by the transmitting device 702, network congestion, and the average processing time of preparation actions.

[0139] When the local pacing timer expires, transmitting device 702 sends a first message 706A, indicating that first receiving device 704A and second receiving device 704B have performed preparation actions. Transmitting device 702 also sends a second message 706B, a third message 708A, and a fourth message 708B, while maintaining a specific message transmission order. Therefore, the local pacing timer (e.g., local pacing timer 204) gives first receiving device 704A and second receiving device 704B sufficient time to prepare their memory and achieve a successful remote direct memory access operation. Furthermore, the local pacing timer reduces the total transmission time of the first message 706A, second message 706B, third message 708A, and fourth message 708B by eliminating the full round-trip time of the preparation action acknowledgment message, thus reducing latency.

[0140] In one exemplary aspect, the present invention provides a system (e.g., RDMA system 100 or 700). The system (i.e., RDMA system 100 or 700) includes a transmitting device (e.g., transmitting device 102 or 702) and a receiving device (e.g., receiving device 104, 704A, or 704B). In one implementation, the system (i.e., RDMA system 100 or 700) may further include a transmitting device 102 and a receiving device 104, the transmitting device 102 including a memory 202, a communication interface 208, and a software module 210, and the receiving device 104 including a memory 216, a communication interface 212, and a software module 218. For example, in... Figure 1 , Figure 2A and Figure 2B The various operations of the transmitting device 102 and the receiving device 104 and their components have been described in detail in the previous section.

[0141] Modifications to the embodiments of the invention described above may be made without departing from the scope of the invention as defined by the appended claims. Words such as “comprising,” “in combination,” “having,” and “is” used to describe and claim the invention are intended to be interpreted in a non-exclusive manner, allowing for the presence of items, components, or elements not explicitly described. Singular references should also be interpreted in relation to the plural. The word “exemplary” as used herein means “as an example, instance, or illustration.” Any embodiment described as “exemplary” is not necessarily to be construed as more preferred or advantageous than other embodiments, and / or excludes combinations of features from other embodiments. The word “optionally” as used herein means “provided in some embodiments but not in others.” It should be understood that some features of the invention described in the context of a single embodiment for clarity may also be provided in combination in a single embodiment. Conversely, various features of the invention described in the context of a single embodiment for brevity may also be provided individually, in any suitable combination, or suitably for any other described embodiment of the invention.

Claims

1. A transmitting device (102, 502, 602, 702), characterized by, The sending device (102, 502, 602, 702) comprises a memory (202) for storing a local pacing timer (204), a controller (206) for: generating a first message (502A) comprising an indication of a first operation type; sending the first message (502A) to a receiving device (104, 504, 604, 704A, 704B) via the communication interface (208); generating a second message (502B) comprising an indication of a second operation type; determining whether the second operation type is associated with the first operation type, and if so: determining whether the local pacing timer (204) exceeds a timer duration since sending the first message (502A), wherein the local pacing timer (204) is a first operation type timer; and if so: sending the second message (502B) to the receiving device (104, 504, 604, 704A, 704B) via the communication interface (208); the timer duration is associated with an operation type, wherein a first timer duration is associated with one operation type and a second timer duration is associated with another operation type.

2. The transmitting device (102, 502, 602, 702) according to claim 1, characterized in that, The controller (206) is further configured to: determining whether the second operation type is not associated with the first operation type, and if not: sending the second message (502B) to the receiving device (104, 504, 604, 704A, 704B) via the communication interface (208) regardless of whether the local pacing timer (204) exceeds the timer duration since sending the first message (502A).

3. The transmitting device (102, 502, 602, 702) according to claim 1 or 2, characterized by, The controller (206) is further configured to: generating a third message (502C) comprising an indication of a third operation type, wherein the third operation type is associated with the first operation type; determining whether the local pacing timer (204) exceeds the second timer duration since sending the second message (502B); and if so: sending the third message (502C) to the receiving device (104, 504, 604, 704A, 704B) via the communication interface (208).

4. The sending device (102, 502, 602, 702) of claim 3, wherein: the third operation type is the second operation type.

5. The sending device (102, 502, 602, 702) of any one of claims 1, 2, 4, wherein: the first operation type is the second operation type.

6. The transmitting device (102, 502, 602, 702) of any one of claims 1, 2, 4, wherein, The controller (206) is further configured to: receiving an acknowledgement message from the receiving device (104, 504, 604, 704A, 704B); and in response to this, The second message (502B) is sent to the receiving device (104, 504, 604, 704A, 704B) through the communication interface (208) regardless of whether the local pacing timer (204) exceeds the timer duration.

7. The transmitting device (102, 502, 602, 702) of any one of claims 1, 2, 4, wherein, The controller (206) is further configured to adjust the timer duration.

8. The transmitting device (102, 502, 602, 702) according to claim 7, c h a r a c t e r i z e d b y The controller (206) is further configured to adjust the timer duration according to one or more of: a number of active connections established through the communication interface (208); information about an operating state of the receiver (104, 504, 604, 704A, 704B); a number of negative acknowledgement [NAK] messages received; an indication received from the receiver (104, 504, 604, 704A, 704B).

9. The transmitting device (102, 502, 602, 702) of any one of claims 1, 2, 4, 8, wherein, The first operation type is associated with a memory region, and the second operation type is associated with the same memory region.

10. The transmitting device (102, 502, 602, 702) of any one of claims 1, 2, 4, 8, wherein, The controller (206) is further configured to, in addition to determining whether the local pacing timer (204) exceeds the timer duration since sending the first message (502A), determine whether a traffic congestion timer exceeds a congestion timer duration since sending the first message (502A), and if so: send the second message (502B) to the receiving device (104, 504, 604, 704A, 704B) through the communication interface (208).

11. The transmitting device (102, 502, 602, 702) of any one of claims 1, 2, 4, 8, wherein, The controller (206) is further configured to, in addition to determining whether the local pacing timer (204) exceeds the timer duration since sending the first message (502A), determine whether the local pacing timer (204) exceeds a congestion timer duration since sending the first message (502A), and if so: send the second message (502B) to the receiving device (104, 504, 604, 704A, 704B) through the communication interface (208).

12. The transmitting device (102, 502, 602, 702) of any one of claims 1, 2, 4, 8, wherein, The sending device (102, 502, 602, 702) serves as a requester in a remote direct memory access system (100, 700).

13. The transmitting device (102, 502, 602, 702) according to claim 12, characterized by, The first operation type indicates a preparation action to be performed by the receiving device (104, 504, 604, 704A, 704B), and the second operation type indicates an operation of performing the preparation action.

14. The transmitting device (102, 502, 602, 702) of any one of claims 1, 2, 4, 8, wherein, The sending device (102, 502, 602, 702) serves as a responder in a remote direct memory access system (100, 700).

15. The sending device (102, 502, 602, 702) of claim 14, wherein: the first operation type indicates a read operation, and the first message (502A) indicates a read result, the second operation type indicates the read operation, and the second message (502B) indicates a subsequent read result.

16. A transmitting device (102, 502, 602, 702), characterized by The sending device (102, 502, 602, 702) comprises a memory (202), a communication interface (208), and a software module (210), the software module (210) comprising: a first message generator software module (210a) for generating a first message (502A), the first message (502A) comprising an indication of a first operation type; a first message transmitter software module (210b) for transmitting the first message (502A) to a receiving device (104, 504, 604, 704A, 704B) via the communication interface (208); a second message generator software module (210c) for generating a second message (502B), the second message (502B) comprising an indication of a second operation type; an operation type association determination software module (210d) for determining whether the second operation type is associated with the first operation type, and if so, performing a duration determination software module (210e) for determining whether a local pacing timer (204) exceeds a timer duration since transmission of the first message (502A), wherein the local pacing timer (204) is a first operation type timer; and if so, performing a second message transmitter software module (210f) for transmitting the second message (502B) to the receiving device (104, 504, 604, 704A, 704B) via the communication interface (208); the timer durations being associated with operation types, wherein a first timer duration is associated with one operation type and a second timer duration is associated with another operation type.

17. A method (300) for a transmitting device (102, 502, 602, 702), characterized by, The method (300) comprises: generating a first message (502A), the first message (502A) comprising an indication of a first operation type; transmitting the first message (502A) to a receiving device (104, 504, 604, 704A, 704B) via a communication interface (208); generating a second message (502B), the second message (502B) comprising an indication of a second operation type; determining whether the second operation type is associated with the first operation type, and if so: determining whether a local pacing timer (204) exceeds a timer duration since transmission of the first message (502A), wherein the local pacing timer (204) is a first operation type timer; and if so: transmitting the second message (502B) to the receiving device (104, 504, 604, 704A, 704B) via the communication interface (208); the timer durations being associated with operation types, wherein a first timer duration is associated with one operation type and a second timer duration is associated with another operation type. The computer instructions, when loaded into and executed by a controller (206) of a transmitting device (102, 502, 602, 702), enable the transmitting device (102, 502, 602, 702) to implement the method (300) according to claim 17.

18. A computer readable medium carrying computer instructions, characterized in that, comprise a communication interface (212) 19. A receiving device (104, 504, 604, 704A, 704B), characterized by, and a controller (214), wherein the controller (214) is configured to: ​ receiving a first message (502A) from a sending device (102, 502, 602, 702) via the communications interface (212), the first message (502A) indicating a first operation; performing the first operation; receiving a second message (502B) from the sending device (102, 502, 602, 702) via the communications interface (212), the second message (502B) indicating a second operation, wherein the second operation is associated with the first operation; determining whether the first operation is being performed, and if so: generating an indication message including an indication of this result, sending the indication message to the sending device (102, 502, 602, 702); timer durations are associated with operation types, wherein a first timer duration is associated with one operation type and a second timer duration is associated with another operation type.

20. A receiving device (104, 504, 604, 704A, 704B), characterized by, The receiving device (104, 504, 604, 704A, 704B) includes a memory (216), a communications interface (212), and software modules (218), the software modules (218) including: a first message receiving software module (218a) for receiving a first message (502A) from a sending device (102, 502, 602, 702) via the communications interface (212), the first message (502A) indicating a first operation; an operation performing software module (218b) for performing the first operation; a second message receiving software module (218c) for receiving a second message (502B) from the sending device (102, 502, 602, 702) via the communications interface (212), the second message (502B) indicating a second operation, wherein the second operation is associated with the first operation; an operation performing determining software module (218d) for determining whether the first operation is being performed, and if so an indication message generating software module (218e) for generating an indication message including an indication of this result, an indication message transmitter software module (218f) for sending the indication message to the sending device (102, 502, 602, 702); timer durations are associated with operation types, wherein a first timer duration is associated with one operation type and a second timer duration is associated with another operation type.

21. A method (400) for a receiving device (104, 504, 604, 704A, 704B), characterized by, The method (400) includes: receiving a first message (502A) from a sending device (102, 502, 602, 702) via the communications interface (212), the first message (502A) indicating a first operation; performing the first operation; receiving a second message (502B) from the sending device (102, 502, 602, 702) via the communications interface (212), the second message (502B) indicating a second operation, wherein the second operation is associated with the first operation; determining whether the first operation is being performed, and if so: generating an indication message including an indication of this result, sending the indication message to the sending device (102, 502, 602, 702); timer durations are associated with operation types, wherein a first timer duration is associated with one operation type and a second timer duration is associated with another operation type. The indication message is sent to the sending device (102, 502, 602, 702); the timer duration is associated with the operation type, wherein the first timer duration is associated with one operation type and the second timer duration is associated with another operation type.

22. A computer readable medium carrying computer instructions, characterized in that, When the computer instructions are loaded into and executed by the controller (214) of the receiving device (104, 504, 604, 704A, 704B), the receiving device (104, 504, 604, 704A, 704B) enables the receiving device (104, 504, 604, 704A, 704B) to implement the method (400) according to claim 21.

23. A system (100, 700) characterized by include: The transmitting device (102, 502, 602, 702) according to any one of claims 1 to 15 and the receiving device (104, 504, 604, 704A, 704B) according to claim 19, or The transmitting device (102, 502, 602, 702) according to claim 16 and the receiving device (104, 504, 604, 704A, 704B) according to claim 20.

Images