RDMA-based message sending and processing method and system, and electronic equipment

Abstract

The invention provides a message sending and processing method and system based on RDMA (Remote Direct Memory Access) and electronic equipment. The message sending method comprises the steps that a first message is sent to a receiving end, the first message carries indication information and identification information of a message to which the first message belongs, and the indication information is used for indicating whether disordered receiving between messages is allowed or not. The sending end carries the identification information of the message and the indication information used for indicating whether to allow the out-of-order receiving between the messages in the first message, so that for a receiving end, under the condition that the indication information carried in the first message indicates that the out-of-order receiving between the messages is allowed, the receiving end only needs to preserve the order of the messages in the message, and the receiving end does not need to perform the out-of-order receiving between the messages. According to the technical scheme of the invention, the method does not need to carry out order preservation between messages, that is, a receiving end can trigger retransmission only under the condition that packet loss or disorder occurs to the messages in the messages without triggering retransmission under the condition that the messages are out of order, so that the retransmission range can be reduced (that is, the retransmission range is limited in the messages), and the transmission efficiency of the network is improved.

Description

Technical Field

[0001] The present application relates to the field of network communication technology, and in particular to a method, system and electronic device for sending and processing messages based on RDMA. Background Art

[0002] Remote Direct Memory Access (RDMA) is a network communication technology that allows computers to transfer data between memories directly on the network. RDMA technology can provide multiple transmission modes at the transport layer, and the commonly used one is the order-preserving transmission mode, such as the Reliable Connect (RC) mode used by RoCEv2, a remote direct memory access based on converged Ethernet, and the Reliable Ordered Delivery (ROD) mode used by the Ultra Ethernet Consortium (UEC). In related technologies, when using the order-preserving transmission mode provided by RDMA technology for data transmission, the sender can divide the data to be sent into messages, and the receiver will check whether the received messages are continuous. If not, it is considered that packet loss or disorder occurs and retransmission is triggered, thereby ensuring the order-preserving transmission of messages. However, in actual applications, this transmission mode has a high retransmission cost when packet loss or disorder occurs, which will reduce the transmission efficiency of the network. Summary of the invention

[0003] The present application provides a method, system and electronic device for sending and processing messages based on RDMA, which are used to solve the problem of low transmission efficiency when messages are transmitted using the order-preserving transmission mode provided by the RDMA technology.

[0004] To solve the above technical problems, this application is implemented as follows: In a first aspect, a method for sending a message based on RDMA is provided, which is applied to a sending end and includes: A first message is sent to a receiving end, wherein the first message carries indication information and identification information of a message to which the first message belongs, and the indication information is used to indicate whether out-of-order reception of messages is allowed.

[0005] In a second aspect, a method for processing a message based on RDMA is provided, which is applied to a receiving end and includes: Receive a first message sent by a sending end, where the first message carries indication information and identification information of a message to which the first message belongs, and the indication information is used to indicate whether out-of-order reception of messages is allowed; The first message is processed according to the indication information and the identification information, wherein the processing includes receiving messages to which the first message belongs out of order or receiving messages in an order-preserving manner.

[0006] In a third aspect, a message sending and processing system based on RDMA is provided, comprising a sending end and a receiving end, wherein; The sending end sends a first message to the receiving end, where the first message carries indication information and identification information of a message to which the first message belongs, and the indication information is used to indicate whether out-of-order reception of messages is allowed; The receiving end receives the first message, and processes the first message according to the indication information and the identification information, wherein the processing includes receiving messages to which the first message belongs out of order or receiving messages in an order-preserving manner.

[0007] In a fourth aspect, an electronic device is provided, including: processor; a memory for storing instructions executable by the processor; The processor is configured to execute the instructions to implement the method as described in the first aspect or the second aspect.

[0008] According to a fifth aspect, a computer-readable storage medium is provided. When instructions in the storage medium are executed by a processor of an electronic device, the electronic device can execute the method described in the first aspect or the second aspect.

[0009] In a sixth aspect, a computer program product is provided, the computer program product comprising a non-transitory computer-readable storage medium storing a computer program, the computer program being operable to cause a computer to execute part or all of the steps in the method described in the first aspect, or to execute part or all of the steps in the method described in the second aspect.

[0010] In an embodiment of the present application, when the sending end and the receiving end transmit messages based on RDMA, they can transmit messages based on messages. Specifically, when the sending end sends a first message to the receiving end, the first message can carry identification information of the message to which the first message belongs and indication information for indicating whether out-of-order reception between messages is allowed. After receiving the first message, the receiving end can process the message based on the identification information and indication information carried in the first message. In this way, when the first message sent by the sending end indicates that out-of-order reception between messages is allowed, the receiving end only needs to maintain the order of the messages in the message, and there is no need to maintain the order between messages. In other words, the receiving end can not trigger retransmission when there is out-of-order between messages, and only trigger retransmission when packets are lost or out-of-order in the message, thereby reducing the retransmission range (i.e., limiting the retransmission range to the message), thereby improving transmission efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0011] In order to more clearly illustrate the technical solutions in the present application or the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0012] Figure 1 It is a schematic diagram of an embodiment of the present application in which the application layer does not require order preservation between messages; Figure 2 It is a flowchart of a method for sending a message based on RDMA according to an embodiment of the present application; Figure 3 is a schematic diagram of an embodiment of the present application in which the application layer sends a first parameter through an extended programming interface; Figure 4 It is a schematic diagram of load balancing performed by a sending end in one embodiment of the present application; Figure 5 It is a schematic diagram of a sending end retransmitting a message in an embodiment of the present application; Figure 6 It is a flowchart of a message processing method based on RDMA in an embodiment of the present application; Figure 7 This is a schematic diagram of a flow chart of message processing in an embodiment of the present application; Figure 8 It is a schematic diagram of load balancing based on message granularity in one embodiment of the present application; Fig. 9 This is a schematic diagram of the basic encapsulation format of a message in an embodiment of the present application; Fig.10 It is a schematic diagram of an embodiment of the present application for extending the basic encapsulation format of a message; Fig.11 It is a schematic diagram of an embodiment of the present application for extending the semantic header encapsulation format of a message; Fig.12 It is a schematic diagram of extending the encapsulation format of a NACK message according to an embodiment of the present application; Fig.13 is a schematic diagram of the structure of an electronic device according to an embodiment of the present application; Fig.14 It is a structural diagram of a message sending device based on RDMA in an embodiment of the present application; Fig.15 is a schematic diagram of the structure of an electronic device according to an embodiment of the present application; Fig.16It is a structural diagram of a message processing device based on RDMA in an embodiment of the present application. DETAILED DESCRIPTION

[0013] RDMA is a network communication technology that uses mechanisms such as "zero copy", kernel bypass, and hardware offload to allow computers to transfer data between memories directly on the network. By using RDMA technology for data transmission, the throughput can be increased while reducing end-to-end communication latency, making RDMA one of the important technologies in high-performance computing and intelligent computing data center networks.

[0014] RDMA technology provides multiple transmission modes at the transport layer. Taking the widely used RoCEv2 as an example, it includes Reliable Connect (RC), Unreliable Datagram (UD) and other modes. Among them, RC is the most commonly used mode, which can provide reliable and sequence-preserving transmission at the message level. For example, the data to be sent is divided into messages at the sending end, and each message carries a sequence number (PSN). If the RC mode is used, the receiving end will check whether the received PSN is continuous. If it is not continuous, it will be regarded as a packet loss by default and trigger retransmission. Other RDMA technologies are similar. For example, the currently popular UEC provides reliable ordered delivery (ROD) and reliable unordered delivery (RUD) modes at the transport layer. When the ROD mode is used, the receiving end transport layer will keep the order of the received messages. When the RUD mode is used, the receiving end directly delivers the messages to the host memory. At this time, the messages in the same message may arrive out of order. When the transport layer determines that packet loss has occurred (typically based on the out-of-order threshold to determine whether there is packet loss), a selective retransmission mechanism is required. Due to the shortcomings of the selective retransmission mechanism, current commercial RDMA network cards still use the go-back-N retransmission mechanism in production environments.

[0015] In the related art, when using an order-preserving transmission mode (such as the RC mode used by the above-mentioned RoCEv2 or the ROD mode used by UE C) to transmit messages, since the messages need to be ordered, once the messages are lost or out of order, the receiving end will trigger retransmission, resulting in a high retransmission cost, affecting the transmission efficiency of the network.

[0016] In some application scenarios, the transport layer provides a programming interface (Verbs, libfabric, etc.) to the application layer. The application layer program may require order preservation within messages, but not between messages. Figure 1As shown in the figure, the application layer program at the sending end needs to send 2 high-performance computing messages, each message is divided into 4 packets, each message carries a matrix data, and the application layer program at the receiving end needs to receive these two messages and perform matrix addition operations. In this case, the order of the messages in the message must be preserved, otherwise the data carried will be wrong. However, due to the commutative law of matrix addition, the arrival order of the messages does not affect the calculation results. Therefore, there is no need to preserve the order between messages. Other scenarios such as mathematical multiplication operations and linear equation solving meet the commutative law and also do not require order preservation between messages. In this way, if the transport layer can provide a processing method that only preserves the order within the message but not between messages, it can greatly reduce the scope of packet loss or out-of-order retransmission of traditional reliable connections while meeting the above application scenarios, thereby improving the transmission efficiency of the network. However, the current RDMA technology still lacks such a fine-grained transmission processing mechanism.

[0017] The embodiment of the present application provides a method, system and electronic device for sending and processing messages based on RDMA. When the sending end and the receiving end transmit messages based on RDMA, the messages can be transmitted based on messages. Specifically, when the sending end sends a first message to the receiving end, the first message can carry the identification information of the message to which the first message belongs and the indication information for indicating whether out-of-order reception between messages is allowed. After receiving the first message, the receiving end can process the message based on the identification information and indication information carried in the first message. In this way, when the first message sent by the sending end indicates that out-of-order reception between messages is allowed, the receiving end only needs to maintain the order of the messages in the message, and there is no need to maintain the order between messages. That is to say, the receiving end can not trigger retransmission when there is out-of-order between messages, and only trigger retransmission when packets are lost or out-of-order in the message, thereby reducing the retransmission range (i.e. limiting the retransmission range to the message), thereby improving transmission efficiency.

[0018] It should be noted that the technical solution provided in the embodiments of the present application can be applied to a router or a switch and a corresponding configuration unit.

[0019] In order to enable those skilled in the art to better understand the technical solutions in this application, the technical solutions in this application will be described clearly and completely below in conjunction with the drawings in one or more embodiments of this application. Obviously, the described embodiments are only part of the embodiments of this application, not all of them. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of this application.

[0020] The terms "first", "second", etc. in the present application and claims are used to distinguish similar objects, and are not used to describe a particular order or precedence. It should be understood that the data used in this way can be interchanged where appropriate so that the present application can be implemented in an order other than those illustrated or described herein. In addition, "and / or" in the present application and claims means at least one of the connected objects, and the character " / " generally means that the objects associated with each other are in an "or" relationship.

[0021] The technical solutions provided by various embodiments of the present application are described in detail below in conjunction with the accompanying drawings.

[0022] Figure 2 It is a flowchart of a method for sending messages based on RDMA according to an embodiment of the present application. Figure 2 The message sending method shown is applied to the sending end, that is, Figure 2 The message sending method shown can be executed by software or hardware installed on the sending end. In some embodiments, Figure 2 The message sending method shown can be specifically applied to the transport layer of the sending end. Figure 2 The message sending method shown is described as follows.

[0023] S202: Send a first message to a receiving end, where the first message carries indication information and identification information of a message to which the first message belongs, and the indication information is used to indicate whether out-of-order reception of messages is allowed.

[0024] When the sender transmits messages based on RDMA, it can transmit messages based on messages. For example, Figure 1 When the sending end shown sends msg1 and msg2 to the receiving end, msg1 and msg2 can be split into 4 messages respectively, and then these messages can be sent to the receiving end in sequence. In an embodiment of the present application, when the sending end sends a message to the receiving end, taking the sending of the first message as an example, the indication information and identification information can be carried in the first message. The indication information is used to indicate whether out-of-order reception between messages is allowed. For example, when the sending end sends msg1 and msg2 to the receiving end, the sent message carries the indication information, and the indication information is used to indicate whether the receiving end is allowed to receive msg1 and msg2 out of order. The identification information is the identification information of the message to which the first message belongs. For example, if the first message is a message obtained by splitting msg1, the identification information is the identification information of msg1, such as the ID of msg1.

[0025] In this way, since the sending end carries the identification information of the message and the indication information for indicating whether out-of-order reception between messages is allowed in the first message, for the receiving end, when the indication information carried in the first message indicates that out-of-order reception between messages is allowed, the receiving end only needs to maintain the order of the messages within the message, and there is no need to maintain the order between messages. In other words, the receiving end can not trigger retransmission when there is out-of-order reception between messages, and only trigger retransmission when packet loss or out-of-order occurs in the message. This can reduce the retransmission range (i.e., limit the retransmission range to the message), thereby improving the transmission efficiency of the network.

[0026] In some implementations, the sending end sending the first message to the receiving end may include: receiving a first parameter sent by the application layer, where the first parameter is used to indicate whether out-of-order reception of messages is allowed; A first message is sent to a receiving end according to the first parameter.

[0027] Specifically, the application layer at the sending end may pass a first parameter to the transport layer at the sending end according to whether the order preservation between messages is required, and the first parameter may indicate whether out-of-order reception between messages is allowed. For example, if the application layer does not require the order preservation between messages, the first parameter may indicate that out-of-order reception between messages is allowed; if the application layer requires the order preservation between messages, the first parameter may indicate that out-of-order reception between messages is not allowed.

[0028] In order to facilitate the application layer to pass the first parameter, in some embodiments, the programming interface provided by the transport layer to the application layer can be extended, so that the application layer can send the first parameter to the transport layer through the extended programming interface, and correspondingly, the transport layer can receive the first parameter sent by the application layer based on the extended programming interface. The extended programming interface may include an extended first field, which can be used to carry the first parameter. In other words, when the programming interface is extended, the first field can be extended in the programming interface so that the application layer can pass the requirement of whether to maintain order between messages to the transport layer through the first field.

[0029] It should be noted that different RDMA technologies use different programming interfaces, such as RoCEv2 uses libverbs, UEC uses libfabric, Intel uses PSM2, Cisco uses usNIC, etc. Different programming interfaces have different details, but the extension methods are similar. For example, for the IB libverbs interface, when extending the interface, the new transmission sub-mode can be extended in the ib_create_qp interface parameter ibv_qp_init_attr to pass the application layer requirements. Figure 3As shown, when the qp_type value is RC and qp_sub_type=true, it can indicate that the application layer has no requirement for order preservation between messages, that is, the application layer allows out-of-order reception of messages.

[0030] After receiving the first parameter sent by the application layer, the sending end may send the first message to the receiving end according to the first parameter when sending the first message to the receiving end. Here, sending the first message to the receiving end according to the first parameter may be to set whether out-of-order reception between messages is allowed according to the first parameter according to the indication information carried in the first message. For example, if the first parameter indicates that out-of-order reception between messages is allowed, the indication information carried in the first message is specifically used to indicate that out-of-order reception between messages is allowed; if the first parameter indicates that out-of-order reception between messages is not allowed, the indication information carried in the first message is specifically used to indicate that out-of-order reception between messages is not allowed.

[0031] In some embodiments, in order to carry the identification information of the message and the indication information for indicating whether out-of-order reception between messages is allowed in the first message, the first message can be extended as needed. For example, for the identification information of the message, since the identification information for carrying the message has not been defined in the RoCEv2 service message, and the identification information for carrying the message has been defined in the UEC transport layer message, it is necessary to extend the RoCEv2 service message accordingly, and it is not necessary to extend the UEC transport layer message accordingly. For the indication information for indicating whether out-of-order reception between messages is allowed, since such indication information is not currently defined in the RoCEv2 service message and the UEC transport layer message, it is necessary to extend the RoCEv2 service message and the UEC transport layer message accordingly so that the RoCEv2 service message and the UEC transport layer message can carry the indication information.

[0032] When the message is extended, the corresponding extension can be made according to the message format of the RDMA technology used. Among them, for the indication information used to indicate whether out-of-order reception between messages is allowed, a 1-bit flag bit can be optionally used for indication. For example, when the flag bit is true (or the flag bit value is 1), the indication information is used to indicate that out-of-order reception between messages is allowed, and when the flag bit is false (or the flag bit value is 0), the indication information is used to indicate that out-of-order reception between messages is not allowed.

[0033] In some implementations, the first message may also carry an operation code, which is used to characterize the current transmission mode, such as a reliable mode or an unreliable mode. When the sender sends the first message to the receiver, it may also perform load balancing based on the identification information in the first message if the first condition is met. The first condition includes that the operation code is a reliable mode (such as RC0), the first message carries identification information and indication information of the message, and the indication information is used to indicate that out-of-order reception between messages is allowed. In other words, when the operation code carried in the first message is a reliable mode, carries identification information and indication information of the message, and the indication information is used to indicate that out-of-order reception between messages is allowed, the sender is allowed to perform load balancing based on message granularity.

[0034] The currently widely used flow-level hash routing algorithm is based on the business flow IP quintuple, which has many problems such as hash conflict polarization. When the sender determines that load balancing can be done based on message granularity, the identification information of the message can be used as the input of the hash algorithm. At this time, the messages belonging to the same message (that is, the messages with the same identification information) can be hashed to one path, thereby ensuring the message order within the message, while the messages belonging to different messages (that is, the messages with different identification information) may be sent to different paths, thereby improving the load balancing effect. In other words, when the sender performs load balancing based on the identification information, it can specifically send the messages with the same identification information to the same transmission path, and send the messages with different identification information to the same or different transmission paths.

[0035] See also Figure 4 , the sender sends msg1, msg2, msg3 and msg4 to the receiver, each of which includes multiple messages ( Figure 4 These messages are not shown), then, when the sender determines that load balancing can be performed based on message granularity, the sending method of each message can be as follows: Figure 4 shown. Figure 4 In the case of load balancing, the sender sends all packets belonging to the same message to one transmission path, and sends packets belonging to different messages to the same or different transmission paths. Figure 4 Taking the first switch connected to the sending end as an example, when forwarding msg1, msg2, msg3 and msg4, each message is forwarded through a transmission path, so that the messages in the same message are sent to one transmission path and are not sent separately. Different messages are forwarded through the same or different transmission paths, that is, msg1 and msg4 are sent to one transmission path, and msg2 and msg3 are sent to another transmission path, so that the messages in different messages can be sent to the same or different transmission paths.

[0036] It should be noted that when the sender performs load balancing based on message granularity, the load balancing effect is related to the size of the message. Generally, the smaller the message, the better the load balancing effect. The selection of the message size can be set according to different application characteristics and is not specifically limited here.

[0037] In some implementations, when the transmitting end sends the first message to the receiving end, the first message may also carry a packet sequence number (PSN), a start of Msg (SOM), an end of Msg (EOM), a source queue pair (SQP) identifier, and a destination queue pair (DQP) identifier. This information may be used by the receiving end to process the first message. For a specific processing flow, see Figure 6 The embodiment shown is not described in detail here. Among them, the packet sequence number can represent the order of the messages and can be used for the receiving end to save the messages. The first packet mark is used to indicate whether the message is the first message in the message to which it belongs. Optionally, the first packet mark can be indicated by 1 bit, and the bit value is 1 to indicate that the message is the first message in the message to which it belongs, and the bit value is 0 to indicate that the message is not the first message in the message to which it belongs. The tail packet mark is used to indicate whether the message is the last message in the message to which it belongs. Optionally, the tail packet mark can also be indicated by 1 bit, and the bit value is 1 to indicate that the message is the last message in the message to which it belongs, and the bit value is 0 to indicate that the message is not the last message in the message to which it belongs. The identifier of the source queue pair can be the ID of the source queue pair. The identifier of the destination queue pair can be the ID of the destination queue pair.

[0038] In some implementations, when the indication information carried in the first message is used to indicate that out-of-order reception between messages is allowed, the sending end may further perform the following operations: receiving a NACK message sent by a receiving end, where the NACK message is sent by the receiving end when it is determined that a message in a message to which the first message belongs is lost or out of order; Retransmit the message according to the NACK message.

[0039] Specifically, when the indication information carried by the first message is used to indicate that out-of-order reception between messages is allowed, for the receiving end, only order preservation within the message can be performed, and order preservation between messages is not performed, that is, multiple messages within the message to which the first message belongs need to be received in order, and the message to which the first message belongs can be received out of order. Among them, when the receiving end is performing order preservation within the message, for multiple messages within the message to which the first message belongs, there may be packet loss or disorder. In this case, the receiving end can trigger retransmission and send a NACK message to the sending end. After receiving the NACK message, the sending end can retransmit the message according to the NACK message.

[0040] In some implementations, in order for the sender to limit the message retransmission range to within the message, the NACK message may carry the identification information of the message to which the retransmitted message belongs (i.e., the identification information of the message to which the first message belongs) and the retransmission start packet sequence number. In this way, when the sender retransmits the message according to the NACK message, it can send only the message starting from the retransmission start packet sequence number in the message corresponding to the identification information to the receiver according to the identification information carried in the NACK message and the retransmission start packet sequence number, thereby limiting the retransmission range to within the message, reducing the number of retransmitted messages, reducing the retransmission cost, and improving the transmission efficiency of the network.

[0041] In order to make the NACK message carry the identification information of the message, in some implementations, the NACK message can be extended as needed, that is, if the current NACK message does not define the identification information of the message, the NACK message needs to be extended so that the NACK message can carry the identification information of the message. Optionally, the NACK message can also carry indication information for indicating that out-of-order reception between messages is allowed. In this case, the NACK message also needs to be extended accordingly so that the NACK message can carry the indication information.

[0042] Figure 5 It is a schematic diagram of a sending end retransmitting a message according to an embodiment of the present application.

[0043] Figure 5In the example, the sender sends msg1 and msg2 to the receiver. msg1 includes 4 messages with PSNs of 1, 2, 3, and 4 respectively, and msg2 includes 5 messages with PSNs of 5, 6, 7, 8, and 9 respectively. The sender's application layer allows messages to be received out of order. When receiving the message, the receiver finds that the message with PSN=3 is lost, triggers retransmission, and sends a NACK message to the sender. The NACK message carries the message identifier msg1, the retransmission start packet sequence number PSN=3, and the indication information msg_order_flag=true used to indicate that out of order reception between messages is allowed. After receiving the NACK message, the sender can only retransmit the messages with PSN=3 and 4 in msg1 when retransmitting the message, that is, the retransmission range is PSN={3~4}. It should be noted that since out of order reception between messages is allowed, msg2 may reach the receiving end transport layer before the {3,4} messages of msg1 and be delivered to the receiving end host memory.

[0044] Assuming that the sending application layer does not allow out-of-order reception of messages, then, Figure 5 In the figure, when the message with PSN=3 is lost, the retransmission range of the sender is PSN={3~9}, while when out-of-order reception between messages is allowed, the retransmission range of the sender is PSN={3~4}. It can be seen that when the sender's application layer allows out-of-order reception between messages, the sender's retransmission range can be reduced and the transmission efficiency can be improved.

[0045] In the embodiment of the present application, since the sending end carries the identification information of the message and the indication information for indicating whether out-of-order reception between messages is allowed in the first message, for the receiving end, when the indication information carried in the first message indicates that out-of-order reception between messages is allowed, the receiving end only needs to maintain the order of the messages in the message, and there is no need to maintain the order between messages. That is to say, the receiving end can trigger retransmission only when packets are lost or out of order in the message, without triggering retransmission when there is out-of-order reception between messages. This can reduce the retransmission range (i.e., limit the retransmission range to the message), thereby improving the transmission efficiency of the network. In addition, since the sending end can perform load balancing based on the message granularity when out-of-order reception between messages is allowed, message blocking can be avoided, further improving the transmission efficiency and overall network utilization of the network.

[0046] Figure 6 It is a flowchart of a message processing method based on RDMA according to an embodiment of the present application. Figure 2 The message sending method shown is applied to the receiving end, that is, Figure 6 The message processing method shown can be executed by software or hardware installed at the receiving end. In some embodiments, Figure 6 The message processing method shown can be specifically applied to the transport layer of the receiving end. Figure 6 The message processing method shown is described as follows.

[0047] S602: Receive a first message sent by a sending end, where the first message carries indication information and identification information of a message to which the first message belongs, and the indication information is used to indicate whether out-of-order reception of messages is allowed.

[0048] After the sending end sends the first message to the receiving end based on RDMA, the receiving end can receive the first message. The first message carries indication information for indicating whether out-of-order reception between messages is allowed and identification information of the message to which the first message belongs. The specific implementation method of the sending end sending the first message to the receiving end can be found in Figure 2 The corresponding contents in the illustrated embodiment will not be described in detail here.

[0049] S604: Process the first message according to the indication information and the identification information, where the processing includes receiving messages to which the first message belongs out of order or receiving messages in an order-preserving manner.

[0050] After receiving the first message, the receiving end may process the first message according to the indication information and identification information carried in the first message, and the processing may include receiving the message to which the first message belongs out of order or receiving the message in order. Among them, if the receiving end receives the message to which the first message belongs out of order, that is, the receiving end does not maintain the order between messages to which the first message belongs, then in the case of disordered messages, the receiving end may not trigger retransmission, thereby reducing retransmission events and improving transmission efficiency. In addition, the processing of the first message may also include receiving the first message in order, that is, maintaining the order of multiple messages in the message to which the first message belongs. In this way, when the receiving end maintains the order of the first message and does not maintain the order between messages for the message to which the first message belongs, the receiving end can trigger retransmission only in the case of packet loss or disorder of the message in the message, and does not trigger retransmission in the case of disorder between messages, thereby reducing the retransmission range (that is, limiting the retransmission range to the message) and improving transmission efficiency.

[0051] In some implementation manners, the receiving end processes the first message according to the indication information and the identification information carried in the first message, which may include: When the indication information indicates that out-of-order reception between messages is allowed, performing a first process on the first message according to the identification information, the processing principle of the first process includes maintaining the order of multiple messages in the message to which the first message belongs and not maintaining the order between messages of the message to which the first message belongs; When the indication information indicates that out-of-order reception between messages is not allowed, a second process is performed on the first message. The processing principle of the second process includes maintaining the order of multiple messages within the message to which the first message belongs and maintaining the order between messages of the message to which the first message belongs.

[0052] The content indicated by the indication information carried in the first message includes two situations, one situation is that the indication information indicates that out-of-order reception between messages is allowed, and the other situation is that the indication information indicates that out-of-order reception between messages is not allowed. In the case where the indication information indicates that out-of-order reception between messages is allowed, the receiving end needs to maintain the order within the message when processing the message, and may not maintain the order between messages, that is, the receiving end can perform a first process on the first message according to the identification information, and the processing principle of the first process includes maintaining the order of multiple messages within the message to which the first message belongs and not maintaining the order between messages for the message to which the first message belongs. In the case where the indication information indicates that out-of-order reception between messages is not allowed, the receiving end needs to maintain the order within the message when processing the message, and also needs to maintain the order between messages, that is, the receiving end can perform a second process on the first message, and the processing principle of the second process includes maintaining the order of multiple messages within the message to which the first message belongs and maintaining the order between messages for the message to which the first message belongs. Among them, when the receiving end performs the second line processing on the first message, the specific implementation method can refer to the specific implementation in the relevant technology, which will not be described in detail here. The embodiment of the present application focuses on how the receiving end performs the first processing on the first message according to the identification information.

[0053] In some implementations, the first message may also carry a packet sequence number, a first packet mark, a last packet mark, an identifier of a source queue pair, and a destination queue pair. For detailed description of this information, see Figure 2 The corresponding contents in the illustrated embodiment will not be repeated here. In the case where the first message carries the packet sequence number, the first packet mark, the last packet mark, the identifier of the source queue pair and the destination queue pair, the receiving end performs the first processing on the first message according to the identification information, which may be the first processing of the first message according to the identification information, the packet sequence number, the first packet mark, the last packet mark, the identifier of the source queue pair and the identifier of the destination queue pair carried in the first message. Specifically, it may include: According to the first packet mark and the last packet mark, determine whether the first packet is the first packet or the last packet in the message; The first message is first processed according to the judgment result, the identification information, the packet sequence number, the identification of the source queue pair and the identification of the destination queue pair.

[0054] The first packet mark is used to indicate whether the first message is the first message in the message to which it belongs. For example, the first packet mark can be indicated by 1 bit, and the bit value is 1 to indicate that the message is the first message in the message to which it belongs, and the bit value is 0 to indicate that the message is not the first message in the message to which it belongs. The last packet mark is used to indicate whether the first message is the last message in the message to which it belongs. The last packet mark can also be indicated by 1 bit, and the bit value is 1 to indicate that the message is the last message in the message to which it belongs, and the bit value is 0 to indicate that the message is not the last message in the message to which it belongs. When the receiving end determines whether the first message is the first message and the last message in the message to which it belongs based on the first packet mark and the last packet mark, it can make the judgment based on the value of the first packet mark and the value of the last packet mark. For example, if the value of the first packet mark is 1, it can be determined that the first message is the first message in the message to which it belongs. If the value of the last packet mark is 1, it can be determined that the first message is the last message in the message to which it belongs. If the value of the first packet mark is 0 and the value of the last packet mark is also 0, it can be determined that the first message is neither the first message nor the last message in the message to which it belongs, but is a message other than the first and last messages in the message to which it belongs.

[0055] After determining whether the first message is the first message or the last message in the message, the first message can be processed first based on the determination result, identification information carried in the first message, the packet sequence number, the identification of the source queue pair, and the identification of the destination queue pair.

[0056] As mentioned above, the judgment result may include three situations. The first situation is that the first message is the first message in the message to which it belongs, the second situation is that the first message is neither the first message in the message to which it belongs nor the last message in the message to which it belongs, and the third situation is that the first message is the last message in the message to which it belongs. For different judgment results, the receiving end may perform different processing operations when processing the first message. The following will describe in detail how to process the first message for different judgment results.

[0057] When the judgment result includes that the first message is the first message in the message, in some implementations, performing a first process on the first message according to the judgment result, the identification information, the packet sequence number, the identification of the source queue pair, and the identification of the destination queue pair may include: Generate a corresponding first variable and a second variable according to the identification information, the identification of the source queue pair and the identification of the destination queue pair, the first variable being used to record the packet sequence number of the last message received in sequence in the message, and the second variable being used to indicate whether the first message in the message has been successfully received; Initializing a first variable and a second variable, wherein the value of the initialized first variable is a packet sequence number of the first message, and the value of the initialized second variable is used to indicate that the first message in the message has been successfully received; Deliver the first message to the host memory.

[0058] Specifically, in the case where the first message is the first message in the message, two variables can be generated for the message, one variable is used to record the packet sequence number of the last message received in order in the message, that is, the above-mentioned first variable (optionally, it can be expressed as msg_psn_cur), and the other variable is used to indicate whether the first message in the message has been successfully received, that is, the above-mentioned second variable (optionally, it can be expressed as som_received). In an embodiment of the present application, there can be a one-to-one mapping relationship between the first variable and the message identifier, the identifier of the source queue pair, and the identifier of the destination queue pair. Optionally, the mapping relationship can be stored as a table, the key is the message identifier, the identifier of the source queue pair, and the identifier of the destination queue pair, and the value is the first variable. Similarly, the second variable can also have a one-to-one mapping relationship with the message identifier, the identifier of the source queue pair, and the identifier of the destination queue pair.

[0059] After the first variable and the second variable are generated, the two variables may be initialized. Since the first message is the first message in the message and has been successfully received by the receiving end, after the first variable and the second variable are initialized, the value of the first variable is the packet sequence number of the first message, and the value of the second variable is used to indicate that the first message in the message has been successfully received. For example, the second variable is represented by 1 bit, so after the second variable is initialized, the value of the second variable may be true or 1.

[0060] After the first variable and the second variable are initialized, the first message may be delivered to the host memory.

[0061] When the judgment result includes that the first message is not the first message or the last message in the message, in some implementations, performing a first process on the first message according to the judgment result, the identification information, the packet sequence number, the identification of the source queue pair, and the identification of the destination queue pair may include: Finding a second variable corresponding to the identification information, the identification of the source queue pair, and the identification of the destination queue pair; In the case where the second variable exists and is used to indicate that the first packet in the message has been successfully received, searching for the first variable corresponding to the identification information, the identification of the source queue pair and the identification of the destination queue pair; When the packet sequence number of the first message is equal to the sum of the value of the first variable and 1, the first message is delivered to the host memory, and the value of the first variable is updated to the packet sequence number of the first message; When the packet sequence number of the first message is not equal to the sum of the value of the first variable and 1, it is determined that a message in the message to which the first message belongs is lost or out of order, and a NACK message is sent to the sender.

[0062] Specifically, when the first message is neither the first message nor the last message in the message, the corresponding second variable (used to indicate whether the first message in the message has been successfully received) can be found according to the identification information carried in the first message, the identification of the source queue pair, and the identification of the destination queue pair. If the second variable is not found or the value of the second variable is used to indicate that the first message in the message has not been successfully received (for example, the value of the second variable is false), it can be said that the receiving end has not successfully received the first message in the message to which the first message belongs. At this time, the first message can be processed according to the processing method in the relevant technology, which is not described in detail here. If the second variable exists and the value of the second variable is used to indicate that the first message in the message has been successfully received (for example, the value of the second variable is true), it can be said that the receiving end has successfully received the first message in the message to which the first message belongs. At this time, the subsequent operation can be continued, that is, according to the identification information carried in the first message, the identification of the source queue pair, and the identification of the destination queue pair, the corresponding first variable (used to record the packet sequence number of the last message that has been transmitted in sequence in the message) is found. Since the first variable and the second variable are generated by the sender according to the identification information, the identification of the source queue pair and the identification of the destination queue pair when the sender receives the first packet in the message, when the second variable is found, the first variable can also be found.

[0063] After finding the first variable, it can be determined whether the packet sequence number of the first message is equal to the sum of the value of the first variable and 1. If the packet sequence number of the first message is equal to the sum of the value of the first variable and 1, it can be explained that the packet sequence number of the first message is continuous with the packet sequence number of the previous message that the receiving end has received in order, and no packet loss or disorder has occurred. At this time, the first message can be delivered to the host memory, and the value of the first variable is updated to the packet sequence number of the first message, so that when the receiving end receives the next message, it can determine whether the packet sequence number of the first message and the packet sequence number of the next message are continuous according to the updated first variable, and then determine whether packet loss or disorder has occurred. If the packet sequence number of the first message is not equal to the sum of the value of the first variable and 1, it can be explained that the packet sequence number of the first message is discontinuous with the packet sequence number of the previous message that the receiving end has received in order, and the message in the message to which the first message belongs has been lost or disordered. At this time, the receiving end can trigger retransmission and send a NACK message to the sending end.

[0064] When the sender sends a NACK message to the receiver, in order to facilitate limiting the message retransmission range of the sender within the message, in some implementations, the NACK message may carry identification information of the message to which the retransmitted message belongs (i.e., identification information of the message to which the first message belongs) and the starting packet sequence number of the retransmission. The starting packet sequence number of the retransmission may be equal to the sum of the value of the first variable and 1, indicating that packet loss or disorder occurs starting from the starting packet sequence number, and the message starting from the starting packet sequence number in the message needs to be retransmitted. For the sender, when the NACK message is received and the message is retransmitted, based on the identification information carried in the NACK message and the starting packet sequence number of the retransmission, only the message starting from the starting packet sequence number of the retransmission in the message corresponding to the identification information can be sent to the receiver, thereby limiting the retransmission range to within the message, reducing the number of retransmitted messages, reducing the retransmission cost, and improving the transmission efficiency of the network. For details, please refer to Figure 5 The embodiments shown are not further described here.

[0065] When the judgment result includes that the first message is the last message in the message, in some implementations, performing a first process on the first message according to the judgment result, the identification information, the packet sequence number, the identification of the source queue pair, and the identification of the destination queue pair includes: Finding a first variable and a second variable corresponding to the identification information, the identification of the source queue pair, and the identification of the destination queue pair; Release the first variable and the second variable; Delivering the first message to the host memory; Send an ACK message to the sender.

[0066] In order to facilitate the understanding of the receiving end's processing flow of the message, the following will be Figure 7 A more specific implementation method shown in FIG. Figure 7 As shown, when processing a message, the following steps may be included.

[0067] Step 1: Extract the identification information msg_id, packet sequence number PSN, first packet mark SOM, last packet mark EOM, indication information msg_order_flag used to indicate whether out-of-order reception of messages is allowed, source queue pair SQP, destination queue pair DQP and other fields from the message.

[0068] Step 2: Check the value of msg_order_flag. If msg_order_flag=false, that is, out-of-order reception of messages is not allowed, process the message according to the existing process and go to step 8. If msg_order_flag=true, that is, out-of-order reception of messages is allowed, go to step 3.

[0069] Step 3: Check the value of SOM. If SOM=1, maintain a variable msg_psn_cur for the msg_id. The initial value of msg_psn_cur is the PSN of the message. Also maintain a variable som_received=true. Then, go to step 5. If SOM=0, determine whether the som_received corresponding to the msg exists and is true. If it exists and is true, find the corresponding msg_psn_cur according to msg_id and execute step 4. If it does not exist or is not true, process the message according to the existing process and go to step 8.

[0070] Step 4: Determine whether the PSN of the message is equal to msg_psn_cur+1. If so, update msg_psn_cur to the PSN of the message, and then execute step 5. If not, execute step 7.

[0071] Step 5: Deliver the message to the host memory and determine whether the message is the tail packet. If it is the tail packet, execute step 6; if not, execute step 8.

[0072] Step 6: If the message EOM=1, release the current msg_psn_cur variable, send ACK to the source end (i.e., the sending end), generate a completion queue entry (Complete Queue Entry, CQE) (this processing is the same as the existing process), and go to step 8.

[0073] Step 7: If the PSN of the message is not equal to msg_psn_cur+1, retransmission is triggered, that is, a NACK message is sent to the source end and the retransmission starting sequence number is indicated as msg_psn_cur+1, and then go to step 8.

[0074] Step 8: The current message processing is completed.

[0075] In an embodiment of the present application, since the first message received by the receiving end carries identification information of the message to which the first message belongs and indication information for indicating whether out-of-order reception between messages is allowed, for the receiving end, when the first message sent by the sending end indicates that out-of-order reception between messages is allowed, the receiving end only needs to maintain the order of the messages within the message, and there is no need to maintain the order between messages. In other words, the receiving end may not trigger retransmission when there is out-of-order transmission between messages, and may trigger retransmission only when packet loss or out-of-order transmission occurs within the message. This may reduce the retransmission range (i.e., limit the retransmission range to within the message), thereby improving transmission efficiency.

[0076] In order to facilitate understanding of the technical solution provided in the embodiments of the present application, three more specific implementation methods will be used as examples for explanation below.

[0077] Example 1: This example illustrates how to perform load balancing based on message granularity when the application layer at the sending end allows messages to be out of order. This example takes a typical spine-leaf architecture as an example. Assume that the switch device has 4 ports and the port bandwidth is 25Gbps. When there is no bandwidth convergence networking, each leaf switch has 2 uplinks available. Assume that the remaining available bandwidth of the link from spine1 node to leaf2 node is 10Gbps.

[0078] Send an elephant flow flow1 from server 1 to server 2, assuming that the elephant flow rate is 16Gbps. The hashing route selection based on the flow ID (typical quintuple) on the leaf switch is random, and it is impossible to perform optimization such as WCMP based on the spine export link. At this time, the first uplink may be hashed to send the elephant flow to the spine1 node. Since there is only 10Gbps left from spine1 to leaf2, congestion will occur at the spine1->leaf2 export and trigger congestion control, flow control and other mechanisms, which will eventually limit the sending rate of flow1 to less than 10Gbps and increase the tail delay of the flow. For example, Figure 8 (a) shown.

[0079] If the message carries the msg_order_flag=true flag, the leaf switch can identify the service type of the message and flexibly perform hash routing based on msg_id+flow ID. Due to the randomness of the hash, in this way, each uplink of leaf1 shares 8Gbps of traffic, thereby preventing congestion of the outbound port of spine1->leaf2. In the end, flow1 can be sent at the full rate of 16Gbps without losing throughput. Figure 8 (b) shown.

[0080] When the message arrives at server 2, the receiving network card transport layer determines that it is not necessary to maintain the order between messages based on the msg_order_flag flag carried by the message, so there is no need to reorder the out-of-order messages, thus no overhead of reordering the messages is incurred.

[0081] Example 2: This example illustrates the overall processing flow of the fine-grained receiving transport layer based on messages. Assume that a service flow flow1 does not require order preservation between messages, and the message carries the msg_order_flag=true flag. When the receiving transport layer parses the message, the transport layer receives and processes it based on the message granularity. This example uses two situations to illustrate the advantages of fine-grained processing based on message granularity.

[0082] Case 1: flow1 contains two msgs, msg1 and msg2, and msg1 and msg2 are each divided into 4 messages. The psn of msg1 is 1~4, and the psn of msg2 is 5~8. The messages are sent in the order of psn1->psn8 at the source. Assume that the two msg messages arrive out of order (the order is still maintained within the message) due to the use of a load balancing strategy based on msg granularity, but no message is lost. Assume that the order of the PSNs of the messages arriving at the receiving end is: 1,2,5,3,6,4,7,8. The receiving end then processes the messages as follows: 1) When a PSN1 message arrives, the msg number is 1 and the SOM mark is 1, the variables msg1_psn_cur=1 and msg1_som_received=true are created, the PSN1 message is delivered to the host memory, the EOM mark is 0, and no message completion related processing is performed; 2) The PSN2 message arrives, the msg number is 1, and the SOM mark is 0. According to msg1, msg1_psn_cur=1 and msg1_som_received=true are found. Since the PSN of the PSN2 message is 2, which is equal to msg1_psn_cur+1, the current message sequence is correct. msg1_psn_cur is updated to 2, and the PSN2 message is delivered to the host memory. The EOM mark is 0, and no message completion related processing is performed; 3) When the PSN5 message arrives, the msg number is 2 and the SOM mark is 1, the variables msg2_psn_cur=5 and msg2_som_received=true are created, the PSN5 message is delivered to the host memory, the EOM mark is 0, and no message completion related processing is performed; 4) PSN3 message arrives, msg number is 1, SOM mark is 0, according to msg1, msg1_psn_cur=2, msg2_som_received=true is found, because the PSN of PSN3 message is 3, which is equal to msg1_psn_cur+1, the current message sequence is correct, msg1_psn_cur is updated to 3, PSN3 message is delivered to host memory, EOM mark is 0, and no message completion related processing is performed; 5) PSN6 message arrives, msg number is 2, SOM mark is 0, according to msg2, find msg2_psn_cur=5, msg2_som_received=true, because the PSN of PSN6 message is 6, which is equal to msg2_psn_cur+1, the current message sequence is correct, msg2_psn_cur is updated to 6, PSN6 message is delivered to host memory, EOM mark is 0, and no message completion related processing is performed; 6) PSN4 message arrives, msg number is 1, SOM mark is 0, according to msg1, msg1_psn_cur=3, msg1_som_received=true are found. Since the PSN of PSN4 message is 4, which is equal to msg1_psn_cur+1, the current message sequence is correct, and PSN4 message is delivered to host memory. Since EOM mark is 1, message completion related processing is performed, and msg1_psn_cur variable is cleared; 7) PSN7 message arrives, msg number is 2, SOM mark is 0, according to msg2, msg2_psn_cur=6, msg2_som_received=true are found. Since the PSN of PSN7 message is 7, which is equal to msg2_psn_cur+1, the current message sequence is correct, msg2_psn_cur is updated to 7, PSN7 message is delivered to host memory, EOM mark is 0, and no message completion related processing is performed; 8) PSN8 message arrives, msg number is 2, SOM mark is 0, according to msg2, msg2_psn_cur=7, msg2_som_received=true are found. Since the PSN of PSN8 message is 8, which is equal to msg2_psn_cur+1, the current message sequence is correct, and PSN8 message is delivered to host memory. Since EOM mark is 1, message completion related processing is performed, and msg2_psn_cur variable is cleared.

[0083] In this case, although the message PSN arrives at the receiving end out of order, since there is no need to maintain the order between messages, the disorder is determined only within the msg, so the packet loss and retransmission process will not be triggered.

[0084] Case 2: flow1 contains four msgs, namely msg1, msg2, msg3 and msg4. msg1, msg2 and msg4 are each divided into 4 messages, and msg3 is divided into one message. The message psn of msg1 is 1~4, the message psn of msg2 is 5~8, the message psn of msg3 is 9, and the message psn of msg4 is 10~13. The messages are sent in the order of psn1->psn13 at the source. Assume that messages PSN3, PSN9, PSN10 are lost due to bit errors during transmission, and the PSN order of the messages received by the receiving end is: 1,2,4,5,6,7,8,11,12,13. Then the receiving end processes the messages as follows: 1) When a PSN1 message arrives, the msg number is 1 and the SOM mark is 1, the variables msg1_psn_cur=1 and msg1_som_received=true are created, the PSN1 message is delivered to the host memory, the EOM mark is 0, and no message completion related processing is performed; 2) The PSN2 message arrives, the msg number is 1, and the SOM mark is 0. According to msg1, msg1_psn_cur=1 and msg1_som_received=true are found. Since the PSN of the PSN2 message is 2, which is equal to msg1_psn_cur+1, the current message sequence is correct. msg1_psn_cur is updated to 2, and the PSN1 message is delivered to the host memory. The EOM mark is 0, and no message completion related processing is performed; 3) PSN4 message arrives, msg number is 1, SOM flag is 0, according to msg1, msg1_psn_cur=2, msg1_som_received=true are found. Since the PSN of PSN4 message is 4, which is not equal to msg1_psn_cur+1 (3), the current message sequence is incorrect, the current PSN4 message is discarded, and a NACK message is returned to the sender, carrying msg1, psn3 and msg_order_flag=true information; 4) When the PSN5 message arrives, the msg number is 2 and the SOM mark is 1, the variables msg2_psn_cur=5 and msg2_som_received=true are created, the PSN5 message is delivered to the host memory, the EOM mark is 0, and no message completion related processing is performed; 5) PSN6 message arrives, msg number is 2, SOM mark is 0, according to msg2, find msg2_psn_cur=5, msg2_som_received=true, because the PSN of PSN6 message is 6, which is equal to msg2_psn_cur+1, the current message sequence is correct, msg2_psn_cur is updated to 6, PSN6 message is delivered to host memory, EOM mark is 0, and no message completion related processing is performed; 6) PSN7 message arrives, msg number is 2, SOM mark is 0, according to msg2, msg2_psn_cur=6, msg2_som_received=true is found. Since the PSN of PSN7 message is 7, which is equal to msg2_psn_cur+1, the current message sequence is correct, msg2_psn_cur is updated to 7, PSN7 message is delivered to the host memory, EOM mark is 0, and no message completion related processing is performed; 7) PSN8 message arrives, msg number is 2, SOM mark is 0, according to msg2, find msg2_psn_cur=7, msg2_som_received=true, because PSN of PSN8 is 8, which is equal to msg2_psn_cur+1, the current message sequence is correct, and PSN8 message is delivered to the host memory. Because EOM mark is 1, the message is completed and the msg2_psn_cur variable is cleared; 8) PSN11 message arrives, msg number is 4, SOM mark is 0, extract msg4 and no msg4_som_received information is found, indicating that the first packet is lost, and the original process is followed; 9) PSN12 message arrives, msg number is 4, SOM mark is 0, extract msg4 and no msg4_som_received information is found, indicating that the first packet is lost, and the original process is followed; 10) PSN13 message arrives, msg number is 4, SOM mark is 0, extract msg4 and no msg4_som_received information is found, indicating that the first packet is lost, and the original process is followed; 11) PSN3 message arrives, msg number is 1, SOM mark is 0, according to msg1, msg1_psn_cur=2, msg1_som_received=true is found, because the PSN of PSN3 is 3, which is equal to msg1_psn_cur+1, the current message sequence is correct, msg1_psn_cur is updated to 3, PSN3 message is delivered to the host memory, EOM mark is 0, and no message completion related processing is performed; 12) PSN4 message arrives, msg number is 1, SOM mark is 0, according to msg1, msg1_psn_cur=3, msg1_som_received=true are found, since PSN4's PSN is 4, which is equal to msg1_psn_cur+1, the current message sequence is correct, and PSN4 message is delivered to the host memory. Since EOM mark is 1, the message is completed and the msg1_psn_cur variable is cleared; 13) The timed-out retransmitted PSN9 message arrives, with the msg number 3. Since it carries the SOM and EOM tags, the message is delivered to the host memory for message completion related processing; 14) The timed-out retransmitted messages of PSN10~PSN13 arrive, and their processing flow is exactly the same as that of msg2, so it will not be repeated here.

[0085] In this case, although packet loss occurs, since the packet loss retransmission range is limited to msg, only two messages need to be retransmitted, which reduces the packet loss retransmission rate and improves transmission efficiency.

[0086] Embodiment 3: This embodiment illustrates an encapsulation extension method in which a service message carries msg_id (if necessary) and msg_order_flag information, and a NACK message carries msg_id (if necessary) and msg_order_flag, taking the currently popular Infiniband (IB) and UEC protocols as examples.

[0087] For the Infiniband protocol, the basic encapsulation format of the message is as follows Fig. 9 As shown in the figure, Opcode is used to indicate the type of the current message. Opcode has 8 bits in total. When Opcode starts with 000, it indicates that the current message is an RC service message or an ACK / NACK message. The last 5 bits are used to indicate the specific type of the message.

[0088] In the basic encapsulation format, there are two Reserved fields that can be reused, of which 1 bit can be used to represent the msg_order_flag information, and the remaining 12 bits can be used to identify the msg_id information. This extension method is applicable to both service messages and NACK messages, and the only difference is the definition of Opcode. For example, Fig.10 shown.

[0089] A new RC message type can be defined by modifying the Opcode. In the Opcode field of the RC operation, values ​​such as 00011000~00011111 are reserved values ​​and can be used to define a new encapsulation format. For example, the present invention takes 00011000 to indicate that the message is a Write message, and does not require order preservation between messages. Similarly, 00011111 can be taken to indicate that the message is a NACK message, and it is also the content of the present invention. The NACK message carries an AETH header (Ack Extension Transport Header), and this part of the definition remains unchanged.

[0090] For the UEC protocol, the semantic header encapsulation format is extended as follows Fig.11 As shown, the message already carries a 16-bit messageid field, which does not need to be expanded. In addition, there are 3 reserved fields in the message, totaling 13 bits, which can be used to carry msg_order_flag. The example in the figure uses 1 bit of the first reserved field to carry msg_order_flag. Opcode indicates the message encapsulation format of the present invention in the reserved value, such as 0x11.

[0091] In the NACK message, the extended encapsulation format is as follows Fig.12As shown. The messge_id is carried in the semantic header, which is a general return header encapsulation. The PSN is carried in the delivery sublayer header. When the Code value is not equal to 0, it indicates that the current message is a NACK message. The index generation field in the general semantic header of the NACK message is an invalid field and can be used to carry msg_order_flag. Opcode redefines the encapsulation type of the semantic header and the delivery sublayer message in the reserved value, for example, the value is 0x12.

[0092] The three embodiments described above are only preferred embodiments of the present application and are not intended to limit the present application. For those skilled in the art, the present application may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application shall be included in the protection scope of the present application.

[0093] The above describes specific embodiments of the present application. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recorded in the claims can be performed in an order different from that in the embodiments and still achieve the desired results. In addition, the processes depicted in the accompanying drawings do not necessarily require the specific order or continuous order shown to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0094] Fig.13 This is a schematic diagram of the structure of an electronic device according to an embodiment of the present application. Fig.13 At the hardware level, the electronic device includes a processor, and optionally also includes an internal bus, a network interface, and a memory. The memory may include a memory, such as a high-speed random access memory (RAM), and may also include a non-volatile memory (non-volatile memory), such as at least one disk storage. Of course, the electronic device may also include hardware required for other services.

[0095] The processor, network interface and memory can be interconnected through an internal bus, which can be an ISA (Industry Standard Architecture) bus, a PCI (Peripheral Component Interconnect) bus or an EISA (Extended Industry Standard Architecture) bus. The bus can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Fig.13Only one bidirectional arrow is used in the diagram, but this does not mean that there is only one bus or only one type of bus.

[0096] The memory is used to store the program. Specifically, the program may include a program code, and the program code includes a computer operation instruction. The memory may include a memory and a non-volatile memory, and provides instructions and data to the processor.

[0097] The processor reads the corresponding computer program from the non-volatile memory into the memory and then runs it, forming an RDMA-based message sending device at the logical level. The processor executes the program stored in the memory and is specifically used to perform the following operations: A first message is sent to a receiving end, wherein the first message carries indication information and identification information of a message to which the first message belongs, and the indication information is used to indicate whether out-of-order reception of messages is allowed.

[0098] The above application Fig.13 The method performed by the RDMA-based message sending device disclosed in the illustrated embodiment can be applied to a processor or implemented by a processor. The processor may be an integrated circuit chip with signal processing capabilities. In the implementation process, each step of the above method can be completed by an integrated logic circuit of hardware in the processor or an instruction in software form. The above processor can be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it can also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components. The methods, steps and logic block diagrams disclosed in this application can be implemented or executed. The general-purpose processor can be a microprocessor or the processor can also be any conventional processor, etc. The steps of the method disclosed in this application can be directly embodied as being executed by a hardware decoding processor, or being executed by a combination of hardware and software modules in the decoding processor. The software module can be located in a storage medium mature in the art, such as a random access memory, a flash memory, a read-only memory, a programmable read-only memory, or an electrically erasable programmable memory, a register, etc. The storage medium is located in the memory, and the processor reads the information in the memory and completes the steps of the above method in combination with its hardware.

[0099] The electronic device may also perform Figure 2The method is implemented in a packet sending device based on RDMA. Figure 2 The functions in the illustrated embodiments will not be described in detail in this application.

[0100] Of course, in addition to software implementation methods, the electronic device of the present application does not exclude other implementation methods, such as logic devices or a combination of software and hardware, etc. That is to say, the execution subject of the following processing flow is not limited to each logic unit, but can also be hardware or logic devices.

[0101] The present application also proposes a computer-readable storage medium, which stores one or more programs, wherein the one or more programs include instructions, which, when executed by a portable electronic device including a plurality of application programs, enable the portable electronic device to execute Figure 2 The method of the embodiment shown is specifically used to perform the following operations: A first message is sent to a receiving end, wherein the first message carries indication information and identification information of a message to which the first message belongs, and the indication information is used to indicate whether out-of-order reception of messages is allowed.

[0102] Fig.14 This is a schematic diagram of the structure of a message sending device 140 based on RDMA in an embodiment of the present application. Fig.14 In a software implementation, the RDMA-based message sending device 140 may include: a sending module 141, wherein: The sending module 141 sends a first message to a receiving end, wherein the first message carries indication information and identification information of a message to which the first message belongs, and the indication information is used to indicate whether out-of-order reception of messages is allowed.

[0103] In some implementations, the sending module 141 sends the first message to the receiving end, including: Receive a first parameter sent by the application layer, where the first parameter is used to indicate whether out-of-order reception of messages is allowed; Sending a first message to the receiving end according to the first parameter; Among them, when the first parameter indicates that out-of-order reception between messages is allowed, the indication information carried in the first message is used to indicate that out-of-order reception between messages is allowed; when the first parameter indicates that out-of-order reception between messages is not allowed, the indication information carried in the first message is used to indicate that out-of-order reception between messages is not allowed.

[0104] In some implementations, the sending module 141 receives the first parameter sent by the application layer, including: The first parameter sent by the application layer based on an extended programming interface is received, where the extended programming interface includes an extended first field, and the first field is used to carry the first parameter.

[0105] In some implementations, the first message also carries an operation code; the sending module 141 also performs load balancing according to the identification information when the first condition is met; Among them, the first condition includes that the operation code is in reliable mode and the indication information is used to indicate that out-of-order reception between messages is allowed; the load balancing according to the identification information includes sending the messages with the same identification information to the same transmission path, and sending the messages with different identification information to the same or different transmission paths.

[0106] In some implementations, when the indication information is used to indicate that out-of-order reception between messages is allowed, the sending module 141 further includes: receiving a NACK message sent by the receiving end, where the NACK message is sent by the receiving end when it is determined that a message in the message to which the first message belongs is lost or out of order; The message is retransmitted according to the NACK message.

[0107] In some implementations, the NACK message carries the identification information and the sequence number of the starting packet of the retransmission; the sending module 141 retransmits the message according to the NACK message, including: According to the identification information and the starting packet sequence number, the message starting from the starting packet sequence number in the message corresponding to the identification information is sent to the receiving end.

[0108] The RDMA-based message sending device 140 provided in the present application can also execute Figure 2 The method is implemented by the RDMA-based message sending device 140. Figure 2 The functions of the illustrated embodiments will not be described in detail in this application.

[0109] Fig.15 This is a schematic diagram of the structure of an electronic device according to an embodiment of the present application. Fig.15 At the hardware level, the electronic device includes a processor, and optionally also includes an internal bus, a network interface, and a memory. The memory may include a memory, such as a high-speed random access memory (RAM), and may also include a non-volatile memory (non-volatile memory), such as at least one disk storage. Of course, the electronic device may also include hardware required for other services.

[0110] The processor, network interface and memory can be interconnected through an internal bus, which can be an ISA (Industry Standard Architecture) bus, a PCI (Peripheral Component Interconnect) bus or an EISA (Extended Industry Standard Architecture) bus. The bus can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Fig.15 Only one bidirectional arrow is used in the diagram, but this does not mean that there is only one bus or only one type of bus.

[0111] The memory is used to store the program. Specifically, the program may include a program code, and the program code includes a computer operation instruction. The memory may include a memory and a non-volatile memory, and provides instructions and data to the processor.

[0112] The processor reads the corresponding computer program from the non-volatile memory into the memory and then runs it, forming an RDMA-based message processing device at the logical level. The processor executes the program stored in the memory and is specifically used to perform the following operations: Receive a first message sent by a sending end, where the first message carries indication information and identification information of a message to which the first message belongs, and the indication information is used to indicate whether out-of-order reception of messages is allowed; The first message is processed according to the indication information and the identification information, wherein the processing includes receiving messages to which the first message belongs out of order or receiving messages in an order-preserving manner.

[0113] The above application Fig.15The method performed by the RDMA-based message processing device disclosed in the illustrated embodiment can be applied to a processor or implemented by a processor. The processor may be an integrated circuit chip with signal processing capabilities. In the implementation process, each step of the above method can be completed by an integrated logic circuit of hardware in the processor or an instruction in the form of software. The above processor can be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it can also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components. The methods, steps and logic block diagrams disclosed in this application can be implemented or executed. The general-purpose processor can be a microprocessor or the processor can also be any conventional processor, etc. The steps of the method disclosed in this application can be directly embodied as being executed by a hardware decoding processor, or being executed by a combination of hardware and software modules in the decoding processor. The software module can be located in a storage medium mature in the art such as a random access memory, a flash memory, a read-only memory, a programmable read-only memory, or an electrically erasable programmable memory, a register, etc. The storage medium is located in the memory, and the processor reads the information in the memory and completes the steps of the above method in combination with its hardware.

[0114] The electronic device may also perform Figure 6 The method is to realize a message processing device based on RDMA. Figure 6 The functions in the illustrated embodiments will not be described in detail in this application.

[0115] Of course, in addition to software implementation methods, the electronic device of the present application does not exclude other implementation methods, such as logic devices or a combination of software and hardware, etc. That is to say, the execution subject of the following processing flow is not limited to each logic unit, but can also be hardware or logic devices.

[0116] The present application also proposes a computer-readable storage medium, which stores one or more programs, wherein the one or more programs include instructions, which, when executed by a portable electronic device including a plurality of application programs, enable the portable electronic device to execute Figure 6 The method of the embodiment shown is specifically used to perform the following operations: Receive a first message sent by a sending end, where the first message carries indication information and identification information of a message to which the first message belongs, and the indication information is used to indicate whether out-of-order reception of messages is allowed; The first message is processed according to the indication information and the identification information, wherein the processing includes receiving messages to which the first message belongs out of order or receiving messages in an order-preserving manner.

[0117] Fig.16 This is a schematic diagram of the structure of a message processing device 160 based on RDMA in an embodiment of the present application. Fig.16 In a software implementation, the RDMA-based message processing device 160 may include: a receiving module 161 and a processing module 162, wherein: The receiving module 161 receives a first message sent by a sending end, wherein the first message carries indication information and identification information of a message to which the first message belongs, and the indication information is used to indicate whether out-of-order reception of messages is allowed; The processing module 162 processes the first message according to the indication information and the identification information, wherein the processing includes receiving messages to which the first message belongs out of order or receiving messages in an order-preserving manner.

[0118] In some implementations, the processing module 162 processes the first message according to the indication information and the identification information, including: When the indication information indicates that out-of-order reception between messages is allowed, performing a first process on the first message according to the identification information, wherein a processing principle of the first process includes maintaining the order of multiple messages in the message to which the first message belongs and not maintaining the order between messages of the message to which the first message belongs; When the indication information indicates that out-of-order reception between messages is not allowed, a second processing is performed on the first message, and the processing principles of the second processing include maintaining the order of multiple messages within the message to which the first message belongs and maintaining the order between messages of the message to which the first message belongs.

[0119] In some implementations, the first message further carries a packet sequence number, a first packet mark, a last packet mark, an identifier of a source queue pair, and an identifier of a destination queue pair; the processing module 162 performs a first process on the first message according to the identifier information, including: Determine, according to the first packet mark and the last packet mark, whether the first message is the first message or the last message in the message; The first message is subjected to a first processing according to the judgment result, the identification information, the packet sequence number, the identification of the source queue pair and the identification of the destination queue pair.

[0120] In some implementations, the judgment result includes that the first message is the first message in the message to which it belongs; the processing module 162 performs a first process on the first message according to the judgment result, the identification information, the packet sequence number, the identification of the source queue pair, and the identification of the destination queue pair, including: Generate a corresponding first variable and a second variable according to the identification information, the identification of the source queue pair and the identification of the destination queue pair, wherein the first variable is used to record the packet sequence number of the last message received in sequence in the message, and the second variable is used to indicate whether the first message in the message has been successfully received; Initializing the first variable and the second variable, wherein the value of the first variable after initialization is the packet sequence number of the first message, and the value of the second variable after initialization is used to indicate that the first message in the message has been successfully received; Deliver the first message to the host memory.

[0121] In some implementations, the judgment result includes that the first message is not the first message or the last message in the message to which it belongs; the processing module 162 performs a first process on the first message according to the judgment result, the identification information, the packet sequence number, the identification of the source queue pair, and the identification of the destination queue pair, including: Finding a second variable corresponding to the identification information, the identification of the source queue pair, and the identification of the destination queue pair; In the case where the second variable exists and is used to indicate that the first message in the message has been successfully received, searching for a first variable corresponding to the identification information, the identification of the source queue pair and the identification of the destination queue pair; When the packet sequence number is equal to the sum of the value of the first variable and 1, delivering the first message to the host memory, and updating the value of the first variable to the packet sequence number; When the packet sequence number is not equal to the sum of the value of the first variable and 1, it is determined that a packet in the message to which the first message belongs is lost or out of order, and a NACK message is sent to the sending end.

[0122] In some implementations, the NACK message carries the identification information and a start packet sequence number of the retransmission, where the start packet sequence number is equal to the sum of the value of the first variable and 1.

[0123] In some implementations, the judgment result includes that the first message is the last message in the message to which it belongs; the processing module 162 performs a first process on the first message according to the judgment result, the identification information, the packet sequence number, the identification of the source queue pair, and the identification of the destination queue pair, including: Searching for a first variable and a second variable corresponding to the identification information, the identification of the source queue pair, and the identification of the destination queue pair; releasing the first variable and the second variable; Delivering the first message to a host memory; Send an ACK message to the sending end.

[0124] The RDMA-based message processing device 160 provided in the present application can also execute Figure 6 The method is implemented by the RDMA-based message processing device 160. Figure 6 The functions of the illustrated embodiments will not be described in detail in this application.

[0125] The embodiment of the present application also provides a message sending and processing system based on RDMA, which includes a sending end and a receiving end. The sending end sends a first message to the receiving end, and the first message carries indication information and identification information of the message to which the first message belongs, and the indication information is used to indicate whether out-of-order reception between messages is allowed. The receiving end receives the first message and processes the first message according to the indication information and the identification information, and the processing includes out-of-order reception between messages or order-preserving reception between messages for the message to which the first message belongs.

[0126] In the embodiment of the present application, the specific implementation of each step performed by the sending end can be found in Figure 2 The specific implementation of the corresponding steps in the embodiment shown can achieve the same technical effect, so it will not be described in detail here. The specific implementation of each step performed by the receiving end can be found in Figure 6 The specific implementation of the corresponding steps in the illustrated embodiment can achieve the same technical effect and will not be described in detail here.

[0127] The present application also proposes a computer program product, which includes a non-transitory computer-readable storage medium storing a computer program, and the computer program is operable to enable a computer to execute some or all of the steps in the above-mentioned RDMA-based message sending method embodiment, or to execute some or all of the steps in the above-mentioned RDMA-based message processing method embodiment.

[0128] In short, the above description is only a preferred embodiment of the present application and is not intended to limit the protection scope of the present application. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application shall be included in the protection scope of the present application.

[0129] The systems, devices, modules or units described in the above embodiments may be implemented by computer chips or entities, or by products with certain functions. A typical implementation device is a computer. Specifically, the computer may be, for example, a personal computer, a laptop computer, a cellular phone, a camera phone, a smart phone, a personal digital assistant, a media player, a navigation device, an email device, a game console, a tablet computer, a wearable device, or a combination of any of these devices.

[0130] Computer readable media include permanent and non-permanent, removable and non-removable media that can be implemented by any method or technology to store information. Information can be computer readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, magnetic tape magnetic disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer readable media does not include temporary computer readable media (transitory media), such as modulated data signals and carrier waves.

[0131] It should also be noted that the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, commodity or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, commodity or device. In the absence of more restrictions, the elements defined by the sentence "comprises a ..." do not exclude the existence of other identical elements in the process, method, commodity or device including the elements.

[0132] Each embodiment in this application is described in a progressive manner, and the same or similar parts between the embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the system embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.

Claims

1. A method for sending a message based on RDMA, applied to a sending end, comprising: A first message is sent to a receiving end, wherein the first message carries indication information and identification information of a message to which the first message belongs, and the indication information is used to indicate whether out-of-order reception of messages is allowed.

2. The method according to claim 1, wherein sending the first message to the receiving end comprises: Receive a first parameter sent by the application layer, where the first parameter is used to indicate whether out-of-order reception of messages is allowed; Sending a first message to the receiving end according to the first parameter; Among them, when the first parameter indicates that out-of-order reception between messages is allowed, the indication information carried in the first message is used to indicate that out-of-order reception between messages is allowed; when the first parameter indicates that out-of-order reception between messages is not allowed, the indication information carried in the first message is used to indicate that out-of-order reception between messages is not allowed.

3. The method according to claim 2, wherein receiving the first parameter sent by the application layer comprises: The first parameter sent by the application layer based on an extended programming interface is received, where the extended programming interface includes an extended first field, and the first field is used to carry the first parameter.

4. The method according to claim 1, wherein the first message also carries an operation code; the method further comprises: When the first condition is met, load balancing is performed according to the identification information; Among them, the first condition includes that the operation code is in reliable mode and the indication information is used to indicate that out-of-order reception between messages is allowed; the load balancing according to the identification information includes sending the messages with the same identification information to the same transmission path, and sending the messages with different identification information to the same or different transmission paths.

5. The method according to claim 1, wherein the indication information is used to indicate that out-of-order reception between messages is allowed, the method further comprises: receiving a NACK message sent by the receiving end, where the NACK message is sent by the receiving end when it is determined that a message in the message to which the first message belongs is lost or out of order; The message is retransmitted according to the NACK message.

6. The method according to claim 5, wherein the NACK message carries the identification information and the sequence number of the starting packet of the retransmission; and the message retransmission according to the NACK message comprises: According to the identification information and the starting packet sequence number, the message starting from the starting packet sequence number in the message corresponding to the identification information is sent to the receiving end.

7. A message processing method based on RDMA, applied to a receiving end, comprising: Receive a first message sent by a sending end, where the first message carries indication information and identification information of a message to which the first message belongs, and the indication information is used to indicate whether out-of-order reception of messages is allowed; The first message is processed according to the indication information and the identification information, wherein the processing includes receiving messages to which the first message belongs out of order or receiving messages in an order-preserving manner.

8. The method according to claim 7, wherein the processing of the first message according to the indication information and the identification information comprises: When the indication information indicates that out-of-order reception between messages is allowed, performing a first process on the first message according to the identification information, wherein a processing principle of the first process includes maintaining the order of multiple messages in the message to which the first message belongs and not maintaining the order between messages of the message to which the first message belongs; When the indication information indicates that out-of-order reception between messages is not allowed, a second processing is performed on the first message, and the processing principles of the second processing include maintaining the order of multiple messages within the message to which the first message belongs and maintaining the order between messages of the message to which the first message belongs.

9. The method according to claim 8, wherein the first message further carries a packet sequence number, a first packet mark, a last packet mark, an identifier of a source queue pair, and an identifier of a destination queue pair; The performing a first process on the first message according to the identification information includes: Determine, according to the first packet mark and the last packet mark, whether the first message is the first message or the last message in the message; The first message is subjected to a first processing according to the judgment result, the identification information, the packet sequence number, the identification of the source queue pair and the identification of the destination queue pair.

10. The method according to claim 9, wherein the judgment result includes that the first message is the first message in the message; and performing a first process on the first message according to the judgment result, the identification information, the packet sequence number, the identification of the source queue pair and the identification of the destination queue pair, comprises: Generate a corresponding first variable and a second variable according to the identification information, the identification of the source queue pair and the identification of the destination queue pair, wherein the first variable is used to record the packet sequence number of the last message received in sequence in the message, and the second variable is used to indicate whether the first message in the message has been successfully received; Initializing the first variable and the second variable, wherein the value of the first variable after initialization is the packet sequence number of the first message, and the value of the second variable after initialization is used to indicate that the first message in the message has been successfully received; Deliver the first message to the host memory.

11. The method according to claim 9, wherein the judgment result includes that the first message is not the first message or the last message in the message; and performing a first process on the first message according to the judgment result, the identification information, the packet sequence number, the identification of the source queue pair and the identification of the destination queue pair, comprising: Finding a second variable corresponding to the identification information, the identification of the source queue pair, and the identification of the destination queue pair; In the case where the second variable exists and is used to indicate that the first message in the message has been successfully received, searching for a first variable corresponding to the identification information, the identification of the source queue pair and the identification of the destination queue pair; When the packet sequence number is equal to the sum of the value of the first variable and 1, delivering the first message to the host memory and updating the value of the first variable to the packet sequence number; When the packet sequence number is not equal to the sum of the value of the first variable and 1, it is determined that a packet in the message to which the first packet belongs is lost or out of order, and a NACK message is sent to the sending end.

12. The method according to claim 11, wherein the NACK message carries the identification information and a starting packet sequence number of the retransmission, and the starting packet sequence number is equal to the sum of the value of the first variable and 1.

13. The method according to claim 9, wherein the judgment result includes that the first message is the last message in the message; and the first processing of the first message according to the judgment result, the identification information, the packet sequence number, the identification of the source queue pair and the identification of the destination queue pair comprises: Searching for a first variable and a second variable corresponding to the identification information, the identification of the source queue pair, and the identification of the destination queue pair; releasing the first variable and the second variable; Delivering the first message to a host memory; Send an ACK message to the sending end.

14. A message sending and processing system based on RDMA, comprising a sending end and a receiving end, wherein; The sending end sends a first message to the receiving end, where the first message carries indication information and identification information of a message to which the first message belongs, and the indication information is used to indicate whether out-of-order reception of messages is allowed; The receiving end receives the first message, and processes the first message according to the indication information and the identification information, wherein the processing includes receiving messages to which the first message belongs out of order or receiving messages in an order-preserving manner.

15. An electronic device, comprising: processor; a memory for storing instructions executable by the processor; The processor is configured to execute the instructions to implement the method according to any one of claims 1 to 13. 16 . A computer-readable storage medium, when instructions in the storage medium are executed by a processor of an electronic device, the electronic device is enabled to execute the method according to claim 1 .

17. A computer program product, comprising a non-transitory computer-readable storage medium storing a computer program, wherein the computer program is operable to cause a computer to execute part or all of the steps in the method according to any one of claims 1 to 13.

Images