Session table item establishment method and device, equipment and medium

Abstract

The embodiment of the invention provides a session table item establishing method, device, equipment and medium, relates to the technical field of communication, is applied to an RDMA gateway, and comprises the following steps: receiving a first RDMA data message, taking a source address as an address of a first host, taking a destination address as an address of a second host, and carrying a QPN of the second host; establishing a first table item in the RDMA session table; a first ACK message is received, the source address is the address of the third host, the destination address is the address of the fourth host, and the QPN of the fourth host is carried; and if the address of the third host is the same as the address of the second host and the address of the fourth host is the same as the address of the first host, adding the QPN of the fourth host in the first table item to obtain a second table item. According to the scheme, the table item can be established by adopting hardware, the processing and forwarding performance of the RDMA gateway is improved, and the realizability and universality of the wide area network RDMA scheme based on the RDMA gateway are improved.

Description

Technical Field

[0001] This application relates to the field of communication technologies, and in particular, to a method, apparatus, device, and medium for establishing session entries. Background Art

[0002] To implement Wide Area Network (WAN) Remote Direct Memory Access (RDMA) and meet the requirements of high-speed access and transmission of data, RDMA gateways are respectively set at two host ends of communication. The RDMA gateway establishes an RDMA session table. After receiving a data packet sent by the local host, the RDMA gateway replies with a pseudo-acknowledgment (ACK) / negative acknowledgment (NACK) packet to the local host according to the RDMA session table, and retransmits lost packets to the peer host to accelerate data transmission and solve the problem of lost packets.

[0003] The process for the RDMA gateway to establish an RDMA session table is as follows: Listen for packets used to establish an RDMA session that are exchanged between two hosts, obtain information from the packets, and establish an entry corresponding to the RDMA session in the RDMA session table according to the obtained information. The packets used to establish an RDMA session can be Connection Manager (CM) packets or Transmission Control Protocol (TCP) packets.

[0004] When using CM packets to establish an RDMA session, since CM packets are relatively complex, it is difficult for the RDMA gateway to parse CM packets, and thus it is difficult to establish entries. When using TCP packets to establish an RDMA session, since TCP packets do not have a fixed message type and packet format, the RDMA gateway cannot establish entries by listening to TCP packets. Therefore, a method for establishing session entries is needed to establish entries using hardware, improve the processing and forwarding performance of the RDMA gateway, and improve the feasibility and generality of the WAN RDMA solution based on the RDMA gateway. Summary of the Invention

[0005] The purpose of the embodiments of this application is to provide a method, apparatus, device, and medium for establishing session entries to establish entries using hardware, improve the processing and forwarding performance of the RDMA gateway, and improve the feasibility and generality of the WAN RDMA solution based on the RDMA gateway. The specific technical solutions are as follows:

[0006] In a first aspect, the embodiments of this application provide a method for establishing session entries, which is applied to an RDMA gateway. The method includes:

[0007] Receive a first RDMA data packet, where the source address of the first RDMA data packet is the address of a first host, the destination address of the first RDMA data packet is the address of a second host, and the Queue Pair Number (QPN) carried in the first RDMA data packet is the QPN of the second host;

[0008] Establish a first entry in the RDMA session table, where the first entry includes the address of the first host, the address of the second host, and the QPN of the second host;

[0009] Receive a first ACK packet, where the source address of the first ACK packet is the address of a third host, the destination address of the first ACK packet is the address of a fourth host, and the QPN carried in the first ACK packet is the QPN of the fourth host;

[0010] If the address of the third host is the same as the address of the second host, and the address of the fourth host is the same as the address of the first host, then add the QPN of the fourth host to the first entry to obtain a second entry.

[0011] In some embodiments, the first RDMA data packet further carries an ACK flag; the method further includes:

[0012] Determine whether the ACK flag carried in the first RDMA data packet is a preset value;

[0013] If so, perform the step of establishing the first entry in the RDMA session table;

[0014] If not, forward the first RDMA data packet according to the address of the second host.

[0015] In some embodiments, the first RDMA data packet further carries a first Packet Sequence Number (PSN), the first ACK packet further carries a second PSN, and the first entry further includes the first PSN;

[0016] The step of, if the address of the third host is the same as the address of the second host, and the address of the fourth host is the same as the address of the first host, then add the QPN of the fourth host to the first entry to obtain a second entry, includes:

[0017] If the address of the third host is the same as the address of the second host, the address of the fourth host is the same as the address of the first host, and the second PSN is the same as the first PSN, then add the QPN of the fourth host to the first entry to obtain a second entry.

[0018] In some embodiments, after obtaining the second entry, the method further includes:

[0019] Add a timestamp to the second entry;

[0020] When the duration from the timestamp to the current moment exceeds a preset duration, delete the second entry.

[0021] In some embodiments, after obtaining the second entry, the method further includes:

[0022] Receive a second RDMA data packet;

[0023] If the source address of the second RDMA data packet is the address of the first host, the destination address of the second RDMA data packet is the address of the second host, and the QPN carried by the second RDMA data packet is the QPN of the second host, then update the timestamp of the second entry to the current moment;

[0024] Forward the second RDMA data packet according to the address of the second host.

[0025] In some embodiments, if the source address of the second RDMA data packet is the address of the first host, the destination address of the second RDMA data packet is the address of the second host, and the QPN carried by the second RDMA data packet is the QPN of the second host, then the method further includes:

[0026] Send a second ACK packet corresponding to the second RDMA data packet according to the address of the first host, the source address of the second ACK packet is the address of the second host, the destination address of the second ACK packet is the address of the first host, and the QPN carried by the second ACK packet is the QPN of the fourth host.

[0027] In some embodiments, after obtaining the second entry, the method further includes:

[0028] Receive a third ACK packet;

[0029] If the source address of the third ACK packet is the address of the second host, the destination address of the third ACK packet is the address of the first host, and the QPN carried by the third ACK packet is the QPN of the fourth host, then delete the third RDMA data packet corresponding to the third ACK packet, where the source address of the third RDMA data packet is the address of the first host, the destination address of the third RDMA data packet is the address of the second host, and the QPN carried by the third RDMA data packet is the QPN of the second host.

[0030] In some embodiments, before obtaining the second entry, the method further includes:

[0031] Receiving a fourth RDMA data packet;

[0032] If the source address of the fourth RDMA data packet is the address of the first host, the destination address of the fourth RDMA data packet is the address of the second host, and the QPN carried by the fourth RDMA data packet is the QPN of the second host, then cache the fourth RDMA data packet or discard the fourth RDMA data packet.

[0033] In a second aspect, an embodiment of the present application provides a session entry establishment device, which is applied to an RDMA gateway. The device includes:

[0034] A first receiving module, configured to receive a first RDMA data packet, where the source address of the first RDMA data packet is the address of a first host, the destination address of the first RDMA data packet is the address of a second host, and the QPN carried by the first RDMA data packet is the QPN of the second host;

[0035] An establishment module, configured to establish a first entry in the RDMA session table, where the first entry includes the address of the first host, the address of the second host, and the QPN of the second host;

[0036] A second receiving module, configured to receive a first ACK packet, where the source address of the first ACK packet is the address of a third host, the destination address of the first ACK packet is the address of a fourth host, and the QPN carried by the first ACK packet is the QPN of the fourth host;

[0037] An adding module, configured to add the QPN of the fourth host to the first entry to obtain a second entry if the address of the third host is the same as the address of the second host and the address of the fourth host is the same as the address of the first host.

[0038] In some embodiments, the first RDMA data packet further carries an ACK flag; the apparatus further includes: a determination module, configured to determine whether the ACK flag carried in the first RDMA data packet is a preset value; if so, perform the step of establishing a first entry in the RDMA session table; if not, forward the first RDMA data packet according to the address of the second host.

[0039] In some embodiments, the first RDMA data packet further carries a first PSN, the first ACK packet further carries a second PSN, and the first entry further includes the first PSN; specifically, the second receiving module is configured to, if the address of the third host is the same as the address of the second host, the address of the fourth host is the same as the address of the first host, and the second PSN is the same as the first PSN, add the QPN of the fourth host to the first entry to obtain a second entry.

[0040] In some embodiments, the second receiving module is further configured to, after obtaining the second entry, add a timestamp to the second entry; when the duration from the timestamp to the current moment exceeds a preset duration, delete the second entry.

[0041] In some embodiments, the second receiving module is further configured to, after obtaining the second entry, receive a second RDMA data packet; if the source address of the second RDMA data packet is the address of the first host, the destination address of the second RDMA data packet is the address of the second host, and the QPN carried in the second RDMA data packet is the QPN of the second host, update the timestamp of the second entry to the current moment; forward the second RDMA data packet according to the address of the second host.

[0042] In some embodiments, the second receiving module is further configured to, if the source address of the second RDMA data packet is the address of the first host, the destination address of the second RDMA data packet is the address of the second host, and the QPN carried in the second RDMA data packet is the QPN of the second host, send a second ACK packet corresponding to the second RDMA data packet according to the address of the first host, the source address of the second ACK packet is the address of the second host, the destination address of the second ACK packet is the address of the first host, and the QPN carried in the second ACK packet is the QPN of the fourth host.

[0043] In some embodiments, the second receiving module is further configured to receive a third ACK packet after obtaining the second entry; if the source address of the third ACK packet is the address of the second host, the destination address of the third ACK packet is the address of the first host, and the QPN carried in the third ACK packet is the QPN of the fourth host, then delete the third RDMA data packet corresponding to the third ACK packet, where the source address of the third RDMA data packet is the address of the first host, the destination address of the third RDMA data packet is the address of the second host, and the QPN carried in the third RDMA data packet is the QPN of the second host.

[0044] In some embodiments, the second receiving module is further configured to receive a fourth RDMA data packet before obtaining the second entry; if the source address of the fourth RDMA data packet is the address of the first host, the destination address of the fourth RDMA data packet is the address of the second host, and the QPN carried in the fourth RDMA data packet is the QPN of the second host, then cache the fourth RDMA data packet or discard the fourth RDMA data packet.

[0045] In a third aspect, an embodiment of the present application provides an electronic device, including a processor, a communication interface, a memory, and a communication bus. Among them, the processor, the communication interface, and the memory complete communication with each other through the communication bus;

[0046] The memory is used to store a computer program;

[0047] The processor is configured to implement the session entry establishment method described in any one of the above when executing the program stored in the memory.

[0048] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, in which a computer program is stored, and the computer program implements the session entry establishment method described in any one of the above when executed by a processor.

[0049] In a fifth aspect, an embodiment of the present application provides a computer program product containing instructions, which when running on a computer, causes the computer to execute the session entry establishment method described in any one of the above.

[0050] Beneficial effects of the embodiments of the present application:

[0051] In the technical solution provided by the embodiment of the present application, the RDMA gateway establishes an entry in the RDMA session table according to the RDMA data packet sent by the first host (i.e., the source host) to the second host (i.e., the destination host) and the ACK packet replied by the second host. When the RDMA gateway receives the RDMA data packet sent by the first host to the second host, the RDMA gateway establishes a first entry in the RDMA session table according to the source address of the RDMA data packet (i.e., the address of the first host), the destination address (i.e., the address of the second host), and the destination QPN (i.e., the QPN of the second host). When the RDMA gateway receives the ACK packet, the RDMA gateway searches for an entry in the RDMA session table with the source address being the address of the fourth host and the destination address being the address of the third host according to the source address (i.e., the address of the third host) and the destination address (i.e., the address of the fourth host) of the ACK packet. If the found first entry indicates that the first host and the fourth host are the same host, and the second host and the third host are the same host, then the QPN of the fourth host (i.e., the QPN of the first host) carried in the ACK packet, that is, the source QPN, is added to the found first entry to obtain a complete entry.

[0052] Applying the technical solution provided by the embodiment of the present application, the RDMA gateway uses the RDMA data packet and the ACK packet with simple message formats to establish entries, and then establishes the RDMA session table, which can realize establishing entries using the hardware in the RDMA gateway, improve the processing and forwarding performance of the RDMA gateway, and the RDMA gateway does not need to listen for the messages used to establish the RDMA session, improving the implementation and generality of the wide-area network RDMA solution based on the RDMA gateway.

[0053] Of course, implementing any product or method of the present application does not necessarily need to achieve all the above-mentioned advantages simultaneously. BRIEF DESCRIPTION OF THE DRAWINGS

[0054] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the following will briefly introduce the drawings required for use in the description of the embodiments or the prior art. Obviously, the drawings in the following description are only some embodiments of the present application, and those of ordinary skill in the art can also obtain other embodiments according to these drawings.

[0055] Figure 1 It is a flowchart of the RDMA gateway working;

[0056] Figure 2 It is a first schematic flowchart of the session entry establishment method provided by the embodiment of the present application;

[0057] Figure 3 It is an interaction diagram of the session entry establishment method provided by the embodiment of the present application;

[0058] Figure 4 It is the second process schematic diagram of the session table entry establishment method provided by the embodiments of the present application;

[0059] Figure 5 It is a schematic diagram of wide area network RDMA communication between a source host and a destination host provided by the embodiments of the present application;

[0060] Figure 6 It is a schematic diagram of an RDMA data packet provided by the embodiments of the present application;

[0061] Figure 7 It is a schematic diagram of an ACK packet provided by the embodiments of the present application;

[0062] Figure 8 It is a flowchart of establishing an RDMA session table provided by the embodiments of the present application;

[0063] Figure 9 It is a schematic structural diagram of a session table entry establishment device provided by the embodiments of the present application;

[0064] Figure 10 It is a schematic structural diagram of an electronic device provided by the embodiments of the present application. Detailed implementation manners

[0065] Next, the technical solutions in the embodiments of the present application will be clearly and completely described in conjunction with the accompanying drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art based on the present application belong to the scope of protection of the present application.

[0066] RDMA is a network technology that allows a host to directly access the memory of another host without the intervention of their respective operating systems. RDMA has advantages and characteristics such as low latency, high throughput, and low central processing unit (CPU) overhead, and generally requires the support of a network card (such as an RDMA network card) in terms of hardware and software. RDMA is mainly applied inside a data center or an intelligent computing center to support services such as high performance computing (HPC), artificial intelligence (AI) training, and data storage. Through the RDMA network card and the InfiniBand (IB) network or the RDMA over Converged Ethernet (RoCE) network, high-speed access and transmission of data are achieved.

[0067] RDMA packets based on the RoCE network all have a Base Transport Header (BTH). The Opcode field in the BTH is used to identify the communication mode and packet type of RDMA. Among them, bits 7-5 of the Opcode field represent the communication mode, and bits 4-0 represent the packet type. For example, the 7th-5th bits of the Opcode field corresponding to the Reliable Connection (RC) mode defined by the RDMA standard are 000 (represented as 0x0). In the RC mode, the 4th-0th bits of the Opcode field of the data packet for Send First are 00000 (represented as 0x0), and the 4th-0th bits of the Opcode field of the data packet for Write Last are 01000 (represented as 0x8). The specific correspondence between the communication mode and packet type of RDMA and the Opcode field can be found in the RDMA standard and will not be elaborated here for the time being.

[0068] Before RDMA data access and transmission between RDMA hosts (i.e., hosts that support RDMA, for ease of description, RDMA hosts will be referred to as hosts hereinafter), it is generally necessary to establish an RDMA session (which can also be called an RDMA connection). For ease of description, among the two hosts that perform RDMA communication in the RDMA session, the host that sends the data packet is called the source host, and the host that receives the data packet is called the destination host. By establishing an RDMA session, the source host and the destination host exchange information such as each other's QPN and initial PSN. An RDMA connection is uniquely identified by the Internet Protocol (IP) address of the source host (i.e., the source IP address), the IP address of the destination host (i.e., the destination IP address), the QPN of the source host (i.e., the source QPN), and the QPN of the destination host (i.e., the destination QPN). When the source host sends an RDMA packet to the destination host, the RDMA packet needs to carry the QPN of the destination host (i.e., the destination QPN) and the PSN of the destination host (i.e., the destination PSN). Generally, it is required that the destination PSN starts from the initial PSN of the destination host and increases sequentially. When the destination host replies to the corresponding RDMA packet to the source host, the corresponding RDMA packet needs to carry the QPN of the source host (i.e., the source QPN) and the destination PSN. That is, the PSNs carried in the RDMA packet sent by the source host to the destination host and the RDMA packet sent by the destination host to the source host are the same.

[0069] With the development of services such as edge computing, multi-data center interconnection, and east-west data computing, the demand for high-speed access and transmission of data has extended from within the data center to the wide area network scenario, and the advantages of RDMA technology have also been extended to the wide area network scenario. The wide area network has characteristics such as large latency, high packet loss rate, and bandwidth limitation. The hardware and software of traditional RDMA network cards cannot directly support the wide area network. To implement wide area network RDMA, one method is to redesign and develop new hardware and software, but the development difficulty is great, the cost is also high, and it is not compatible with traditional RDMA network cards. Another method is to use an RDMA gateway (which can also be called an RDMA proxy).

[0070] As Figure 1 shown in the working flow chart of the RDMA gateway, two communicating host ends (i.e., RDMA host 1 and RDMA host 2) are respectively set with RDMA gateways (i.e., GW1 and GW2), and an RDMA session table (which can also be called an RDMA flow table) is established on the RDMA gateway. The RDMA session table generally records information such as source IP, destination IP, source QPN, destination QPN, destination PSN, and message cache. The data message of the host is first sent to the RDMA gateway, and the RDMA gateway replies with a pseudo ACK / NACK message to the local host according to the RDMA session table, and retransmits the lost packets to the peer host to accelerate the transmission of the data message and solve the packet loss problem.

[0071] Figure 1 In, RDMA host 1 sends data messages (message, msg) to GW1, such as Figure 1 msg1, msg2, and msg3 in. When GW1 receives the data message sent by RDMA host 1, GW1 sends the data message to GW2 according to the RDMA session table, and replies with a pseudo ACK / NACK message to RDMA host 1, reducing the waiting time of RDMA host 1 for the ACK / NACK message, thereby accelerating the sending of the data message of RDMA host 1. After receiving the data message, GW2 sends the data message to RDMA host 2, receives the real ACK / NACK message sent by RDMA host 2, and then forwards the real ACK / NACK message to GW1.

[0072] The RDMA gateway will also cache the data message according to the RDMA session table. When packet loss occurs on the wide area network path between the two RDMA gateways (such as Figure 1 msg2 in), GW1 retransmits msg2 according to the RDMA session table and the cached data message, thereby solving the problem of packet loss on the wide area network path of wide area network RDMA.

[0073] The process for the RDMA gateway to establish an RDMA session table is as follows: Listen for the packets used to establish an RDMA session that are exchanged between two hosts, obtain information from the packets, and create an entry corresponding to the RDMA session in the RDMA session table based on the obtained information. The packets used to establish an RDMA session can be CM packets or TCP packets.

[0074] 1) The host uses CM packets to establish an RDMA session. CM packets include connect request packets, connect reply packets, and ready to use packets, such as Figure 1 . Among them, the connect request packet includes the source QPN and the initial PSN of the source host, and the connect reply packet includes the destination QPN and the initial PSN of the destination host.

[0075] The RDMA gateway listens for CM packets, parses the CM packets, and establishes an RDMA session table. Since the packet format of CM packets is relatively complex and includes a Datagram Extended Transport Header (DETH) and a Management Datagram Header in addition to general data packets, it is difficult for the RDMA gateway to parse CM packets, which in turn makes it difficult to establish entries.

[0076] 2) The host uses TCP packets to establish an RDMA session. Before the source host and the destination host perform RDMA communication, they first establish a TCP connection and exchange information such as the QPN and the initial PSN of the peer through the TCP connection. In actual applications, there are also many applications that establish an RDMA session through a TCP connection and do not use CM packets, such as the NVIDIA Collective Communications Library (NCCL). Since TCP packets do not have a fixed message type and packet format, the RDMA gateway cannot establish entries by listening for TCP packets, which affects the generality of the RDMA gateway for wide area network RDMA.

[0077] Therefore, a method for establishing session entries is needed to establish entries using hardware, improve the processing and forwarding performance of the RDMA gateway, and improve the feasibility and generality of the wide area network RDMA solution based on the RDMA gateway.

[0078] To solve the above problems, an embodiment of the present application provides a method for establishing a session entry, which is applied to an RDMA gateway. The path where the RDMA gateway is located also includes a first host and a second host. An RDMA session has been established between the first host and the second host for RDMA communication. For the convenience of description, hereinafter, the first host is taken as the source host for sending data packets, and the second host is taken as the destination host for receiving data packets as an example for illustration, which does not play a limiting role. The RDMA session can be established using CM packets or TCP packets. The establishment method of the RDMA session is not limited herein. In the embodiment of the present application, the RDMA gateway can be an RDMA gateway at the first host end or an RDMA gateway at the second host end, which is not limited herein. Refer to Figure 2 , which is the first flow schematic diagram of the method for establishing a session entry provided by the embodiment of the present application. The above method for establishing a session entry includes the following steps.

[0079] Step S21: Receive a first RDMA data packet. The source address of the first RDMA data packet is the address of the first host, the destination address of the first RDMA data packet is the address of the second host, and the QPN carried by the first RDMA data packet is the QPN of the second host.

[0080] Step S22: Establish a first entry in the RDMA session table. The first entry includes the address of the first host, the address of the second host, and the QPN of the second host.

[0081] Step S23: Receive a first ACK packet. The source address of the first ACK packet is the address of the third host, the destination address of the first ACK packet is the address of the fourth host, and the QPN carried by the first ACK packet is the QPN of the fourth host.

[0082] Step S24: If the address of the third host is the same as the address of the second host, and the address of the fourth host is the same as the address of the first host, then add the QPN of the fourth host to the first entry to obtain a second entry.

[0083] In the technical solution provided by the embodiment of the present application, the RDMA gateway establishes an entry in the RDMA session table according to the RDMA data packet sent by the first host (i.e., the source host) to the second host (i.e., the destination host) and the ACK packet replied by the second host. When the RDMA gateway receives the RDMA data packet sent by the first host to the second host, the RDMA gateway establishes a first entry in the RDMA session table according to the source address of the RDMA data packet (i.e., the address of the first host), the destination address (i.e., the address of the second host), and the destination QPN (i.e., the QPN of the second host). When the RDMA gateway receives the ACK packet, the RDMA gateway searches for an entry in the RDMA session table with the source address being the address of the fourth host and the destination address being the address of the third host according to the source address (i.e., the address of the third host) and the destination address (i.e., the address of the fourth host) of the ACK packet. If the first entry found indicates that the first host and the fourth host are the same host, and the second host and the third host are the same host, then the QPN of the fourth host carried in the ACK packet (i.e., the QPN of the first host), that is, the source QPN, is added to the found first entry to obtain a complete entry.

[0084] Applying the technical solution provided by the embodiment of the present application, the RDMA gateway uses the RDMA data packet and the ACK packet with a simple message format to establish an entry, and then establishes an RDMA session table, which can realize establishing the entry by using the hardware in the RDMA gateway, improving the processing and forwarding performance of the RDMA gateway. The RDMA gateway also does not need to listen for the packets used to establish the RDMA session, improving the implementation and generality of the wide area network RDMA solution based on the RDMA gateway.

[0085] In the above step S21, the first host sends an RDMA data packet to the second host through the RDMA gateway. The source address of the RDMA data packet is the address of the first host, the destination address is the address of the second host, and the carried QPN is the QPN of the second host. In the embodiment of the present application, the source address is the source IP address, and the destination address is the destination IP address.

[0086] In the embodiment of the present application, the RDMA data packet may include an Ethernet II header, an IP header, a User Datagram Protocol (UDP) header, and an IB. The IB includes a BTH and an Invariant Cyclic Redundancy Check (ICRC) field. The IP header may be an Internet Protocol Version 6 (IPv6) header or an Internet Protocol Version 4 (IPv4) header, which is not limited herein.

[0087] The source address and destination address of the RDMA data packet are carried in the source IP address field and destination IP address field in the IP header respectively, and the QPN is carried in the destination queue pair (DPN) field in the BTH.

[0088] In the embodiment of the present application, the RDMA gateway at the first host end receives the RDMA data packet sent by the first host, and the RDMA gateway at the second host end receives the RDMA data packet forwarded by the RDMA gateway at the first host end. The first RDMA data packet received by the above RDMA gateway is the first RDMA data packet.

[0089] In the above step S22, the RDMA gateway stores an RDMA session table, and the RDMA session table includes entries corresponding to each RDMA session. The entry corresponding to an RDMA session includes the source address, destination address, source QPN, and destination QPN corresponding to the RDMA session.

[0090] After receiving the first RDMA data packet, the RDMA gateway (i.e., the first RDMA gateway and the second RDMA gateway) can parse the first RDMA data packet to obtain the source address, destination address, and QPN from the first RDMA data packet. That is, the address of the first host, the address of the second host, and the QPN of the second host are obtained. The RDMA gateway creates a new entry in the RDMA session table and writes the address of the first host, the address of the second host, and the QPN of the second host as the source address, destination address, and destination QPN respectively into the new entry. The entry after writing is the first entry.

[0091] In the embodiment of the present application, after receiving the RDMA data packet, the RDMA gateway at the second host end continues to forward the received RDMA data packet to the second host.

[0092] After receiving the RDMA data packet, the second host can parse the received RDMA data packet and generate a corresponding ACK packet according to the RDMA data packet. The source address of the ACK packet is the address of the second host, the destination address is the address of the first host, and the carried QPN is the QPN of the first host. Corresponding to the above RDMA data packet, the source address and destination address of the ACK packet in the embodiment of the present application are carried in the source IP address field and destination IP address field in the IP header respectively, and the QPN is carried in the DPN field in the BTH.

[0093] The second host sends the generated ACK packet to the RDMA gateway at the second host side, and the RDMA gateway at the second host side then forwards the ACK packet to the RDMA gateway at the first host side, and the RDMA gateway at the first host side can continue to forward the ACK packet to the first host.

[0094] In the embodiments of the present application, the RDMA gateway at the second host side can also generate a corresponding ACK packet, that is, the RDMA gateway at the second host side can parse the received RDMA data packet by itself and generate a corresponding ACK packet according to the RDMA data packet, and forward the generated ACK packet to the RDMA gateway at the first host side. At this time, the second host does not need to generate an ACK packet. For ease of description, hereinafter, the case where the second host generates an ACK packet is taken as an example for illustration, which does not play a limiting role.

[0095] In the above step S23, the RDMA gateway continues to receive packets. The ACK packet received after receiving the first RDMA data packet is the first ACK packet. The source address of the first ACK packet is the address of the third host, the destination address is the address of the fourth host, and the carried QPN is the QPN of the fourth host, that is, the first ACK packet is an ACK packet sent by the third host to the fourth host through the RDMA gateway.

[0096] The first ACK packet and the first RDMA data packet can belong to the same RDMA session, that is, the destination address of the first ACK packet is the same as the source address of the first RDMA data packet, and the fourth host and the first host are the same host; the source address of the first ACK packet is the same as the destination address of the first RDMA data packet, and the third host and the second host are the same host.

[0097] The first ACK packet and the first RDMA data packet can also belong to different RDMA sessions. In this case, the RDMA gateway can also be located on other paths where the third host, the fourth host, and the other RDMA gateway are located, which is not limited herein.

[0098] When the RDMA gateway is the RDMA gateway at the third host side, the first ACK packet can be an ACK packet sent by the third host or a locally generated ACK packet; when the RDMA gateway is the RDMA gateway at the fourth host side, the first ACK packet can also be an ACK packet forwarded by the RDMA gateway at the third host side.

[0099] In the embodiments of the present application, the RDMA gateway can also receive a NACK packet. The specific processing process is the same as that of the ACK packet. Hereinafter, only the ACK packet is taken as an example for illustration.

[0100] In the above step S24, after obtaining the first ACK packet, the RDMA gateway can parse the first ACK packet to obtain the source address, destination address, and QPN from the first ACK packet, that is, obtain the address of the third host, the address of the fourth host, and the QPN of the fourth host.

[0101] The RDMA gateway can search for an entry in the RDMA session table with the source address being the address of the fourth host and the destination address being the address of the third host to match the entry of the first ACK packet in the RDMA session table.

[0102] If an entry is found in the RDMA session table and the found entry is the first entry, then the first host and the fourth host are the same host, the second host and the third host are the same host, and the first ACK packet and the first RDMA data packet belong to the same RDMA session.

[0103] The RDMA gateway can write the QPN of the fourth host (that is, the QPN of the first host) as the source QPN into the first entry to obtain a complete entry including the source address, destination address, source QPN, and destination QPN, and the obtained entry is the second entry.

[0104] If the found entry is not the first entry, then the first host and the fourth host are not the same host or the second host and the third host are not the same host, and the first ACK packet and the first RDMA data packet do not belong to the same RDMA session. When the found entry in the RDMA session table does not include the source QPN, the RDMA gateway can write the QPN of the fourth host as the source QPN into the found entry, or continue to forward the first ACK packet to the other end RDMA gateway or the fourth host according to the actual processing flow.

[0105] If no entry is found in the RDMA session table, indicating that the RDMA session to which the first ACK packet belongs has not established an entry in the RDMA session table, the RDMA gateway can forward the first ACK packet to the other end RDMA gateway or the fourth host according to the actual processing flow.

[0106] As Figure 3 An interaction diagram of the session entry establishment method provided by the embodiment of the present application as shown. The first host sends a first RDMA data packet to the RDMA gateway at the first host end. The RDMA gateway at the first host end receives the first RDMA data packet, establishes a first entry in the RDMA session table, and sends the first RDMA data packet to the RDMA gateway at the second host end through the wide area network. The RDMA gateway at the second host end receives the first RDMA data packet, establishes a first entry in the RDMA session table, and sends the first RDMA data packet to the second host.

[0107] The second host sends a first ACK packet to the RDMA gateway at the second host side. The RDMA gateway at the second host side receives the first ACK packet, matches the first entry of the first ACK packet in the RDMA session table, adds the QPN of the first host (i.e., the QPN of the fourth host) to the first entry to obtain a second entry, and sends the first ACK packet to the RDMA gateway at the first host side through the wide area network. The RDMA gateway at the first host side receives the first ACK packet, matches the first entry of the first ACK packet in the RDMA session table, adds the QPN of the first host to the first entry to obtain a second entry, and sends the first ACK packet to the first host. For details, please refer to the description in the above Figure 1 section and will not be elaborated here.

[0108] In some embodiments, the first RDMA data packet may also carry an ACK flag, that is, the first RDMA data packet is the first RDMA data packet carrying the ACK flag received by the RDMA gateway. The above method for establishing a session table entry may further include the following steps: before performing the above step S22, determine whether the ACK flag carried in the first RDMA data packet is a preset value; if so, perform the step of establishing the first entry in the RDMA session table, that is, perform the above step S22; if not, forward the first RDMA data packet according to the address of the second host.

[0109] In the embodiments of the present application, the ACK flag is carried in the acknowledge request (ack req) field in the BTH of the RDMA data packet, indicating whether the RDMA data packet needs to reply with a corresponding ACK packet.

[0110] After receiving the first RDMA data packet, the RDMA gateway first parses the first RDMA data packet and determines whether the ACK flag carried in the first RDMA data packet is a preset value, that is, determines whether the value of the acknowledge request field is a preset value. The preset value can be 1, 0, etc. Hereinafter, the preset value is taken as 1 as an example, which is not restrictive.

[0111] If the ACK flag is a preset value and the second host needs to reply with an ACK packet corresponding to the first RDMA data packet to the first host, the RDMA gateway performs the above step S22 to establish the first entry.

[0112] If the ACK flag is not a preset value and the second host does not need to reply with an ACK packet corresponding to the first RDMA data packet to the first host, the RDMA gateway forwards the first RDMA data packet. That is, the RDMA gateway at the first host side forwards the first RDMA data packet to the RDMA gateway at the second host side, and the RDMA gateway at the second host side forwards the first RDMA data packet to the second host.

[0113] In the technical solution provided by the embodiment of the present application, the RDMA gateway establishes an entry in the RDMA session table only when it is necessary to reply to the ACK message according to the ACK flag, so as to establish a complete entry according to the replied ACK message.

[0114] In some embodiments, the first RDMA data message may further carry a first PSN, the first ACK message may further carry a second PSN, and the first entry may further include the first PSN; the above step S24 may be implemented by the following steps: if the address of the third host is the same as the address of the second host, the address of the fourth host is the same as the address of the first host, and the second PSN is the same as the first PSN, then add the QPN of the fourth host to the first entry to obtain a second entry.

[0115] The RDMA data message and the ACK message may also carry a PSN to indicate the sequence number of the message. The PSN may be carried in the PSN field in the BTH. The PSN carried by the first RDMA data message is the first PSN. The PSN carried by the first ACK message is the second PSN. The entry corresponding to an RDMA session includes the source address, destination address, source QPN, destination QPN, and destination PSN corresponding to the RDMA session.

[0116] For the first RDMA data message, after receiving the first RDMA data message, the RDMA gateway may further parse the PSN from the first RDMA data message, that is, obtain the first PSN. After establishing a new entry, the RDMA gateway may further write the first PSN as the destination PSN into the new entry to obtain a first entry including the source address, destination address, destination QPN, and destination PSN.

[0117] For the first ACK message, after obtaining the first ACK message, the RDMA gateway may further parse the PSN from the first ACK message, that is, obtain the second PSN. Furthermore, the RDMA gateway may further search for an entry in the RDMA session table according to the second PSN to determine that the received first ACK message is the ACK message corresponding to the first RDMA data message. The RDMA gateway searches for an entry in the RDMA session table with the source address being the address of the fourth host, the destination address being the address of the third host, and the destination PSN being the second PSN to match the first ACK message in the RDMA session table.

[0118] If an entry is found in the RDMA session table and the found entry is the first entry, then the first host and the fourth host are the same host, the second host and the third host are the same host, the first PSN is the same as the second PSN, the first ACK packet and the first RDMA data packet belong to the same RDMA session, and the first ACK packet is the packet corresponding to the first RDMA data packet replied by the second host to the first host. The RDMA gateway can write the QPN of the fourth host as the source QPN into the first entry to obtain the second entry.

[0119] If the found entry is not the first entry, or no entry is found in the RDMA session table, the RDMA gateway can process it according to the actual processing flow. For details, please refer to the relevant description of step S24 above. Figure 2 partial description of step S24.

[0120] In the technical solution provided by the embodiments of the present application, the RDMA gateway further establishes an entry according to the PSNs carried in the RDMA data packet and the ACK packet, and only when the PSNs are the same, determines that the ACK packet is the ACK packet corresponding to the RDMA data packet, ensuring the accuracy of the establishment of the RDMA session table.

[0121] In some embodiments, before obtaining the second entry after establishing the first entry, the source QPN is not included in the first entry. The RDMA gateway can continue to receive RDMA data packets. Any RDMA data packet received by the RDMA gateway after receiving the first RDMA data packet and before obtaining the second entry is the fourth RDMA data packet. The above session table entry establishment method may further include the following steps: receiving the fourth RDMA data packet; if the source address of the fourth RDMA data packet is the address of the first host, the destination address of the fourth RDMA data packet is the address of the second host, and the QPN carried by the fourth RDMA data packet is the QPN of the second host, then cache the fourth RDMA data packet or discard the fourth RDMA data packet.

[0122] In the embodiments of the present application, the fourth RDMA data packet and the first RDMA data packet may belong to the same RDMA session or different RDMA sessions, and this is not limited.

[0123] The RDMA gateway can parse the fourth RDMA data packet, obtain the source address (i.e., the first source address), the destination address (i.e., the first destination address), and the QPN (i.e., the first QPN) from the fourth RDMA data packet, and search for an entry in the RDMA session table with the source address being the first source address, the destination address being the first destination address, and the destination QPN being the first QPN, so as to match the entry of the fourth RDMA data packet in the RDMA session table.

[0124] If an entry is found in the RDMA session table and the found entry is the first entry, then the first source address is the same as the address of the first host, the first destination address is the same as the address of the second host, the first QPN is the same as the QPN of the second host, and the fourth RDMA data packet belongs to the same RDMA session as the first RDMA data packet.

[0125] If the RDMA gateway determines that the source QPN is not included in the first entry, indicating that the RDMA session has not been established, the RDMA gateway can either directly discard the fourth RDMA data packet or cache the fourth RDMA data packet in the buffer corresponding to the first entry and wait to continue forwarding after the second entry is established.

[0126] In the embodiments of the present application, after the RDMA gateway at the first host end receives the fourth RDMA data packet sent by the first host, it caches or discards the fourth RDMA data packet, so the RDMA gateway at the second host end will not receive the fourth RDMA data packet.

[0127] If the found entry is not the first entry, the RDMA gateway can forward the fourth RDMA data packet according to the actual processing flow based on the first destination address, etc., which will not be elaborated here for the time being.

[0128] If no entry is found in the RDMA session table, indicating that the RDMA session to which the fourth RDMA data packet belongs has not established an entry in the RDMA session table, the RDMA gateway can use the fourth RDMA data packet as the first RDMA data packet, use the host that sends the fourth RDMA data packet as the first host, and use the host that receives the fourth RDMA data packet as the second host, and continue to execute the above steps S22 to S24 to establish the first entry in the RDMA session table.

[0129] In the technical solution provided by the embodiments of the present application, after the RDMA gateway receives an RDMA data packet belonging to the same RDMA session as the first RDMA data packet, it can first suspend forwarding the RDMA data packet until the second entry is established.

[0130] In the embodiments of the present application, before the RDMA gateway executes the above step S22, it may search for an entry in the RDMA session table whose source address is the address of the first host, destination address is the address of the second host, and destination QPN is the QPN of the second host, so as to match the entry of the first RDMA data packet. If an entry is found in the RDMA session table and the found entry includes the source QPN, the entry for the established RDMA session to which the first RDMA data packet belongs does not need to be established again, that is, step S22 is not executed. If the found entry does not include the source QPN, the first RDMA data packet is equivalent to the fourth RDMA data packet, and the RDMA gateway caches the first RDMA data packet. If no entry is found in the RDMA session table, the RDMA gateway may continue to execute the above steps S22 to S24 to establish the first entry and the second entry in the RDMA session table.

[0131] In the embodiments of the present application, before the RDMA gateway executes the above step S24, it may search for an entry in the RDMA session table whose source address is the address of the fourth host, destination address is the address of the third host, and source QPN is the QPN of the fourth host, so as to match the entry of the first ACK packet. If an entry is found in the RDMA session table, the entry for the established RDMA session to which the first ACK packet belongs is already there, and the RDMA gateway does not execute step S24 anymore. If no entry for the first ACK packet is found in the RDMA session table, the RDMA gateway may continue to execute step S24, such as searching for an entry in the RDMA session table whose source address is the address of the fourth host, destination address is the address of the third host, and destination PSN is the second PSN. For specific details, reference may be made to the relevant description of step S24 above.

[0132] In some embodiments, the RDMA gateway may preset an automatic deletion duration (i.e., the preset duration) for the entries in the RDMA session table to automatically age the established entries and improve the security and availability of the RDMA session table. The above method for establishing session table entries may further include the following steps: after obtaining the second entry in step S24, adding a timestamp to the second entry; when the duration from the timestamp to the current moment exceeds the preset duration, deleting the second entry.

[0133] In the embodiments of the present application, the preset duration may be 10 minutes, 1 hour, etc., and there is no limitation in this regard. The RDMA gateway may add a timestamp to the second entry after obtaining the second entry, or may add a timestamp to the first entry after establishing the first entry and update the added timestamp to the moment when the second entry is obtained after obtaining the second entry. There is no limitation on the way the RDMA gateway adds timestamps here.

[0134] When the duration from the timestamp of the second entry (or the first entry) determined by the RDMA gateway to the current moment exceeds the preset duration, the RDMA gateway deletes the second entry (or the first entry).

[0135] In some embodiments, after obtaining the second entry, where the source QPN is included in the second entry and a timestamp has been added, the RDMA gateway can continue to receive RDMA data packets. Refer to Figure 4 , which is the second flowchart of the session entry establishment method provided by the embodiments of the present application. The above session entry establishment method may include the following steps.

[0136] Step S41: Receive a first RDMA data packet. The source address of the first RDMA data packet is the address of the first host, the destination address of the first RDMA data packet is the address of the second host, and the QPN carried by the first RDMA data packet is the QPN of the second host. This is the same as step S21 above.

[0137] Step S42: Establish a first entry in the RDMA session table. The first entry includes the address of the first host, the address of the second host, and the QPN of the second host. This is the same as step S22 above.

[0138] Step S43: Receive a first ACK packet. The source address of the first ACK packet is the address of the third host, the destination address of the first ACK packet is the address of the fourth host, and the QPN carried by the first ACK packet is the QPN of the fourth host. This is the same as step S23 above.

[0139] Step S44: If the address of the third host is the same as the address of the second host, and the address of the fourth host is the same as the address of the first host, then add the QPN of the fourth host to the first entry to obtain a second entry. This is the same as step S24 above.

[0140] Step S45: Receive a second RDMA data packet.

[0141] Step S46: If the source address of the second RDMA data packet is the address of the first host, the destination address of the second RDMA data packet is the address of the second host, and the QPN carried by the second RDMA data packet is the QPN of the second host, then update the timestamp of the second entry to the current moment; forward the second RDMA data packet according to the address of the second host.

[0142] In the technical solution provided by the embodiments of the present application, the RDMA gateway continues to receive RDMA data packets, and updates the timestamp of the entry each time it matches an entry in the RDMA session table, ensuring that the required entries will not be deleted.

[0143] In the above step S45, the second RDMA data packet is any RDMA data packet received by the RDMA gateway after receiving the first RDMA data packet and obtaining the second entry. The second RDMA data packet and the first RDMA data packet may belong to the same RDMA session or different RDMA sessions, and this is not limited.

[0144] In the above step S46, the RDMA gateway can parse the second RDMA data packet, obtain the source address (i.e., the second source address), the destination address (i.e., the second destination address), and the QPN (i.e., the second QPN) from the second RDMA data packet, and search for an entry in the RDMA session table whose source address is the second source address, destination address is the second destination address, and destination QPN is the second QPN, so as to match the entry of the second RDMA data packet in the RDMA session table.

[0145] If an entry is found in the RDMA session table and the found entry is the second entry, then the second source address is the same as the address of the first host, the second destination address is the same as the address of the second host, the second QPN is the same as the QPN of the second host, and the second RDMA data packet and the first RDMA data packet belong to the same RDMA session.

[0146] When the RDMA gateway determines that the establishment of the second entry corresponding to the RDMA session is completed (the entry includes the source QPN), the RDMA gateway can update the timestamp of the second entry to the current moment and forward the second RDMA data packet according to the address of the second host (i.e., the second destination address). That is, the RDMA gateway at the first host end forwards the second RDMA data packet to the RDMA gateway at the second host end, and the RDMA gateway at the second host end forwards the second RDMA data packet to the second host.

[0147] If the found entry is not the second entry, then the second source address is different from the address of the first host, the second destination address is different from the address of the second host, or the second QPN is different from the QPN of the second host, and the second RDMA data packet and the first RDMA data packet do not belong to the same RDMA session. The RDMA gateway can process the second RDMA data packet according to the actual processing flow, such as forwarding the second RDMA data packet according to the second destination address, which will not be elaborated here for the time being.

[0148] If no entry is found in the RDMA session table, indicating that the entry for the RDMA session to which the second RDMA data packet belongs has not been established in the RDMA session table, the RDMA gateway may use the second RDMA data packet as the first RDMA data packet, the host that sends the second RDMA data packet as the first host, and the host that receives the second RDMA data packet as the second host, and continue to execute the above steps S42 to S44 to establish the first entry in the RDMA session table.

[0149] In the embodiment of the present application, the second RDMA data packet may also carry a PSN (i.e., the third PSN), and the second entry may also include a destination PSN and an expected PSN, where the expected PSN is the destination PSN plus one. The RDMA gateway may search for an entry in the RDMA session table with the source address being the second source address, the destination address being the second destination address, and the destination QPN being the second QPN, and determine the entry with the expected PSN being the third PSN therefrom as the found entry.

[0150] In the embodiment of the present application, the destination PSN included in the second entry starts from the first PSN (or the second PSN), and the expected PSN starts from the first PSN plus one. Whenever an entry for the received RDMA data packet is found in the RDMA session table and the received RDMA data packet is normally forwarded, both the destination PSN and the expected PSN of the second entry are incremented by one. If there is no entry with the expected PSN being the third PSN among the entries with matching source address, destination address, and destination QPN, it indicates that the transmission order of the RDMA data packet is incorrect, and packet loss and retransmission are required to ensure the correct transmission order of the RDMA data packet.

[0151] In some embodiments, the ACK flag carried in the second RDMA data packet is a preset value. The above method for establishing a session entry may further include the following steps: after finding the second entry, send a second ACK packet corresponding to the second RDMA data packet according to the address of the first host, where the source address of the second ACK packet is the address of the second host, the destination address of the second ACK packet is the address of the first host, and the QPN carried in the second ACK packet is the QPN of the fourth host.

[0152] After the RDMA gateway at the first host end finds the second entry, it may generate a pseudo ACK packet with the source address being the address of the second host, the destination address being the address of the first host, and carrying the QPN of the fourth host according to the source QPN, source address, and destination address included in the second entry. This pseudo ACK packet is the second ACK packet corresponding to the second RDMA data packet. The RDMA gateway at the first host end may send the second ACK packet to the first host.

[0153] After the RDMA gateway of the second host end finds the second entry, it can receive the ACK packet (true ACK packet) corresponding to the second RDMA data packet sent by the second host. This ACK packet is the second ACK packet corresponding to the second RDMA data packet. The RDMA gateway of the second host end can also generate an ACK packet (true ACK packet) with the source address being the address of the second host, the destination address being the address of the first host, and carrying the QPN of the fourth host according to the source QPN, source address, and destination address included in the second entry. This ACK packet is the second ACK packet corresponding to the second RDMA data packet. The RDMA gateway of the second host end can send the second ACK packet to the RDMA gateway of the first host end.

[0154] In the embodiments of the present application, after the RDMA gateway finds the second entry, it can determine whether to send a pseudo ACK packet or a true ACK packet according to whether the received second RDMA data packet comes from a wide area network or a local area network. When the received second RDMA data packet comes from a wide area network, the RDMA gateway (such as the RDMA gateway of the second host end) replies with a true ACK packet; when the received second RDMA data packet comes from a local area network, the RDMA gateway (such as the RDMA gateway of the first host end) replies with a pseudo ACK packet. The RDMA gateway can determine whether the second RDMA data packet comes from a wide area network or a local area network through the port type of the received second RDMA data packet or the routing table for forwarding the second RDMA data packet, and this is not limited.

[0155] In the embodiments of the present application, after the RDMA gateway finds the second entry, it can also cache the second RDMA data packet, that is, cache the second RDMA data packet in the buffer area corresponding to the second entry for subsequent packet loss retransmission.

[0156] In the technical solution provided by the embodiments of the present application, the RDMA gateway generates a pseudo ACK packet and sends it to the first host according to the second entry, or forwards the ACK packet to the other end RDMA gateway to achieve the accelerated transmission of the RDMA data packet.

[0157] In some embodiments, after obtaining the second entry, the RDMA gateway can also receive a third RDMA data packet. The source address of the third RDMA data packet is the address of the first host, the destination address of the third RDMA data packet is the address of the second host, and the QPN of the third RDMA data packet is the QPN of the second host. That is, the second RDMA data packet belonging to the same RDMA session as the first RDMA data packet is the third RDMA data packet.

[0158] In an embodiment of the present application, the RDMA gateway may cache the third RDMA data packet in the buffer area corresponding to the second entry. For the specific processing procedure of the third RDMA data packet by the RDMA gateway, reference may be made to the relevant description in the above step S46.

[0159] After obtaining the second entry, the RDMA gateway may continue to receive ACK packets. The above method for establishing a session entry may further include the following steps: receiving a third ACK packet; if the source address of the third ACK packet is the address of the second host, the destination address of the third ACK packet is the address of the first host, and the QPN carried by the third ACK packet is the QPN of the fourth host, then deleting the third RDMA data packet corresponding to the third ACK packet.

[0160] In an embodiment of the present application, the third ACK packet is any ACK packet received by the RDMA gateway after obtaining the second entry. The third ACK packet and the first ACK packet may belong to the same RDMA session or different RDMA sessions, and this is not limited.

[0161] The RDMA gateway may parse the third ACK packet, obtain the source address, destination address, and QPN from the third ACK packet, and search for an entry in the RDMA session table whose source address is the third source address, destination address is the third destination address, and destination QPN is the third QPN, so as to match the entry of the third ACK packet in the RDMA session table.

[0162] If an entry is found in the RDMA session table and the found entry is the second entry, then the third source address is the same as the address of the second host, the third destination address is the same as the address of the first host, the third QPN is the same as the QPN of the fourth host, and the third ACK packet and the first ACK packet belong to the same RDMA session. After receiving the third ACK packet, the RDMA gateway may determine that the corresponding RDMA data packet (such as the third RDMA data packet received before receiving the third ACK packet) has been accurately transmitted to the second gateway and no longer needs to retransmit lost packets. Then, according to the position indicated by the second entry, the cached third RDMA data packet is deleted. The RDMA gateway may also update the timestamp of the second entry to the current moment.

[0163] In an embodiment of the present application, the third ACK packet may carry a PSN, and the expected PSN included in the second entry is the same as the PSN carried by the third ACK packet. The RDMA gateway may also delete the third RDMA data packets cached in the RDMA gateway whose PSN is less than or equal to the PSN carried by the third ACK packet, and this is not limited.

[0164] If the found table entry is not the second table entry, or no table entry is found in the RDMA session table, the RDMA gateway can process it according to the actual processing flow, which will not be elaborated here for the time being.

[0165] The following will Figures 5 to 8 describe in detail the method for establishing a session table entry provided by the embodiments of the present application.

[0166] Figure 5 In the figure, RDMA host 1 and RDMA host 2 perform wide area network RDMA communication. RDMA host 1 sends data to RDMA host 2, that is, RDMA host 1 is the source host with an IP address of 13.0.0.4, and RDMA host 2 is the destination host with an IP address of 12.0.0.1, as Figure 5 shown by the data flow direction. RDMA host 1 and RDMA host 2 are respectively connected to an RDMA gateway, namely GW1 and GW2, to implement wide area network RDMA communication through the RDMA gateway.

[0167] RDMA host 1 and RDMA host 2 have established an RDMA session through TCP packets or CM packets. For this RDMA session: the source IP address is 13.0.0.4, the source QPN is 0x000172, the destination IP address is 12.0.0.1, the destination QPN is 0x000170, and the initial destination PSN is 10903959. The detailed process for the RDMA gateway (i.e., GW1 and GW2) to establish an RDMA session table is as follows:

[0168] Step 1) RDMA host 1 sends an RDMA data packet, and the PSN carried in the RDMA data packet increases sequentially, which are 10903959 (ack req = 0), 10903960 (ack req = 0), 10903961 (ack req = 0), 10903962 (ack req = 1), 10903963 (ack req = 0), etc.

[0169] As Figure 6 shown in the RDMA data packet, the PSN of this RDMA data packet is 10903962, and it includes frame 50, Ethernet header, IPv4 header, UDP header, IB, and data. Among them, the IPv4 header includes the source IP address (i.e., 13.0.0.4) and the destination IP address (12.0.0.1), the UDP header includes the source port (i.e., 65203) and the destination port (i.e., 4791), the IB includes BTH and ICRC, and the BTH includes the operation code, partition key (i.e., 65535), reserved bits, DQP (i.e., the QPN of RDMA host 2, 0x000170), acknowledgment request (i.e., ack req = 1), and PSN.

[0170] Step 2) The RDMA gateways (i.e., GW1 and GW2) listen for packets, determine that the packet is an RDMA packet based on the destination port number 4791 in the UDP header, and determine whether the packet is an RDMA data packet or an ACK packet based on the opcode in the BTH. For example, Figure 6 The opcode of the RDMA data packet shown is 01000000 (i.e., 0x08), which is an RDMA data packet for the last packet write operation in RC mode.

[0171] After determining that the received packet is an RDMA data packet, the RDMA gateway can first look up an entry in the RDMA session table based on the source IP address, destination IP address, and destination QPN indicated by the DQP to determine whether the entry corresponding to the RDMA session has been established.

[0172] When no entry is found, the RDMA gateway determines whether to create a new entry based on the value of ack req. If ackreq = 0, that is, execute Step 3), and forward the RDMA data packet normally; if ack req = 1, that is, execute Step 4), and create a new entry.

[0173] When an entry is found, the RDMA gateway determines whether the found entry includes the source QPN. If the source QPN is not included, that is, execute Step 5), and cache or discard the RDMA data packet; if the source QPN is included, it means that the entry corresponding to the RDMA session has been established. The RDMA gateway processes it according to the actual processing flow, such as determining whether the PSN carried in the RDMA data packet is the same as the destination PSN included in the found entry. If they are the same, the RDMA data packet is forwarded; if they are not the same, packet loss and retransmission are performed.

[0174] Step 3) The RDMA gateway normally forwards the RDMA data packets 10903959 (ack req = 0), 10903960 (ackreq = 0), 10903961 (ack req = 0).

[0175] Step 4) When the RDMA gateway receives the first RDMA data packet (PSN = 10903962) with ack req = 1 (i.e., carrying the ACK flag), it creates an entry A in an incomplete RDMA session table: the source IP address is 13.0.0.4, the source QPN is to be determined (the source QPN is not in the RDMA data packet), the destination IP address is 12.0.0.1, the destination QPN is 0x000170, and the destination PSN is 10903962. Then it continues to forward the data packet 10903962 (ack req = 1).

[0176] Step 5) The RDMA gateway continues to receive RDMA data packet 10903963 (ack req = 0) etc. from RDMA host 1, and first caches the received packets (the packets can also be discarded).

[0177] Step 6) The RDMA gateway waits for the ACK packet corresponding to RDMA data packet 10903962 (ack req = 1).

[0178] Step 7) RDMA host 2 sequentially receives RDMA data packets 10903959 (ack req = 0), 10903960 (ack req = 0), 10903961 (ack req = 0), 10903962 (ack req = 1).

[0179] Step 8) RDMA host 2 checks that ack req = 1 for RDMA data packet 10903962 and replies with the corresponding ACK packet, such as Figure 7 the ACK packet shown, which includes frame 67, Ethernet header, IPv4 header, UDP header, and IB. Among them, IB includes BTH, ACK Extended Transport Header (AETH), and ICRC. The source IP address of the ACK packet is 12.0.0.1, the destination IP address is 13.0.0.4, DQP (i.e., the QPN of RDMA host 1) is 0x000172, PSN is 10903962, and the operation code is 01000000 (i.e., 017), which is an ACK packet for confirmation response in RC mode.

[0180] Step 9) The RDMA gateway receives the ACK packet corresponding to RDMA data packet 10903962, uses the source IP address of the ACK packet as the destination IP address, uses the destination IP address of the ACK packet as the source IP address, and adds the PSN. By querying the RDMA session table, it can find entry A in the RDMA session table. Then, it fills the destination QPN = 0x000172 indicated by DQP into entry A as the source QPN. Finally, a complete entry A is obtained: the source IP address is 13.0.0.4, the source QPN is 0x000172, the destination IP address is 12.0.0.1, the destination QPN is 0x000170, the destination PSN is 10903962, and a timestamp is added to entry A.

[0181] Before performing a lookup using the source IP address, destination IP address, and PSN, the RDMA gateway can first determine whether the packet is an RDMA data packet or an ACK packet based on the operation code in the BTH. After determining that the received packet is an ACK packet, the RDMA gateway can look up an entry in the RDMA session table based on the source IP address, destination IP address, and destination QPN indicated by the DQP to determine whether the entry corresponding to the RDMA session has been established.

[0182] When no entry is found, the RDMA gateway can query the RDMA session table based on the source IP address, destination IP address, and PSN, which is the above step 9).

[0183] When an entry is found, the RDMA gateway further determines whether the PSN carried in the ACK packet is the same as the destination PSN included in the found entry. If they are the same, the ACK packet is forwarded; if not, packet loss retransmission is performed.

[0184] Step 10) The RDMA gateway can forward the ACK packet from the RDMA host 2, and after establishing entry A, continue to cache and forward RDMA data packets such as 10903963 (ack req = 0), and perform packet loss retransmission.

[0185] The RDMA gateway can continue to receive RDMA data packets, look up an entry in the RDMA session table based on the source IP address, destination IP address, destination QPN indicated by the DQP, and PSN of the RDMA data packet, and perform entry matching.

[0186] After finding an entry, the RDMA gateway can determine whether a pseudo-ACK packet needs to be replied based on the port type of the received RDMA packet. When the port type of the received RDMA packet is a local area network port, the RDMA gateway (such as GW1) can generate a pseudo-ACK packet and reply to the source host (such as RDMA host 1) with the pseudo-ACK packet. When the port type of the received RDMA packet is a wide area network port, the RDMA gateway (such as GW2) does not need to generate a pseudo-ACK packet.

[0187] The RDMA gateway can continue to receive ACK packets, look up an entry in the RDMA session table based on the source IP address, destination IP address, and destination QPN indicated by the DQP of the ACK packet, and perform entry matching. The RDMA gateway can delete the RDMA data packets with PSN less than or equal to the PSN carried in the true ACK packet and release the cache after receiving the true ACK packet.

[0188] After the RDMA host 1 and the RDMA host 2 finish the wide - area network RDMA communication, the timestamp of the table entry will no longer be updated. If the RDMA gateway checks that the timestamp in the RDMA session table has not been updated for a long time (for example, the duration from the timestamp to the current moment exceeds 10 minutes), the table entry in the RDMA session table will be aged out.

[0189] Such as Figure 8 The flowchart for establishing the RDMA session table as shown is applied to the RDMA gateway and includes the following steps.

[0190] Step S81, parse the RDMA message.

[0191] In the embodiment of the present application, the RDMA gateway parses the source IP address, destination IP address, destination QPN, and PSN from the RDMA data packet; and parses the source IP address, destination IP address, source QPN, and PSN from the ACK packet.

[0192] Step S82, determine whether it is a data packet or an ACK packet according to the operation code. If the RDMA packet is a data packet, execute step S83; if the RDMA packet is an ACK packet, execute step S86.

[0193] Step S83, check the ACK flag. If the ACK flag (i.e., ack req) is 1, execute step S84; if the ACK flag is 0, execute step S85.

[0194] Step S84, establish entry A.

[0195] In the embodiment of the present application, the RDMA gateway establishes entry A including the source IP address, destination IP address, destination QPN, and destination PSN, and executes step S85 to forward the RDMA packet.

[0196] Step S85, forward the packet. The RDMA gateway temporarily ends the establishment of the RDMA session entry, waits to receive the corresponding ACK packet to update the entry, and completes the establishment.

[0197] Step S86, search for entry A. If entry A is found, execute step S87; if entry A is not found, it means that the currently received ACK packet is not the corresponding ACK packet, then temporarily end the establishment of the RDMA session entry, wait to receive the corresponding ACK packet to update the entry, and complete the establishment.

[0198] Step S87, update entry A. The RDMA gateway ends the establishment of the RDMA session entry.

[0199] In the embodiment of the present application, when the RDMA gateway finds entry A, it adds the source QPN to entry A to obtain the updated entry and completes the establishment of the RDMA session entry.

[0200] Applying the technical solution provided by the embodiments of the present application, the embodiments of the present application propose a method for establishing an RDMA session table, which does not need to listen for CM messages or TCP messages, but directly establishes the RDMA session table according to the RDMA data message and the ACK message, so that the wide-area network RDMA solution based on the RDMA gateway has better feasibility and versatility. Since the BTH header lengths of the RDMA data message and the ACK message are short and fixed, it is easier to parse the message with hardware. The RDMA session table can be generated by hardware such as an Application-Specific Integrated Circuit (ASIC), a Field-Programmable Gate Array (FPGA), or a Network Processor (NP), thereby improving the processing and forwarding performance of the RDMA message.

[0201] In the technical solution provided by the embodiments of the present application, without relying on listening for TCP messages or CM messages, the RDMA session table is directly established according to the RDMA data message and the ACK message, which can be used in scenarios such as data centers and intelligent computing centers, and can also perform visualization, QoS guarantee, traffic scheduling, etc. of the RDMA message based on the RDMA session table.

[0202] Corresponding to the above session table entry establishment method, the embodiments of the present application also provide a session table entry establishment device, such as Figure 9 , which is applied to the RDMA gateway. The above device includes:

[0203] A first receiving module 91, configured to receive a first RDMA data message, where the source address of the first RDMA data message is the address of the first host, the destination address of the first RDMA data message is the address of the second host, and the QPN carried by the first RDMA data message is the QPN of the second host;

[0204] A establishing module 92, configured to establish a first entry in the RDMA session table, where the first entry includes the address of the first host, the address of the second host, and the QPN of the second host;

[0205] A second receiving module 93, configured to receive a first ACK message, where the source address of the first ACK message is the address of the third host, the destination address of the first ACK message is the address of the fourth host, and the QPN carried by the first ACK message is the QPN of the fourth host;

[0206] An adding module 94 is used to add the QPN of the fourth host to the first entry to obtain a second entry if the address of the third host is the same as the address of the second host and the address of the fourth host is the same as the address of the first host.

[0207] In the technical solution provided by the embodiment of the present application, the RDMA gateway establishes an entry in the RDMA session table according to the RDMA data packet sent by the first host (i.e., the source host) to the second host (i.e., the destination host) and the ACK packet replied by the second host. When the RDMA gateway receives the RDMA data packet sent by the first host to the second host, the RDMA gateway establishes a first entry in the RDMA session table according to the source address (i.e., the address of the first host), the destination address (i.e., the address of the second host), and the destination QPN (i.e., the QPN of the second host) of the RDMA data packet. When the RDMA gateway receives the ACK packet, the RDMA gateway searches for an entry in the RDMA session table whose source address is the address of the fourth host and whose destination address is the address of the third host according to the source address (i.e., the address of the third host) and the destination address (i.e., the address of the fourth host) of the ACK packet. If the found first entry indicates that the first host and the fourth host are the same host and the second host and the third host are the same host, then add the QPN of the fourth host (i.e., the QPN of the first host) carried in the ACK packet, that is, add the source QPN, to the found first entry to obtain a complete entry.

[0208] Applying the technical solution provided by the embodiment of the present application, the RDMA gateway uses the RDMA data packet and the ACK packet with a simple message format to establish an entry, and then establishes an RDMA session table, which can implement establishing an entry by using the hardware in the RDMA gateway, improving the processing and forwarding performance of the RDMA gateway. The RDMA gateway also does not need to listen for the packets used to establish the RDMA session, improving the implementation and generality of the wide-area network RDMA solution based on the RDMA gateway.

[0209] In some embodiments, the first RDMA data packet may also carry an ACK flag; the device may further include: a judging module, configured to judge whether the ACK flag carried in the first RDMA data packet is a preset value; if so, execute the step of establishing the first entry in the RDMA session table; if not, forward the first RDMA data packet according to the address of the second host.

[0210] In some embodiments, the first RDMA data packet may further carry a first PSN, the first ACK packet may further carry a second PSN, and the first entry may further include the first PSN; the second receiving module 93 may specifically be configured to, if the address of the third host is the same as the address of the second host, the address of the fourth host is the same as the address of the first host, and the second PSN is the same as the first PSN, add the QPN of the fourth host to the first entry to obtain a second entry.

[0211] In some embodiments, the second receiving module 93 may further be configured to, after obtaining the second entry, add a timestamp to the second entry; when the duration from the timestamp to the current moment exceeds a preset duration, delete the second entry.

[0212] In some embodiments, the second receiving module 93 may further be configured to, after obtaining the second entry, receive a second RDMA data packet; if the source address of the second RDMA data packet is the address of the first host, the destination address of the second RDMA data packet is the address of the second host, and the QPN carried by the second RDMA data packet is the QPN of the second host, update the timestamp of the second entry to the current moment; forward the second RDMA data packet according to the address of the second host.

[0213] In some embodiments, the second receiving module 93 may further be configured to, if the source address of the second RDMA data packet is the address of the first host, the destination address of the second RDMA data packet is the address of the second host, and the QPN carried by the second RDMA data packet is the QPN of the second host, send a second ACK packet corresponding to the second RDMA data packet according to the address of the first host, the source address of the second ACK packet is the address of the second host, the destination address of the second ACK packet is the address of the first host, and the QPN carried by the second ACK packet is the QPN of the fourth host.

[0214] In some embodiments, the second receiving module 93 may further be configured to, after obtaining the second entry, receive a third ACK packet; if the source address of the third ACK packet is the address of the second host, the destination address of the third ACK packet is the address of the first host, and the QPN carried by the third ACK packet is the QPN of the fourth host, delete the third RDMA data packet corresponding to the third ACK packet, the source address of the third RDMA data packet is the address of the first host, the destination address of the third RDMA data packet is the address of the second host, and the QPN carried by the third RDMA data packet is the QPN of the second host.

[0215] In some embodiments, the second receiving module 93 may further be configured to receive a fourth RDMA data packet before obtaining the second table entry; if the source address of the fourth RDMA data packet is the address of the first host, the destination address of the fourth RDMA data packet is the address of the second host, and the QPN carried in the fourth RDMA data packet is the QPN of the second host, then buffer the fourth RDMA data packet or discard the fourth RDMA data packet.

[0216] Embodiments of the present application further provide an electronic device, such as the above-mentioned RDMA gateway, as Figure 10 shown, including a processor 101, a communication interface 102, a memory 103, and a communication bus 104, where the processor 101, the communication interface 102, and the memory 103 complete communication with each other through the communication bus 104;

[0217] The memory 103 is used to store a computer program;

[0218] When the processor 101 is configured to execute the program stored on the memory 103, it implements the steps of any of the above session table entry establishment methods.

[0219] The communication bus mentioned in the above electronic device may be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. This communication bus can be divided into an address bus, a data bus, a control bus, etc. For the sake of simplicity, only a thick line is shown in the figure, but it does not mean that there is only one bus or one type of bus.

[0220] The communication interface is used for communication between the above electronic device and other devices.

[0221] The memory may include a Random Access Memory (RAM), and may also include a Non-Volatile Memory (NVM), such as at least one disk memory. Optionally, the memory may also be at least one storage device located far from the aforementioned processor.

[0222] The above-mentioned processor may be a general-purpose processor, including a Central Processing Unit (CPU), a Network Processor (NP), etc.; it may also be a Digital Signal Processor (DSP), an Application Specific Integrated Circuit (ASIC), a Field-Programmable Gate Array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components.

[0223] In another embodiment provided by the present application, there is also provided a computer-readable storage medium, in which a computer program is stored, and when the computer program is executed by a processor, the steps of any of the above session entry establishment methods are implemented.

[0224] In another embodiment provided by the present application, there is also provided a computer program product containing instructions, which when running on a computer, causes the computer to execute the session entry establishment method in any of the above embodiments.

[0225] In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware, or any combination thereof. When implemented using software, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the processes or functions described in the embodiments of the present application are generated in whole or in part. The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices. The computer instructions may be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions may be transmitted from one website, computer, server, or data center to another website, computer, server, or data center by wire (such as coaxial cable, optical fiber, Digital Subscriber Line (DSL)) or wireless (such as infrared, wireless, microwave, etc.). The computer-readable storage medium may be any available medium that can be accessed by a computer, or a data storage device such as a server or data center that includes one or more integrated available media. The available medium may be a magnetic medium (for example, a floppy disk, a hard disk, a magnetic tape), an optical medium (for example, a DVD), or a semiconductor medium (for example, a Solid State Disk (SSD)), etc.

[0226] It should be noted that in this text, relational terms such as first and second are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any actual relationship or order between these entities or operations. Moreover, the terms "comprising", "including" or any other variants thereof are intended to cover non-exclusive inclusion, such that a process, method, article or device comprising a series of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, method, article or device. Without further limitation, an element defined by the statement "comprising an..." does not exclude the presence of additional identical elements in the process, method, article or device comprising the said element.

[0227] Each embodiment in this specification is described in a related manner. For the same or similar parts among the embodiments, reference can be made to each other. Each embodiment focuses on the differences from other embodiments. In particular, for the embodiments of the device, electronic device, computer storage medium, and computer program product, since they are basically similar to the method embodiments, the description is relatively simple, and reference can be made to the corresponding parts of the method embodiments for the relevant content.

[0228] The above description is only a preferred embodiment of the present application and is not intended to limit the protection scope of the present application. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application are included within the protection scope of the present application.

Claims

1. A method for establishing a session entry, characterized in that, Applied to a Remote Direct Memory Access (RDMA) gateway, the method includes: Receiving a first RDMA data packet, where the source address of the first RDMA data packet is the address of a first host, the destination address of the first RDMA data packet is the address of a second host, and the Queue Pair Number (QPN) carried by the first RDMA data packet is the QPN of the second host; Establishing a first entry in the RDMA session table, where the first entry includes the address of the first host, the address of the second host, and the QPN of the second host; Receiving a first Acknowledgment (ACK) packet, where the source address of the first ACK packet is the address of a third host, the destination address of the first ACK packet is the address of a fourth host, and the QPN carried by the first ACK packet is the QPN of the fourth host; If the address of the third host is the same as the address of the second host, and the address of the fourth host is the same as the address of the first host, then add the QPN of the fourth host to the first entry to obtain a second entry.

2. The method according to claim 1, wherein The first RDMA data packet also carries an ACK flag; the method further includes: Determining whether the ACK flag carried in the first RDMA data packet is a preset value; If so, perform the step of establishing the first entry in the RDMA session table; If not, forward the first RDMA data packet according to the address of the second host.

3. The method according to claim 1, wherein The first RDMA data packet also carries a first Packet Sequence Number (PSN), the first ACK packet also carries a second PSN, and the first entry also includes the first PSN; The step of, if the address of the third host is the same as the address of the second host, and the address of the fourth host is the same as the address of the first host, then add the QPN of the fourth host to the first entry to obtain a second entry, includes: If the address of the third host is the same as the address of the second host, the address of the fourth host is the same as the address of the first host, and the second PSN is the same as the first PSN, then add the QPN of the fourth host to the first entry to obtain a second entry.

4. The method according to claim 1, wherein After obtaining the second entry, the method further includes: Adding a timestamp to the second entry; When the duration from the timestamp to the current moment exceeds a preset duration, delete the second entry.

5. The method according to claim 4, wherein After obtaining the second entry, the method further includes: Receiving a second RDMA data packet; If the source address of the second RDMA data packet is the address of the first host, the destination address of the second RDMA data packet is the address of the second host, and the QPN carried by the second RDMA data packet is the QPN of the second host, then update the timestamp of the second entry to the current moment; Forward the second RDMA data packet according to the address of the second host.

6. The method according to claim 5, wherein If the source address of the second RDMA data packet is the address of the first host, the destination address of the second RDMA data packet is the address of the second host, and the QPN carried by the second RDMA data packet is the QPN of the second host, then the method further includes: According to the address of the first host, send a second ACK packet corresponding to the second RDMA data packet, the source address of the second ACK packet is the address of the second host, the destination address of the second ACK packet is the address of the first host, and the QPN carried by the second ACK packet is the QPN of the fourth host.

7. The method according to claim 5, characterized in that, After obtaining the second entry, the method further includes: Receive a third ACK packet; If the source address of the third ACK packet is the address of the second host, the destination address of the third ACK packet is the address of the first host, and the QPN carried by the third ACK packet is the QPN of the fourth host, then delete the third RDMA data packet corresponding to the third ACK packet, the source address of the third RDMA data packet is the address of the first host, the destination address of the third RDMA data packet is the address of the second host, and the QPN carried by the third RDMA data packet is the QPN of the second host.

8. The method according to claim 5, wherein Before obtaining the second entry, the method further includes: Receive a fourth RDMA data packet; If the source address of the fourth RDMA data packet is the address of the first host, the destination address of the fourth RDMA data packet is the address of the second host, and the QPN carried by the fourth RDMA data packet is the QPN of the second host, then cache the fourth RDMA data packet or discard the fourth RDMA data packet.

9. A session entry establishment device, characterized in that Applied to a Remote Direct Memory Access (RDMA) gateway, the apparatus includes: A first receiving module, configured to receive a first RDMA data packet, the source address of the first RDMA data packet is the address of a first host, the destination address of the first RDMA data packet is the address of a second host, and the Queue Pair Number (QPN) carried by the first RDMA data packet is the QPN of the second host; A establishing module, configured to establish a first entry in an RDMA session table, the first entry includes the address of the first host, the address of the second host, and the QPN of the second host; A second receiving module, configured to receive a first Acknowledgment (ACK) packet, the source address of the first ACK packet is the address of a third host, the destination address of the first ACK packet is the address of a fourth host, and the QPN carried by the first ACK packet is the QPN of the fourth host; An adding module, configured to if the address of the third host is the same as the address of the second host, and the address of the fourth host is the same as the address of the first host, then add the QPN of the fourth host to the first entry to obtain a second entry.

10. An electronic device, characterized in that, It includes a processor, a communication interface, a memory, and a communication bus. Among them, the processor, the communication interface, and the memory complete their mutual communication through the communication bus; The memory is used for storing computer programs; The processor is used to implement the method according to any one of claims 1-8 when executing the programs stored on the memory.

11. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program, and when the computer program is executed by the processor, the method according to any one of claims 1-8 is implemented.

Images